Something to read

The main advantage of relational models is the ability to build dependencies between objects. That enables the structuring of the source data. The relational model can be represented in the form of graphs, in which objects and events are represented as nodes, while relationships show dependencies between objects and events.

more...

In the previous articles (Part 1, Part 2, Part 3), we experimented with shapes and angles whose values were passed to the perceptron and the neural network built on the basis of the DeepNeuralNetwork.mqh library. We also conducted experiments on optimization methods in the strategy tester.
An important task in the current experiments was to track the influence of the amount of transmitted data and the depth of history we take this data from. In addition, we needed to reveal patterns, whether short or long templates are better, as well as whether we should use fewer or more parameters for passing.

more...
-------------------

• Experiments with neural networks (Part 1): Revisiting geometry
• Experiments with neural networks (Part 2): Smart neural network optimization
• Experiments with neural networks (Part 3): Practical application

All traders hope to maximize the percentage return on their investment by as much as possible, however higher returns usually come at a higher risk. This is the reason why risk adjusted returns are the main measure of performance in the investment industry. There are many different measures of risk adjusted return, each one with its own set of advantages and disadvantages. The Sharpe ratio is a popular risk return measure famous for imposing unrealistic preconditions on the distribution of returns being analyzed. This has inevitably lead to the development of alternative performance metrics that seek to provide the same ubiquity of the Sharpe ratio without its shortcommings. In this article we provide the implementation of alternative risk return metrics and generate hypothetical equity curves to analyze their characteristics.

more...

In the previous article "Neural networks made easy (Part 39): Go-Explore, a different approach to exploration", we familiarized ourselves with the Go-Explore algorithm and its ability to explore the environment.

In this article, we will take a closer look at possible optimization methods for the Go-Explore algorithm to improve its efficiency over longer training periods.

more...

In this article, we will explore the application of hierarchical reinforcement learning in trading. We propose using this approach to create a hierarchical trading model that will be able to make optimal decisions at different levels and adapt to different market conditions.

In this article, we will consider the architecture of the hierarchical model, including various levels of decision making, such as determining entry and exit points for trades. We also present hierarchical model learning methods that combine global-level reinforcement learning and local-level reinforcement learning.

more...

In the field of reinforcement learning, neural network models often face the problem of procrastination when the learning process slows down or gets stuck. Model procrastination can have serious consequences for achieving goals and requires taking appropriate measures. In this article, we will look at the main reasons for model procrastination and propose methods for solving them.

more...

In this article, we introduce the concept of "Diversity is All You Need", which allows you to teach a model a skill without an explicit reward function. Variety of actions, exploration of the environment, and maximizing the variability of interactions with the environment are key factors for training an agent to behave effectively.

more...

In the previous article, we got acquainted with the DIAYN method, which allows you to train separable skills. This makes it possible to build a model that can change the agent behavior depending on the current state.

In this paradigm, a question arises of learning skills whose behavior would be easily predictable. At the same time, we are not ready to sacrifice the diversity of their behavior. A similar problem is solved by the authors of the Dynamics-Aware Discovery of Skills (DADS) method presented in 2020. Unlike DIAYN, the DADS method seeks to teach skills that not only have variety in behavior, but are also predictable.

more...

In this article, I propose to get acquainted with the alternative method of teaching skills Explore, Discover and Learn (EDL). EDL approaches the problem from a different angle, which allows it to overcome the problem of limited state coverage and offer more flexible and adaptive agent behavior.

more...

"Goal-conditioned reinforcement learning" sounds a little unusual or even strange. After all, the basic principle of reinforcement learning is aimed at maximizing the total reward during the interaction of the agent with the environment. But in this context, we are looking at achieving a specific goal at a specific stage or within a specific scenario.

more...