-
Quantization in machine learning (Part 1): Theory, sample code, analysis of implementation in CatBoost
Quote:
The article considers the theoretical application of quantization in the construction of tree models No complex mathematical equations are used. While writing the article, I discovered the absence of established unified terminology in the scientific works of different authors, so I will choose the terminology options that, in my opinion, best reflect the meaning. Besides, I will use the terms of my own in the matters left unattended by other researchers. This article will use terms and concepts I have previously described in the
article "CatBoost machine learning algorithm from Yandex without learning Python or R". Therefore, I recommend that you familiarize yourself with it before reading the current article.
more...
-
Neural networks made easy (Part 62): Using Decision Transformer in hierarchical models
Quote:
Previously, we considered hierarchical models for solving problems with, so to speak, the classical approach of the Markov process. However, the advantages of using hierarchical approaches also apply to sequence analysis problems. One such algorithm is the Control Transformer presented in the article "
Control Transformer: Robot Navigation in Unknown Environments through PRM-Guided Return-Conditioned Sequence Modeling". The method authors position it as a new architecture designed to solve complex control and navigation problems based on reinforcement learning. This method combines modern methods of reinforcement learning, planning and machine learning, which allows us to create adaptive control strategies in a variety of environments.
Control Transformer opens new perspectives for solving complex control problems in robotics, autonomous driving and other fields. I propose to look at the prospects for using this method in solving our trading problems.
more...
-
Quantization in machine learning (Part 2): Data preprocessing, table selection, training CatBoost models
Quote:
The article considers the practical application of quantization in the construction of tree models No complex mathematical equations are used. This is the second part of the article "
Quantization and other methods of preprocessing input data in machine learning", so I strongly recommend starting your acquaintance with it. Here we will talk about the following:
- In the first part, we will consider the methods for preprocessing sample data implemented in MQL5.
- In the second part, we will conduct an experiment that will provide information on the feasibility of data quantization.
more...
-
Neural networks made easy (Part 63): Unsupervised Pretraining for Decision Transformer (PDT)
Quote:
PDT jointly learns an embedding space of future trajectory as well as a future prior conditioned only on past information.. By conditioning action prediction on the target future embedding, PDT is endowed with the ability to "reason over the future". This ability is naturally task-independent and can be generalized to different task specifications.
To achieve efficient online fine-tuning in downstream tasks, you can easily adapt the framework to new conditions by associating each future embedding to its return, which is realized by training a reward prediction network for each future embedding.
more...
-
Neural networks made easy (Part 64): ConserWeightive Behavioral Cloning (CWBC) method
Quote:
The Decision Transformer and all its modifications, which we discussed in recent articles, belong to the methods of Behavior Cloning (BC). We train models to repeat actions from "expert" trajectories depending on the state of the environment and the target outcomes. Thus, we teach the model to imitate the behavior of an expert in the current state of the environment in order to achieve the target.
more...
-
Neural networks made easy (Part 65): Distance Weighted Supervised Learning (DWSL)
Quote:
Behavior cloning methods, largely based on the principles of supervised learning, show fairly good results. But their main problem remains the search for ideal role models, which are sometimes very difficult to collect. In turn, reinforcement learning methods are able to work with non-optimal raw data. At the same time, they can find suboptimal policies to achieve the goal. However, when searching for an optimal policy, we often encounter an optimization problem that is more relevant in high-dimensional and stochastic environments.
To bridge the gap between these two approaches, a group of scientists proposed the
Distance Weighted Supervised Learning (DWSL) method and presented it in the article "
Distance Weighted Supervised Learning for Offline Interaction Data". It is an offline supervised learning algorithm for goal-conditioned policy. Theoretically, DWSL converges to an optimal policy with a minimum return boundary at the level of trajectories from the training set. The practical examples in the article demonstrate the superiority of the proposed method over imitation learning and reinforcement learning algorithms. I suggest taking a closer look at this DWSL algorithm. We will evaluate its strengths and weaknesses in solving our practical problems.
more...
-
Neural networks made easy (Part 68): Offline Preference-guided Policy Optimization
Quote:
Reinforcement learning is a universal platform for learning optimal behavior policies in the environment under exploration. Policy optimality is achieved by maximizing the rewards received from the environment during interaction with it. But herein lies one of the main problems of this approach. The creation of an appropriate reward function often requires significant human effort. Additionally, rewards may be sparse and/or insufficient to express the true learning goal. As one of the options for solving this problem, the authors if the paper "
Beyond Reward: Offline Preference-guided Policy Optimization" suggested the OPPO method (OPPO stands for the Offline Preference-guided Policy Optimization). The authors of the method suggest the replacement of the reward given by the environment with the preferences of the human annotator between two trajectories completed in the environment under exploration. Let's take a closer look at the proposed algorithm.
more...
-
Neural networks made easy (Part 69): Density-based support constraint for the behavioral policy (SPOT)
Quote:
Offline reinforcement learning allows the training of models based on data collected from interactions with the environment. This allows a significant reduction of the process of interacting with the environment. Moreover, given the complexity of environmental modeling, we can collect real-time data from multiple research agents and then train the model using this data.
At the same time, using a static training dataset significantly reduces the environment information available to us. Due to the limited resources, we cannot preserve the entire diversity of the environment in the training dataset.
more...
-
Neural networks made easy (Part 70): Closed-Form Policy Improvement Operators (CFPI)
Quote:
The approach to optimizing the Agent policy with constraints on its behavior turned out to be promising in solving offline reinforcement learning problems. By exploiting historical transitions, the Agent policy is trained to maximize a learned value function.
Behavior constrained policy can help to avoid a significant distribution shift in relation to Agent actions, which provides sufficient confidence in the assessment of the action costs. In the previous article we got acquainted with the
SPOT method, which exploits this approach. As a continuation of the topic, I propose to get acquainted with the Closed-Form Policy Improvement (CFPI) algorithm, which was presented in the paper "
Offline Reinforcement Learning with Closed-Form Policy Improvement Operators".
more...
-
Neural networks made easy (Part 71): Goal-Conditioned Predictive Coding GCPC)
Quote:
Goal-Conditioned Behavior Cloning (BC) is a promising approach for solving various offline reinforcement learning problems. Instead of assessing the value of states and actions, BC directly trains the Agent behavior policy, building dependencies between the set goal, the analyzed environment state and the Agent's action. This is achieved using supervised learning methods on pre-collected offline trajectories. The familiar Decision Transformer method and its derivative algorithms have demonstrated the effectiveness of sequence modeling for offline reinforcement learning.
more...
-
Neural networks made easy (Part 72): Trajectory prediction in noisy environments
Quote:
The noise prediction module solves the auxiliary problem of identifying noise in the analyzed trajectories. This helps the movement prediction model better model potential spatial diversity and improves understanding of the underlying representation in movement prediction, thereby improving future predictions.
The authors of the method conducted additional experiments to empirically demonstrate the critical importance of the spatial consistency and noise prediction modules for SSWNP. When using only the spatial consistency module to solve the movement prediction problem, suboptimal performance of the trained model is observed. Therefore, they integrate both modules in their work.
more...
-
Neural networks made easy (Part 73): AutoBots for predicting price movements
Quote:
The proposed method is based on the Encoder-Decoder architecture. It was developed to solve problems of safe control of robotic systems. It allows the generation of sequences of trajectories for multiple agents consistent with the scene. AutoBots can predict the trajectory of one ego-agent or the distribution of future trajectories for all agents in the scene. In our case, we will try to apply the proposed model to generate sequences of price movements of currency pairs consistent with market dynamics.
more...
-
Neural networks made easy (Part 74): Trajectory prediction with adaptation
Quote:
Building a trading strategy is inseparable from analyzing the market situation and forecasting the most likely movement of a financial instrument. This movement often correlated with other financial assets and macroeconomic indicators. This can be compared with the movement of transport, where each vehicle follows its own individual destination. However, their actions on the road are interconnected to a certain extent and are strictly regulated by traffic rules. Also, due to the individual perception of the road situation by vehicle drivers, a share of stochasticity remains on the roads.
In this article I want to introduce you to a method for effectively jointly predicting the trajectories of all agents on the scene with dynamic learning of weights
ADAPT, which was proposed to solve problems in the field of navigation of autonomous vehicles. The method was first presented in the article "
ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation".
more...
-
Neural networks made easy (Part 75): Improving the performance of trajectory prediction models
Quote:
Forecasting the trajectory of the upcoming price movement probably plays one of the key roles in the process of constructing trading plans for the desired planning horizon. The accuracy of such forecasts is critical. In an attempt to improve the quality of trajectory forecasting, we complicate our trajectory forecasting models.
more...
-
Developing a multi-currency Expert Advisor (Part 4): Pending virtual orders and saving status
Quote:
In the previous
article, we have significantly revised the code architecture to build a multi-currency EA with several parallel working strategies. Trying to achieve simplicity and clarity, we have so far only considered a certain minimum set of functionality. Even considering the limitations of our task, we have significantly altered the code from the previous
articles.
Now hopefully we have the groundwork that is sufficient enough to increase functionality without radical changes to the already written code. We will try to make a minimum number of edits only where it is really necessary.
more...
-
Neural networks made easy (Part 79): Feature Aggregated Queries (FAQ) in the context of state
Quote:
Object detection in video has a number of certain characteristics and must solve the problem of changes in object features caused by motion, which are not encountered in the image domain. One of the solutions is to use temporal information and combine features from adjacent frames. The paper "
FAQ: Feature Aggregated Queries for Transformer-based Video Object Detectors" proposes a new approach to detecting objects in video. The authors of the article improve the quality of queries for Transformer-based models by aggregating them. To achieve this goal, a practical method is proposed to generate and aggregate queries according to the features of the input frames. Extensive experimental results provided in the paper validate the effectiveness of the proposed method. The proposed approaches can be extended to a wide range of methods for detecting objects in images and videos to improve their efficiency.
more...
-
Neural networks made easy (Part 80): Graph Transformer Generative Adversarial Model (GTGAN)
Quote:
The recently published paper "
Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation" introduces the algorithm for the graph transformer generative adversarial model (
GTGAN), which succinctly combines both of these approaches. The authors of the GTGAN algorithm address the problem of creating a realistic architectural design of a house from an input graph. The generator model they presented consists of three components: a message passing convolutional neural network (
Conv-MPN), Graph Transformer encoder (GTE) and generation head.
Qualitative and quantitative experiments on three complex graphically constrained architectural layout generations with three datasets that were presented in the paper demonstrate that the proposed method can generate results superior to previously presented algorithms.
more...
-
Neural Networks Made Easy (Part 81): Context-Guided Motion Analysis (CCMR)
Quote:
A particularly interesting method entitled CCMR was presented in the paper "
CCMR: High Resolution Optical Flow Estimation via Coarse-to-Fine Context-Guided Motion Reasoning". It is an approach to optical flow estimation that combines the advantages of attention-oriented methods of motion aggregation concepts and high-resolution multi-scale approaches. The CCMR method consistently integrates context-based motion grouping concepts into a high-resolution coarse-grained estimation framework. This allows for detailed flow fields that also provide high accuracy in occluded areas. In this context, the authors of the method propose a two-stage motion grouping strategy where global self-attentional contextual features are first computed and them used to guide motion features iteratively across all scales. Thus, context-directed reasoning about
XCiT-based motion provides processing at all coarse-grained scales. Experiments conducted by the authors of the method demonstrate the strong performance of the proposed approach and the advantages of its basic concepts.
more...
-
Matrix Factorization: The Basics
Quote:
In this article, we will talk about matrix calculations. Dear readers, do not rush to refuse reading this article, thinking that we will talk about something purely mathematical and too complicated. Contrary to what many people think, a good programmer is not someone who writes a giant program that only they can understand, or someone who writes code in a trendy programming language. A real and good programmer understands that a computer is nothing more than a computing machine to which we can tell how computations should be performed. It doesn't matter what exactly we are creating, it can be a simple text editor that does not contain any mathematical code. But this is just an illusion. Even a text editor contains a fair amount of math in its code, especially if it has a spell checker built into it.
more...
-
Neural Networks Made Easy (Part 83): The "Conformer" Spatio-Temporal Continuous Attention Transformer Algorithm
Quote:
The unpredictability of financial market behavior can probably be compared to the volatility of the weather. However, humanity has done quite a lot in the field of weather forecasting. So, we can now quite trust the weather forecasts provided by meteorologists. Can we use their developments to forecast the "weather" in financial markets? In this article, we will get acquainted with the complex algorithm of the "Conformer" Spatio-Temporal Continuous Attention Transformer, which was developed for the purposes of weather forecasting and is presented in the paper "
Conformer: Embedding Continuous Attention in Vision Transformer for Weather Forecasting". In their work, the authors of the method propose the
Continuous Attention algorithm. They combine it with those we discussed in the previous article on
Neural ODE.
more...
-
Neural Networks Made Easy (Part 84): Reversible Normalization (RevIN)
Quote:
In the previous article, we discussed the
Conformer method, which was originally developed for weather forecasting. This is quite an interesting method. When testing the trained model, we got a pretty good result. But did we do everything right? Is it possible to get a better result? Let's look at the learning process. We are clearly not using the model forecasting the next most probable timeseries values for its intended purpose. By feeding the model input data from a timeseries, we trained it by propagating the error gradient from models using the prediction results. We started with the Critic's results.
RevIN — is a flexible, trainable layer that can be applied to any arbitrarily chosen layers, effectively suppressing non-stationary information (mean and variance of an instance) in one layer and restoring it in another layer of nearly symmetric position, such as input and output layers.
more...
-
Neural Networks Made Easy (Part 85): Multivariate Time Series Forecasting
Quote:
Forecasting timeseries is one of the most important elements in building an effective trading strategy. When performing a trading operation in one direction or another, we proceed from our own vision (forecast) of the upcoming price movement. Recent advances in deep learning models, especially architecture-based
Transformer models, have demonstrated significant progress in this area, offering a great potential for solving the multifaceted problems associated with long-term timeseries forecasting.
more...
-
Neural Networks Made Easy (Part 86): U-Shaped Transformer
Quote:
Forecasting long-term timeseries is of specifically great importance for trading. The
Transformer architecture, which was introduced in 2017, has demonstrated impressive performance in the areas of Natural Language Processing (
NLP) and Computer Vision (
CV). The use of
Self-Attention mechanisms allows the effective capturing of dependencies over long time intervals, extracting key information from the context. Naturally, quite quickly a large number of different algorithms based on this mechanism were proposed for solving problems related to timeseries.
more...
-
Matrix Factorization: A more practical modeling
Quote:
In the previous article
"Matrix Factorization: The Basics", I talked a little about how you, my dear readers, can use matrices in your general calculations. However, at that time I wanted you to understand how the calculations were done, so I didn't pay much attention to creating the correct model of the matrices.
more...
-
Neural networks made easy (Part 89): Frequency Enhanced Decomposition Transformer (FEDformer)
Quote:
Long-term forecasting of time series is a long-standing problem in solving various applied problems.
Transformer-based models show promising results. However, high computational complexity and memory requirements make it difficult to use the
Transformer for modeling long sequences. This has given rise to numerous studies devoted to reducing computational costs of the
Transformer algorithm.
Despite the progress made by
Transformer-based time series forecasting methods based, in some cases they fail to capture the common features of the time series distribution. The authors of the paper "
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting" have made an attempt to solve this problem. They compare the actual data of a time series with its predicted values obtained from the vanilla
Transformer. Below is a screenshot from that paper.
more...
-
Neural Networks Made Easy (Part 92): Adaptive Forecasting in Frequency and Time Domains
Quote:
The article presents the results of experiments on eight real data sets, according to which ATFNet shows promising results and outperforms other state-of-the-art time series forecasting methods on many datasets.
more...
-
Neural Networks Made Easy (Part 93): Adaptive Forecasting in Frequency and Time Domains (Final Part)
Quote:
In the previous
article, we got acquainted with the
ATFNet algorithm, which is an ensemble of 2 time series forecasting models. One of them works in the time domain and constructs predictive values of the studied time series based on the analysis of signal amplitudes. The second model works with the frequency characteristics of the analyzed time series and records its global dependencies, their periodicity and spectrum. Adaptive merging of two independent forecasts, according to the author of the method, generates impressive results.
more...
-
Developing a multi-currency Expert Advisor (Part 14): Adaptive volume change in risk manager
Quote:
In one of the
previous articles of the series, I touched on the topic of risk control and developed a risk manager class that implements basic functionality. It allows setting a maximum daily loss level and a maximum overall loss level, upon reaching which trading stops and all open positions are closed. If a daily loss was reached, trading was resumed the next day, and if an overall loss was reached, it was not resumed at all.
more...
-
From Basic to Intermediate: IF ELSE
Quote:
In the previous article,
From Basic to Intermediate: Passing by Value or by Reference, we provided a practical and objective explanation of the concepts, risks, and precautions you should take when transferring data between different programs.
Based on that discussion, as well as the content covered previously, we can now begin to explore more advanced topics. This is because, in programming itself, we do not work solely with mathematical expressions. Doing so would not only be a tremendous waste of the computational power and factorization capabilities that a computer offers but would also limit the possibilities of what we can truly achieve.
more...
