The backtesting platform: The architecture and its requirements

Trading strategies are the core part of each algorithmic trading system. Designing and developing them requires an infrastructure to test, validate and verify the performance and accuracy. These processes usually utilize historical financial data to learn the obvious and hidden market price patterns, extract simple or complex rules, and apply the acquired knowledge to future decisions. These processes are called backtesting.

A perfect, fast, and flexible backtesting platform is essential for algorithmic trading product development. Many backtesting packages, libraries, or platforms are available and accessible online. Although these tools cover most of the requirements for developing a trading system, they are usually too general to be used by firms or financial corporations. Hence, designing and developing a customized backtesting platform according to particular requirements is essential. At Eveince, we do our best to build our customized backtesting platform to target our necessities based on our technology development procedures. This article describes the architecture of a customized backtesting platform, which make the process more flexible and faster, and discusses its elements in detail.

This given multi-layer backtesting platform logically isolates each layer as separated modules. This kind of architecture helps segregate different parts, which makes the development process more optimized and easier. The only requirement that should be considered in this approach is defining standard inputs and outputs for each module or layer to make the communication between them more manageable and link the data effortlessly. Regardless of the complexity of each layer, this architecture provides an opportunity for parallel development for quantitative team members. Figure 1 shows the multi-layer architecture.

Figure 1- The multi-layer architecture of backtesting platform

As you can see, most layers consist of multiple sub-layers, which are submodules of a more extensive functionality. Like the modules connected logically, all the separated data are also associated logically with a time index, making it possible to merge different parts of data.

Layer one: Fetch Data

Exchanges often provide traders with standard APIs to get historical data. The priority is to have access to price data with different time intervals and precise information to build a backtesting platform. This layer implements the functions or methods to retrieve data from various exchanges through standard APIs and store them in the OLHCV candle price format. In this part, the user just needs to determine the start and end time of data, the list of symbols, and the time interval. While we fix the input and output format in this layer, we can change the internal APIs and exchange logic. We have two different functions for fetching data. One is based on the time range, and the other is based on the number of data points. More detail is shown in figure.2.

figure 2- The first layer of backtesting architecture: The fetch data layer

Layer two: Feature Extraction

This layer consists of two different modules. The first is feature extraction, and the second one is labeling. Each machine learning model needs data as a feature vector to work and also requires labels if supervised methods are employed. At Eveince, the data team aims to develop quantitative trading strategies with a particular concentration on machine learning methods and models. Hence, feature vectors are one of the most critical aspects of our work. This layer uses technical indicators, fundamental data, and augmented variables as features. You can enrich your indicator library by adding the most useful technical indicators and designing new indicators to represent further aspects of price data in your indicators pool.

One profound facility in this layer is a config file describing the feature vector as a manual sheet. The trader defines the features, parameters, and normalization methods in this file, making new features much more straightforward than other approaches.

The second sub-layer is related to labeling. All the classification problems, or generally all the supervised models, need labels to distinguish different classes to sort out the problem. Sometimes the labels are directly calculated by the detailed data, but we usually use complicated labels, which discriminate the space more accurately. All the required methods for calculating the labels are implemented in this layer.

In summary, the inputs of this layer are raw price data as CSV files, and the outputs are the final feature vectors and labels, which are separately stored in CSV files. All the details of the feature extraction layer are shown in figure.3.

figure 3- The second layer of backtesting architecture: The feature extraction layer

Layer three: Model Fit and Predict

This layer consists of three super important sub-layers in our backtesting procedure. In the first sub-layer, the activities for training and inferencing are done thoroughly, which is the most vital part of our platform. The way that we recognize patterns from feature vectors and how to analyze them quantitatively is determined in this step. We just change the logic in these procedures to explore new ideas or models. To run this sub-layer, we need to determine train and test data, the time for backtesting, and the model’s parameters. The training phase should be executed when we build new models, but the inferencing phase is always performed during backtesting. The outputs are the probabilities and confidence scores of inferences; most of the time, we cannot use them directly to make final decisions. Based on this fact, we need another sub-layer that gets the inferences outputs and convert them into the expectations, which can be used as data for creating final orders. This layer is a mapping between inferences and the expectation of market behavior.

The last layer, called stop-loss, is used to add stop-loss order into the execution based on fixed rules. It adds flags that indicate the time that stop-loss is triggered, and we should consider these time points in execution. The details of this layer are shown in figure.4.

figure 4- The third layer of backtesting architecture: The model fit and predict layer

Layer four: Execution

Execution is the primary layer for creating and executing orders which generate portfolios. It consists of two sub-layers which are execution and portfolio re-balancing. The first one uses the expectation values from the previous layer and converts them into executable orders in the exchange. Like other backtesting platforms, the portfolio values are calculated by simulating the execution without volume restriction. The execution can be long or short, and we support both by implementing spot and future execution methods. In this section, the function uses parameters from the configuration file and the price data for execution. Every order executed in the simulation will update the portfolio value. It means the exact asset value should be calculated according to the profit or loss from the trade and the commission that must be reduced from the asset based on the amount of the trade. In this section, we also manage risk by adjusting the position size based on the risk we can tolerate by measuring the asset risk and the future expectation.

In the second sub-layer, which is optional, we re-balance the portfolio and reallocate assets according to the performance and risk. This layer needs one complete portfolio for the first iteration and then calculates the metrics point by point and triggers the re-balancing methods. In this layer, the execution process runs multiple times based on trigger points, generating a new portfolio each time.

The outputs are the portfolio values and other calculated metrics, such as trade count, buy and hold value, and cash amount. The details of this layer are shown in figure.5.

figure 5- The fourth layer of backtesting architecture: The execution model

Layer five: Analytics

This layer is developed to observe and analyze the backtesting results and the portfolio’s performance metrics. It consists of two sub-layer, and the first one is responsible for generating the experiment data for the visualizer segment. This layer divides the output into broker and individual symbols performance.

The second layer visualizes the experiment data with different charts and tables or graphs. We also have a comparator to compare the output and performance of multiple portfolios simultaneously.

Farhad Azadjoo, Machine learning engineer & quantitative researcher, Eveince.