Backtesting: A Comprehensive Guide to Strategy Validation

Backtesting involves applying a strategy or predictive model to historical data to determine its accuracy. It can be used to test and compare the viability of trading strategies so tradersSix Essential Skills of Master TradersJust about anyone can become a trader, but to be one of the master traders takes more than investment capital and a three-piece suit. Keep in mind: there is a sea of individuals looking to join the ranks of master traders and bring home the kind of money that goes with that title. can employ and tweak successful strategies.

Summary

• Backtesting involves applying a strategy or predictive model to historical data to determine its accuracy.
• It allows traders to test trading strategies without the need to risk capital.
• Common backtesting measures include net profit/loss, return, risk-adjusted return, market exposure, and volatility.

How Backtesting Works

Analysts use backtesting as a way to test and compare various trading techniques without risking money. The theory is that if their strategy performed poorly in the past, it is unlikely to perform well in the future (and vice versa). The two main components looked at during testing are the overall profitability and the risk level taken.

However, a backtest will look at the performance of a strategy relative to many different factors. A successful backtest will show traders a strategy that’s proven to show positive results historically. While the market never moves the same, backtesting relies on the assumption that stocks move in similar patterns as they did historically.

Implementation

A backtest is usually coded by a programmerProgrammingProgramming is the process of writing instructions for a computer to perform. It is similar to a recipe for humans. A recipe contains a list of actions running a simulation on the trading strategy. The simulation is run using historical data from stocks, bonds, and other financial instruments. The person facilitating the backtest will assess the returns on the model across several different datasets.

It is also essential that the model is tested across many different market conditions to assess performance objectively. Variables within the model are then tweaked for optimization against several different backtesting measures.

Common Backtesting Measures

• Net Profit/Loss
• Return: The total return of the portfolio over a given time frame
• Risk-Adjusted ReturnRisk-Adjusted Return RatiosThere are a number of risk-adjusted return ratios that help investors assess existing or potential investments. The ratios can be more helpful: The return of the portfolio adjusted for a level of risk
• Market Exposure: the degree of exposure to different segments of the market
• VolatilityVolatilityVolatility is a measure of the rate of fluctuations in the price of a security over time. It indicates the level of risk associated with the price changes of a security. Investors and traders calculate the volatility of a security to assess past variations in the prices: The dispersion of returns on the portfolio

Backtesting Bias

When creating a trading model to be backtested, traders must avoid bias in creating the model. In order to ensure objectivity, the strategy must be tested on several different time periods with an unbiased and representative sample of stocks.

If a trader were to pick and choose the stocks and time period in which their strategy is backtested against, the model would be fundamentally flawed. While the test may yield positive results, this would only be because the model was created to fit this data perfectly. Therefore, it is essential that different datasets are used throughout the process.

Look-Ahead Bias

Another mistake when backtesting is look-ahead bias. Look-ahead bias involves incorporating information into the model being backtested that normally wouldn’t be available when the model is actually implemented.

For example, assume you’re backtesting a trading model that relies on financial information available at fiscal year-end. In the model, you enter the information as of December 31^st; however, the information generally isn’t available until a couple of weeks after the end of the year. Implementing the data in a backtest would cause the return on the model to be artificially high due to look-ahead bias.

• A – Fiscal year-end (time at which backtesting model assumes annual report released)
• B – Annual report released
• C – Time at which the backtesting model assumes first-quarter report release
• D – First quarter report released

The graph above shows a timeline of how a backtesting model could become flawed due to look-ahead bias. The model assumes that information becomes available at points A and C, while in reality, the information becomes available at points B and D. The result of a properly constructed backtest would likely yield an entirely different result than the one that makes the same assumptions as above.

Who Uses Backtesting?

Anyone can perform their own backtest; however, backtests are usually run by institutional investors and money managers. Backtesting uses data that can be expensive to obtain and requires complex modeling.

Institutional traders and investment companies possess the human and financial capital necessary to employ backtesting models in their trading strategies. Additionally, with large amounts of money on the line, institutional investorsInstitutional InvestorAn institutional investor is a legal entity that accumulates the funds of numerous investors (which may be private investors or other legal entities) to are often required to backtest to assess risk.

Example

Suppose you’re an analyst at an investment firm, and you’ve been asked to backtest a strategy against a set of historical data given to you. The strategy involves buying a stock if it hits a 90-day low. The first step in backtesting would be choosing unbiased historical data.

You then apply the strategy to the data and find that the strategy yielded a return of 150 basis points better than the current strategy used by the company. The backtest helped to solidify the research performed in creating the trading strategy. The investment firm can decide whether the backtest is reason enough to employ the strategy.

Related Readings

CFI is the official provider of the global Capital Markets & Securities Analyst (CMSA)®Program Page - CMSAEnroll in CFI's CMSA® program and become a certified Capital Markets &Securities Analyst. Advance your career with our certification programs and courses. certification program, designed to help anyone become a world-class financial analyst. To keep advancing your career, the additional resources below will be useful:

• AlgorithmsAlgorithms (Algos)Algorithms (Algos) are a set of instructions that are introduced to perform a task. They automate trading to generate profits at a frequency impossible to a human trader.
• Clustering IllusionClustering IllusionClustering illusion refers to a cognitive bias in behavioral finance in which an investor observes patterns in what are actually random events. In other
• Hypothesis TestingHypothesis TestingHypothesis Testing is a method of statistical inference. It is used to test if a statement regarding a population parameter is correct. Hypothesis testing
• Sample Selection BiasSample Selection BiasSample selection bias is the bias that results from the failure to ensure the proper randomization of a population sample. The flaws of the sample selection