Last Updated on 24 April, 2024 by Abrahamtolle
Backtesting is when you make strict trading rules and settings and apply those rules to historical financial data. Thus, you can easily conclude if you have a trading idea that might be reliable or not. If the strategy has not worked in the past, there is not much point in trading it.
This article answers some of the most frequently asked questions (FAQ) about backtesting: 30 facts about historical testing every trader should know.
Before you start reading we would like to inform you that we have an online inexpensive course that covers the most important aspect of backtesting:
Backtesting course
We have been trading (successfully) and backtesting since the end of the 1990s and thus we have developed pretty extensive knowledge about backtesting and historical analysis of trading strategies. We have compiled that knowledge into an inexpensive backtesting course. You can read more about the course and order by clicking on the banner below:
Why is backtesting important?
Backtesting is important because it helps you determine facts – the opposite of anecdotal evidence (which financial markets are full of). By analyzing trading statistics and historical performance you get to falsify or confirm your trading idea. You get to know the historical probability of a trade being a success or not.
Why would you trade your money if you really have no idea if the strategy has worked in the past? Our experience is that most traders don’t have any positive statistical edge in the first place, thus making most of the focus on psychology and money management a wasteful exercise. Trading is very much a game of statistics and probabilities, and thus you need to know the odds of your strategy. When you have a tradable strategy, you just need to push buttons and execute.
Furthermore, backtesting helps you to get other and perhaps better trading ideas. Trading is about trial and error!
Any strategy that has not worked in the past, is unlikely to yield many positive results in the future. Opposite, a strategy that has shown promise in the past, might be an inefficiency that will work in the future, and you might even be able to optimize and improve the strategy without being liable to curve fitting.
How to backtest a trading strategy?
To backtest a strategy you need to do the following:
- You need a trading idea
- The trading idea needs to be formulated into specific trading rules and settings
- Decide which markets/assets to backtest
- Get/download historical data
- Divide the data into two parts (in-sample and out-of-sample (more below))
- Test the trading rules on the in-sample data
- Does the strategy work? Can it be improved?
- Test the (modified) trading rules on the out-of-sample data. Does it still work well?
- If not, scrap it and move on (and you hopefully learned something)
- If yes, paper trade the strategy for some months
- If paper trading works, consider trading it live
In short, backtesting is as simple as that. It’s no rocket science but requires a lot of work, procedures, and feedback loops.
How long should you backtest a trading system?
We use the following rules of thumb:
- At least 100 trades (with a few exceptions)
- Must include different business cycles (at least a bear market)
These two are the only two rules we use. However, we like to test as far back in time as possible.
Strategy Shop
Backtested trading strategies
Any end-of-day trading strategy is normally backtested decades back, while we use about ten years for intraday data (for day trading).
Backtesting day trading strategies (how to)
In theory, there is no difference between backtesting on daily bars or intraday bars (5 min bars, for example).
However, backtesting on intraday historical data is more demanding than using end-of-day data. There are two reasons for that:
- It’s technically more different (both code and rules). Some rules are actually very difficult to quantify.
- Intraday data are less likely to work well in the future. There is more randomness involved in the time series data.
Is backtesting a waste of time?
Of course not!
Most backtests end up worthless, but at least you are improving your backtesting, and you get to see what is not working, which is also a great asset to have. Trading is about compounding knowledge!
Best backtesting software
There is no answer that fits all traders. It depends on many factors. What works for you, is not necessarily the best choice for other traders.
For example, we use Amibroker. It’s a fantastic program for backtesting (fast and cheap), but it’s a bit cumbersome if you want to trade your backtested strategies live. You need to write a few extra lines of code to both connect to Interactive Brokers (or the broker of your choice) and to keep track of your strategies.
Other quantitative traders might stick to Tradestation, Multicharts, Ninjatrader, etc.
The fact is that trading platforms are kind of a commodity product that is hard to differentiate from.
How to backtest option strategies
You backtest option strategies just the same way you do with stocks or futures. That said, it’s a bit more complicated due to the many different strikes and expirations. You certainly would need experience from end-of-day backtesting before you venture into backtesting options.
Backtest forex – how do you do it?
Again, the process and principles are the same for any backtest – no matter the market. The main advantage of forex is that data is abundant.
But sadly, we believe forex is mostly a waste of time. It’s a low bar to trade it, but most quantitative forex traders lose and stop trading. We believe it’s a bad idea to start forex trading. Please read our reasons why:
Is backtesting accurate?
No! Nothing is accurate in trading or predictions about the future. The future is unknown – also with a backtest.
But in our opinion, it’s the best estimation of the future you’ll ever have.
Backtesting in Excel
Excel and spreadsheets are very underrated backtesting software. We both did backtesting and live trading (VBA script) for our very successful day trading until 2018.
That said, spreadsheets have their limitations, especially if you want to backtest portfolios.
Additionally, Excel has limitations when it comes to trading history analysis and trading performance metrics. With a software you get all this with the click of a button, while in Excel you need to “code” it.
Strategy backtesting software
If you are serious about trading, we don’t recommend Excel but that you purchase some specific trading software, like Amibroker, for example. It’s cheap, reliable, and lightning fast.
Backtest trading strategy Python
Python is extremely popular among quant traders, something that we find a bit odd, but we are not coders, either. We suspect it’s mainly programmers and coders that like using Python.
But why start from “scratch” when you can use Amibroker, Tradestation, etc.? To us, it doesn’t make sense, even though Python has a ton of free resources. You should make an analysis of trades, not spend time coding. Any strategy test should be made as fast as possible.
Please read our take on this in separate articles:
How many trades should you backtest?
We believe it’s more important to include different business cycles than to specifically focus on the number of trades, even though the latter should be at least 100 – preferably more.
Technical analysis backtesting
We have a rule: if it can’t be backtested, don’t trade it. This applies to technical analysis as well. We mention it because it’s very tricky to backtest technical chart patterns. Most of the patterns are not backtested, and thus you should avoid them.
In our list of trading strategies we quantify some technical patterns:
The best platform to backtest trading strategies
We use Amibroker because we believe it’s the best platform to backtest trading strategies. But as you might guess, other traders might disagree.
The most used platform among professional and independent traders is probably Tradestation because of the simplicity to use it for BOTH backtesting and live trading.
Thus, the best platform boils down to what you’ll use it for.
Backtesting crypto
Crypto and Bitcoin can easily be backtested. But all cryptos have so far limitations because of their short and erratic history. It’s unlikely the past predicts the future very well because of this.
An example of two crypto backtests can be found here:
- Trend following and momentum strategies on bitcoin (crypto)
- Does RSI work on crypto or Bitcoin trading? Is RSI good for crypto?
Backtesting website
We are biased, but we believe the blog and website you are reading now is one of the best. Apart from this one, we can recommend the quantocracy website which contains a nice summary of quant articles.
Backtesting vs forward testing
Backtesting is when you backtest known data while forward testing is when you test the trading rules made on the known data on future unknown data.
Backtesting – pros and cons
Let’s briefly list the pros (and advantages):
- You can confirm or falsify a trading idea
- Backtesting lets you automate
- You can exploit the law of large numbers
- Backtesting reduces emotions
- Backtesting saves time
These are the cons (disadvantages):
- The backtest might be liable to favorable conditions
- Backtests work because you know the after the fact
- Backtesting involves elements of curve fitting
- Survivorship bias (survivorship bias in trading)
- Chance, luck, and randomness should not be ignored
- Backtesting involves garbage in, garbage out
- Transaction costs are unknown
- Markets change – no backtest can accommodate change
- All traders do behavioral mistakes
Despite having more bullet points containing disadvantages than advantages, we still believe backtesting is the far superior way of making money in the markets. If you have no idea if your strategy has made money in the past, how can you be sure it’s profitable?
All of the above points are further discussed in our backtesting course:
Automated backtesting
Some programs allow you to look for a trading strategy that is based on a zillion trading rules.
For example, you can program the software to look for the best settings for any technical trading indicator. But we believe such software requires a TON of experience due to all the curve fitting that takes place.
Backtesting trading strategies free
On the web, you can find web-based software that lets you backtest for free. That is an option if you’d like to find out if backtesting is for you, but it’s not a viable long-term solution. The best option is to spend some time on finding the right software and then purchasing it.
Walk forward backtesting
A walk-forward optimization is when you use both in-sample and out-of-sample tests frequently.
It’s done like this:
If you have 20 years of data, you might divide the data into ten equal parts (in length). You then have two years of data for each set. The first year is the in-sample backtest, and the second year is the out-of-sample test (where you test the trading rules made in year one).
This is repeated ten times and the final results are evaluated to make the final parameters for the strategy.
It’s a bit cumbersome, but most software platforms offer this ability. We don’t use it ourselves because we have not found it very useful.
Anchored and non-anchored backtesting
Anchored and unanchored (non-anchored) are two forms of walk-forward backtesting.
An anchored walk-forward test uses a fixed start date in each simulation, while a non-anchored uses a start date that is moved gradually forward by an amount equal to the length of each out-of-sample period.
Backtesting trading strategy – Tradingviews
Tradingview is a very popular platform that has gained many users with the rise of crypto. It lets you connect to a wide range of brokers if you want to place trades.
Backtesting data
You need data to backtest. We recommend Norgate for end-of-day data because they have “clean” and error-free data. Any quantified and data-driven strategy needs good data, otherwise, the backtest is useless.
You can also download free data from Yahoo and Google, but beware of incorrect High and Low readings. The open and close are mostly correct, though, at least on the most liquid tickers.
Backtesting trading strategies
If you want to backtest several trading strategies in one backtest, most of the platforms let you do that. Amibroker has, for example, a lightning-fast portfolio option.
Backtest stock strategy (how to)
If you want to backtest stocks you need to make a watch list or a filter that lets you backtest those stocks you want. Again, this is simple with specific trading software.
FAQ
Why is backtesting important for traders?
Backtesting is crucial for traders as it provides factual insights into the historical performance of a trading strategy. It helps confirm or falsify trading ideas, allowing traders to understand the historical probability of success and make informed decisions.
How does backtesting contribute to trading strategy development?
Backtesting contributes to the development of trading strategies by helping traders formulate specific rules and settings based on a trading idea. It allows for the testing of these rules on historical data, optimizing and improving strategies for better future performance.
How do you backtest a trading strategy?
To backtest a trading strategy, you need a clear trading idea formulated into specific rules. Select the markets/assets to backtest, obtain historical data, and divide it into in-sample and out-of-sample parts. Test the rules on in-sample data, optimize if needed, and validate on out-of-sample data.
What is the recommended duration for backtesting a trading system?
We recommend backtesting a trading system for at least 100 trades, with exceptions based on specific circumstances. It’s essential to include different business cycles, such as a bear market, to ensure the strategy’s robustness.
Are there disadvantages to backtesting?
Yes, there are disadvantages to backtesting, including the potential for the backtest to be influenced by favorable conditions, the challenge of working with after-the-fact data, and the risk of curve fitting. It’s important to be aware of these limitations.