Last Updated on 24 September, 2020 by Samuelsson
Many traders would agree that the trading strategy is the most important thing in trading. Without it, you cannot make money in the markets, and are destined for losses.
In this article, we’re going to look at how you create a trading strategy from scratch. We’ll cover the steps, the pitfalls, and a lot more that’s pertinent to the strategy creation process!
Here are the steps to building a trading strategy that we’ll cover:
- Step 1: Choosing the Market for Your Strategy (Or Not)
- Step 2: Coming Up With a Trading Idea
- Step 3: Backtesting the Idea
- Step 4: Evaluating and Further Testing the Strategy
- Step 5: Going live With a Strategy
However, before going through the steps, let first have a look at the things you in order to create a strategy.
More specifically, that’s market data and a backtesting platform that will let you see how the strategy has performed historically.
Requirements: Backtesting Platform & Market Data
When building trading strategies, we use backtesting to see how the idea has fared historically. That is, we test the strategy on historical data to see if it has worked or not. Doing so we know what is worth our attention and what we can throw away!
First of all, you need to use a backtesting platform. While backtesting a strategy manually is possible, it’s a tedious task that won’t take you very far. With the help of a computer and a good backtesting platform, you could speed up the process significantly, and in the end, find many more trading strategies.
Here at The Robust Trader, we build a lot of strategies, and with the amount of analysis we do, we can say with certainty that we wouldn’t be able to have nearly as many strategies if we had used manual backtesting
The other thing you need is market data to do your backtesting on. Ensuring that the data is of good quality is crucial. Poor data quality could produce misleading backtest results, which is why data quality is important to consider!
Now, there is quite a lot to say on both these topics, so let’s have a closer look at both!
When it comes to backtesting platforms, there are many alternatives on the market. However, in the list below we’ve decided to include those platforms that we have had the best experience with.
Still, we haven’t tried all the options out there, and we are sure that there are other good options on the market. Having that said, you really cannot go wrong with any of the options below.
TradeStation is our favorite platform. It has all the features that an advanced trader needs, both in terms of backtesting, customization, and live trading. It also has the benefit of coming with market data, which isn’t the case with the other options on this list. We’ll come back to market data in just a bit!
Even if TradeStation is a powerful trading platform, it’s not the fastest of them all. However, it’s still fast enough to cover most people’s needs, as long as you’re not doing portfolio backtests. Then there are better alternatives available, such as Multicharts or Amibroker.
TradeStation comes with Easylanguage, which is a powerful yet easy coding language! Another great benefit of the platform is that TradeStation, in addition to being a data provider and trading platform, also is a broker. Of the solutions on this list, TradeStation is the only that’s a plug-and-play solution!
Multicharts is quite similar to TradeStation, and if you have used the TradeStation, you will be familiar with its interface and features. However, it doesn’t come with market data, so you’ll have to connect an external data feed.
Multicharts is faster than TradeStation when it comes to backtesting. While the differences are noticeable when optimizing on a single symbol, they become even more apparent when backtesting a portfolio of symbols. The difference is striking, and to the advantage of the Multicharts platform.
Multicharts is a great platform that we like a lot as well. The coding language resembles that of TradeStation in every detail, which in itself is a great plus! You won’t have any issues coding in this language, and it will let you move on rapidly and not waste your time on unnecessary coding acrobatics!
Amibroker is a very competent backtesting platform that’s the fastest of the ones on this list. It runs a slightly more advanced coding language, but it’s still manageable, even for a beginner.
With Amibroker you will be able to backtest strategies on portfolios in a matter of seconds! In that sense, it beats both Multicharts and TradeStation, hands down.
Which Software Should You Choose?
All three platforms that we’ve listed here are good choices and will get the job down. If you just want to build strategies on single market and have autotrading capabilities, TradeStation is your best choice
However, if you’re going to test on baskets of symbols then Amibroker or Multicharts probably are your top choices. Of these two, Multicharts will get you started quicker than Amibroker, since its coding language is very beginner-friendly.
Now, if you want to create trading strategies on one market, then you could probably go with either of the platforms on the list.
However, of the ones covered, TradeStation is the one that will cost you the least.
As a TradeStation client you get free access to the platform AND market data. This could save you quite some money in the long run, since market data could be quite expensive.
Another benefit of the TradeStation platform is that you won’t have to spend time connecting to a broker. You already have that inbuilt with TradeStation!
It’s important that the data your using to build your strategy is of good quality. With most premium market data, this isn’t an issue. However, with free market data, there could be strange data errors that might mess things up. More than once I’ve experienced how free data suddenly has the close price set to zero, which could have unwelcomed effects for the accuracy of your tests!
If your trading platform doesn’t come with premium data, you should consider subscribing to a premium data vendor. Then you will save yourself the pain of having to manually go in and change the data when it contains an error. And most likely you won’t be able to find most errors, and will have your results impacted without noticing.
However, free market data is perfectly functional if you just want to try out backtesting, especially on the daily timeframe. If you want to do some analysis on lower timeframes, you should, however, look into getting premium market data. Daily data usually is acceptable, while intra-day data usually doesn’t hold the same quality.
Here is a good list with a lot of market data providers.
Below follow some common errors in market data that could give misgiving results in a backtest:
Now if you haven’t started building a strategy yet, don’t get too bogged down with these very technical aspects of market data. The most important thing right now is that you get started! We suggest that you skip this part if you’re just starting out!
Adjustments for Stock Splits and Dividends (Applies to stocks)
If the data hasn’t been adjusted for things like dividend payments, stock splits, or other corporate action, then the backtest results could be misleading.
For instance, if the stock underwent a stock split, and your data doesn’t take that into account, you will end up with a huge gap in the chart, like the blue line in the chart below.
If your strategy happened to be in a short position during that fall, then you would get a huge profit in your backtest, which of course never happened.
For dividends, not having them included could potentially have your strategy perform a little worse in the backtest, assuming that it’s a strategy that goes long. However, if you have a profitable strategy, then this won’t be an issue.
If your strategy behaves acceptably only with the dividends included, then that’s nothing you would like to trade anyways.
Survivorship Bias (stocks)
Survivorship bias is another issue you need to cope with when you’re building trading strategies on stocks. If you’re testing your strategy on an index, then there will be stocks that have fallen out of the index, and others that have been included.
Now, since the current composition of the index only consists of the stocks that have made it to this point, you will, in fact, be testing your strategy on those stocks that have performed the best. And this could make your results overly optimistic, since those stocks that were part of the index in the past but performed poorly, were taken out from it.
However, in our experience, this isn’t as much of an issue as many people make it seem. Backtesting still works even if there is survivorship bias included, and as you design a trading strategy, you’re making sure that the strategy is profitable enough to survive a slight survivorship bias.
Is the data back-adjusted? (For futures)
Futures contracts expire, and as such, there isn’t one futures contract for you to do all your backtesting on. Instead, you’ll be backtesting on a back-adjusted contract, where they basically have stitched together many future contracts, to give us enough data to test on.
Now, some data vendors just merge the contracts, without considering that the expiring contract might be trading higher or lower than the coming contract. This is due to backwardation and contango, which means that different futures contract can be priced differently, since their delivery months are different. As such, the market’s expectation, as well as storage costs, if it’s a physically deliverable contract, are priced into the individual contracts.
If you don’t take this into account, you’ll get big gaps in price as there is a rollover to the next month, which will render the data unusable for trades that last more than one day.
This issue is dealt with by using a continuous contract. There the differences in price between the contract months are accounted for by back-adjusting the data. So, if the new front contract trades lower than than the current month, then all the previous data is lowered to match the price level of the new contract.
While this makes sure there aren’t big gaps between the contract months, it also means that the price levels aren’t completely accurate. If a market is in backwardation, it means that all previous market data is lowered for each rollover. In some markets, like the Soybean Meal futures market, this leads to that price back in time even goes negative!
Step 1: Choosing the Market for Your Strategy (Or not)
Some people like to start building a strategy by choosing what market to test on.
This is completely fine if you specifically are looking to develop a strategy for one market. For example, if you want to find a new swing trading strategy for the stock market, then it’s natural to go with S&P-500 or S&P-100 stocks, just to give an example.
In case your choosing this approach, you might want to ask yourself what tends to work best on that particular market. For example, mean reversion tends to do very well with stocks, while momentum strategies could be harder to find.
Knowledge about what works well in the markets is something that you build with time, as you test your ideas, and see how they perform.
Not Choosing a market
What we often find, is that many traders are too preoccupied with finding strategies for one market only. In most cases, that market tends to be the S&P-500.
Now, there are many other great markets. For example, there are many ETFs with commodities as their underlying, on which you can build excellent strategies. And if you trade futures, as we do, there are countless markets you can choose from, all behaving in their own manner. You really don’t want to miss these edges just because you decided beforehand to only narrow in on one certain market.
Most times, we don’t choose a market. We are looking for trading strategies, preferably on some market that we aren’t that exposed to already, but we also know that we cannot impose our will on the market. Therefore, we test our our strategy on several markets, and then concentrate on where we suspect there might be an edge. That way we find many more strategies, and achieve better results long term.
One dollar earned trading bizarre markets like “lean hogs” is worth exactly as much as one dollar coming from trading the S&P! Quite obvious isn’t it, but many seem to forget this simple truth!
Step 2: Coming Up With a Trading Idea
A strategy, of course, is based on a few conditions, and as such it’s your task as the strategy designer to come up with ideas for the strategy. This step really is the creative phase. It’s where you want to unleash your creative spirit and hatch new ideas, regardless of how unconventional they might be! Many times it’s the strangest ideas that go against the common perception that work the best.
Now, in this phase, we typically test whatever we come up with. The important thing is to keep the ideas coming which becomes easier the more you test. You will soon come to a point where the number of ideas you have isn’t the limiting factor, but the time at your disposal!
Now, as to the trading idea itself, in our view, you could start by:
- Deciding what trading style your strategy should be, and then test ideas that match
- Just test random patterns, ideas, or whatever you come up with.
Let’s go a little deeper into each of the above approaches!
1. Deciding the Strategy Style First
If you go with number one, you could, for example, start by first defining the market tendency you’re looking for. The very same behavior can be defined in many ways, of which some are more successful than others.
For example, if we wanted to test mean reversion in a market, we could start by asking ourselves when a market would revert back to it’s mean.
The obvious answer here, in line with the general tendency of mean-reverting markets, is that we want to buy when the market has overextended itself to the downside, and vice versa.
However, while it’s clear when we want to buy, the overextension could be defined in many ways. It could be that the RSI goes under 5, that we’ve had three consecutive lower closes, or anything. The variations you could come up with here are unlimited!
And you will certainly find that some variations work much better than others. If mean reversion works with the market you’re investigating, that is!
2. Coming Up With Random Patterns and Ideas
Maybe you don’t want to search for a specific type of strategy? Well, that’s completely fine, and most of our own testing it carried out this way. We take what the market gives us, because it won’t give us anything else. As basic and rudimentary it might sound, it’s something many beginning traders haven’t taken in fully.
Now, one of the things we like with not confining ourselves to a specific trading style is that you really can unleash your creative spirit! You have NOTHING holding you back.
Why not try the Stochastic indicator together with the ADX?
Or try and use volume with the RSI?
Or compare Monday close prices to Friday close prices?
The possibilities are endless!
However, this also brings up an interesting question!
Does the trading Idea Need to Be Logical?
There are many people who advocate that a trading strategy should be based on something that’s logical and easy to understand.
We understand that trusting the strategy is vital if you’re going to keep trading it throughout its less profitable periods. And if that trust comes from you understanding the logic of the strategy, then that’s fine.
However, what we can say with 100% certainty, is that you’re missing a lot of good, and perfectly tradable strategies by only trading strategies that you understand. Markets aren’t logical, and sometimes we need to accept that it’s impossible for us to know why a strategy works. The important thing is to know that it works, which we’ll ascertain with some different robustness testing methods, which we’ll cover later in the article.
Where to Find Trading Ideas – Good Sources
Many beginners find it hard to know what to test. As we’ve discussed before, you will become better at this with practice and hours spent building trading strategies.
Below are a couple of different places where you can find ideas to test. We ourselves have built a lot of strategies with ideas we’ve got from these sources:
1. Expose Yourself to Market Data
If you spend some time watching the market, you will soon begin to notice quite a lot of small patterns that you could use to build great trading strategies. The most important thing here isn’t to make sure that the idea you get actually holds some merit. The main purpose is to spark ideas that hopefully will branch uncontrollably, leaving you with more things to test than you have time for!
2. Take Notes
When you watch the market, make sure to take notes. It could be questions that arise, or just plain observations. Soon you’ll find that you have quite a lot to test!
3. Trading Podcasts and Trading Forums
Listen to trading podcast and trading forums to get new ideas. Just be careful with what you hear and read. Most things are outright garbage, so don’t trust anything before you’ve tested it.
Short Tip for Trading Ideas: Keep them Simple!
Many new traders believe that complex ideas work best.
I will be very outright on this one
This is NOT the case
Simple things and ideas work remarkably well in the markets. You would be surprised if we showed you the logic of some of our strategies. Quite often they consist of two or three conditions, and that’s it. For example, this strategy consists of only two conditions!
Now, the ideas that work this well most times aren’t those that you can read about in a book, or anywhere on the Internet. Instead, they are unconventional and unusual logics that the masses haven’t thought of or yet discovered.
Being inventive and not dismissing ideas before testing them is key to creating trading strategies and survive in the game!
Common Trading Styles To Base Your Ideas On
Now, as you remember, we outlined two ways you could go about to create a strategy. The first one was to try to build a specific type of trading strategy, while the other option was to test random ideas.
Now, if want to start with a specific type of strategy, you might want to know the most common strategy types. Let’s have a look at three of them!
1. Mean Reversion
Mean reversion is the tendency of markets to revert to their mean. This means that those markets usually overextend to either side, to then revert back to the mean. If a market has moved too much to the downside we say that it’s oversold, and if it has moved too much too the upside, we say that it’s overbought.
When we build mean reversion strategies, we aim to define the oversold and overbought thresholds. That’s where the market is likely to soon turn around!
2. Trend Following
Trend following is the opposite of mean reversion. Instead of assuming that an extension to either side will be followed a reversion to the mean, we see it as a sign of market strength. And if the market is strong and vital, we want to follow along in its direction.
Trend following strategies still work to some extent, but not as well as they have in the past. If you’ve read the market wizard books, then you’ve probably heard about traders like Ed Seykota, who used simple trend following trading strategies to make huge amounts of money.
While that isn’t possible anymore in the same way as before, there are many great trend following trading strategies left to be found.
In general, it’s harder to apply trend following to stocks than to commodities like Energies, which tend to trend quite a lot.
3. Breakout Strategies
Breakout strategies are quite similar to trend following strategies, and sometimes you can’t separate the two. Both trade on market strength and assume that once momentum has picked up, the market is likely to continue in the dominant direction.
However, if you were to make any distinction, it would be the following:
- Breakouts refer to the action of breaking a certain price level. It could be the high of the previous bar, a pivot point, or any other level you decide. Once the market goes beyond that level, it’s expected to continue in the direction of the breakout.
- Trend following and momentum strategies refer more to the large swings that occur after a breakout. Typically, these kinds of strategies aim to milk the trend for as long as possible, but aren’t concerned with trying to get in right at the start of the trend. Instead, they aim to capture the large bulk of the market movement in the middle.
Trend following strategies often have very few winning trades. Sometimes, the winning trade/losing trade ratio could be as low 1 to 5, while the strategy remains perfectly profitable due to its outsize winning trades.
Step 3: Backtesting the Idea
Now that you have your trading idea ready, you need to backtest it to see if it holds or not. If you have a trading platform this means that you simply run the backtest and see what results it brings.
Don’t expect a majority of the ideas to show merit. In fact, most of the things you test are going to prove themselves useless. Just to give you some perspective, it is likely that it will take somewhere between 50-200 trading ideas to create one trading strategy!
Testing ideas in search of trading strategies takes much time, but at the same time, it’s a very creative process that’s very inspiring and fun, especially as you start to find things. Building trading strategies indeed is one of those processes that rely on intermittent reinforcement to remain enjoyable. Each new timeframe, market and trading idea could become your next trading strategy!
How to Backtest a Strategy Manually
If you haven’t got access to a trading platform, then backtesting your idea could become a little trickier. Then you will have to carry out the backtest manually, which won’t work time-wise if you have a lot to test.
However, if you want to test a couple of ideas, this could still be a viable option.
In order to backtest the strategy manually, you need to have access to charting software. Then these are the steps you have to take
- Formulate the strategy
- Journal the trades
Once you know the strategy rules, you have to start from the beginning of the testing period. Roll the chart forward, and look for the entry signal you defined. Once you find one, jot down the entry price and other information you want to keep, and then roll the chart window forward. Once your exit signal is effectuated, note the exit price, and calculate the profit for that trade. Then you just continue to roll your chart window forward, and log the trades as they occur!
One mistake that many beginners make at this stage, is to exclude trades, even if they had a buy signal. They assume that they never would have taken it anyway.
Doing so is dangerous, and invalidates the backtests. Signals that seem obvious with hindsight are seldom as clear when the trade is entered! Many times the reason for not including a test in the backtest only is that it was a losing trade, and nothing else.
You can read more about these steps in our article on how to manually backtest a trading strategy!
Unstructured or Creative Strategy Testing?
Some people advocate a structured testing process from start to finish. They set up the ideas they want to test, the markets, timeframes, and every other detail. All before running the test itself.
This seems to work well for them, and while we also might do this at certain times, we don’t think you should limit yourself before you have explored the idea and seen how it works. You need to have an open mind, and test on as many markets and timeframes as you possibly can. We often let the “edges guide us”. That is, we focus on the markets that seem the most interesting. This is something that works very well for us, and that has yielded many unexpected trading strategies!
Step 4: Evaluating and Further Testing the Strategy
If you find that the strategy indeed works you could either keep it as is, or try to improve on it.
In general, you at least want there to be some type of positive tendency that you can continue to build on. Often, the plain trading idea produces a quite rough equity curve, which then needs to be enhanced by adding a filter. Below you see an example of this.
However, when adding filters, it is important to not add too many conditions. If you have 10 different filters to force the strategy to work, then you’re probably doing something wrong.
And even worse, the strategy isn’t likely to work in live trading, since it’s curve fit!
Watch Out For Curve Fitting!
Let’s say that you have created a strategy that you are content with.
Well, you should definitely not go and trade the strategy immediately. You first have to ensure that the strategy is robust, and likely to work in live trading. There certainly is a good reason why trading material and videos always start with “historical performance is not indicative of future performance. The backtest results could easily fool you, and you might have created a strategy that just was a result of random luck.
This tendency is often referred to as curve fitting, and is something that is vital to know if you want to make any money at all in the markets!
Let’s have a closer look at curve fitting, and why it’s so important to be aware of!
What Is Curvefitting and Why Should You Care?
When it comes to backtesting, our testing is made under the assumption that we can derive what works in the markets from historical data. We believe that the security we’re analyzing behaves in a specific behavior, that’s mirrored in the market data that we’re examining.
This is an assumption that, at least to some extent, holds true. It is possible to find recurrent behavior that can form a trading strategy by examining market data. However, what also is true, if not even more so, is that most market action is random. Most of the movements are completely random, and hold no predictive value whatsoever.
Now, when we’re dealing with market data and testing our ideas, the number of possible definitions of trading ideas are nearly endless. And with so many variations, we’re bound to have randomness come into play and occlude our vision. Some of the things that we found looked promising may only have worked out of random luck.
And if something works just out of random luck, then the performance of that strategy is going to be completely random going forward. In other words, it’s the opposite of having an edge, which we want to have in a trading strategy.
This is often what it looks like:
The strategy works well during the test period, and then falls apart completely when subjected to new market data.
Of course, this is something we want to avoid, and that’s why we need to use some methods to validate our strategies and ensure that they are robust enough to trade live.
Ways to Avoid Curve Fitting
Let’s have a look at the most common approaches that traders use to build robust trading strategies!
1.Robust Parameter Combinations
When building a trading strategy, you could look at the different parameter combinations. In general, you want a strategy to work with as many parameter combinations as possible. The more the better. It indicates that the edge persists across a wide range of settings, which makes it less likely that the strategy is curve fit.
Now, a strategy could still work well even if it only shows good results with a few parameter combinations. However, it’s always a good rule of thumb to look for strategies that work with many parameter combinations.
2. Out of Sample and In Sample
Out of sample testing is another method that can be used to gauge the robustness of a strategy. It’s one of the most common and well-known methods.
Out of sample testing simply means that you divide your data into two sets, which are:
- The training set (In Sample)
- The validation set (Out of Sample)
You then test your strategy on the training set, and when you feel done, you insert the validation set, to see how it performs ther. If it holds, you can be a little more certain that your strategy actually works.
The general idea is that random market behavior won’t persist throughout both data sets, while a real edge, which isn’t random, will.
Just be careful! If you start going back and forth between the in-sample and out of sample sets, then you’ve in fact converted the out of sample to in sample. Out of sample data needs to be unseen to remain relevant!
Out of sample testing could also be made with live data, which means that you let the strategy sit for a while.
3. Walk Forward Testing
Walk Forward Testing is a more advanced form of out of sample and in sample testing.
Instead of just using just one validation and training set, you use several. This is how it works.
If you have 10 years of data, you divide those 10 years into 10 chunks. Then you optimize the strategy on year 1 , and apply those settings to year 2. Then you do this for all the 10 chunks until there is no data left.
The image below from amibroker illustrates this nicely.
The last step is to take all of those out of sample periods and stitch them together, which at least in theory creates an equity curve that’s completely out of sample.
4. Test on Other Markets
If the strategy trades successfully on several markets, that could be a sign of robustness. It shows that the strategy seems to be based on a strong tendency that’s prevalent across many markets.
However, most of our own robust trading strategies don’t work very well with other markets than the ones they were originally designed for.
When building trading strategeis, some traders like to use a monte Carlo simulator. In short, a monte Carlo simulation means that you change the order of the trades, and see what happens. If the trading strategy copes with having its trades in another order, then that’s believed to be a sign of robustness.
To run a monte carlo simulation on your strategy, you first need to get all the trades, with their P&L in a list. Most software will export this for you automatically, so it shouldn’t be too much of a hassle.
Then the monte Carlo simulator will rearrange all the trades several thousand times, to give you statistically viable results. For example, if you chose to run the simulation 10000 times, then you will have 10000 different equity curves, and every one of those will have slightly different performance metrics.
From these 10000 equity curves, you can now calculate the confidence intervals for different strategy metrics, such as drawdown.
Why Use Monte Carlo?
The monte Carlo simulator helps to answer what happens if the trades came in another order than in the backtest. For example, if the two worst trades happened in a row, then perhaps the maximum drawdown had been much bigger.
Some traders choose to not trade a strategy if it doesn’t perform well in monte carlo. While this might seem like a sound approach, we’re not very fond of it. Besides, after having done many monte carlo simulations, you often can tell what strategies will pass and not.
Considering that this could make you unconsciously tweak your strategies to work well in monte carlo, it adds little value to the process. However, some traders wouldn’t agree with us on this statement!
Which Method is Best?
This really depends on which trader you ask. None of the methods outlined above are foolproof, and all have their shortcomings. Consistently building robust trading strategies becomes a balancing act, where you need to take several factors into account, to get the whole picture.
In our algorithmic trading course, we teach our process of building robust trading strategies!
Step 5: Going live With a strategy
Once you are sure that you have a robust trading strategy, it’s time to go live. Many beginning traders are too eager to launch their trading strategies, and start trading their strategies far too soon.
It’s important to remember that trading is a long term commitment. Getting rich as a trader takes many years of hard work, and you shouldn’t feel stressed about launching a strategy today.
Take your time, even if that means not trading this year, or the coming year. Watch the strategies you’ve created and learn from them. It will provide some valuable lessons, and we can assure you that looking back, you will be thankful that you didn’t start trading right away.
What Almost Nobody Tells You
Many traders believe that once that one trading strategy is all they need to become rich. Even if this is a compelling though, it isn’t true. And there is one reason for that.
Trading strategies stop working. Regularly.
Markets are in constant change, and regardless of how well prepared we are and how rigorous our robustness testing is, our strategies are going to fail. If somebody claims something else, you should stop listening to them immediately. It probably means that you’re dealing with a scammer.
Now, as a result of this, we cannot rely fully on one single strategy. We’ll need to spread our risk, to ensure that the failure of one single strategy doesn’t cost us too much of our capital. In effect, this also means that we constantly have to look for new strategies, to replace those that fail.
In this guide to building a trading strategy we’ve shared some tips on how you could go about when creating your very own robust trading strategy.
As you probably realize, creating something that works isn’t an easy task.
The best thing you can do right now is to not rush into trading the strategies you’ve created. Wait some time and see what happens with the strategy once you’ve created it.
Trust us, you will not regret it!