Emotions destroy returns. The answer is removing human biases from the equation entirely. In this chapter, we explore how to build and validate systematic logic that works out-of-sample across different regimes.
A backtest simulates how an algorithmic trading strategy would have performed using historical data. While many proclaim "past performance does not guarantee future results", systematic past performance helps you understand probability distributions of strategies.
"An algorithmic strategy with no backtest is a pure guess. An algorithmic strategy with a 20-year backtest is an educated probability."
Many amateur quant traders use platforms like TradingView or Python backtrader to test strategies. But naive backtesting is virtually worthless.
Curve-fitting (also known as overfitting), is the mortal enemy of quant traders. It involves creating a rule set so specific to the past data that it models noise rather than predicting the future trend.
A strategy must work across a neighborhood of parameters to be statistically valid. For example, a 200-day moving average strategy must also show positive alpha at 180-days, 190-days, 210-days, and 220-days. If the performance completely craters around a hyper-specific parameter, what you have is a curve-fitted system, not a robust edge. We call this searching for "Parameter Space Plateaus".
Instead of running a backtest over 20 years and optimizing variables, institutions employ Walk-Forward Analysis (WFA). This mitigates curve fitting.
How it works:
| Iteration | In-Sample Tuning Period | Out-of-Sample Testing Period (The "Walk") |
|---|---|---|
| 1 | Jan 2005 - Dec 2008 | Jan 2009 - Dec 2009 |
| 2 | Jan 2006 - Dec 2009 | Jan 2010 - Dec 2010 |
| 3 | Jan 2007 - Dec 2010 | Jan 2011 - Dec 2011 |
The model is trained dynamically. Only the "out-of-sample" 1-year test blocks are chained together to build the final equity curve. This simulates exactly what you would have experienced live!
A mean-reversion strategy will get decimated in a trending risk-off crash. A trend-following strategy will sustain enormous drawdowns in choppy sideways regimes (whipsaw). This means the best quantitative strategies employ mathematical Regime Filters to turn off logic when market weather is bad.
Example of Regime Filters: