Backtesting: what it's good for, and how it fools you

Backtesting — running a strategy's rules against historical data to see how it would have done — is genuinely useful. It's also one of the easiest ways in all of finance to fool yourself.

The seduction

Give a computer enough freedom to tweak a strategy's parameters and it will find a version that looks spectacular on the past. That's not skill; it's selection. As research on backtest overfitting by López de Prado and colleagues shows, if you try enough variations, a strikingly good-looking result is essentially guaranteed by chance — and it tends to fall apart the moment it meets new data.

A backtest that's been tuned until it shines is often just a record of how well you fit the noise.

Using it honestly

The discipline is about resisting hindsight:

• Decide the rules before you look, not by hunting for what would have worked.
• Hold out data the strategy never sees during development, and judge it there.
• Be suspicious of perfection. A curve that's too clean is a warning sign, not a triumph.
• Account for real costs — spreads, slippage, fees — which quietly erase many paper-perfect edges.

What it can and can't tell you

A careful backtest can tell you a rule isn't obviously broken and roughly how it behaves through different conditions. It cannot promise the future will resemble the past, and it can't substitute for live, paper-first validation.

Treated as a sanity check, backtesting earns its place. Treated as a crystal ball, it becomes an elaborate way to be confidently wrong.

---

This article is educational and does not constitute investment advice or a recommendation. Options trading involves substantial risk and is not suitable for every investor. Autopilot Options does not guarantee profits or prevent losses. Past performance and historical data do not guarantee future results.