Contents
What a backtest isThe knobs that move resultsThe biases that flatter resultsA discipline for reading reportsRed flags in a reportWhat a backtest is
A backtest replays a strategy against historical candles and reports what would have happened. It is a simulator with a known input and a deterministic strategy, which makes it excellent for answering "what did this rule do to that data" and poor for answering "what will this rule do next month". The distinction is the entire discipline.
The knobs that move results
| Knob | What it changes | Direction of temptation |
|---|---|---|
| Fee assumption | Cost per round trip | Zero fees, infinite confidence |
| Slippage model | Fill price realism | Omitted entirely |
| Pair list | What is traded | Curated after the fact |
| Period | Which history | Only the kind year |
| Timeframe | Granularity | Whatever looked steepest |
Every one of these knobs is legitimate and every one has a direction in which results improve. A backtest report that does not state each knob's setting is a report that cannot be audited, whatever its total profit.
The biases that flatter results
- Overfitting. Rules tuned until they fit the past perfectly; the future declines to cooperate.
- Survivorship in pairs. A pair list of today's winners tested against history in which some of them did not exist.
- Look-ahead. Indicators that quietly use a candle that has not closed yet. One library bug of this kind is enough to make every result fiction.
- Fee and slip optimism. A round trip that costs nothing in the report and real money in the market.
A discipline for reading reports
- Read the trade count and the period before the profit line. Three hundred trades over two years outranks thirty trades over a fortnight.
- Read the maximum drawdown next to the profit. Profit is what the strategy makes in a good moment; drawdown is what it asks you to endure on the way.
- Re-run the same strategy on a period it was not tuned on. The drop from in-sample to out-of-sample is the honest number.
- Turn fees up and see if the strategy survives. If profit vanishes at realistic fees, it was never there.
Red flags in a report
- A total return quoted with no period and no trade count.
- Results that improve monotonically with every added indicator — that is memorization, not edge.
- One pair doing all the work while the report implies a portfolio.
- No mention of what happens when the strategy is wrong, which is the only certainty in the business.