Edge Builder
Glossary
The validation report uses a handful of quant terms. Here they are in plain language — no statistics degree required.
- In-sample / Out-of-sample
- In-sample is the data the strategy was built and tuned on — the “study set”. Out-of-sample is fresh data it never saw — the “exam”. A strategy that only works in-sample has likely memorised the past rather than learned a real edge.
- Overfitting
- When a strategy fits the random noise in historical data instead of a genuine, repeatable pattern. It looks fantastic in a backtest and falls apart live. Killing overfitting is the whole point of validation.
- Walk-forward analysis
- Splitting history into rolling windows and repeatedly tuning on one slice then testing on the next, unseen slice. It checks whether an edge persists through time rather than in one lucky period.
- Sharpe Ratio
- Return measured against the risk taken to get it — higher means smoother, more efficient gains. The “Deflated” version corrects for the fact that trying many variations makes the best one look better than it really is.
- Max drawdown
- The largest peak-to-trough fall in the equity curve — the worst losing stretch you would have had to sit through. It tells you how painful the strategy gets before it recovers.
- Profit factor
- Gross profit divided by gross loss. Above 1 means the strategy made more than it lost; the higher the number, the more cushion it has.
- Monte Carlo permutation
- Reshuffling the order of the trades many times to see whether the result holds up or depended on one lucky sequence. It produces a range of likely outcomes instead of a single number.
- Probability of Backtest Overfit (PBO)
- An estimate of how likely it is that the “best” backtest is just chance. Lower is better — a high PBO is a red flag that the result won’t survive live.
- Expectancy
- The average profit (or loss) you can expect per trade over the long run. Positive expectancy is the foundation of any viable strategy.
- Risk of ruin
- The probability of losing enough capital to be knocked out of the game. Even a profitable strategy can have an uncomfortable risk of ruin if position sizing is too aggressive.
Keep reading