Survivorship Bias in Strategies

The survivorship bias entry in this library covers data — backtesting on assets that still exist while ignoring those that failed. This entry covers a different and equally dangerous form: survivorship bias in strategy selection.

A strategy creator tests 50 parameter combinations. One produces a strong backtest. That one gets published. The buyer sees one strategy with impressive results. They do not see the 49 that were discarded. The strong backtest is not evidence of edge — it is evidence of search. Given enough parameter combinations, at least one will fit the historical data well by chance alone. This is sometimes called "data mining bias" or "p-hacking" in the academic literature.

The antidote is to ask: how many strategies were tested before this one was selected? If the answer is "many" and only the best one is being shown, the reported performance is inflated. If the answer is "one, derived from a theoretical model and tested once," the evidence is stronger — though still not conclusive. When the botwir3 marketplace launches, this question becomes the most important due diligence a buyer can perform. The seller's track record is not the strategy's track record — it is the track record of the best strategy the seller found. The distinction is the difference between evidence and advertising.