LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
A smooth backtest claims that each decision used only information you had then, that the settings were not shopped, and that failed firms stayed in the universe. Future data, a searched parameter, and a survivor-only list each break one claim. The lecture's examples look strong until the break is repaired, and then the result falls apart.
Why it matters
Strategies that fail after they go live were often already broken in the test. A regime change is the story people reach for. The lecture's point is that the arithmetic was often wrong first. A pretty curve is a reason to inspect the pipeline, not a reason to size up.
Costs, the bid-ask spread, and slippage matter too. The lecture calls them accomplices. If the cleaned test cannot produce a positive Sharpe ratio, real-world frictions will not rescue it.
The idea in plain words
Look-ahead bias means the trade used something that was not known when the order would have been sent. Picture two envelopes. The first holds everything available at or before the decision. The second holds what arrived later. Only the first envelope is a legal input. The lecture ranks an alternative-data signal into buckets, goes long the top, shorts the bottom, and calls the book market-neutral. Before the leak is closed, the Sharpe ratio is three and the buckets line up with later returns. After it is closed, performance drops hard. A ratio around 0.9 or 1.2 can already be polluted. The leak can be a number from a week ahead, or today's signal scaled with next week's return.
Data snooping is shopping the settings. Two moving-average crossovers are compared. A three-and-five pair loses to a seven-and-twelve pair, so the winner looks like the rule to trade. Try hundreds of pairs and a high in-sample Sharpe ratio shows up by construction. Traded forward, that seven-and-twelve crossover has a Sharpe ratio of negative 1.8.
Survivorship bias is the missing wreckage. Wartime analysts mapped holes in planes that returned and were tempted to armor the holes. Planes that did not return never contributed a map. The same mistake is a universe of firms that still exist. The lecture's long-short signal looks fine until bankrupt and delisted names are restored. Some had large weights. Longs that should have helped lost, shorts that should have hurt gained, and the Sharpe ratio turns negative.
How to use it
Keep every input in the first envelope. Parameters, scalers, and alternative data must be fit on information that existed at the decision, including when that data would have arrived.
For settings, chop history into slices. The lecture's sketch is a couple of months to choose, then the next couple of months unseen, rolled forward. A model that learned structure should not die because seven and twelve becomes eight and twelve. You can still overfit that ritual, so the point is stability, not a new high score.
Define universe membership at each decision date from a rule you could have run that day, such as an index or a market-cap screen. Include names that later fail.
What usually goes wrong
A Sharpe ratio of three feels like proof and becomes a reason to skip the audit. The lecture treats it as a reason to look for a leak. A score near one feels humble, so a quieter leak is ignored.
The parameter search gets narrated as research. Without a reason for that window, the forward test did not rhyme with the in-sample winner. Dead firms get left out because a vendor's current list is convenient. The repaired test is the one that went negative.
Questions people ask
Does look-ahead bias always create an absurd Sharpe ratio?
No. The leaked test printed a Sharpe ratio of three, but a ratio near 0.9 or 1.2 can be tainted too. The leak does not have to look ridiculous.
Is the best in-sample window the one to trade?
Not if you found it by trying many windows. The seven-and-twelve pair won in sample and then posted a Sharpe ratio of negative 1.8 out of sample. Nearby settings should be dull, not disastrous.
Why include firms that went bankrupt?
A decision made today cannot see who survives. The lecture's signal looked healthy on survivors and turned to a negative Sharpe ratio once delisted names were restored.