LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
A neural network trained to "predict" a fair die does not learn the next face. In the lecture it settles near 3.52, against a true mean of 3.5, and the plain average beats it. What it learns is an expectation. With extra inputs that expectation becomes conditional, and it is only as stable as the problem.
Why it matters
Shops such as Citadel, Jane Street, and Two Sigma still test probability more often than they test a model. The dice example is the reason. If the best answer for an independent roll is the mean, the skill is knowing that mean, when it is conditional, and when the distribution will not hold still.
The idea in plain words
A random variable is a distribution. A die is discrete and flat. A bell curve puts more weight in the center. Five rolls are a rough histogram. A thousand, then ten thousand, fill in the theoretical bars, because the rolls are independent and the die is not reweighted.
A trading signal does not hand you the theoretical curve. You only have the draws, and those draws can come from a shape that changes. Keep sampling and the histogram need never match one fixed bell. A die is also different from a price in another way. No fact you condition on changes the next roll. A price can jump when information arrives, which is why trading on material nonpublic information is illegal. Condition on new information, or wait, and the distribution you were approximating moves.
Train a network on rolls by feeding it a constant and asking for the best guess. The lecture's fit prints 3.52. The mean of the faces is 3.5. On the squared error the mean wins. Under independent draws from that fixed die, no network beats the mean in the limit. The network learned the average.
The same split-the-difference rule is a conditional expectation once inputs vary. Two 25-year-olds in the lecture's sketch can be 68 inches or 72 inches. The best single height, given only age, lies between them. Gender or ethnicity can move that level. It is still an average inside a slice of the data, not a promise about one person. The lecture's name for a flexible version of that slice is a nonlinear conditional expectation, and a network is one way to approximate it.
How to use it
A market-making sketch in the lecture makes the level concrete. Bids and offers are uniform noise around a mid, and the history of mids is the training set. The network returns about 3.99. The expectation of the bid and offer is 4. The useful decision is what you do around that level, not the label "prediction."
Probabilities are the same object. A Bernoulli random variable has a mean equal to its success probability. Mark even rolls as 1 and odd rolls as 0. Five, then a thousand, then ten thousand marks move toward one half. A network trained on 10,000 such marks learns 0.49. The true probability is 0.5. Up or down for a price, if you force it into that 0-1 form, is the same kind of level.
Face unlock is the stable case. Above a 50 percent probability the phone opens. Sunglasses, a mask, an odd angle, or a new beard can leave it locked. The face still changes slowly enough to be usable. A firm in 2000 and the same firm in 2025 are not draws from one curve.
The lecture's practical split is to let a model build a signal, then test the signal with ordinary statistics. A picture with clean separation puts positive average returns on one side of a cut and negative averages on the other, long one basket and short the other. That picture can break as the market moves. Options, earnings, and a hedged short-volatility book are named as places where people inspect average gain and loss directly. The network is not required for that inspection.
What usually goes wrong
A flexible model is not a cure for an expectation that will not sit still. A chat model is also a conditional expectation, and the language stays put. Stock returns do not. A separator fit on one window, then trusted as a lasting long-short book, repeats the dice error on a die that is being recut.
Questions people ask
Why did the network print 3.52?
It was approximating the mean of a fair die, which is 3.5. With independent rolls from a fixed die, the lecture's error comparison favors the mean itself.
Is a probability a different kind of output?
In this account, no. Even versus odd is a Bernoulli mean. The network's 0.49 on 10,000 rolls is an estimate of the 0.5 probability.
Where does the lecture still use a model?
To build a signal. The test of that signal, in the lecture, can be a classical average of returns in buckets, not another network mapping the signal onto the return.