LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
A neural network, like other supervised models, learns an expectation: the value that shrinks average error against observed targets. It is not a crystal ball. That expectation helps when the target stays put, as with a known option formula, and it comes apart when the same machinery is aimed at prices or returns whose distribution keeps moving.
Why it matters
The menu is large: linear and logistic regression, random forests, support-vector machines, neural nets. The task is shared. You pick a model and a parameter set, often called theta. Data go in, and an output comes out. Training, fitting, and calibration all mean choosing theta to shrink error.
The lecture starts with football throws, before any price series. Features might be height, weight, bench press, and age, recorded before the throw. The target is distance. Mean squared error sums the squared gaps. Only theta is free. A convex loss lets you slide to the single low point. Real problems are non-convex, so a downhill search can stop in a local valley you cannot see from outside.
The idea in plain words
A fit that hugs yesterday's points can miss tomorrow's. An underfit curve misses the shape. An overfit curve threads old points and then misses new ones. In the lecture's comparison, the overfit error on new points was almost three times the robust model's. The control is a split. With 1,000 throws, 60 percent can set theta, 20 percent can tell you when to stop, and 20 percent stays unseen. Training error can keep falling after validation error turns up. Stop at that turn.
A fair die has expectation 3.5. After 10,000 rolls, a network trained to name the next face produced 3.5. A die cannot land there. The number minimizes squared error, so it is an expectation, and as a face it is always wrong.
A feed-forward network is a stack of matrix products, added biases, and element-wise activations. One hidden layer is the shallow case. More than one is deep learning, and theta is the full set of weights and biases. The universal approximation theorem says that, on a compact set, this stack can approximate any continuous function as closely as you demand. A straight line cannot follow a curved map from body size to throwing distance. A network can.
How to use it
Aim it at an expectation that is stable enough to learn. The lecture builds many Black-Scholes parameter sets, prices each option with the analytical formula, and trains the network to map parameters to price. After a few epochs the two prices are already pulling together. The formula is both the solution of a hedging equation and a discounted risk-neutral expectation, so a universal approximator is a natural student.
The practical case is a model with no closed form. Simulate prices from a rough or non-Markovian model, teach the network offline, and reuse it for fast calibration or exotic extrapolation. Nearby uses include signals from alternative data, and experimental risk measures built with generative cousins such as variational autoencoders and GANs.
A direct price-path forecast is different. In sample, a curve can hug the path. Past the line that marks unseen data, it comes apart, because the source of returns moved. A die's distribution stays fixed. A return distribution does not, and the network has learned a statistic of the old source.
What usually goes wrong
A lower training loss is not a forecast of one outcome. The die makes that plain. Training after validation error has turned up is the second miss. Feeding prices or returns in as the target, and hoping depth will outrun non-stationarity, is the third. Stability of the source sets the ceiling.
Face recognition is the lecture's contrast. Classes separate cleanly enough for a phone lock. A false reject is retried, and a beard may force a rescan, yet that drift is mild beside returns. Supervised learning is also only one family. Unsupervised methods and reinforcement learning, including optimal delta hedging, are separate jobs. A network that prices an option has not been asked to hedge it.
Questions people ask
Is the output a forecast of one result?
It is an expectation chosen to shrink error. The die experiment landed on 3.5, the theoretical mean, which no face can show.
Why can a network follow a curved price map?
On compact sets it can approximate any continuous function. A line of best fit cannot. The lecture names this the universal approximation theorem.
Should the target be the price itself?
Keep the price path out of the target. Approximate a stable pricing function, or build a signal from other data, because the return distribution moves.