LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
Deep hedging is a reinforcement-learning approach, associated with Hans Bühler, that learns a hedging policy instead of rehedging on a fixed clock or a hand-made no-trade band. The policy maps a state, a vector of inventory, prices, volatility, and path information, to a hedge. The target is a risk-adjusted result, not the largest possible gain. It does not remove model risk, and it does not make continuous delta hedging possible.
Why it matters
A market maker’s theoretical price comes from hedging, not from a forecast of the risk premium. Under Black-Scholes, if the underlying follows geometric Brownian motion and the market is complete, delta hedging replicates the option. The fair price is the discounted risk-neutral expected payoff. Trade at that price, hedge continuously, and the average gain and loss is zero. You earn the riskless rate on average in that argument, not on every path. Students hear “riskless” and think every day pays the T-bill. The animation in the lecture shows the opposite: individual hedged paths wander, and the average of many of them settles near zero.
The idea in plain words
Two idealizations break in practice.
The model can be wrong about future dynamics. If the true paths were known, there would be nothing to model. Local volatility, stochastic volatility, and rough-volatility models exist because geometric Brownian motion is a choice. Everyone who prices, on the sell side or the buy side, accepts some model risk.
You cannot hedge continuously. Continuous trading would pile up infinite transaction costs. The lecture compares a near-continuous hedge with rehedging on a coarse fixed schedule over a one-year simulation. The coarse schedule shows a lower average and a wider scatter. Discrete hedging adds variance even before costs.
Fixed clocks are not optimal either. A gap down might deserve a hedge. A gap down followed by a gap up might not, and you do not know that in advance. Gamma bands and no-trade zones are practical rails. Deep hedging asks whether a learned policy can do better, on a risk measure the desk actually cares about, by seeing inventory and new information.
Reinforcement learning frames that question as a Markov decision process. A state goes in, an action comes out, a new state and a reward come back. The objective is the expected reward under the policy. In a toy grid, a reward of plus ten sat on a star and minus ten on a skull. A small network needed on the order of a hundred episodes to stop walking into the skull. The market version is the same loop with a worse state: positions, the underlying, path-dependent features such as signatures, realized volatility, time, and, for several assets, implied-volatility surfaces. Text has to be turned into numbers before it can enter the vector.
The reward is deliberately not raw gain. Entropy-style risk measures and expected shortfall, a tail-loss average, keep the agent from “solving” the problem by levering the noisiest coin. A single lucky discrete-hedging path can beat the learned policy. One path is not the objective. Tail risk and inventory are.
How to use it
Start from a price you can hedge, then add a spread you can defend. If you move to a learned hedge, pick the simulator first: a classical stochastic-volatility or jump model, or a generative model used to invent tail events that history has not shown. Train the network to approximate the policy. Compare its gain distribution with a static delta rule on the same paths, and look at the tails, not only the mean.
What usually goes wrong
Maximizing expected gain with no risk penalty. Treating one good path as proof the policy dominates delta. Believing the network removes the need for a model. The model is the simulator it trained on. And hedging on a timer because the textbook hedge is continuous, then acting surprised when costs and gap risk show up.
Questions people ask
If I trade at the fair price and delta-hedge, do I earn the riskless rate every day?
No. In the replication argument the average hedged result is flat. Single paths move. The spread is what pushes the average above zero, and only if the hedge is good enough.
What does the network actually learn?
A map from a numeric state vector to a hedge. That map is the policy. Deep hedging is the name for fitting it with a neural network under a risk-adjusted reward.
Can this replace the pricing model?
No. You still choose the simulator, the features, and the risk measure. Wrong dynamics remain wrong dynamics.