LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
Yesterday's S&P 500 return does not tell you today's. A histogram of returns sits near zero with fat tails, and a scatter of today against yesterday, over about five years, is a round cloud. A linear regression fit on an earlier slice had a mean squared error near 0.99 and a test error of 33.50. A neural network fit the training slice a bit harder, at 0.952, and did worse on the test slice, at 33.75. Extra model flexibility did not create a signal that the plot did not already show was missing.
Why it matters
It is easy to start a forecasting project at the model and skip the picture. The lecture's picture is the model. If yesterday's return is zero, the vertical slice of the cloud is spread on both sides of zero. A yesterday of 0.25 still leaves today anywhere in a wide band, on the order of plus or minus a couple of percent inside the axes the lecture drew, with outliers possibly off the chart. There is no upward or downward stripe. A correlation you could trade would have drawn one. Price-only lags omit rates, inflation, and news. The lecture's line is that a model cannot invent those missing inputs, whether the model is a line or a network.
The idea in plain words
The response is today's return. The only input is the previous session's return. Training and test rows are cut in time, not shuffled, because the rows are a series. Shuffle them and a later day can leak into the fit. The linear model is ordinary least squares on the training rows. Mean squared error near 1 on that slice, against 33.50 on the later slice, is the lecture's picture of fitting noise. The test number is not a universal constant of markets. It is what this split produced, and it is huge next to 0.99. Repeating the split, the lecture says, keeps the same pattern: a tolerable training score, a bad test score. The network is the same task with a more flexible function. Its training error, 0.952, is a little lower than the line's, which the lecture reads as a better fit to noise. Its test error, 33.75, is a little higher. That is the overfitting warning in one pair of numbers. Networks can keep lowering training error after the test error has started to rise. A validation slice, held out from both, is the usual place to stop. The lecture names that practice and does not run it here. The comparison does not need it. Both models fail out of sample, and the flexible one does not fail less. Economic variables might matter contemporaneously. The lecture's example is inflation surprising the people who had expected it to rise, and returns moving that day.
How to use it
Plot the scatter before you train. If it is a blob, expect a training score that looks calm and a later score that does not. Use a time split. Report both errors. A model that is "better" only on the training slice has not earned a forecast. If you add inputs, demand a reason they could move the next return, then look at whether the test error actually falls. The lecture's jab at common technical overlays, such as Bollinger bands, is the same point: a transformation of past prices is still past prices. Complexity helps when the relevant shape is real and a straight line cannot see it, for example when several series need a transformation you do not know in advance. Complexity does not help when the input is empty.
What usually goes wrong
Reading 0.99 as accuracy. It is a training mean squared error on a series whose raw variation the test error says was not captured. Comparing a network's 0.952 with the line's 0.99 and shipping the network. The test numbers go the other way, 33.75 against 33.50. Shuffling a time series into a random train-test split, which the lecture refuses, lets the fit see neighbors of the days it claims to predict. And treating a fat-tailed histogram centered at zero as a small detail. The lecture calls excess kurtosis a standard fact of returns. The center says the typical day is near flat. The tails say the atypical day is fatter than a bell curve. Neither fact is a lag you can trade.
Questions people ask
What were the four errors?
Linear model: about 0.99 in sample, 33.50 out of sample. Network: 0.952 in sample, 33.75 out of sample. The lecture calls 33.50 enormous relative to 0.999, the same training result described to three digits.
What data were plotted?
S&P 500 returns, a histogram centered near zero, and about five years of today against yesterday. The lecture says the same emptiness would show up for a single stock or an ETF if the only input were a lagged return.
Would a validation set have saved the network?
It might have stopped training earlier. It would not have created a relationship the scatter does not contain. The lecture's point is the missing information, not the choice of optimizer.