LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
Tetlock’s 2007 study turned one Wall Street Journal column into a pessimism score and compared it with later Dow Jones returns. A one-standard-deviation rise in the score lined up with 8.1 basis points of next-day return left on the table. A strategy built from the score showed a 7.3 percent annualized return in the paper, and Tetlock’s own caveat, which the lecture repeats, is that modest round-trip trading costs wipe that result out.
Why it matters
The column is “Abreast of the Market.” The word lists come from a Harvard psychosocial dictionary with 77 categories. Seventy-seven counts are too many to regress one by one, so the study compresses them. Principal component analysis, a compression that keeps the direction of greatest shared variation, puts the heaviest loadings on weak, uncertain, and negative words. That direction is the pessimism factor. The lecture says the same negative direction stayed dominant when the compression was redone over time, which is why the score is usable as a series rather than as a one-year accident.
Earlier text work is thin in the lecture’s telling. Antweiler and Frank had already scored bullish versus bearish posts on Yahoo Finance and Raging Bull, and they had noted that a large share of financial social posts, described as some 80 percent, were positive. Their bullishness measure, as summarized here, did not show implications for the market variables they studied. Tetlock’s object is a newspaper column and a pessimism score, not a chat-room bullishness score.
The idea in plain words
The score used in the return regression is built only from information available at the previous date, so a reader of that day’s column could have computed it before acting. The regression links Dow Jones returns to that lagged pessimism. The reported association is the 8.1 basis-point next-day figure for a one-standard-deviation shock. The lecture calls that economically noticeable and immediately adds the paper’s warning, tied to Campbell, Grossman, and Wang in 1993: the push fades, and nearly the whole move unwinds before that trading week is over.
The strategy that trades the factor is reported at 7.3 percent annualized. The lecture compares that with an average market return described as barely higher, and it reports positive estimated expected returns in twelve years out of the fifteen in the sample. A binary-style argument in the paper is summarized as suggesting the pattern was unlikely to be chance. The lecture’s later aside, that sentiment strategies looked less stable through the pandemic period, is offered as hindsight the 2007 paper could not have had.
The cost sentence is the one that changes the decision. Transaction costs, market frictions, slippage, scalability, and taxes were not in the 7.3 percent. Tetlock notes, and the lecture agrees, that even modest round-trip costs obliterate the profitability. A basis point is one hundredth of one percent, so 8.1 basis points is a small next-day gap. Small gaps and round-trip costs collide.
How to use it
Treat the pessimism factor as a template for a text project, not as a system to copy. The template has three parts: a fixed corpus (one column), a dictionary with too many bins, and a compression whose leading piece is stable and interpretable as negativity. If your corpus is different, you still have to check that the leading piece is negative words and that it does not flip sign from year to year. The lecture’s reason for trusting the factor was that stability.
What usually goes wrong
Quoting 7.3 percent or 8.1 basis points without the reversal and the costs turns a temporary, fragile association into a plan. The lecture’s own caution is not to treat the result as spendable. Twelve positive years out of fifteen is a robustness note inside the sample, not a description of later decades. The pandemic remark in the lecture is a warning that sentiment rules have broken, not a new backtest.
Questions people ask
What did a one-standard-deviation pessimism shock line up with?
With 8.1 basis points of next-day Dow Jones return in the regression the lecture describes. The same lecture says the influence fades and nearly unwinds before the week ends.
What return did the paper’s strategy show?
An annualized 7.3 percent, with positive estimated expected returns in 12 of 15 years. Modest round-trip costs, as Tetlock notes and the lecture repeats, wipe out that profitability.
Why compress 77 word categories?
Because 77 series are too many to use directly. The first principal component concentrated on weak, uncertain, and negative words, and that negative reading stayed stable over time in the lecture’s account.