LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
Text can be turned into a number, tied to a company, and used to rank a cross-section of stocks. Easy data with an easy score is something many people can trade. Harder text, with a score that is harder to extract, is where the lecture looks for opportunity. The score is still full of noise, and a historical pattern in returns is not a promise that the same ranking will keep working.
Why it matters
The lecture states an inverse link: when a data set is easy to get and the signal is easy to pull out, more people can trade it. Alternative text is harder. Firms the speaker worked at bought and sold whole companies for proprietary data. A public tweet can still move a price. A Dogecoin post from Elon Musk is the lecture's short illustration, set beside the longer point that causality in markets is messy, endogenous, and hard to isolate in an experiment.
Documents are not equal. That same post carries more weight than an unknown account repeating a slogan. Text also misfires. A post about Doge the organization is not a post about the coin. A famous mix-up had algorithms buying Berkshire Hathaway when Anne Hathaway was in the news. A unique source still needs a filter.
The idea in plain words
A corpus is a collection, such as a site's articles, a set of posts, or a decade of filings. A document is one item. A token is a small meaningful piece: a word, a symbol, punctuation, or an emoji. Tokenization splits a document into those pieces. Counts of tokens across documents form a document-term matrix, a grid with words as columns and documents as rows. It is usually huge and sparse. Cutting rare words, using term frequency–inverse document frequency, or storing the grid sparsely are later refinements.
With only the words buy, stock, and crypto, each document is a point you can plot. Straight-line distance failed the lecture's three examples: every pair was the same distance apart, yet one note was about crypto, one about stock, and one about both. Angle works better. Cosine of 0 degrees is 1, and cosine of 90 degrees is 0. The crypto note and the stock note sat at 90 degrees, cosine 0. Each sat about 54.7 degrees from the note that mentioned both, cosine about 0.577. The same cosine extends to more words even when you cannot draw the space.
A signal vector that lives only on the buy direction then scores each document against "buy." The two notes with no buy-word sat at 90 degrees to that vector, score 0. The mixed note, at 54.7 degrees, scored about 0.577. The trap is negation. "Don't buy" can leave that geometry unchanged.
How to use it
A score is not yet a trade. Named-entity recognition has to decide which instrument the document is about. A spaCy pass on "Apple announced new products today" marked Apple as an organization and "today" as a date. "Apple was delicious" can still tag the fruit as a company. A language model can judge one sentence. Millions of documents make a slow model a poor production choice, which is why the lecture favors fast vectorized methods and points to a separate write-up where networks beat dictionary and cosine baselines on sentiment labels.
What usually goes wrong
Negation, entity mix-ups, and a generic word list all manufacture false scores. So does treating one document as the whole view of a firm. Even a clean monotonic chart is an artifact of history. The lecture lists reasons it can fade: the idea gets crowded, or the data become easy enough that more people trade it. Return distributions also move through time, so a bucket that ranked well in one sample need not rank the same way later.
Questions people ask
Does a viral post prove text causes returns?
The lecture treats formal causality as weak in this setting and uses the Dogecoin post as a plain illustration that some text shows up in the price.
Why not measure similarity by distance?
In the three-document example, every pair had the same straight-line distance, while the crypto note and the stock note had nothing in common. Cosine of the angle separated them.
What do you trade once a document has a score?
You still need the entity. Then you average many documents per name, rank the cross-section, and study whether top and bottom buckets had different average future returns.