LessonThis is a lesson restated in new words. Any figure in it is an example from the source, not this desk's record.
A 10,000 by 10,000 matrix product of standard normal draws took about 23 seconds on the CPU in this lecture and about 3 seconds on a free Colab GPU, close to a tenfold cut, but only after the timer waited for the GPU to finish. Neural-net training is mostly that kind of arithmetic, so the same move, pushing the model and the tensors onto the device, is why a movie-review network’s ten updates felt almost immediate after they had been painful on the CPU.
Why it matters
The speedup is parallel work. The lecture’s picture is a bag of marbles. One person counting every marble is the single thread. Splitting the bag with a friend and adding the two counts at the end is the parallel version. A GPU does that split across many cores, which helps when the work is a big array operation. It does not help a chain of tiny loops that must happen in order. Knowing which of those you wrote matters more than knowing the word “GPU.”
Colab’s free tier is the setup in the lecture. The runtime panel shows RAM and disk. Changing the runtime type to a GPU, saving, and confirming that a GPU RAM line appears means the kernel can see the device. If a message says too many sessions are active, closing the extras is the fix they used. They did not buy extra compute for the demo.
The idea in plain words
Import PyTorch and the time module. Draw a 10,000 by 10,000 matrix from a standard normal law, mean 0 and standard deviation 1, with torch.normal. That matrix lives on the CPU. A second tensor, created by sending the first to CUDA, lives on the GPU. You cannot mix them in one product. The location is part of the value.
Time the CPU product by recording the clock, multiplying the matrix by itself, recording the clock again, and subtracting. That difference was about 23 seconds. The GPU version needs an extra call, torch.cuda.synchronize, before the second clock read. Without it, the Python line that launches the product returns immediately while the device is still working, and the elapsed time collapses toward zero. Synchronization makes the main thread wait until the product is actually done. With that wait, the lecture measured about 3 seconds.
The network example is a classifier for movie reviews. The notebook downloads text, cleans it, sets aside labels, and builds a document-term matrix, a table of documents against word counts. Dimensionality reduction is skipped. A train-test split separates rows. The network uses a sigmoid activation, so its output is read as a probability between 0 and 1 of the positive class. The loss is binary cross-entropy. The optimizer is stochastic gradient descent. On the CPU, each pass lowered the error slowly, which is what backpropagation is supposed to do, and the pace was too slow for the lecture’s taste if the next job is retuning settings.
The GPU copy of that notebook changes where objects live, not the architecture. The sequential model, the loss, and the tensors are each sent to CUDA. Ten iterations that had been a wait on the CPU were described as almost instant on the free GPU.
How to use it
Confirm the device before any timing. A missing GPU RAM line means you are still on the CPU and a later .cuda call will fail. Generate or load tensors, then move the ones that must meet each other. Keep a naming habit so a CPU tensor and its GPU twin are not multiplied by accident.
What usually goes wrong
Trusting a GPU time that omitted synchronization understates the product by a huge factor and makes a three-second kernel look free. The lecture pauses on this because it is the measurement bug, not a property of the hardware.
Questions people ask
Why did the GPU timer nearly hit zero without an extra call?
The launch returned before the device finished. torch.cuda.synchronize blocks until the queued GPU work is done, which is what made the lecture’s second clock read land near 3 seconds instead of near 0.
What were the two measured times?
About 23 seconds for the CPU product of the 10,000 by 10,000 matrix with itself, and about 3 seconds for the CUDA product after synchronization. The lecture calls that almost a tenfold difference on that matrix.
What changed in the review network besides the device?
The hyperparameters, the split, the sigmoid, the binary cross-entropy loss, and stochastic gradient descent stayed. The model, the loss, and the tensors were sent to CUDA, and ten updates went from slow CPU passes to a near-immediate GPU run.