Demystifying attention, the key mechanism inside transformers and LLMs.

Visualizing Attention, a Transformer's Heart | Chapter 6, Deep Learning

A visual introduction to transformers. This chapter focusses on the overall structure, and word embeddings

But what is a GPT?  Visual intro to Transformers | Deep learning, chapter 5

Answering a few viewer questions about the refractive index.

How light can appear faster than c, why it bends, and other questions | Optics puzzles 4

The origins of the index of refraction, and why it depends on color

Why light can “slow down”, and why it depends on color | Optics puzzles 3

Winners for the first summer of math exposition

Summer of Math Exposition 1 results

Winners for the second summer of math exposition

Summer of Math Exposition 2 results

How shining polarized light through sugar water results in colored diagonal stripes

The barber pole effect | Optics puzzles 1

How light can be described by a force due to the delayed effect of an accelerating charge, and how it can be used to explain the demo from the previous lesson.

Explaining the barber pole effect from origins of light | Optics puzzles, 2

Winners for the third summer of math exposition

Summer of Math Exposition 3 results

A visual trick for computing the sum of two normally-distributed variables, and how it fits into the Central Limit Theorem

Why are normal distributions "central limits"?

An apparent pattern that breaks, and the reason behind it.

Explaining the absurd circle division pattern | Moser's circle problem

How to add random varaibles, with a focus on two distinct ways to visualize the continuous case

Convolutions and adding random variables

Summer of Math Exposition 2 announcement

A classic proof explaining the pi in a Gaussian distribution, combined with a derivation of that distribution, the Herschel-Maxwell derivation, that explains why the proof is reasonable.

Why π is in the normal distribution (beyond integral tricks)

A visual introduction to one of probabilty's most important theorems

But what is the Central Limit Theorem?

From probability to image processing and FFTs, an overview of discrete convolutions

But what is a convolution?

A curious pattern of integrals that all equal pi...until they don't.

Researchers thought this was a bug (Borwein integrals)

Three false proofs and what lessons they teach.

How to Lie With Visual Proofs

Generating functions, as applied to a hard puzzle used for IMO training.

Olympiad level counting

Following up on the previous video about Wordle and information theory

Oh, wait, actually the best Wordle opener is not “crane”…

A lesson on information theory and Shannon entropy, told through an exploration of writing a Wordle solver.

Solving Wordle using information theory

A tale of two problem solvers, looking for the average area of a cube's shadow

Alice, Bob, and the average shadow of a cube

An introduction to holomorphic dynamics, which studies iterative processes in the complex plane.

How a Mandelbrot set arises from Newton’s work

A surprising fractal that arises from Newton's method for finding roots of polynomials.

Newton's Fractal (which Newton knew nothing about)

Summer of Math Exposition 1 announcement

A quick way to compute eigenvalues of a 2x2 matrix

A quick trick for computing eigenvalues

Exponentiating matrices, and the kinds of linear differential equations this solves.

How (and why) to raise e to the power of a matrix

The medical test paradox: Can redesigning Bayes rule help?

Hamming codes part 2, the elegance of it all

A discovery-oriented introduction to error correction codes.

Hamming codes and error correction

An introduction to symmetry, group theory, isomorphisms, and the monster group.

Group theory, abstraction, and the 196,883-dimensional monster

An information puzzle with an interesting twist

The impossible chessboard puzzle

Tips to be a better problem solver

Intuition for i to the power i

An overview of a simplified version of the DP-3T algorithm for privacy-first contact-tracing

The DP-3T algorithm for contact tracing (via Nicky Case)

The power tower puzzle

What makes the natural log "natural"?

Logarithm Fundamentals

Imaginary interest rates

What is Euler's formula actually saying?

Complex number fundamentals

Trigonometry fundamentals

The simpler quadratic formula

Lockdown math announcement

Introduction to probability density functions.

Why “probability of 0” does not mean “impossible” | Probabilities of probabilities, part 2

SIR models for epidemics, showing how tweakign behavior can change an outbreak.

Simulating an epidemic

The binomial distribution, introduced as setup to talk about the beta distribution

Binomial distributions | Probabilities of probabilities, part 1

A primer on exponential and logistic growth, with epidemics as a central example

Exponential growth and epidemics

A short explanation of why Bayes' theorem is true, together with discussion on a common misconception in probability

The quick proof of Bayes' theorem

A visual way to think about Bayes' theorem, and strategies for making probability more intuitive.

Bayes' theorem

Q&A with Grant (3blue1brown), windy walk edition

A game of darts yields an interesting connection to higher dimensional geometry.

Darts in Higher Dimensions

A curious pattern in polar plots with prime numbers, together with discussion of Dirichlet's theorem

Why do prime numbers make these spirals?

Problem 2 from the 2011 IMO, a seemingly easy question that turned out to be ridiculously difficult

The unexpectedly hard windmill question (2011 IMO, Q2)

A quick explanation of e^(pi i) in terms of motion and differential equations

e^(iπ) in 3.14 minutes, using dynamics

A montage of "fourier series" drawings, in which the sum of many rotated vectors traces an image

Pure Fourier series animation montage

How Fourier series arose from studying heat flow, and how they can be thought of as decomposing any drawing in 2d into a sum of rotations.

But what is a Fourier series? From heat flow to circle drawings

Solving the heat equation

The heat equation, as an introductory PDE.

But what is a partial differential equation?

What is a differential equation, the pendulum equation, and some basic numerical methods

Differential equations, studying the unsolvable

What Cramer's rule is, and a geometric reason it's true

Cramer's rule, explained geometrically

The third and final part of the block collision sequence.

How colliding blocks act like a beam of light...to compute pi.

A solution to the puzzle involving two blocks, sliding fricionlessly, where the number of collisions mysteriously computes pi

Why do colliding blocks compute pi?

A puzzle involving colliding blocks where the number pi, vey unexpectedly, shows up.

The most unexpected answer to a counting puzzle

Two proofs for the surface area of a sphere

But why is a sphere's surface area four times its shadow?

Solving a discrete math puzzle, namely the stolen necklace problem, using topology, namely the Borsuk Ulam theorem

Sneaky Topology | The Borsuk-Ulam theorem and stolen necklaces

A look at what turbulence is (in fluid flow), and a result by Kolmogorov regarding the energy cascade of turbulence.

Why 5/3 is a fundamental constant for turbulence

An introduction to an interactive experience on why quaternions describe 3d rotations

Quaternions and 3d rotation, explained interactively

How to visualize quaternions, a 4d number system, in our 3d world

Visualizing quaternions (4d numbers) with stereographic projection

Q&A with Grant Sanderson (3blue1brown)

A beautiful proof of why slicing a cone gives an ellipse.

Why slicing a cone gives an ellipse

This video recounts a lecture by Richard Feynman giving an elementary demonstration of why planets orbit in ellipses.  See the excellent book by Judith and David Goodstein, "Feynman's lost lecture”, for the full story behind this lecture, and a deeper dive into its content.

Feynman's Lost Lecture (ft. 3Blue1Brown)

Divergence, curl, and their relation to fluid flow and electromagnetism

Divergence and curl:  The language of Maxwell's equations, fluid flow, and more

A visual for derivatives which generalizes more nicely to topics beyond calculus. Thinking of a function as a transformation, the derivative measure how much that function locally stretches or squishes a given region.

The other way to visualize derivatives

A proof of the Wallis product for pi, together with some neat tricks using complex numbers to analyze circle geometry.

The Wallis product for pi, proved geometrically

An algorithm for solving continuous 2d equations using winding numbers.

Winding numbers and domain coloring

A bit of the history behind how we came to use the symbol "pi" to represent what it does today, and how Euler used it to refer to several different circle constants.

How pi was almost 6.283185...

A beautiful solution to the Basel Problem (1+1/4+1/9+1/16+...) using Euclidian geometry.  Unlike many more common proofs, this one makes it very clear why pi is involved in the answer.

Why is pi here?  And why is it squared?  A geometric answer to the Basel problem

The general uncertainty principle, about the concentration of a wave vs the concentration of its fourier transform, applied to two non-quantum examples before showing what it means for the Heisenberg uncertainty principle.

The more general uncertainty principle, beyond quantum

An animated introduction to the Fourier Transform, winding graphs around circles.

But what is the Fourier Transform?  A visual introduction.

A classic puzzle in graph theory, the "Utilities problem", a description of why it is unsolvable on a plane, and how it becomes solvable on surfaces with a different topology.

The three utilities puzzle with math/science YouTubers

A challenging question of geometry and probability on the Putnam.  The shift in perspective used for the solution here carries with it broader loessons for problem-solving.

The hardest problem on the hardest test

The math of backpropagation, the algorithm by which neural networks learn.

Backpropagation calculus

An overview of backpropagation, the algorithm behind how neural networks learn.

What is backpropagation really doing?

An overview of gradient descent in the context of neural networks. This is a method used widely throughout machine learning for optimizing how a computer performs on certain tasks.

Gradient descent, how neural networks learn

Analyzing our neural network

An overview of what a neural network is, introduced in the context of recognizing hand-written digits.

But what is a Neural Network?

An introduction to the quantum behavior of light, specifically the polarization of light.  The emphasis is on how many ideas that seem "quantumly weird" are actually just wave mechanics, applicable in a lot of classical physics.

Some light quantum mechanics (with minutephysics)

A method for thinking about high-dimensional spheres, introduced in the context of a classic example involving a high-dimensional sphere inside a high-dimensional box.

Thinking outside the 10-dimensional box

Drawing curves that fill all of space, and a philosophical take on why mathematics about infinite objects can still be useful in finite contexts.

Hilbert's Curve: Is infinite math useful?

When a piece of cryptography is described as having "256-bit security", what exactly does that mean?  Just how big is the number 2^256?

How secure is 256 bit security?

How does bitcoin work? What is a "block chain"? What problem is this system trying to solve, and how does it use the tools of cryptography to do so?

But how does bitcoin actually work?

There are a few special right triangles many of us learn about in school, like the 3-4-5 triangle or the 5-12-13 triangle.  Is there a way to understand all triplets of numbers (a, b, c) that satisfy a^2 + b^2 = c^2?  There is!  And it uses complex numbers in a clever way.

All possible pythagorean triples, visualized

A beautiful derivation of a formula for pi.  At first, 1-1/3+1/5-1/7+1/9-.... seems unrelated to circles, but in fact, there is a circle hiding here, as well as some interesting facts about prime numbers in the context of complex numbers.

Pi hiding in prime regularities

What is the second derivative?  Third derivative?  How do you think about these?

Higher order derivatives

A different perspective of Taylor Series that's related to the fundamental theorem of calculus.

Taylor series (geometric view)

Taylor series are extremely useful in engineering and math, but what are they?  This video shows why they're useful, and how to make sense of the formula.

Taylor series

Derivatives are about slope, and integration is about area. These ideas seem completely different, so why are they inverses?

What does area have to do with slope?

What is integration?  Why is it computed as the opposite of differentiation?  What is the fundamental theorem of calculus?

Integration and the fundamental theorem of calculus

How does (ε, δ) "epsilon delta" help us formalize what exactly it means for one value to approach another?

(ε, δ) "epsilon delta" definitions of limits

Challenging questions whose solutions have interesting twists and broader lessons

Puzzles