Field notes

A language model ends every step with a score — a logit — for each token in its vocabulary; softmax turns those scores into the probabilities you see as bars.

Temperature divides the logits before softmax: below 1 it widens the gaps (sharper), above 1 it shrinks them (flatter). At 0 the softmax collapses to argmax — greedy decoding.

Top-p (nucleus) sampling keeps the smallest set of tokens whose probabilities sum to p and renormalizes; the tail simply cannot be drawn. It displaced fixed top-k because the nucleus adapts to how confident the model is.

Surprise is the mean −log₂ p of recent draws — a per-token cross-entropy in bits. Pleasing text tends to live in a middle band: neither 0 (rote) nor maximal (noise).

The dashed outline never moves: it is the raw distribution at T = 1. You never change the model — only the draw.