From Likelihoods to an Answer

The clues in a prompt do not point to one inevitable answer. They shape a set of possibilities. At each step, the model calculates how plausible many possible next tokens are, given the context so far. The prompt supplies the current clues; the model’s learned patterns help connect them to possible continuations.

Imagine the prompt: “The key opened the…” The model might assign a high score to door, a lower one to lock, and a much lower one to watermelon. These scores are not little judgments of truth or intention. They are numerical signals produced by the model’s learned settings, reflecting how well each candidate fits the patterns it has learned and the current context.

The technical name for the raw scores is logits. You do not need the mathematics behind them to grasp their role: they are like entries on a ranking board before the system chooses what to say. A further calculation turns the scores into probabilities—a distribution across possible next tokens. Perhaps door receives the largest share, while lock and other candidates receive smaller shares. The exact numbers depend on the model and the prompt; the important point is that the model weighs alternatives rather than retrieving a complete sentence from a shelf.

A selection rule then turns that distribution into one token. The simplest rule is to take the highest-probability option every time. With our example, that might mean choosing door. This is predictable, but it can also make the output repetitive or overly safe. Another approach allows the model to select among several likely candidates, giving less likely options a chance. A small surprise can make a story livelier; too much surprise can make it wander into watermelon territory.

A setting often called temperature controls how concentrated or spread out the choices are. Lower temperature makes the most likely candidates more dominant, so responses tend to be steadier and more predictable. Higher temperature gives lower-ranked candidates more opportunity, so responses tend to vary more. Temperature does not add knowledge or make the model understand a prompt better. It changes how the available options are selected. And it is only one part of the process: products may use other settings or selection methods, and may keep some details behind the curtain.

Once a token is chosen, it is added to the context. The model then calculates a fresh set of scores for the next token, now taking the new token into account. If it chooses door, the next continuation may be “to,” “with,” or “and”—each choice changing what fits after it. Generation proceeds in this repeated cycle: score possible next tokens, select one, add it to the text, and calculate again.

Small changes to the prompt can therefore change the result. “The key opened the…” invites one set of continuations; “The old brass key opened the…” supplies extra clues and shifts their relative scores. So might a sentence earlier in the conversation, if it changes what the model should pay attention to. Even when the words in the prompt stay the same, a selection process that allows variation may choose a different token on another run.

This is not the model weighing a sentence as a whole and then pressing “send.” It is building the sentence through a sequence of local choices, each shaped by the prompt, the text already generated, the model’s learned patterns, and the selection settings. A tiny choice near the beginning can steer the next choice, and then the next. The answer unfolds one token at a time.

Return to the request that opened this book: “Can you make this warmer?” Picture it arriving after the user has pasted a rather chilly email: “Your proposal has been rejected. Do not contact us again.” We can now follow the request without pretending to inspect the model’s actual hidden calculations.

First, a tokenizer divides the text and the instruction into pieces and gives them IDs. The model looks up numerical embeddings for those pieces and adds information about where they occur. In the opening chapters, these were our reusable tiles and our invisible map: not meanings in little boxes, but useful numerical starting points.

Next, the representations pass through transformer layers. Their learned parameters—the countless adjusted knobs—shape the transformations. Attention lets the request about making something “warmer” draw on the preceding email. Here, the clues “proposal,” “rejected,” and “contact us” make an interpersonal meaning more relevant than changing the temperature. The model need not inspect a real room to make that connection.

Then the model assigns scores to possible next tokens. A continuation such as “Of course” may fit a helpful response better than “Set the thermostat.” A selection process chooses a token; the process repeats until the reply is complete. The assistant might offer: “Thank you for sharing your proposal. We won’t be moving forward, but we appreciate the time you put into it.” The example is illustrative, not a claim about the exact words any particular model would generate.

Where did the apparent understanding come from? Not from a stored file labeled warmer email, and not from one clever step. It came from learned patterns meeting current context, transformed through layers and expressed through many successive token choices. If an application had left out the email, the same instruction might have produced a clarifying question instead. If the model had missed the tone, its answer could still have sounded polished while failing the request.

That is the whole route in miniature: text → tokens → numerical representations → layers and attention → next-token scores → a reply. It is not every engineering detail, but it is a useful picture of where the work happens. When we reach harder problems, the model may produce additional intermediate material or use tools before the final reply—but this basic machinery remains underneath.

Now take the next-token choices from that journey and picture how an answer can develop in different directions. Imagine the prompt ending, “The old brass key opened the…” One possible continuation leads toward a box; another toward a gate. From there, the passage might become a mystery about what the box contained or a story about a garden beyond the gate.

Picture a path splitting again and again through a forest. Some trails look more likely than others, and an early turn can shape the character of the route that follows. A sentence may develop a tone, a topic, or an argument through many such turns. The image helps explain why a small change can have a large effect: one different word can make some later continuations fit better and others less well. Each chosen token becomes part of the context that shapes the next choice, so the answer’s direction accumulates as it grows.

But the forest is an analogy, not a map of a literal search taking place inside the model. In ordinary generation, the model need not lay out every possible sentence, compare all the branches, and then travel down the best one. Some systems and settings do explore or compare multiple candidate continuations, but that is not required for the basic process described in the previous section.

Nor is there always one objectively “best” trail. Several continuations can be reasonable, and the selection method affects how varied or predictable the result is. A less expected turn can make writing feel fresh, but it can also send an answer toward a dead end: a confident-sounding claim, an irrelevant detail, or a sentence that wanders away from the question.

The branching image captures the accumulating effect of choices. It does not mean the model sees a forest, plans a whole journey, or knows where every path ends. It is a way to picture how different possible directions can lead to different kinds of answers.

One early turn can make the rest of an answer look inevitable. That is part of what makes a fluent mistake so persuasive: once a response begins down a plausible path, each new token can fit neatly with the ones before it—even if the path started from a false assumption.

Imagine asking for the year a particular old bridge opened. The model may have encountered many descriptions of bridges, dates, and local history. It can produce a sentence with the familiar shape of a reference-book answer: “The bridge opened in 1896 and was designed by…” The date may be correct. It may also be a convincing guess, assembled from patterns that often appear together. Smooth grammar and a precise-sounding number do not tell you which.

The reason is built into the prediction task. During ordinary generation, the model calculates which tokens are likely to fit the prompt and the words already produced. It has learned patterns that often carry knowledge: names tend to accompany certain places; explanations tend to follow questions; dates often appear in historical summaries. Those patterns let it give useful answers. But the immediate calculation is not a separate, guaranteed test of whether every claim matches the world.

A sentence can therefore be likely language without being verified fact. These are related, but they are not the same. If a model has learned that a certain kind of question is commonly answered with a neat date and a brief explanation, it can produce that form even when its support for the specific date is weak. A polished answer is evidence of fluency, not proof of certainty.

This is also why invented details can arrive in the same confident tone as accurate ones. In ordinary generation, there is no universal alarm that sounds whenever a claim lacks reliable support. The model may continue a plausible pattern instead of stopping to say, “I don’t know.” It can even build on an earlier error: once a mistaken date enters the answer, later sentences may fit around it, like carefully arranged furniture in the wrong room.

That does not mean the model’s answers are random, or that prediction is useless for finding facts. Much of what it has learned reflects real information, and prediction can bring relevant details together with remarkable skill. But its confidence in the wording should not be mistaken for a dependable meter of truth. A model can state something correctly, state it incorrectly, or hedge—and the tone alone may not reliably distinguish those cases.

The branching path from the previous section has no automatic fact-checking gate at every turn. Some systems can consult tools or apply extra checks, but those are additional processes, not a guarantee supplied by fluent wording itself. When a detail matters, the sensible question is not “Does this sound authoritative?” but “What supports it?” The answer’s ease of reading tells us how well the language flows. It does not, by itself, tell us whether the world agrees.