Skip to content

Main Index

Dr. Neil's Notes

Software > Development > AI in Delivery Systems

The Average Answer

Introduction

Large language models can produce an answer that sounds considered, confident, and complete. The mechanism underneath is not a person thinking through the problem. It is a statistical model estimating which sequence of words is likely to follow the context it has been given.

That is why an LLM is often useful and often ordinary at the same time. It is very good at producing a plausible answer from familiar patterns. It is much less reliable at knowing when the familiar answer is the wrong one.

A probability engine, not a search for truth

During training, a language model learns statistical patterns from large amounts of text. In use, it takes the prompt and the text already generated, then estimates a probability distribution for the next token. A token might be a word, part of a word, or punctuation. It chooses a next token, adds it to the context, and repeats the process.

The selection can involve randomness, so the same prompt can produce different answers. This is the stochastic part. Settings such as temperature can make the selection more or less varied, but they do not turn the model into a source of facts or judgment. The model is estimating what fits the context, not checking what is true in the world.

Why the answer tends toward the average

Here, "average" does not mean the model literally calculates the arithmetic mean of every answer it has seen. It means that when many continuations are plausible, familiar and commonly represented patterns tend to have an advantage. The result often sounds like the conventional response, fluent, tidy, and close to what people have said before.

This is a consequence of learning to predict language from examples. Common ways of explaining a problem, structuring an argument, or solving a familiar task leave strong patterns for the model to follow. Unusual approaches may be possible, but they are often less likely unless the context gives the model a reason to take them.

The model's randomness does not fix this. It can produce variation, but variation is not the same as originality, and neither is the same as correctness. A different answer can simply be a different plausible guess.

Why breakthrough products are unlikely to come from an LLM alone

A product that is substantially different from anything people have seen before is unlikely to emerge from an LLM acting on its own. The model learns from patterns in existing material and generates continuations from those patterns. It can combine ideas in surprising ways, but it does not independently discover which unmet need matters, decide that the conventional approach should be abandoned, or learn from customers using a product in the real world.

Breakthroughs are not just unusual combinations. They involve recognising a valuable problem, making decisions with incomplete evidence, building something, seeing how it behaves, and changing direction when reality disagrees with the plan. Those activities depend on goals, context, judgment, and feedback from the world. A model can assist with research, exploration, prototypes, and analysis, but someone still has to choose what is worth pursuing and take responsibility for the learning.

An LLM can therefore be part of an innovative team without being the source of the innovation by itself. Treating a plausible combination of familiar ideas as a breakthrough is an easy way to mistake novelty of wording for value.

Why it can look like super intelligence

People judge a tool partly by comparing its output with what they could produce themselves. If someone usually struggles to draft clearly, find a starting point, or connect ideas across a topic, an LLM that does those things quickly can feel extraordinary. The improvement against that person's usual baseline may be real and valuable.

Doing better than one person's typical result is not the same as being superintelligent. The model can produce a fluent answer without knowing whether it is correct, relevant, or safe. Its confidence of style can make the gap between a useful draft and a reliable decision hard to see, especially when the reader does not have the expertise to check it.

The right comparison is not only "Can it do this better than I can right now?" It is also "Does the result stand up to evidence, expert review, and the consequences of using it?" For some people, an LLM will raise the quality of everyday work substantially. That is a genuine benefit. It does not remove the need for judgment, it makes good judgment more important, because a polished answer is easier to trust than it is to verify.

Useful for a first draft, not a substitute for judgment

This tendency makes LLMs useful for summarising, drafting, transforming text, and producing a starting point for common problems. It also means the first answer can quietly steer a team toward an obvious option, even when the real situation calls for a different one.

When the work needs a novel approach or a decision grounded in local context, make the constraints and relevant evidence explicit. Ask for alternatives and trade-offs, then check those alternatives against the actual system, users, and goals. Do not treat polished language as proof that the answer is sound.

This is one reason human review belongs in the architecture of AI-enabled delivery. The model can produce a plausible candidate quickly. The team still has to decide whether it fits, whether it is safe, and whether it solves the right problem. Use the model to widen or accelerate the work, not to outsource responsibility for the result.


Part of the AI in Delivery Systems series.

Authors: Neil Roodyn