See how an AI tool writes its answers, and what it can see

Recall

Recall: Think of a time an AI tool sounded sure and was wrong. Was it a name, a number or a date? Keep that example in mind as you read.

The idea

An AI tool runs on a language model: a program trained on large amounts of text to predict what comes next.

It writes a reply one likely word at a time. It draws on three things: what it learned in training, what's in your chat, and anything it's connected to, such as search or your files. A fact that comes from none of these is a guess that reads like a fact.

How it works

One likely word at a time

OpenAI's help page says the model predicts "the next most likely word when generating a response, one word at a time." It doesn't look up a stored answer. It builds a new one each time, from patterns in the text it learned from.

Prediction can be right or wrong

NIST, the US standards agency, says language models "predict the next token or word in a sentence or phrase." A token is a small piece of text, often a word or part of one. NIST adds that such prediction "can produce factually accurate and consistent outputs," but "it can also produce outputs that are factually inaccurate or internally inconsistent."

Same question, different answer

OpenAI explains there is "an inherent element of randomness in how the model responds. As a result, the same question may yield different answers across different queries." That's useful to you. Ask twice, and the parts that change are the parts to check.

Training stops at a cutoff

Each model learns from text up to a date, called its knowledge cutoff. Claude's help says models "may not be aware of events or information that occurred after their respective cutoff dates." Ask about last month's news without search, and you may get an older answer that sounds current.

Search and files add what training lacks

With search on, a tool can bring in current pages. Claude's help says every search response includes citations. OpenAI says search responses "may include citations." If you attach a file, the tool can read it. If you didn't attach it, it can't quote it.

A confident tone isn't evidence

NIST describes tools that "generate and confidently present erroneous or false content in response to prompts." Its name for this is "confabulation." The wording sounds just as sure when the fact is wrong.

Practice

Is this part a guess?

Ask these about one fact in the reply.