Decoded #2: How AI works (and why it hallucinates)

When you start a text like "I'll be there in five..." and your phone suggests "minutes," it isn't thinking. It looked at billions of past messages and learned that "minutes" usually follows "five" in that kind of sentence. Sometimes it's right. Sometimes it isn't.

When you ask an AI a question, at its core it doesn't look up an answer. It generates one, one word at a time, picking whatever's most likely based on patterns from its training. No fact-checking. No concept of "true" or "false." Just "what word usually follows these other words?"

Which means it doesn't know when it doesn't know. It still has to produce the next word. So when the patterns aren't there, it produces whatever sounds about right and keeps going.

For some work, that mechanism is exactly what makes AI powerful. The same system that can confidently invent can also read across hundreds of sources, spot the pattern you'd miss reading them one at a time, surface the pearl that connects three guidelines, and turn a wall of evidence into a coherent why. When AI is explaining a decision rather than making one, fluency is the feature.

For other work, that same mechanism becomes a liability. AI doesn't fail loudly, it fails fluently. The dangerous hallucinations aren't the obviously wrong ones. They're the ones that sound exactly like the right ones.

A useful frame: AI is calibrated to sound right, not to be right. Good for explaining decisions. Not for making them.