Skip to content
back
ART-01·UNDERSTAND·3 December 2025·7 min

How an LLM actually works: what nobody explains clearly

By Camilo Rivera

Screenprint poster: seven off-white squares in a row on a violet line, and the eighth, yellow, about to be set down at the end of the row
Generated poster · art direction by OSOM Labs

Everyone talks about ChatGPT, Claude, Gemini. Few people know what GPT stands for, or what actually happens when you type a question. It's not magic. It's not intelligence in the human sense either. It's something else. And understanding this "something else" changes the way you use these tools. Since 24 September 2026, this article also brings together our explanation of the GPT acronym, previously published separately.

0101

What GPT stands for

GPT stands for Generative Pre-trained Transformer. Three words, three ideas:

  • Generative: the model produces new text, one piece after another. It doesn't classify or sort: it writes.
  • Pre-trained: it learnt from a huge amount of text before anyone used it. Whatever it "knows" comes from that phase.
  • Transformer: the technical architecture, invented in 2017, that almost every current model is built on.
  • GPT is the brand of OpenAI's models, but the word has become generic. The rest of this article explains what each of these three ideas covers.

    0202

    The stochastic parrot: an imperfect but useful metaphor

    We've often heard that LLMs are "stochastic parrots": they repeat patterns without understanding. It's partially true, but it's reductive.

    An LLM doesn't store text that it regurgitates. It has learned statistical relationships between words. When it generates "The cat is on the...", it doesn't search a database. It calculates that "mat", "sofa" or "bed" have a high probability of following, based on billions of examples.

    What this actually means: An LLM doesn't "know" anything in the strict sense. It predicts. Very well. But predicting isn't knowing.

    0303

    Attention: the 2017 invention that changed everything

    Before 2017, language models read text word by word, in order. It was slow, and they lost the thread of long sentences.

    The Transformer, introduced that year by Google researchers, brought in a mechanism called "attention": the model looks at every word in a sentence at once and weighs the links between them.

    Example: in "The cat that was on the mat fell asleep", it links "fell asleep" to "cat", not to "mat". Earlier models stumbled over sentences like this.

    That ability to hold the overall context is what makes today's LLMs so fluent.

    0404

    Tokens: why AI chops up your words oddly

    LLMs don't read words. They read "tokens": chunks of words.

    "Unconstitutionally" becomes several tokens: "Un", "constitu", "tion", "ally". The word "the" in English is a single token. The word "Sion" (the Swiss city) might be split into "S" + "ion".

    Why this matters:

  • LLMs have a token limit (context). The longer your conversation, the more they "forget" the beginning.
  • Rare or technical words cost more tokens = more risk of errors.
  • Languages other than English are less efficient: French generally needs more tokens to say the same thing.
  • 0505

    An answer written one token at a time

    An LLM doesn't think before it writes. It produces its answer one token at a time: it reads your request, predicts the first token, adds it to the context, predicts the next one, and repeats until the end.

    That's why:

  • an answer can stop abruptly when the token limit is reached;
  • the model doesn't easily go back;
  • the start of the answer shapes everything that follows, and so does the way you phrase your request.
  • 0606

    Training: billions of texts, zero truth

    An LLM is trained on massive quantities of text: books, websites, articles, forums, source code. Everything written by humans.

    The problem: The internet contains both true and false information. The LLM learns both without distinction. It learns that "the earth is round" AND "the earth is flat" exist as sentences. It doesn't know which one is true; it only knows which one is more frequent in certain contexts.

    This is why LLMs "hallucinate": they generate plausible text, not true text. If you ask for an obscure fact, the model will produce something credible, whether it's accurate or invented.

    And a cut-off date: training stops at a given point. The model knows nothing of what happened afterwards, unless it has access to an online search. Biases present in the training texts carry over into its answers.

    0707

    Temperature: the creativity/precision slider

    When an LLM generates text, it can be more or less "creative". This is controlled by a parameter called temperature.

  • Low temperature (0.1-0.3): predictable, repetitive responses with no surprises
  • High temperature (0.8-1.0): varied, surprising, sometimes incoherent responses
  • In practice:

  • For code or facts: low temperature
  • For creative writing: higher temperature
  • Consumer assistants use an intermediate setting, a compromise between the two
  • 0808

    GPT, Claude, Gemini, Llama: cousins

    Almost every major model is built on the same Transformer architecture:

  • GPT (OpenAI)
  • Claude (Anthropic)
  • Gemini (Google)
  • Llama (Meta)
  • What sets them apart isn't the basic principle, but the training data, the adjustments made afterwards, the safety choices and the size of the model.

    When a company boasts about "its revolutionary proprietary AI", chances are it's a Transformer like the others, with an interface on top.

    0909

    What this changes for you

    Understanding these mechanisms changes the way you interact with an LLM:

    1. Don't trust it for facts

    Always verify. Especially dates, numbers, and citations.

    2. Be precise

    The clearer your request, the less the model has to "guess". The less it guesses, the less it invents.

    3. Context matters

    An LLM with your conversation history performs better than a "cold start" LLM.

    4. The limits aren't bugs

    Hallucination isn't a problem OpenAI will "fix". It's inherent to how it works.

    Key takeaways

    01

    GPT = Generative Pre-trained Transformer: a model that generates, pre-trained, built on the Transformer architecture.

    02

    An LLM predicts probable text, one token at a time; it doesn't "understand" in the human sense.

    03

    Attention, invented in 2017, lets it hold the context of a whole sentence.

    04

    Training on the internet includes both true and false, and stops at a date: hence hallucinations and gaps.

    05

    Temperature sets the slider between creativity and reliability.

    06

    GPT, Claude, Gemini and Llama are cousins: same architecture, different settings.

    Stack

    LLMTransformerTokenizationMachine Learning
    how an llm works · what does gpt mean · gpt meaning · transformer attention · ai tokens · ai hallucination · llm temperature

    More articles in the journal

    We document everything we do.