How AI language models work: a plain-English guide for marketers

A large language model (LLM) is software that has learned the patterns of written language from a huge amount of text, and uses those patterns to predict what text should come next. When you ask ChatGPT, Claude or Gemini a question, the model is generating a reply one small piece at a time, each piece chosen because it is a likely continuation of everything before it.

That one idea explains most of what marketers notice about AI, including why it can sound confident while being wrong. This is part 2 of our AI 101 series for marketing teams. If you haven't read part 1, AI chat vs skills vs agents vs coding agents sorts out the vocabulary first.

What a large language model actually is

A large language model is built in two broad stages. In training, the model reads an enormous collection of text (books, websites, code, articles and more) and repeatedly tries to predict the next word. Each wrong guess slightly adjusts billions of internal settings, which over time come to encode grammar, writing styles, reasoning patterns and a lot of general knowledge.

After that, developers tune the model to follow instructions and hold a conversation, which is why it answers your question instead of just continuing your sentence. The finished model is fixed. It does not learn from your chats in the moment and has no live connection to the internet unless the app around it adds one.

Two practical consequences follow. First, a language model has no database of facts it looks things up in. Second, the model is always producing the most plausible answer, not a verified one. That is a strength for drafting and a risk for anything you publish.

Tokens and context windows: how much a model can see

Language models break text into tokens, which are chunks of characters. A common word may be one token; a long or unusual word may be several. As a rough rule of thumb, a token is a bit less than one English word. Providers measure usage, pricing and limits in tokens.

The context window is the maximum number of tokens the model can take into account at once. It includes your instructions, any documents you paste or attach, the conversation so far and the reply being written. Think of it as the model's desk: whatever is on the desk, it can use. Whatever isn't, it cannot see.

Today's larger context windows can hold long reports or several documents at once. Size still matters in practice:

• In a very long conversation, older messages can fall out of the window or get summarized, so the model may "forget" a brief you gave it an hour ago.
• Models tend to pay closer attention to some parts of a long input than others. Put the most important instructions clearly at the start, and repeat key constraints near the end if the input is long.
• A new chat usually starts with an empty desk. Unless the app has a memory feature turned on, the model does not remember last week's conversation.

Why AI models make things up, and how to reduce it

When a language model states something false with confidence, it is called a hallucination. It happens because the model is built to produce plausible text, and a plausible-sounding statistic, quote or product feature is easy to generate even when no real one exists. The model has no built-in way to tell a true continuation from a merely likely one.

Hallucinations are most common with specific facts: numbers, dates, citations, URLs, recent events and niche topics. They are least common when the answer is sitting in text you have given the model. That points to the fixes:

1. Give it the source. Paste the product sheet, the research report or the brand guidelines and tell the model to answer only from that material.
2. Ask for citations. Request that each claim point to the section or passage it came from, so you can check quickly.
3. Give it permission to say it doesn't know. A line such as "If the answer isn't in the document, say so" reduces invented filler.
4. Check the facts that matter. Anything going to customers, executives or regulators gets verified by a person against the original source.

Training data, your prompt and your documents

An AI model's answer can draw on three sources of information, and each deserves a different level of trust.

• Training data. Everything the model absorbed before it was released. It is broad but has a cutoff date, may be out of date on your industry and knows nothing private about your company.
• What you give it in the chat. Your instructions, pasted text and attached files. This is the most reliable source for your task, because the model can read it directly within the context window.
• Retrieval from your documents. Many business tools can search a connected knowledge base (a shared drive, a help center, a product catalog) and pull the relevant passages into the context window before the model answers. The technical name is retrieval-augmented generation, or RAG. In plain words: look it up first, then write the answer from what was found.

RAG is how most company AI assistants work, including an AI receptionist that answers from your own service details. The quality of the answers depends heavily on the quality of the documents. Outdated or contradictory content produces confident answers built on the wrong page.

The model vs the app around it

ChatGPT, Claude, Gemini and Microsoft Copilot are apps. Inside each is one or more language models, and around the model sits a layer of features the provider built.

Common app features include:

• File uploads, which put documents into the context window.
• Web search, which fetches current pages so the model can answer about recent events (and cite them).
• Memory, which saves facts about you between conversations. Review it now and then.
• Connectors, which link the app to tools such as email, calendars, drives or your CRM.
• Tools and actions, which let the model run code, create files or take steps in other systems. This is where chat starts to become an agent, which we cover in AI agents and automation.

Two people using the same model can get very different results depending on which features are switched on.

What "multimodal" means

A multimodal model can work with more than text. Depending on the model and app, that can mean reading images, screenshots, charts and PDFs, listening to audio, or generating images and speech. For marketers, that means reviewing a landing page screenshot or pulling figures from a chart. Models can still misread small text or misjudge a chart, so check what you rely on.

Privacy basics for marketing teams

What happens to the data you put into an AI tool depends on the provider, the plan and the settings. A few principles hold across most tools:

• Use a business or enterprise plan for company work. These commonly offer stronger data commitments and admin controls than free consumer accounts. Read your provider's actual terms.
• Check the training setting. Many consumer apps have a setting that controls whether your conversations can be used to improve future models. Know where it is and what your team has it set to.
• Keep regulated and sensitive data out unless approved. Customer personal data, health or financial information, unreleased financials and contracts need a clear yes from whoever owns privacy and security.
• Review connectors before switching them on. A connector to your drive or CRM gives the app access to everything that account can see.

These basics belong in a written policy. Part 5 of this series walks through how to write an AI usage policy for a marketing team.

What to expect from an AI model: a quick reference

Use this table to set expectations and decide where a person stays in the loop.

Once your team understands what is happening under the hood, the next step is asking well. Part 3 covers how to write AI prompts that get usable results. If you want help putting language models to work on real marketing processes, AI for marketing from ATL Martech starts with the workflows your team already runs and adds AI where it measurably saves time.