What is an LLM and why does it matter?

Everyone is talking about ChatGPT, Claude and Gemini. Few people know what sits under the hood, and that's a shame, because once you understand it you also understand why one project succeeds and another misses the mark. A Large Language Model (LLM) is not magic and not science fiction. It's a statistical model that has become very good at one boring task: predicting which piece of text is most likely to follow the previous one. From that boring task, a whole generation of new software has emerged. In this article I'll explain what an LLM is, how it works, what it can and cannot do, and where it makes a concrete difference for small and mid-sized businesses.

What is an LLM in one paragraph

A Large Language Model is a neural network trained on enormous quantities of text. "Large" refers to the number of parameters (the adjustable knobs in the model, often hundreds of billions) and to the volume of text it was trained on. The model learns patterns in language: which words follow which words, which sentences fit which contexts, which answers belong to which questions. The result is a system that can write, summarize, translate, generate code and reason step by step, all by repeatedly predicting the most likely next piece of text.

If you'd rather start with the basics of AI, read what is AI: an explainer for SMEs first.

How an LLM works in 60 seconds

Three concepts sum it up: tokens, training and prediction.

Tokens. An LLM doesn't see letters or words, it sees tokens. A token is a piece of text, often 3 to 4 letters or half a syllable. The word "datadream" gets split into two or three tokens, for example. Modern models are trained on hundreds of billions to trillions of tokens. Llama 3 was trained on 15 trillion tokens, GPT-4 on an estimated far more. The entire internet, plus books, plus code, plus scientific papers, plus a lot you'd rather not know was online.

Training. During training, the model sees billions of examples and has to predict the next token each time. Give it "The capital of the Netherlands is" and it has to learn that "Amsterdam" is a more likely next token than "Berlin". This happens trillions of times, and the parameters get adjusted until the model becomes good at predicting. Then comes a second phase: fine-tuning with human feedback (RLHF), where people teach the model what a useful or an unsafe answer looks like. That second phase is what makes the difference between a text predictor and something you can actually ask questions.

Prediction. When you ask a question, the model predicts the most likely answer token by token. It doesn't think, it doesn't know anything in the true sense of the word. It has become very good at recognizing patterns in what a good answer looks like. That's both its strength and its biggest limitation. An LLM that should say "I don't know" will often produce something that looks a lot like a good answer and just say that instead.

Why 2022 was the tipping point

LLMs had been around for a while, but 2022 changed everything. Three things came together.

In 2017 a team at Google published the paper Attention is All You Need. It introduced the transformer architecture, the computational model that lets you process text in parallel instead of word by word. That made training at enormous scale possible.

In 2020 OpenAI launched GPT-3, with 175 billion parameters. Big enough to do astonishing things, but not yet accessible to the general public.

In November 2022 OpenAI launched ChatGPT, based on GPT-3.5. Within five days it had a million users, within two months a hundred million. For the first time everyone, not just researchers, could talk to an LLM in plain language. That was the tipping point: AI moved from the lab to the laptop. And to the devices you use every day.

Since then, new models have appeared from multiple companies, context windows have grown dramatically, and costs have dropped by over 90%. An LLM that was unaffordable for a ten-person company in 2022 now costs a few tens of euros a month.

What an LLM is good at

An LLM is strong at anything involving language and patterns.

Writing and rewriting. Blog posts, emails, product descriptions, summaries, social posts. With a good prompt and your brand voice as input, an LLM produces a first draft in seconds. Not always finished, but a solid starting point. Read more about content at scale at /ai-content.

Summarizing and structuring. Give an LLM a 50-page report and ask for the three most important conclusions. Or turn loose notes into a tidy document. Here an LLM beats a human hands down, on time too. For an accountant in Goes this saves hours a week of reading nobody enjoys.

Translation. LLMs often translate better than Google Translate, especially for longer passages where context matters. Anthropic published a comparison showing that Claude 3 scored higher than earlier models on several multilingual benchmarks.

Writing and debugging code. According to GitHub's Developer Survey 2023, 92% of professional developers were already using AI tools then, and the Stack Overflow Developer Survey 2025 confirms this has stabilized around 84%. LLMs don't just write code, they explain existing code, find bugs and propose refactors. At DataDream I work largely with Claude Code in a terminal, not with an editor with autocomplete.

Step-by-step reasoning. Modern models like Claude Opus 4.7 and GPT-5 can solve more complex problems by writing out intermediate steps explicitly (chain-of-thought). This works well for problems involving analysis or planning.

Where an LLM can go wrong

Equally important: know what an LLM cannot do, because that's where most implementations come unstuck.

Hallucinations. An LLM predicts the most likely answer, not the most correct one. When it doesn't know something, it can confidently produce a made-up fact, quote or source. OpenAI research shows that even the latest models hallucinate between 1% and 30% of the time, depending on the type of question. For publications or legal work: always fact-check. For internal brainstorming: less of an issue, but know it happens.

Knowledge cutoff. A model is trained up to a certain date (the "knowledge cutoff"). Ask about this week and the model knows nothing, unless it has access to live web search. Claude, ChatGPT and Gemini all have web grounding built in now, but you have to enable it explicitly.

No real understanding. An LLM does not "understand" what it writes, it predicts patterns. On arithmetic, logic or complex planning it sometimes makes mistakes a 10-year-old would spot instantly. For those tasks, always combine an LLM with external tools (a calculator, a calendar, a database). That's not a workaround, that's the right architecture.

Cost. API prices have fallen sharply, but at high volume it adds up. An LLM call costs between $0.50 and $30 per million tokens, depending on model and provider. For a chatbot with 100,000 conversations a month that quickly runs into a few thousand euros. Good architecture (caching, smaller models for simple tasks, RAG) can cut costs by 70%.

Privacy. Feeding sensitive company data into an LLM is a point of concern. More on that later.

The big LLMs in 2026

The landscape has settled reasonably by 2026. Four big players and a growing open source movement.

Claude (Anthropic). The top model is now Claude Opus 4.7, with a context window of 1 million tokens. Strong on long documents, code, research and autonomous tasks. Price: around $15 per million input tokens, $75 per million output tokens. Claude has a strong focus on constitutional AI and safety. My personal preference for serious work.

GPT (OpenAI). GPT-5 is broadly available, GPT-4o remains popular for speed and multimodality (image, voice, video). Prices land around $5 to $20 per million tokens, depending on the model. For many people ChatGPT itself is the first point of contact with AI. See the OpenAI documentation.

Gemini (Google). Gemini 2.5 Pro with native multimodality and very large context windows (1M+ tokens). Strong in research thanks to direct Google Search grounding. Pricing is competitive, often slightly cheaper than OpenAI or Anthropic. Google's Gemini documentation.

Mistral (French/European). Mistral Large is the European alternative, strong on multilingualism and often attractive for companies that want European data residency. Priced below the US top tier, with solid quality for most business tasks.

Llama (Meta). Open source. Llama 4 is free to download and self-host, or available via providers like Groq and Together AI. For companies that want full control over their infrastructure, or that have strict privacy requirements, Llama is the default choice. Llama models are also commercially usable under the Meta license.

For most SME use cases a choice between Claude and GPT is fine. If you want European hosting or self-hosting, you'll end up with Mistral or Llama.

How LLMs make a difference for your business

So much for the technology. What does this concretely deliver for an SME? Five applications where you see results within weeks.

1. Customer service that never sleeps. A chatbot with access to your knowledge base and FAQs handles 60 to 80% of standard questions automatically. For the rest it hands off to a human, with the conversation history already filled in. Result: faster responses at a lower cost, freeing up staff for complex problems. Important: this only works with good RAG (see below), otherwise the bot will hallucinate about your products.

2. Content at scale without flat output. With a good prompt library and your brand voice as input, you generate product content, blog posts and emails 5x faster than by hand. An editor reviews and publishes. For a webshop with 200 SKUs that saves 3 to 5 days of work a month.

3. Research and analysis from hours to minutes. Market research, competitive analysis, summaries of reports or contracts: an LLM does in 10 minutes what takes a junior consultant a day. Combine that with live web search and you have a research assistant that also brings in new information.

4. Personal communication at scale. Emails, proposals and presentations genuinely tailored to the recipient. Not generic mail-merge, but content that fits each customer or segment. Open rates and responses go up by tens of percentage points when this is done well.

5. Agents that carry out tasks, not just answer. The difference between a chatbot and a real AI agent is that an agent can also use tools: send emails, pull data, fill in forms, book appointments. Read about it in AI agent vs chatbot: the difference and /ai-agents.

McKinsey estimates that generative AI adds between $2.6 and $4.4 trillion in annual value globally. For SMEs the gain is less spectacular but much more concrete: 20 to 40% productivity gains on specific processes, provided the implementation is right.

Open source vs closed source: what do you pick?

A question that comes up regularly: should I run my AI on a commercial API, or self-host with open source?

Closed source (Claude, GPT, Gemini) is the default for most companies. You pay per use, you get the best model of the moment, and you don't have to manage infrastructure. Downsides: your data goes to the provider (though you can lock this down with enterprise contracts), and you depend on their pricing and availability.

Open source (Llama, Mistral, Qwen) is interesting if you have strict privacy requirements, your own GPUs, or if you want to fine-tune a specific model on your data. Downside: you pay a lot up front (infrastructure, expertise) and the models often trail half a generation behind the top closed source models.

For 90% of SMEs: start with closed source via API. Open source becomes interesting once your volume grows large enough or compliance requires it. Not before.

For 90% of SMEs: start with closed source via API. Open source only becomes interesting at large volume or under strict compliance.

Privacy and data: what you need to know

Two things to keep in mind before feeding sensitive data into an LLM.

1. Data processing agreement and data residency. With a regular consumer chatbot (free ChatGPT, Claude.ai) you have no data processing agreement and your data can be used for training. For business use, always a paid team or API account, with a DPA and the right opt-outs. Anthropic, OpenAI and Google all offer this. For strict EU data residency: Mistral, or an EU region on Azure OpenAI.

2. AI Act and GDPR. The EU AI Act has been in force since 2024 and is being rolled out in phases. For most SME uses (chatbots, content generation, productivity) you fall into the "limited risk" category: you have to make clear that a user is talking to AI. For applications around recruitment, credit scoring or biometric data, much heavier requirements apply. GDPR still applies in full: don't drop personal data into an LLM without a legal basis and proper security.

The standard in 2026 has become Retrieval-Augmented Generation (RAG). Instead of having the model know everything, you feed it the relevant snippets from your own documents at the moment the question is asked. Benefits include providing current knowledge, reducing hallucinations, and controlling the information source. For business chatbots and internal knowledge tools, RAG has become the norm.

Conclusion: LLMs are a new category of software

An LLM is not a search engine or a consciousness. It's a new category of software: a statistical language model that recognizes patterns and continues them. That's less romantic than the marketing sometimes suggests, and at the same time a big step, because so much work in a business consists of processing language and information.

The difference an LLM makes for your business depends on two things: do you pick the right applications where the technology adds value, and do you have the discipline to integrate it cleanly with your data, your brand voice and your processes. Without that discipline it's an expensive gimmick. With it, it can be a significant productivity tool.

Want to know where AI can already make a difference in your business? Take the free AI scan or book a conversation about a concrete AI strategy. We can help you identify potential productivity gains.