What is ChatGPT?

In November 2022, OpenAIOpenAIThe AI research company behind ChatGPT, GPT-4, and DALL-E.Click to learn more → released ChatGPT. Within five days, it had a million users. Within two months, 100 million. It was the fastest-growing consumer application in history.

But what actually is it?

The short version

ChatGPT is a large language modelLarge Language Model (LLM)AI trained on massive text data to understand and generate human language.Click to learn more → (LLM): an AI system trained to predict and generate text.

You type something. It predicts what text should come next. That's the whole trick.

The longer version

Here's what happens when you send a message to ChatGPT:

1. Your text gets converted to numbers. Each word (actually, each chunk of a word called a "token"TokenA chunk of text (word, part of word, or character) that AI models process as a single unit.Article coming soon) becomes a number the model can process.

2. The model processes these numbers through layers. Imagine your message passing through 96 layers of mathematical transformations, each one building understanding of what you said.

3. It predicts the next token. Based on everything it learned during trainingTrainingThe process of teaching an AI model by showing it examples and adjusting its parameters.Click to learn more →, the model calculates a probability distribution: "The next word is 40% likely to be 'the', 20% likely to be 'I', 8% likely to be 'Hello'..."

4. It picks a token and repeats. Usually it picks from the top candidates (with some randomness), adds that token to its response, then predicts the next token after that.

5. This continues until it's done. The model generates tokens one by one until it decides the response is complete.

That's it. There's no database of answers. No rules about how to respond. Just: "given everything I've seen, what text probably comes next?"

How did it learn?

ChatGPT was trained in two phases:

Phase 1: Pre-training

Feed the model a massive amount of text from the internet, books, articles, websites, code, conversations. Hundreds of billions of words.

The training task is simple: predict the next word. Over and over, on every piece of text.

"The cat sat on the ___"

If the model predicts "mat" and the actual word is "mat", good. If it predicts "elephant", bad, adjust the internal numbers and try again.

After doing this trillions of times, the model learns patterns:

• Grammar and syntax
• Facts about the world (some accurate, some not)
• How to write in different styles
• How people typically respond to questions

Phase 2: Fine-tuningFine-tuningCustomizing a pre-trained AI model on specific data to improve its performance for a particular task.Article coming soon with humans

The raw pre-trained model is good at predicting text but not good at being helpful. It might continue your message instead of answering it. It might say harmful things.

So OpenAI had humans rate model responses. "This response is helpful. This one is rude. This one is incorrect."

They used these ratings to adjust the model further, teaching it to be helpful, harmless, and honest.

This human feedback step is what makes ChatGPT feel like an assistant rather than an autocomplete engine.

What it's actually doing

ChatGPT doesn't "know" things the way you know things.

It doesn't have a database of facts it looks up. It doesn't "understand" your question, think about it, then formulate an answer.

It has patterns encoded in billions of numerical parametersParametersThe numerical values a neural network learns during training — GPT-4 has over a trillion.Click to learn more →. When you ask a question, these patterns guide it toward responses that are statistically likely given text like this on the internet.

Usually, statistically likely = correct and helpful.

Sometimes, statistically likely = confidently wrong.

The limitations

It hallucinatesHallucinationWhen AI confidently generates false or made-up information.Click to learn more →. It will make up facts, cite papers that don't exist, invent historical events. Not because it's lying, but because plausible-sounding text is what it generates, whether or not it's true.

Its knowledge is frozen. GPT-4 was trained on data up to a certain date. It doesn't know what happened yesterday.

It has no memory between conversations. Each conversation starts fresh (unless you're using features that simulate memory).

It can be manipulated. Clever promptsPromptThe input text you give to an AI to get a response.Click to learn more → can sometimes get it to say things it shouldn't.

It doesn't actually reason. What looks like thinking is pattern matching. It can solve many problems, but complex novel reasoning still trips it up.

Why it matters

Despite the limitations, ChatGPT changed everything:

• It's a new interface. Instead of clicking buttons and filling forms, you just talk. Ask for what you want.
• It's accessible. No coding required. Anyone can use it.
• It's general-purpose. The same model writes poetry, debugs code, explains science, and drafts emails.

For the first time, AIArtificial IntelligenceComputer systems designed to perform tasks that typically require human intelligence.Click to learn more → felt useful to regular people, not just researchers and engineers.

---

ChatGPT is built on decades of AI research, but the core idea is surprisingly simple: predict the next word, at massive scale. Want to understand the foundation? Start with What is AI?