If you use ChatGPT, Claude, Gemini, or any other AI chatbot regularly, you’ve probably come across terms like tokens, context length, system prompts, and tool calling. This ChatGPT and AI chatbot glossary explains everything you need to know — in plain English, without the technical jargon.
For a broader introduction to AI vocabulary, see our main AI Glossary: 100+ Artificial Intelligence Terms Explained Simply.
Chatbot Basics
Chatbot — A software program that simulates conversation with a human. Modern AI chatbots use large language models and can handle complex, open-ended conversations rather than just responding to preset keywords.
ChatGPT — OpenAI’s AI chatbot built on the GPT family of large language models. It is the most widely used AI chatbot globally, capable of writing, coding, analysis, research, and more.
LLM (Large Language Model) — The AI engine powering modern chatbots. An LLM is trained on massive amounts of text to understand and generate human language. ChatGPT runs on GPT-4; Claude and Gemini run on their own LLMs.
Model — The specific version of the AI you are using. ChatGPT users can choose between different models (GPT-4o, GPT-4, etc.), each with different capabilities, speed, and cost.
API (Application Programming Interface) — A way for developers to connect their own applications to an AI model. Using the OpenAI API, a business can embed ChatGPT’s capabilities into their own website, app, or workflow without users needing to visit ChatGPT.com.
Prompt & Input Terms
Prompt — The text you type to instruct or question an AI chatbot. The quality of your prompt has a direct impact on the quality of the response. Specific, clear prompts produce much better results than vague ones.
System Prompt — A hidden instruction given to the AI before the conversation starts, setting its behaviour, persona, or constraints. For example, a customer service chatbot might have a system prompt that says “You are a helpful assistant for Acme Ltd. Only answer questions about our products.”
User Prompt — The message sent by the user during the conversation. In contrast to the system prompt (which sets behaviour), the user prompt is the actual question or instruction the user types.
Assistant Message — The AI’s response in the conversation. In the API, messages are structured as a conversation history with clear roles: system, user, and assistant.
Conversation Flow — The back-and-forth exchange of messages between a user and an AI chatbot. Good conversation flow design guides users to get useful answers without confusion or dead ends.
Model Settings
Token — The basic unit of text a language model processes, roughly equivalent to a word or part of a word. Token limits determine how long your prompt and response can be. “Hello, how are you?” is about 6 tokens.
Temperature — A setting that controls how creative or varied the chatbot’s responses are. A temperature of 0 gives highly consistent, predictable answers. A higher temperature (e.g., 0.8–1.0) gives more varied, creative replies — useful for brainstorming or writing.
Top-p — Another output-control parameter that limits which tokens the model considers for each word in its response, based on cumulative probability. Used with temperature to fine-tune how focused or creative the output is.
Context & Memory
Context Length — The maximum number of tokens an AI model can “see” at once, including the entire conversation history, system prompt, and current message. If a conversation exceeds the context length, the model loses access to earlier parts of the chat.
Context Window — Another name for context length — the “window” of text the model can read in one go. GPT-4 Turbo has a context window of 128,000 tokens; Claude 3.5 supports up to 200,000 tokens.
Memory — The ability of a chatbot to remember information across conversations. By default, most chatbots start fresh each session. Some platforms (like ChatGPT with Memory enabled) store facts about you between conversations to personalise future responses.
Knowledge Base — A collection of documents or data that an AI chatbot can search to answer questions. Used in customer support bots to ground answers in company-specific information.
Advanced Chatbot Features
Tool Calling — The ability for an AI chatbot to use external tools — like a web search, calculator, or database — to answer a question it couldn’t answer from memory alone. ChatGPT’s web browsing feature is an example of tool calling.
Function Calling — A technical mechanism that lets a chatbot trigger specific actions or API calls in the background — for example, looking up a customer’s order, sending an email, or writing to a database.
AI Agent — A chatbot that can take multi-step autonomous actions — planning, searching, writing, and executing — to complete a complex task without requiring human input at every step. Agents go beyond single-turn conversations.
RAG (Retrieval-Augmented Generation) — A technique that lets the chatbot retrieve relevant content from an external knowledge source before generating its response. This keeps answers grounded in fact and reduces hallucinations. Common in enterprise chatbots and support tools.
Customisation & Safety
Fine-Tuning — Training an existing AI model on your own data to make it specialise in your domain or style. A customer service company might fine-tune a model on their past support conversations to make it sound more on-brand and more accurate.
Safety Filter — A built-in rule that prevents the chatbot from producing harmful, offensive, or inappropriate content. Safety filters can be configured at the model level or added by the application developer.
Guardrails — Broadly, any constraints that limit what an AI chatbot can say or do — safety filters, topic restrictions, tone rules, or output format requirements.
| Term | What It Controls |
|---|---|
| Temperature | Creativity and variation in responses |
| Top-p | Token selection range at each step |
| Context length | How much of the conversation the model sees |
| System prompt | Behaviour, tone, and role of the chatbot |
| Guardrails | Topics and outputs the chatbot will avoid |
Frequently Asked Questions
What is the difference between a system prompt and a user prompt?
The system prompt is set before the conversation begins — usually by the developer or business deploying the chatbot — to define its role, tone, and limits. The user prompt is what the person typing in the chat sends as their message. Users typically cannot see or modify the system prompt.
Why does a chatbot sometimes forget earlier parts of a conversation?
AI chatbots have a context window — a maximum amount of text they can hold in mind at once. When a conversation grows long enough to exceed that limit, the model loses access to the earliest messages. This is not a memory failure in the human sense; it’s a technical limit on how much text the model can process at one time.
What is a token and why does it matter?
A token is the basic unit of text a language model processes — roughly a word or part of a word. Token limits affect how long your prompt can be, how long the response can be, and, for paid API users, how much you’re charged (most AI APIs price by the token). Understanding token limits helps you structure prompts and manage costs.
What is the difference between ChatGPT and the OpenAI API?
ChatGPT is a consumer-facing product you access through a web or mobile interface. The OpenAI API is a developer tool that lets you connect directly to the underlying models to build your own applications. Both use the same underlying GPT models, but the API gives far more control over settings, system prompts, and integration options.
Final Thoughts
Understanding these terms makes you a more effective user of AI chatbots and puts you in a much stronger position if you ever want to build or customise one. The concepts are straightforward once you see how they relate — and most of them come up constantly when working with any modern AI tool.
For broader AI vocabulary, visit our main AI Glossary. If you’re interested in how these models generate content, our Generative AI Glossary covers the underlying concepts in depth.
