As GenAI has evolved, so has the vocabulary used to describe it. Understanding this vocabulary is essential for evaluating solutions and managing the transition from AI as a chatbot to AI as more of a governed digital workforce.
For most insurance professionals, the age of GenAI began in November 2022 when OpenAI introduced ChatGPT.
Suddenly, everyone seemed to be using new technical terms, such as large language models (LLMs), tokens, prompts, and hallucinations. We took our first steps – writing prompts, checking for hallucinations, and learning how LLMs process human language.
This quarterly update explores the evolving vocabulary of GenAI. It starts with a review of the foundational terminology and the changes that have occurred, followed by a look at new terms that are driving today’s AI conversation.
The LLM represents the culmination of several breakthroughs. This model accepts text and produces a textual answer based on knowledge learned from large volumes of data from the web and elsewhere, filtered for safety and acceptability. Now multimodal and other advanced successors can handle other types of data, such as images and audio, as their input/output. We still refer to the core components of modern AI systems as LLM. Every agentic system featured in this article has an LLM-like model at its core.
Originally, a prompt was simply the input provided to a textual LLM, something such as, “Summarize this applicant’s attending physician statement.” This led to prompt engineering to achieve more usable output through clearer input. Such prompts might even incorporate images, such as, “You are an underwriting assistant reviewing an attending physician statement (APS) and the attached X-ray for a life insurance application. Summarize the APS in 150 words or fewer, covering diagnoses, current treatment, and any red flags relevant to mortality risk. Then describe what is visible in the X-ray in plain language and note whether it is consistent with or adds new information beyond the APS text. Do not provide a diagnosis; only describe observable findings for a human underwriter to review.”
The smallest unit of text a human can deal with is typically a letter. LLMs represent text and images as sequences of tokens, which are often a handful or fewer letters that commonly co-occur and may convey meaning. Words such as “underwriting” and “underinsured” share the token “under,” and while this overlap in meaning is coincidental, rather than by design, it can still convey a sense of shared meaning across related words. Because the LLM thinks in terms of tokens, not letters, simple questions such as, “How many r’s are there in the word strawberry?” are notoriously difficult for LLMs. Tokens are also the unit upon which companies are billed, leading many to decry “rising token costs.”
LLMs can often very confidently appear to make up or “hallucinate” output that might seem correct upon light review, yet is not actually supported by the input or factual reality. We may request an LLM to draft a new policy provision based on the rest of the policy text and our company’s guidelines. It can be difficult for an LLM to know when to synthesize information that exists in its input versus generating new content to fill a gap. For example, if we ask an LLM what is written in a policy about a scenario the policy does not actually address, it may draft a plausible-sounding provision, rather than stating that no such provision exists, and a hallucination occurs.
As GenAI has rapidly advanced, so has the vocabulary needed to describe new processes and capabilities.
An LLM has no memory of its own; it simply produces output for the input given. Everything it knows about a specific task must be handed to it in the moment, inside what is called the context window. Think of context as the model’s desk: the documents, instructions, and conversation history currently sitting on it. The desk is large these days, often covered with hundreds of thousands of tokens, but it is still finite, and everything on it costs money. Managing what goes on the desk, and what gets cleared off, has become its own discipline, called context engineering.
If context is the desk, RAG is the bookshelf. A foundation model, such as Claude or ChatGPT, was not trained on an underwriting manual. A RAG system searches a company’s document library, retrieves relevant passages, and places them into context before the model answers. Ask, “What is our preferred build chart for a 45-year-old male?” and RAG pulls the actual chart from the actual manual. RAG 2.0 goes further: It can reformulate queries, search multiple sources, follow citations, retrieve tables and images, and iteratively verify that it found the right evidence. The model derives answers from documents, not just its training data, sharply reducing hallucination. RAG was the headline technique of 2023 and 2024. In 2026, basic RAG is the foundation; RAG 2.0 is the smarter retrieval layer built on top of it.
An LLM on its own can only produce text. A tool is a capability provided to the model that lets it perform tasks such as querying the policy admin system, looking up ICD codes, calculating a substandard rating, or sending an email. The model decides when to use a tool, invokes it, and reads the result back into its context. Tools are what turned AI from a writer into a worker.
Once a model can use tools, users can give it a goal instead of a question. An agent is an LLM placed in a loop: It plans, takes action with a tool, observes the result, and decides what to do next, repeating until the goal is met. Ask an agent to “prepare this case for the underwriter,” and it might retrieve the application, order the MIB check, summarize the APS, flag the elevated A1C, and draft a requirements letter, all as separate self-directed steps. This is the single-biggest shift since 2022. The chatbot answered you; the agent works for you.
Agents are capable but generic. A skill is a packaged bundle of instructions, examples, and reference material that teaches an agent how an organization does a particular job. A “claims correspondence skill” might contain approved letter templates, tone guidelines, and state-specific disclosure language. The agent loads the skill only when the task calls for it. Skills are how institutional knowledge – the kind that used to live in a senior examiner’s head – gets encoded so an agent can follow it. Skills are often created by regular users to impart knowledge to the AI systems they use.
With every carrier wiring agents to admin systems, document stores, and data feeds, the industry needed a standard interface. MCP is that interface, an open protocol that lets any AI application connect to any compliant system the same way. Before MCP, connecting an agent to your claims platform meant custom integration work. Now the vendor ships an MCP server, and any agent can connect to it. It is worth knowing because “Do you have an MCP server?” has become a standard question during vendor due diligence.
Within MCP, a resource is a piece of data that a connected system offers to the model, such as a policy record, a treaty document, or a mortality table. Tools let the agent act; resources let it read. The distinction matters for governance, because read-only resources are far easier to approve than tools that can change data.
Context vanishes when a session ends. Memory is the mechanism that lets an agent retain information across sessions: the cases it worked yesterday, the corrections an underwriter made, and the preferences of a particular examiner. An agent with memory improves with use. An agent without it starts every morning as a new hire. Memory also raises obvious privacy questions, and remembering applicant health information across sessions is as much a data-retention decision as a technical one.
Closely related but distinct, state is where a specific piece of work currently stands. A claim undergoing adjudication, with three documents reviewed and two pending, has been stated. Long-running agentic workflows must track state carefully so a process can pause for human approval and resume without starting over. Memory is what the agent knows; state is where the work is.
Real insurance workflows are too large for a single agent. Orchestration is the coordination layer that assigns work across multiple specialized agents – such as an intake agent, a medical records agent, and a risk assessment agent – and manages the handoffs, the state, and the escalation points where a human must sign off. If the agent is the worker, orchestration is the supervisor. This is where most enterprise AI engineering efforts now live.
The vocabulary of GenAI is evolving as the technology itself changes. We began with models that answered prompts; we are moving toward systems that retrieve evidence, use tools, remember prior work, and coordinate complex processes.
Understanding these terms is therefore more than learning the latest technical jargon. It provides a practical framework for evaluating new products, asking better questions of vendors, and deciding where these systems can be used safely and effectively.
The terminology will continue to change, but the larger direction is clear: GenAI is becoming less like a chatbot we consult and more like a managed workforce we must equip, supervise, and govern.
Contact us today to learn how RGA can help you implement AI safely and responsibly for the biggest benefit to your business.
