M365Con 2027 – free online conference · 11–22 Jan 2027 · 120 sessions m365con.net

Retrieval-augmented generation (RAG)

Concept

A pattern in which an AI model first retrieves relevant documents from a knowledge source and then writes its answer based on them. It reduces made-up answers and lets models use current, private data without retraining.

✦ CONCEPTRetrieval-augmentedgeneration (RAG)M365.FM
Title graphic: Retrieval-augmented generation (RAG) (Concept)

Relationships

Retrieval-augmented generation (RAG) is closely related to AI agent, Knowledge source and Azure AI Search.

Description

Retrieval-augmented generation (RAG) is a pattern in which an AI model first retrieves relevant documents from a knowledge source and then writes its answer based on them. The approach reduces made-up answers and lets a language model work with current, private data without being retrained. It is the usual way to connect a language model to company content, and it sits within the broader field of Generative AI.

The core idea is a two-step flow. When a question arrives, a retrieval step searches a knowledge source for content that is relevant to it. That content is then handed to the model as context, and the model composes its response from those documents rather than relying only on what it learned during training. Because the knowledge lives in the source and not in the model, the answers can reflect business data that changes over time, and no retraining is needed when the content is updated.

Retrieval typically depends on a searchable index of content. The M365.FM episode "Vector Databases - Simply Explained" describes the technology behind this: vector databases search by meaning, so AI can understand context instead of just matching keywords. The episode covers vectors and embeddings, semantic search, similarity search, vector indexes, chunking and hybrid search, and explains that similar concepts are stored close together in vector space. In practice, the quality of a RAG answer depends heavily on the quality of the underlying data and on how that data is indexed and retrieved.

RAG is closely related to the AI agent. Agents often use retrieval as one of their tools to gather knowledge before they act. The episode "Copilot Studio AI Agents and RAG with Nilüfer Doğan [MVP]" looks at this connection in a low-code setting. It describes how Microsoft Copilot Studio can be used to build agents that go beyond simple chatbots, understand business context, access enterprise knowledge and automate workflows, combining Microsoft 365, Power Platform, Dataverse and external data sources.

For Azure-based solutions, "RAG on Azure — Simply Explained" presents RAG in plain English. The episode describes it as a technique behind modern AI applications that enables large language models to answer questions using a organization's own business data instead of relying only on training knowledge, and it discusses how organizations build secure, accurate and trustworthy AI solutions with Microsoft Azure.

RAG also introduces security questions, which the M365.FM episodes treat as a risk surface. "Indirect Prompt Injection Security for Enterprise AI" points out that trusted documents, emails, SharePoint content, Teams conversations and knowledge bases can become attack vectors. The episode says that, unlike traditional software, large language models cannot reliably separate data from instructions, so every piece of retrieved content becomes part of the model's context and can carry hidden commands or poisoned content.

A second concern is covered in "Private RAG Security: Authorization-Aware Data Retrieval". The episode argues that data sovereignty is not the same as data security. When documents leave systems such as SharePoint and Microsoft 365 and are ingested into vector databases, permissions and access controls may not carry over during indexing. The result, according to the episode, can be a capable AI system that unintentionally exposes sensitive information to users who should not have access to it. Decisions around a RAG design therefore include not only which knowledge source to use and how to index it, but also how authorization is preserved at retrieval time.

Overview compiled with AI from M365.FM episodes and the facts on this page. You can improve it with the edit button below.

TypeConcept
KindTechnique
Also known asRAG

Key points about Retrieval-augmented generation (RAG)

Key points about Retrieval-augmented generation (RAG) from the episodes, each with its source episode:

Episodes about Retrieval-augmented generation (RAG)

Episodes in which Retrieval-augmented generation (RAG) is discussed:

Your Copilot Has No Memory — Why RAG Fails and LLM Wiki Fixes It

2026-08-10

Copilot can answer a question perfectly today and fumble the same question tomorrow because it searches instead of remembering. This episode explains why RAG-based Copilot stays shallow and how the LLM Wiki pattern adds compiled, maintained knowledge, for IT…

Timeline of episodes about Retrieval-augmented generation (RAG)

Articles about Retrieval-augmented generation (RAG) (9)

Articles in which Retrieval-augmented generation (RAG) is a topic:

Frequently asked questions

Questions marked with AI were written with AI from the facts on this page.

What is retrieval-augmented generation (RAG)? (AI)

RAG is a pattern in which an AI model first retrieves relevant documents from a knowledge source and then writes its answer based on them. It reduces made-up answers and lets models use current, private data without retraining.

How does RAG relate to vector databases? (AI)

Retrieval usually relies on a searchable index, and vector databases provide search by meaning instead of keyword matching. The episode "Vector Databases - Simply Explained" describes embeddings, semantic search, chunking and hybrid search as part of this.

What security issues does RAG raise? (AI)

The M365.FM episodes point to two: retrieved content such as documents and emails can carry hidden instructions (indirect prompt injection), and permissions may be lost when documents are indexed into vector databases, which can expose data to unauthorized users.

When should I use RAG instead of Microsoft 365 Copilot?

Use RAG when answers must come from your own systems, such as old PDFs, file servers or wikis, with citations. Copilot suits everyday productivity inside Microsoft 365. · From the episode: When to Use RAG vs Microsoft 365 Copilot

What is Retrieval Augmented Generation?

RAG retrieves ground truth from your own content and forces the model to cite it. The episode likens it to a librarian with receipts that refuses to bluff. · From the episode: When to Use RAG vs Microsoft 365 Copilot

Where does Copilot reach its boundary?

Where knowledge lives outside the clean M365 streets, in stale PDFs, forgotten SharePoint stacks, file-server ghosts and unmaintained wikis. That is where RAG takes over. · From the episode: When to Use RAG vs Microsoft 365 Copilot

Why isn't private RAG enough?

When documents are ingested into vector databases, the original permissions and access controls may be lost, creating gaps even though the data stays in a controlled environment. · From the episode: Private RAG Security: Authorization-Aware Data Retrieval

What risks do private RAG systems introduce?

The episode mentions access control gaps and the potential for data poisoning. When documents leave SharePoint for a vector database, the original permissions may not travel with them, so data sovereignty alone does not guarantee data security. · From the episode: Private RAG Security: Authorization-Aware Data Retrieval

Relationship map of Retrieval-augmented generation (RAG): linked topics

Related topics in brief

Topics linked to Retrieval-augmented generation (RAG), each briefly explained:

All relationships of Retrieval-augmented generation (RAG), grouped by type of relationship:

Related to

All 3 topics related to Retrieval-augmented generation (RAG):

Glossary

Community wiki

Short info above, community knowledge below: add practical tips, examples or corrections. New contributions are reviewed first; trusted members publish directly.

Experts and companies for this topic

Community members with expertise in Retrieval-augmented generation (RAG):

Experts on related topics

Experts on topics related to Retrieval-augmented generation (RAG), each with the topic that connects them:

Sources

Maintained by Mirko Peters (Microsoft MVP) · Last updated: