July 16, 2026
RAG on Azure — Simply Explained
Large Language Models like GPT-4o are incredibly powerful, but they have two major limitations. First, their knowledge is frozen in time, meaning they don't automatically know about recent events, changing regulations, or newly created documents. Second, they have no built-in knowledge of your organization's private data, including internal documentation, policies, product manuals, customer information, or business processes. Without additional context, AI models are forced to guess, increasing the risk of inaccurate or hallucinated answers. In this episode of Microsoft Knowledge Nuggets, we explain Retrieval-Augmented Generation (RAG) in simple terms and show how Azure combines Azure AI Search, Azure OpenAI, and Azure AI Foundry to build AI applications that answer questions using your own trusted data instead of relying solely on model memory.
WHY RETRIEVAL-AUGMENTED GENERATION SOLVES THE BIGGEST AI CHALLENGE
Rather than retraining or fine-tuning a language model every time your business information changes, RAG follows a much smarter approach. Before generating an answer, it first retrieves the most relevant information from your documents, databases, SharePoint sites, PDFs, websites, or other enterprise knowledge sources. That information is then added to the user's question before being sent to the language model. The AI generates its response based on the retrieved context instead of relying purely on its training data. This approach dramatically improves accuracy, reduces hallucinations, keeps information current, and ensures sensitive enterprise data never becomes part of the model itself.
VECTOR EMBEDDINGS, SEMANTIC SEARCH, AND AZURE AI SEARCH
One of the most important concepts behind RAG is semantic search. Instead of searching for exact keywords, Azure AI Search converts documents and user questions into vector embeddings—mathematical representations of meaning. This allows the search engine to understand concepts rather than simply matching words. For example, a search for "budget hotels" can successfully find documents discussing "affordable accommodation" because their meanings are closely related. We explain how Azure AI Search indexes enterprise data, creates vector embeddings using embedding models, performs hybrid search, applies semantic ranking, and retrieves the most relevant content within milliseconds before passing it to the language model.
HOW AZURE OPENAI AND AZURE AI FOUNDRY POWER RAG APPLICATIONS
Once Azure AI Search retrieves the relevant knowledge, Azure OpenAI uses models like GPT-4o or GPT-4.1 to generate a natural language response based entirely on the supplied context. Azure AI Foundry then acts as the orchestration layer that connects models, prompts, enterprise knowledge, tools, and deployment into one unified AI development platform. This episode explains how developers create Foundry projects, connect Azure AI Search indexes, configure system prompts, deploy AI agents, and build production-ready RAG solutions without manually wiring together multiple Azure services. Together, Azure AI Search, Azure OpenAI, and Azure AI Foundry provide a complete enterprise architecture for building secure, scalable, and trustworthy generative AI applications.
CLASSIC RAG VS AGENTIC RAG
Not every AI application retrieves information in the same way. We compare Classic RAG, where a single search retrieves relevant documents before generating an answer, with the newer Agentic RAG approach, where AI agents can perform multiple searches, combine information from different sources, reason across datasets, and dynamically decide which knowledge to retrieve. While Classic RAG delivers fast, predictable responses for straightforward question-and-answer scenarios, Agentic RAG offers significantly higher accuracy for complex, multi-step business questions by allowing AI agents to intelligently orchestrate retrieval before generation. Understanding the strengths of both architectures helps organizations choose the right design for their specific AI workloads.
BUILDING YOUR FIRST ENTERPRISE RAG SOLUTION ON AZURE
Getting started with RAG on Azure is simpler than many developers expect. This episode walks through storing enterprise documents in Azure Storage, indexing them with Azure AI Search, generating vector embeddings, deploying GPT-4o through Azure OpenAI, connecting everything inside Azure AI Foundry, and testing AI responses against real business knowledge. Whether you're building customer support assistants, enterprise copilots, document search applications, internal knowledge bots, or AI-powered automation, RAG provides one of the most effective ways to combine generative AI with trusted enterprise data. After listening to this episode, you'll understand why Retrieval-Augmented Generation has become the foundation of nearly every modern enterprise AI solution built on Microsoft Azure.
Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-modern-work-security-and-productivity-with-microsoft-365--6704921/support.
🚀 Want to be part of m365.fm?
Then stop just listening… and start showing up.
👉 Connect with me on LinkedIn and let’s make something happen:
- 🎙️ Be a podcast guest and share your story
- 🎧 Host your own episode (yes, seriously)
- 💡 Pitch topics the community actually wants to hear
- 🌍 Build your personal brand in the Microsoft 365 space
This isn’t just a podcast — it’s a platform for people who take action.
🔥 Most people wait. The best ones don’t.
👉 Connect with me on LinkedIn and send me a message:
"I want in"
Let’s build something awesome 👊
1
00:00:00,000 --> 00:00:03,520
Today's topic is one that almost everyone has heard of, but almost nobody can explain in
2
00:00:03,520 --> 00:00:04,520
plain English.
3
00:00:04,520 --> 00:00:06,840
Ragh, retrieval, augmented, generation.
4
00:00:06,840 --> 00:00:11,400
You've seen the acronym everywhere, Microsoft announcements, AI conference talks, blog posts,
5
00:00:11,400 --> 00:00:12,400
but what does it actually mean?
6
00:00:12,400 --> 00:00:14,040
And more importantly, why should you care?
7
00:00:14,040 --> 00:00:15,040
Here's the thing.
8
00:00:15,040 --> 00:00:17,920
Large language models like GPT-4O are powerful.
9
00:00:17,920 --> 00:00:21,280
They can write essays, summarize documents, even generate code.
10
00:00:21,280 --> 00:00:23,080
But they come with two big problems.
11
00:00:23,080 --> 00:00:26,600
First, their knowledge is frozen in time, ask chat GPT about something that happened yesterday
12
00:00:26,600 --> 00:00:30,240
and it has no idea.
13
00:00:30,240 --> 00:00:32,360
Second, these models never saw your company's internal data, your policies, product details,
14
00:00:32,360 --> 00:00:33,360
customer emails.
15
00:00:33,360 --> 00:00:36,600
So when you ask an LLM a question about your business, it's guessing.
16
00:00:36,600 --> 00:00:38,840
So that's where retrieval, augmented, generation comes in.
17
00:00:38,840 --> 00:00:39,840
Ragh isn't a new model.
18
00:00:39,840 --> 00:00:41,440
It's not a bigger or better GPT.
19
00:00:41,440 --> 00:00:42,840
It's a different approach entirely.
20
00:00:42,840 --> 00:00:46,960
Instead of asking the LLM to pull answers from its training memory, Ragh first looks up
21
00:00:46,960 --> 00:00:51,160
the right information from your own data and hands that information to the model alongside
22
00:00:51,160 --> 00:00:52,320
the question.
23
00:00:52,320 --> 00:00:57,120
The model then uses that context to give you answers grounded in real up-to-date content,
24
00:00:57,120 --> 00:00:58,120
not guesses.
25
00:00:58,120 --> 00:01:01,080
Without Ragh, your AI works from memory that's months or years old.
26
00:01:01,080 --> 00:01:04,080
With Ragh, it works from your actual documents right now.
27
00:01:04,080 --> 00:01:07,760
That's the difference between an AI that sounds confident but might be wrong and one that
28
00:01:07,760 --> 00:01:09,800
shows you exactly where it got its answer.
29
00:01:09,800 --> 00:01:11,320
Here's what we'll cover today.
30
00:01:11,320 --> 00:01:14,000
First we'll look at the two problems that make Ragh necessary.
31
00:01:14,000 --> 00:01:18,360
Then we'll break down what Ragh actually is, the retrieval, the augmentation, the generation.
32
00:01:18,360 --> 00:01:20,400
Next we'll walk through the key Azure services.
33
00:01:20,400 --> 00:01:23,840
Azure AI search, Azure Open AI and Azure AI Foundry.
34
00:01:23,840 --> 00:01:27,200
And we'll finish with a real example so you can see how the pieces fit together.
35
00:01:27,200 --> 00:01:28,680
Let's start with the problem itself.
36
00:01:28,680 --> 00:01:31,480
Once you see that, the solution becomes obvious.
37
00:01:31,480 --> 00:01:33,160
The two problems LLMs have.
38
00:01:33,160 --> 00:01:35,000
So what's wrong with large language models?
39
00:01:35,000 --> 00:01:36,800
Two things actually.
40
00:01:36,800 --> 00:01:38,200
First their knowledge is frozen.
41
00:01:38,200 --> 00:01:39,200
Think about it.
42
00:01:39,200 --> 00:01:42,600
When GPT-40 was trained, it learned from a snapshot of the internet that's now months
43
00:01:42,600 --> 00:01:43,600
or years old.
44
00:01:43,600 --> 00:01:47,320
So it doesn't know about new products, recent regulations, or yesterday's news.
45
00:01:47,320 --> 00:01:51,200
It's got about a policy change from last week and it has no way to know that information
46
00:01:51,200 --> 00:01:53,920
simply isn't in its training data.
47
00:01:53,920 --> 00:01:59,120
Second, LLMs never saw your private data, your internal documents, customer emails, proprietary
48
00:01:59,120 --> 00:02:00,120
databases.
49
00:02:00,120 --> 00:02:02,160
The model has zero access to any of that.
50
00:02:02,160 --> 00:02:06,400
So when an employee asks about your specific business processes, the model can't answer.
51
00:02:06,400 --> 00:02:07,680
It doesn't know what you know.
52
00:02:07,680 --> 00:02:09,440
Some people think fine tuning solves this.
53
00:02:09,440 --> 00:02:12,440
You take the base model and train it a bit more on your own data.
54
00:02:12,440 --> 00:02:14,000
But fine tuning has its own problems.
55
00:02:14,000 --> 00:02:17,320
It's expensive, you're paying for GPU time and engineering effort.
56
00:02:17,320 --> 00:02:19,040
It doesn't fix staleness.
57
00:02:19,040 --> 00:02:22,520
Once you fine tune, the model is still frozen at that point in time.
58
00:02:22,520 --> 00:02:26,920
And it can't handle data that changes frequently like inventory levels or pricing.
59
00:02:26,920 --> 00:02:29,560
Fine tuning is like teaching someone a textbook by heart.
60
00:02:29,560 --> 00:02:32,280
Useful, but they still can't look up today's newspaper.
61
00:02:32,280 --> 00:02:33,960
The real solution is simpler.
62
00:02:33,960 --> 00:02:37,320
Give the model access to your knowledge at the moment it needs to answer.
63
00:02:37,320 --> 00:02:41,240
Instead of trying to cram everything into its memory, don't make it memorize everything.
64
00:02:41,240 --> 00:02:43,800
Let it look things up in real time, from your actual data.
65
00:02:43,800 --> 00:02:46,960
That's what Ragn does, simple idea, but it changes everything.
66
00:02:46,960 --> 00:02:48,400
What Ragn actually is.
67
00:02:48,400 --> 00:02:49,920
So here's Ragn in one sentence.
68
00:02:49,920 --> 00:02:54,120
Instead of asking the model to pull an answer from its training data, we first go and find
69
00:02:54,120 --> 00:02:58,440
the relevant information, then feed that to the model alongside the question.
70
00:02:58,440 --> 00:02:59,640
That's the core idea.
71
00:02:59,640 --> 00:03:00,800
Three simple steps.
72
00:03:00,800 --> 00:03:03,200
Retrieve, augment, and generate.
73
00:03:03,200 --> 00:03:04,720
Let me give you an analogy.
74
00:03:04,720 --> 00:03:05,960
Think of a librarian.
75
00:03:05,960 --> 00:03:08,160
You walk into a librarian, ask a question.
76
00:03:08,160 --> 00:03:10,840
You wouldn't expect the librarian to have memorized every book, right?
77
00:03:10,840 --> 00:03:11,840
That would be ridiculous.
78
00:03:11,840 --> 00:03:16,000
Instead you ask them to find the right book, open it to the right page, and read you the answer.
79
00:03:16,000 --> 00:03:17,960
The librarian doesn't memorize a thing.
80
00:03:17,960 --> 00:03:20,080
They retrieve first and read second.
81
00:03:20,080 --> 00:03:21,840
And that's exactly what Ragn does.
82
00:03:21,840 --> 00:03:23,320
Retrieve will first, generation second.
83
00:03:23,320 --> 00:03:25,280
So here's how it actually works in practice.
84
00:03:25,280 --> 00:03:26,640
The user asks a question.
85
00:03:26,640 --> 00:03:28,600
The system then searches a knowledge base.
86
00:03:28,600 --> 00:03:31,240
Your documents, your data, your private information.
87
00:03:31,240 --> 00:03:35,040
Whatever it finds gets fed into the large language model and the model generates an answer
88
00:03:35,040 --> 00:03:36,320
based on that context.
89
00:03:36,320 --> 00:03:37,600
And here's the part that matters.
90
00:03:37,600 --> 00:03:39,200
The LLM never sees your raw data.
91
00:03:39,200 --> 00:03:41,320
It only sees what the retrieval system hands it.
92
00:03:41,320 --> 00:03:43,800
It's not a drawback, so it's a security feature.
93
00:03:43,800 --> 00:03:46,240
Your sensitive documents stay in your own storage.
94
00:03:46,240 --> 00:03:50,560
At query time, the model only gets the relevant snippets it needs to answer that specific question.
95
00:03:50,560 --> 00:03:53,600
No training on your data, no memorization, no leakage.
96
00:03:53,600 --> 00:03:54,960
Your data stays where it belongs.
97
00:03:54,960 --> 00:03:57,760
So Ragn solves both problems we talked about earlier.
98
00:03:57,760 --> 00:03:58,760
Stale data?
99
00:03:58,760 --> 00:03:59,760
Not an issue.
100
00:03:59,760 --> 00:04:01,280
Your knowledge base stays current.
101
00:04:01,280 --> 00:04:02,280
Private data?
102
00:04:02,280 --> 00:04:04,000
The model only sees what you choose to retrieve.
103
00:04:04,000 --> 00:04:07,480
It's like giving the librarian access to your filing cabinet, but only letting them pull
104
00:04:07,480 --> 00:04:10,600
out the one folder you need right when you need it.
105
00:04:10,600 --> 00:04:12,600
What do you think about your data?
106
00:04:12,600 --> 00:04:13,600
What do you think about the data?
107
00:04:13,600 --> 00:04:14,600
What do you think about the data?
108
00:04:14,600 --> 00:04:15,600
What do you think about the data?
109
00:04:15,600 --> 00:04:16,600
What do you think about the data?
110
00:04:16,600 --> 00:04:17,600
What do you think about the data?
111
00:04:17,600 --> 00:04:18,600
What do you think about the data?
112
00:04:18,600 --> 00:04:19,600
What do you think about the data?
113
00:04:19,600 --> 00:04:20,600
What do you think about the data?
114
00:04:20,600 --> 00:04:21,600
What do you think about the data?
115
00:04:21,600 --> 00:04:22,600
What do you think about the data?
116
00:04:22,600 --> 00:04:23,600
What do you think about the data?
117
00:04:23,600 --> 00:04:24,600
What do you think about the data?
118
00:04:24,600 --> 00:04:25,600
What do you think about the data?
119
00:04:25,600 --> 00:04:26,600
What do you think about the data?
120
00:04:26,600 --> 00:04:27,600
What do you think about the data?
121
00:04:27,600 --> 00:04:28,600
What do you think about the data?
122
00:04:28,600 --> 00:04:29,600
What do you think about the data?
123
00:04:29,600 --> 00:04:30,600
What do you think about the data?
124
00:04:30,600 --> 00:04:31,600
What do you think about the data?
125
00:04:31,600 --> 00:04:32,600
What do you think about the data?
126
00:04:32,600 --> 00:04:33,600
What do you think about the data?
127
00:04:33,600 --> 00:04:34,600
What do you think about the data?
128
00:04:34,600 --> 00:04:35,600
What do you think about the data?
129
00:04:35,600 --> 00:04:36,600
What do you think about the data?
130
00:04:36,600 --> 00:04:37,600
What do you think about the data?
131
00:04:37,600 --> 00:04:38,600
What do you think about the data?
132
00:04:38,600 --> 00:04:39,600
What do you think about the data?
133
00:04:39,600 --> 00:04:40,600
What do you think about the data?
134
00:04:40,600 --> 00:04:41,600
What do you think about the data?
135
00:04:41,600 --> 00:04:42,600
What do you think about the data?
136
00:04:42,600 --> 00:04:43,600
What do you think about the data?
137
00:04:43,600 --> 00:04:44,600
What do you think about the data?
138
00:04:44,600 --> 00:04:45,600
What do you think about the data?
139
00:04:45,600 --> 00:04:46,600
What do you think about the data?
140
00:04:46,600 --> 00:04:47,600
What do you think about the data?
141
00:04:47,600 --> 00:04:48,600
What do you think about the data?
142
00:04:48,600 --> 00:04:49,600
What do you think about the data?
143
00:04:49,600 --> 00:04:50,600
What do you think about the data?
144
00:04:50,600 --> 00:04:51,600
What do you think about the data?
145
00:04:51,600 --> 00:04:52,600
What do you think about the data?
146
00:04:52,600 --> 00:04:53,600
What do you think about the data?
147
00:04:53,600 --> 00:04:54,600
What do you think about the data?
148
00:04:54,600 --> 00:04:55,600
What do you think about the data?
149
00:04:55,600 --> 00:04:56,600
What do you think about the data?
150
00:04:56,600 --> 00:04:57,600
What do you think about the data?
151
00:04:57,600 --> 00:04:58,600
What do you think about the data?
152
00:04:58,600 --> 00:04:59,600
What do you think about the data?
153
00:04:59,600 --> 00:05:00,600
What do you think about the data?
154
00:05:00,600 --> 00:05:01,600
What do you think about the data?
155
00:05:01,600 --> 00:05:02,600
What do you think about the data?
156
00:05:02,600 --> 00:05:03,600
What do you think about the data?
157
00:05:03,600 --> 00:05:04,600
What do you think about the data?
158
00:05:04,600 --> 00:05:05,600
What do you think about the data?
159
00:05:05,600 --> 00:05:06,600
What do you think about the data?
160
00:05:06,600 --> 00:05:07,600
What do you think about the data?
161
00:05:07,600 --> 00:05:08,600
What do you think about the data?
162
00:05:08,600 --> 00:05:09,600
What do you think about the data?
163
00:05:09,600 --> 00:05:10,600
What do you think about the data?
164
00:05:10,600 --> 00:05:11,600
What do you think about the data?
165
00:05:11,600 --> 00:05:12,600
What do you think about the data?
166
00:05:12,600 --> 00:05:13,600
What do you think about the data?
167
00:05:13,600 --> 00:05:14,600
What do you think about the data?
168
00:05:14,600 --> 00:05:15,600
What do you think about the data?
169
00:05:15,600 --> 00:05:16,600
What do you think about the data?
170
00:05:16,600 --> 00:05:17,600
What do you think about the data?
171
00:05:17,600 --> 00:05:18,600
What do you think about the data?
172
00:05:18,600 --> 00:05:19,600
What do you think about the data?
173
00:05:19,600 --> 00:05:20,600
What do you think about the data?
174
00:05:20,600 --> 00:05:21,600
What do you think about the data?
175
00:05:21,600 --> 00:05:22,600
What do you think about the data?
176
00:05:22,600 --> 00:05:23,600
What do you think about the data?
177
00:05:23,600 --> 00:05:24,600
What do you think about the data?
178
00:05:24,600 --> 00:05:25,600
What do you think about the data?
179
00:05:25,600 --> 00:05:26,600
What do you think about the data?
180
00:05:26,600 --> 00:05:27,600
What do you think about the data?
181
00:05:27,600 --> 00:05:28,600
What do you think about the data?
182
00:05:28,600 --> 00:05:29,600
What do you think about the data?
183
00:05:29,600 --> 00:05:30,600
What do you think about the data?
184
00:05:30,600 --> 00:05:31,600
What do you think about the data?
185
00:05:31,600 --> 00:05:32,600
What do you think about the data?
186
00:05:32,600 --> 00:05:33,600
What do you think about the data?
187
00:05:33,600 --> 00:05:34,600
What do you think about the data?
188
00:05:34,600 --> 00:05:35,600
What do you think about the data?
189
00:05:35,600 --> 00:05:36,600
What do you think about the data?
190
00:05:36,600 --> 00:05:37,600
What do you think about the data?
191
00:05:37,600 --> 00:05:38,600
What do you think about the data?
192
00:05:38,600 --> 00:05:39,600
What do you think about the data?
193
00:05:39,600 --> 00:05:41,600
It handles two main tasks and lets walk through them.
194
00:05:41,600 --> 00:05:45,600
First, indexing, that's getting your data ready to be searched, second, searching.
195
00:05:45,600 --> 00:05:48,600
That's actually finding the right content when someone asks a question.
196
00:05:48,600 --> 00:05:50,600
Let's go through the indexing process first.
197
00:05:50,600 --> 00:05:53,600
You start by pointing Azure AI search at your data source,
198
00:05:53,600 --> 00:05:56,600
which could be a blob storage container full of PDFs,
199
00:05:56,600 --> 00:06:00,600
a SQL database with customer records or any number of other sources.
200
00:06:00,600 --> 00:06:04,600
The service reads those documents, breaks them into smaller pieces called chunks,
201
00:06:04,600 --> 00:06:06,600
creates vector embeddings for each chunk,
202
00:06:06,600 --> 00:06:09,600
using that embedding model we talked about earlier,
203
00:06:09,600 --> 00:06:11,600
and stores everything in a search index.
204
00:06:11,600 --> 00:06:14,600
Think of the index as a giant, highly organized catalog.
205
00:06:14,600 --> 00:06:17,600
Every piece of content gets a unique ID, a chunk of text,
206
00:06:17,600 --> 00:06:20,600
and a set of coordinates on that map of meaning.
207
00:06:20,600 --> 00:06:21,600
Now the searching process.
208
00:06:21,600 --> 00:06:23,600
When a user asks a question,
209
00:06:23,600 --> 00:06:26,600
AI search converts that question into a vector using the same embedding model,
210
00:06:26,600 --> 00:06:31,600
then scans the index to find chunks whose coordinates are closest to the questions coordinates.
211
00:06:31,600 --> 00:06:35,600
It returns the top matches, usually the five to ten most relevant ones,
212
00:06:35,600 --> 00:06:37,600
and does all of this in milliseconds.
213
00:06:37,600 --> 00:06:39,600
You get a few strong options here.
214
00:06:39,600 --> 00:06:41,600
Pure vector search finds content by meaning,
215
00:06:41,600 --> 00:06:44,600
so similar concepts show up even if the exact words don't match.
216
00:06:44,600 --> 00:06:48,600
Keyword search works like a traditional search engine, finding exact word matches.
217
00:06:48,600 --> 00:06:51,600
Or you can use hybrid search, which combines both approaches,
218
00:06:51,600 --> 00:06:53,600
and that's usually the best choice.
219
00:06:53,600 --> 00:06:55,600
There's also something called semantic re-ranking,
220
00:06:55,600 --> 00:06:59,600
which takes those initial results and resorts them based on how well they actually answer the question.
221
00:06:59,600 --> 00:07:02,600
It's like a second pass that catches things the first one might have missed.
222
00:07:02,600 --> 00:07:04,600
One thing you should know.
223
00:07:04,600 --> 00:07:06,600
Azure AI search can get expensive.
224
00:07:06,600 --> 00:07:10,600
The basic tier starts around $75 a month and climbs from there.
225
00:07:10,600 --> 00:07:12,600
If you're just experimenting, use the free tier,
226
00:07:12,600 --> 00:07:15,600
or make sure you delete your resources when you're done.
227
00:07:15,600 --> 00:07:18,600
For production use though, the cost usually pays off.
228
00:07:18,600 --> 00:07:21,600
You're buying speed and reliability at scale.
229
00:07:21,600 --> 00:07:25,600
Azure Open AI plus AI Foundry, Generation and Orchestration.
230
00:07:25,600 --> 00:07:27,600
So now your data is indexed and ready for retrieval,
231
00:07:27,600 --> 00:07:31,600
but the LLM still needs to understand your question and generate a solid answer.
232
00:07:31,600 --> 00:07:34,600
That's where Azure Open AI and Azure AI Foundry come in.
233
00:07:34,600 --> 00:07:36,600
Azure Open AI provides the actual language model,
234
00:07:36,600 --> 00:07:40,600
GPT-40, GPT-4.1, or whatever model you choose.
235
00:07:40,600 --> 00:07:42,600
This is the Generation Engine.
236
00:07:42,600 --> 00:07:45,600
It receives a prompt that includes both the original question from the user
237
00:07:45,600 --> 00:07:48,600
and the context that Azure AI search retrieved.
238
00:07:48,600 --> 00:07:52,600
The model reads through that context and produces an answer based on what it finds.
239
00:07:52,600 --> 00:07:55,600
Now you can't just dump everything in and hope for the best.
240
00:07:55,600 --> 00:07:59,600
You need a system prompt, a set of instructions you give the model upfront about how to behave.
241
00:07:59,600 --> 00:08:03,600
You tell it to use only the provided context to cite its sources.
242
00:08:03,600 --> 00:08:08,600
And most importantly to say, I don't know if the relevant information isn't in the retrieve documents.
243
00:08:08,600 --> 00:08:10,600
That's what keeps the model honest.
244
00:08:10,600 --> 00:08:15,600
Without a good system prompt, the model might fall back on its training data or worse, make something up.
245
00:08:15,600 --> 00:08:17,600
So how do you actually wire all these pieces together?
246
00:08:17,600 --> 00:08:19,600
That's where Azure AI Foundry comes in.
247
00:08:19,600 --> 00:08:24,600
It's the portal where you create a project, deploy your models, connect your data sources and test your agent.
248
00:08:24,600 --> 00:08:27,600
Think of it as the control center for your entire rag setup.
249
00:08:27,600 --> 00:08:33,600
You can add your Azure AI search index as a knowledge source to your agent without writing custom code for the basic setup.
250
00:08:33,600 --> 00:08:37,600
You just create a connection, select your index and the agent can start querying it automatically.
251
00:08:37,600 --> 00:08:39,600
The portal handles the orchestration.
252
00:08:39,600 --> 00:08:47,600
When a user asks a question, the agent knows to call AI search, get the relevant chunks and pass them to the LLM along with the original question.
253
00:08:47,600 --> 00:08:48,600
Here's a concrete example.
254
00:08:48,600 --> 00:08:53,600
In one tutorial I came across, someone built a movie recommendation agent using a Netflix data set.
255
00:08:53,600 --> 00:09:01,600
They uploaded a CSV file with movie titles, descriptions and cast information to Azure Blob Storage, then used Azure AI search to index that data.
256
00:09:01,600 --> 00:09:06,600
In Azure AI Foundry, they created an agent with GPT-4O and connected the AI search index.
257
00:09:06,600 --> 00:09:14,600
When a user asked, "I like thrillers, give me some recommendations," the agent searched the index, found thriller movies from that specific data set, and returned recommendations.
258
00:09:14,600 --> 00:09:15,600
And here's the key.
259
00:09:15,600 --> 00:09:17,600
It only recommended movies that were actually in that data set.
260
00:09:17,600 --> 00:09:22,600
It didn't invent anything, didn't pull from its training data and worked exclusively from the content they provided.
261
00:09:22,600 --> 00:09:24,600
That's the power of Ragn.
262
00:09:24,600 --> 00:09:26,600
Classic vs. AgenteGrag.
263
00:09:26,600 --> 00:09:27,600
Now let's talk about setting this up.
264
00:09:27,600 --> 00:09:32,600
There are two main patterns, classic Ragn and AgenteGrag, which one you pick depends on what you're building.
265
00:09:32,600 --> 00:09:34,600
Classic Ragn is the straightforward approach.
266
00:09:34,600 --> 00:09:37,600
One query goes in, one search runs, and one answer comes out.
267
00:09:37,600 --> 00:09:39,600
Simple, fast, predictable.
268
00:09:39,600 --> 00:09:43,600
Think of it like walking up to the reception desk in an office building and asking for one specific file.
269
00:09:43,600 --> 00:09:46,600
The receptionist grabs it, hands it to you, done.
270
00:09:46,600 --> 00:09:49,600
Classic Ragn works great for factual Q&A on a single topic.
271
00:09:49,600 --> 00:09:55,600
Like, what's the company policy on remote work? Or show me the troubleshooting steps for this error code?
272
00:09:55,600 --> 00:09:59,600
It's reliable, cheap, and easy to fix when something goes wrong because the flow is linear.
273
00:09:59,600 --> 00:10:03,600
AgenteGrag is different. The AI agent decides how to retrieve information.
274
00:10:03,600 --> 00:10:10,600
It can ask multiple subquestions, search different indexes, iterate on results, and synthesize across sources before answering.
275
00:10:10,600 --> 00:10:14,600
It's like having a building manager who coordinates with different departments to gather all the pieces.
276
00:10:14,600 --> 00:10:16,600
They don't just grab one file.
277
00:10:16,600 --> 00:10:21,600
They talk to HR, check records, cross-reference policies, and then come back with a complete answer.
278
00:10:21,600 --> 00:10:26,600
Microsoft ran their own tests and found AgenteGrag gives about a 40% improvement in answer relevance for complex questions.
279
00:10:26,600 --> 00:10:30,600
That's significant, but it uses 3 to 10 times more tokens and adds latency.
280
00:10:30,600 --> 00:10:33,600
For simple questions, that extra work is wasted.
281
00:10:33,600 --> 00:10:35,600
Classic Ragn handles those just fine.
282
00:10:35,600 --> 00:10:36,600
So when do you use each?
283
00:10:36,600 --> 00:10:40,600
Classic for routine, high volume queries where speed and cost matter most.
284
00:10:40,600 --> 00:10:45,600
AgenteG for multi-step ambiguous or cross-domain questions where accuracy is worth the extra compute.
285
00:10:45,600 --> 00:10:50,600
Connection section end to end example. Let's see how all these pieces fit together with a real example.
286
00:10:50,600 --> 00:10:55,600
Imagine you're building a customer support agent for a travel agency called Marygold Travels.
287
00:10:55,600 --> 00:11:00,600
You have trip brochures, hotel reviews, and policy documents scattered across your organization.
288
00:11:00,600 --> 00:11:06,600
You want customers to ask questions and get accurate answers based on your actual content, not generic guesses.
289
00:11:06,600 --> 00:11:09,600
Step 1. Store all that content as your blob storage.
290
00:11:09,600 --> 00:11:14,600
Brochures in one folder, reviews in another, policies in a third, just files in a container.
291
00:11:14,600 --> 00:11:19,600
Step 2. Use Azure AI search to index that data by running the import data wizard,
292
00:11:19,600 --> 00:11:22,600
pointing it at your blob storage and letting it work.
293
00:11:22,600 --> 00:11:28,600
The service reads every document, chunks them into pieces, creates vector embeddings, and builds a searchable index.
294
00:11:28,600 --> 00:11:29,600
The whole process takes a few minutes.
295
00:11:29,600 --> 00:11:32,600
Step 3. Go into Azure AI Foundry and create an agent.
296
00:11:32,600 --> 00:11:39,600
Deploy GPT-4.0 as your model, write a system prompt that tells it to use only the provided context and site its sources,
297
00:11:39,600 --> 00:11:42,600
then connect your Azure AI search index as a knowledge source.
298
00:11:42,600 --> 00:11:44,600
A few clicks and it's done.
299
00:11:44,600 --> 00:11:48,600
Now a customer asks, what's the best review hotel in Dubai under $300?
300
00:11:48,600 --> 00:11:51,600
The agent takes that question and sends it to Azure AI search.
301
00:11:51,600 --> 00:11:56,600
Azure AI search converts it to a vector and finds the most relevant chunks from your reviews and brochures.
302
00:11:56,600 --> 00:11:59,600
Those chunks get passed to GPT-4.0 along with the original question.
303
00:11:59,600 --> 00:12:05,600
The model reads through them, identifies the top match, and generates a response with citations pointing back to your source documents.
304
00:12:05,600 --> 00:12:08,600
Here's the thing, the LLM never memorizes that data.
305
00:12:08,600 --> 00:12:15,600
It never trained on your hotel reviews, it works from your specific content every single time, add new reviews tomorrow and the answers update automatically.
306
00:12:15,600 --> 00:12:18,600
Remove an outdated brochure and it stops showing up.
307
00:12:18,600 --> 00:12:21,600
The model doesn't need retraining, it just needs the right retrieval layer.
308
00:12:21,600 --> 00:12:23,600
So how do you actually start building Ragnar Azure?
309
00:12:23,600 --> 00:12:25,600
Let's break it down into three steps.
310
00:12:25,600 --> 00:12:27,600
First things first, know your data.
311
00:12:27,600 --> 00:12:33,600
Before you even pick a service, figure out where your content lives, how it's organized, and who should have access.
312
00:12:33,600 --> 00:12:35,600
Ragnar lives or dies on the quality of your data.
313
00:12:35,600 --> 00:12:38,600
Next, keep it simple, don't overcomplicate the setup.
314
00:12:38,600 --> 00:12:42,600
Use the import data wizard in Azure AI search to index just one set of documents.
315
00:12:42,600 --> 00:12:47,600
Test with basic queries. Make sure your retrieval works perfectly before you add the LLM.
316
00:12:47,600 --> 00:12:51,600
Most Ragn failures aren't model problems, they're retrieval problems.
317
00:12:51,600 --> 00:12:56,600
Last step, add the LLM. Once your search is returning the right results, then you layer in generation.
318
00:12:56,600 --> 00:12:59,600
That's when you connect as you open AI and write your system prompt.
319
00:12:59,600 --> 00:13:03,600
Here's the thing, Ragnar isn't something you buy, it's a pattern you build.
320
00:13:03,600 --> 00:13:08,600
Azure gives you the pieces, storage, search, models, and the glue to connect them.
321
00:13:08,600 --> 00:13:11,600
You just need to put it all together the right way.
322
00:13:11,600 --> 00:13:15,600
If this helped you see the big picture, hit subscribe for more play in English breakdowns.
323
00:13:15,600 --> 00:13:17,600
Drop a comment with your Ragnar use case.
324
00:13:17,600 --> 00:13:19,600
I'd love to know what you're building.
00:00:00,000 --> 00:00:03,520
Today's topic is one that almost everyone has heard of, but almost nobody can explain in
2
00:00:03,520 --> 00:00:04,520
plain English.
3
00:00:04,520 --> 00:00:06,840
Ragh, retrieval, augmented, generation.
4
00:00:06,840 --> 00:00:11,400
You've seen the acronym everywhere, Microsoft announcements, AI conference talks, blog posts,
5
00:00:11,400 --> 00:00:12,400
but what does it actually mean?
6
00:00:12,400 --> 00:00:14,040
And more importantly, why should you care?
7
00:00:14,040 --> 00:00:15,040
Here's the thing.
8
00:00:15,040 --> 00:00:17,920
Large language models like GPT-4O are powerful.
9
00:00:17,920 --> 00:00:21,280
They can write essays, summarize documents, even generate code.
10
00:00:21,280 --> 00:00:23,080
But they come with two big problems.
11
00:00:23,080 --> 00:00:26,600
First, their knowledge is frozen in time, ask chat GPT about something that happened yesterday
12
00:00:26,600 --> 00:00:30,240
and it has no idea.
13
00:00:30,240 --> 00:00:32,360
Second, these models never saw your company's internal data, your policies, product details,
14
00:00:32,360 --> 00:00:33,360
customer emails.
15
00:00:33,360 --> 00:00:36,600
So when you ask an LLM a question about your business, it's guessing.
16
00:00:36,600 --> 00:00:38,840
So that's where retrieval, augmented, generation comes in.
17
00:00:38,840 --> 00:00:39,840
Ragh isn't a new model.
18
00:00:39,840 --> 00:00:41,440
It's not a bigger or better GPT.
19
00:00:41,440 --> 00:00:42,840
It's a different approach entirely.
20
00:00:42,840 --> 00:00:46,960
Instead of asking the LLM to pull answers from its training memory, Ragh first looks up
21
00:00:46,960 --> 00:00:51,160
the right information from your own data and hands that information to the model alongside
22
00:00:51,160 --> 00:00:52,320
the question.
23
00:00:52,320 --> 00:00:57,120
The model then uses that context to give you answers grounded in real up-to-date content,
24
00:00:57,120 --> 00:00:58,120
not guesses.
25
00:00:58,120 --> 00:01:01,080
Without Ragh, your AI works from memory that's months or years old.
26
00:01:01,080 --> 00:01:04,080
With Ragh, it works from your actual documents right now.
27
00:01:04,080 --> 00:01:07,760
That's the difference between an AI that sounds confident but might be wrong and one that
28
00:01:07,760 --> 00:01:09,800
shows you exactly where it got its answer.
29
00:01:09,800 --> 00:01:11,320
Here's what we'll cover today.
30
00:01:11,320 --> 00:01:14,000
First we'll look at the two problems that make Ragh necessary.
31
00:01:14,000 --> 00:01:18,360
Then we'll break down what Ragh actually is, the retrieval, the augmentation, the generation.
32
00:01:18,360 --> 00:01:20,400
Next we'll walk through the key Azure services.
33
00:01:20,400 --> 00:01:23,840
Azure AI search, Azure Open AI and Azure AI Foundry.
34
00:01:23,840 --> 00:01:27,200
And we'll finish with a real example so you can see how the pieces fit together.
35
00:01:27,200 --> 00:01:28,680
Let's start with the problem itself.
36
00:01:28,680 --> 00:01:31,480
Once you see that, the solution becomes obvious.
37
00:01:31,480 --> 00:01:33,160
The two problems LLMs have.
38
00:01:33,160 --> 00:01:35,000
So what's wrong with large language models?
39
00:01:35,000 --> 00:01:36,800
Two things actually.
40
00:01:36,800 --> 00:01:38,200
First their knowledge is frozen.
41
00:01:38,200 --> 00:01:39,200
Think about it.
42
00:01:39,200 --> 00:01:42,600
When GPT-40 was trained, it learned from a snapshot of the internet that's now months
43
00:01:42,600 --> 00:01:43,600
or years old.
44
00:01:43,600 --> 00:01:47,320
So it doesn't know about new products, recent regulations, or yesterday's news.
45
00:01:47,320 --> 00:01:51,200
It's got about a policy change from last week and it has no way to know that information
46
00:01:51,200 --> 00:01:53,920
simply isn't in its training data.
47
00:01:53,920 --> 00:01:59,120
Second, LLMs never saw your private data, your internal documents, customer emails, proprietary
48
00:01:59,120 --> 00:02:00,120
databases.
49
00:02:00,120 --> 00:02:02,160
The model has zero access to any of that.
50
00:02:02,160 --> 00:02:06,400
So when an employee asks about your specific business processes, the model can't answer.
51
00:02:06,400 --> 00:02:07,680
It doesn't know what you know.
52
00:02:07,680 --> 00:02:09,440
Some people think fine tuning solves this.
53
00:02:09,440 --> 00:02:12,440
You take the base model and train it a bit more on your own data.
54
00:02:12,440 --> 00:02:14,000
But fine tuning has its own problems.
55
00:02:14,000 --> 00:02:17,320
It's expensive, you're paying for GPU time and engineering effort.
56
00:02:17,320 --> 00:02:19,040
It doesn't fix staleness.
57
00:02:19,040 --> 00:02:22,520
Once you fine tune, the model is still frozen at that point in time.
58
00:02:22,520 --> 00:02:26,920
And it can't handle data that changes frequently like inventory levels or pricing.
59
00:02:26,920 --> 00:02:29,560
Fine tuning is like teaching someone a textbook by heart.
60
00:02:29,560 --> 00:02:32,280
Useful, but they still can't look up today's newspaper.
61
00:02:32,280 --> 00:02:33,960
The real solution is simpler.
62
00:02:33,960 --> 00:02:37,320
Give the model access to your knowledge at the moment it needs to answer.
63
00:02:37,320 --> 00:02:41,240
Instead of trying to cram everything into its memory, don't make it memorize everything.
64
00:02:41,240 --> 00:02:43,800
Let it look things up in real time, from your actual data.
65
00:02:43,800 --> 00:02:46,960
That's what Ragn does, simple idea, but it changes everything.
66
00:02:46,960 --> 00:02:48,400
What Ragn actually is.
67
00:02:48,400 --> 00:02:49,920
So here's Ragn in one sentence.
68
00:02:49,920 --> 00:02:54,120
Instead of asking the model to pull an answer from its training data, we first go and find
69
00:02:54,120 --> 00:02:58,440
the relevant information, then feed that to the model alongside the question.
70
00:02:58,440 --> 00:02:59,640
That's the core idea.
71
00:02:59,640 --> 00:03:00,800
Three simple steps.
72
00:03:00,800 --> 00:03:03,200
Retrieve, augment, and generate.
73
00:03:03,200 --> 00:03:04,720
Let me give you an analogy.
74
00:03:04,720 --> 00:03:05,960
Think of a librarian.
75
00:03:05,960 --> 00:03:08,160
You walk into a librarian, ask a question.
76
00:03:08,160 --> 00:03:10,840
You wouldn't expect the librarian to have memorized every book, right?
77
00:03:10,840 --> 00:03:11,840
That would be ridiculous.
78
00:03:11,840 --> 00:03:16,000
Instead you ask them to find the right book, open it to the right page, and read you the answer.
79
00:03:16,000 --> 00:03:17,960
The librarian doesn't memorize a thing.
80
00:03:17,960 --> 00:03:20,080
They retrieve first and read second.
81
00:03:20,080 --> 00:03:21,840
And that's exactly what Ragn does.
82
00:03:21,840 --> 00:03:23,320
Retrieve will first, generation second.
83
00:03:23,320 --> 00:03:25,280
So here's how it actually works in practice.
84
00:03:25,280 --> 00:03:26,640
The user asks a question.
85
00:03:26,640 --> 00:03:28,600
The system then searches a knowledge base.
86
00:03:28,600 --> 00:03:31,240
Your documents, your data, your private information.
87
00:03:31,240 --> 00:03:35,040
Whatever it finds gets fed into the large language model and the model generates an answer
88
00:03:35,040 --> 00:03:36,320
based on that context.
89
00:03:36,320 --> 00:03:37,600
And here's the part that matters.
90
00:03:37,600 --> 00:03:39,200
The LLM never sees your raw data.
91
00:03:39,200 --> 00:03:41,320
It only sees what the retrieval system hands it.
92
00:03:41,320 --> 00:03:43,800
It's not a drawback, so it's a security feature.
93
00:03:43,800 --> 00:03:46,240
Your sensitive documents stay in your own storage.
94
00:03:46,240 --> 00:03:50,560
At query time, the model only gets the relevant snippets it needs to answer that specific question.
95
00:03:50,560 --> 00:03:53,600
No training on your data, no memorization, no leakage.
96
00:03:53,600 --> 00:03:54,960
Your data stays where it belongs.
97
00:03:54,960 --> 00:03:57,760
So Ragn solves both problems we talked about earlier.
98
00:03:57,760 --> 00:03:58,760
Stale data?
99
00:03:58,760 --> 00:03:59,760
Not an issue.
100
00:03:59,760 --> 00:04:01,280
Your knowledge base stays current.
101
00:04:01,280 --> 00:04:02,280
Private data?
102
00:04:02,280 --> 00:04:04,000
The model only sees what you choose to retrieve.
103
00:04:04,000 --> 00:04:07,480
It's like giving the librarian access to your filing cabinet, but only letting them pull
104
00:04:07,480 --> 00:04:10,600
out the one folder you need right when you need it.
105
00:04:10,600 --> 00:04:12,600
What do you think about your data?
106
00:04:12,600 --> 00:04:13,600
What do you think about the data?
107
00:04:13,600 --> 00:04:14,600
What do you think about the data?
108
00:04:14,600 --> 00:04:15,600
What do you think about the data?
109
00:04:15,600 --> 00:04:16,600
What do you think about the data?
110
00:04:16,600 --> 00:04:17,600
What do you think about the data?
111
00:04:17,600 --> 00:04:18,600
What do you think about the data?
112
00:04:18,600 --> 00:04:19,600
What do you think about the data?
113
00:04:19,600 --> 00:04:20,600
What do you think about the data?
114
00:04:20,600 --> 00:04:21,600
What do you think about the data?
115
00:04:21,600 --> 00:04:22,600
What do you think about the data?
116
00:04:22,600 --> 00:04:23,600
What do you think about the data?
117
00:04:23,600 --> 00:04:24,600
What do you think about the data?
118
00:04:24,600 --> 00:04:25,600
What do you think about the data?
119
00:04:25,600 --> 00:04:26,600
What do you think about the data?
120
00:04:26,600 --> 00:04:27,600
What do you think about the data?
121
00:04:27,600 --> 00:04:28,600
What do you think about the data?
122
00:04:28,600 --> 00:04:29,600
What do you think about the data?
123
00:04:29,600 --> 00:04:30,600
What do you think about the data?
124
00:04:30,600 --> 00:04:31,600
What do you think about the data?
125
00:04:31,600 --> 00:04:32,600
What do you think about the data?
126
00:04:32,600 --> 00:04:33,600
What do you think about the data?
127
00:04:33,600 --> 00:04:34,600
What do you think about the data?
128
00:04:34,600 --> 00:04:35,600
What do you think about the data?
129
00:04:35,600 --> 00:04:36,600
What do you think about the data?
130
00:04:36,600 --> 00:04:37,600
What do you think about the data?
131
00:04:37,600 --> 00:04:38,600
What do you think about the data?
132
00:04:38,600 --> 00:04:39,600
What do you think about the data?
133
00:04:39,600 --> 00:04:40,600
What do you think about the data?
134
00:04:40,600 --> 00:04:41,600
What do you think about the data?
135
00:04:41,600 --> 00:04:42,600
What do you think about the data?
136
00:04:42,600 --> 00:04:43,600
What do you think about the data?
137
00:04:43,600 --> 00:04:44,600
What do you think about the data?
138
00:04:44,600 --> 00:04:45,600
What do you think about the data?
139
00:04:45,600 --> 00:04:46,600
What do you think about the data?
140
00:04:46,600 --> 00:04:47,600
What do you think about the data?
141
00:04:47,600 --> 00:04:48,600
What do you think about the data?
142
00:04:48,600 --> 00:04:49,600
What do you think about the data?
143
00:04:49,600 --> 00:04:50,600
What do you think about the data?
144
00:04:50,600 --> 00:04:51,600
What do you think about the data?
145
00:04:51,600 --> 00:04:52,600
What do you think about the data?
146
00:04:52,600 --> 00:04:53,600
What do you think about the data?
147
00:04:53,600 --> 00:04:54,600
What do you think about the data?
148
00:04:54,600 --> 00:04:55,600
What do you think about the data?
149
00:04:55,600 --> 00:04:56,600
What do you think about the data?
150
00:04:56,600 --> 00:04:57,600
What do you think about the data?
151
00:04:57,600 --> 00:04:58,600
What do you think about the data?
152
00:04:58,600 --> 00:04:59,600
What do you think about the data?
153
00:04:59,600 --> 00:05:00,600
What do you think about the data?
154
00:05:00,600 --> 00:05:01,600
What do you think about the data?
155
00:05:01,600 --> 00:05:02,600
What do you think about the data?
156
00:05:02,600 --> 00:05:03,600
What do you think about the data?
157
00:05:03,600 --> 00:05:04,600
What do you think about the data?
158
00:05:04,600 --> 00:05:05,600
What do you think about the data?
159
00:05:05,600 --> 00:05:06,600
What do you think about the data?
160
00:05:06,600 --> 00:05:07,600
What do you think about the data?
161
00:05:07,600 --> 00:05:08,600
What do you think about the data?
162
00:05:08,600 --> 00:05:09,600
What do you think about the data?
163
00:05:09,600 --> 00:05:10,600
What do you think about the data?
164
00:05:10,600 --> 00:05:11,600
What do you think about the data?
165
00:05:11,600 --> 00:05:12,600
What do you think about the data?
166
00:05:12,600 --> 00:05:13,600
What do you think about the data?
167
00:05:13,600 --> 00:05:14,600
What do you think about the data?
168
00:05:14,600 --> 00:05:15,600
What do you think about the data?
169
00:05:15,600 --> 00:05:16,600
What do you think about the data?
170
00:05:16,600 --> 00:05:17,600
What do you think about the data?
171
00:05:17,600 --> 00:05:18,600
What do you think about the data?
172
00:05:18,600 --> 00:05:19,600
What do you think about the data?
173
00:05:19,600 --> 00:05:20,600
What do you think about the data?
174
00:05:20,600 --> 00:05:21,600
What do you think about the data?
175
00:05:21,600 --> 00:05:22,600
What do you think about the data?
176
00:05:22,600 --> 00:05:23,600
What do you think about the data?
177
00:05:23,600 --> 00:05:24,600
What do you think about the data?
178
00:05:24,600 --> 00:05:25,600
What do you think about the data?
179
00:05:25,600 --> 00:05:26,600
What do you think about the data?
180
00:05:26,600 --> 00:05:27,600
What do you think about the data?
181
00:05:27,600 --> 00:05:28,600
What do you think about the data?
182
00:05:28,600 --> 00:05:29,600
What do you think about the data?
183
00:05:29,600 --> 00:05:30,600
What do you think about the data?
184
00:05:30,600 --> 00:05:31,600
What do you think about the data?
185
00:05:31,600 --> 00:05:32,600
What do you think about the data?
186
00:05:32,600 --> 00:05:33,600
What do you think about the data?
187
00:05:33,600 --> 00:05:34,600
What do you think about the data?
188
00:05:34,600 --> 00:05:35,600
What do you think about the data?
189
00:05:35,600 --> 00:05:36,600
What do you think about the data?
190
00:05:36,600 --> 00:05:37,600
What do you think about the data?
191
00:05:37,600 --> 00:05:38,600
What do you think about the data?
192
00:05:38,600 --> 00:05:39,600
What do you think about the data?
193
00:05:39,600 --> 00:05:41,600
It handles two main tasks and lets walk through them.
194
00:05:41,600 --> 00:05:45,600
First, indexing, that's getting your data ready to be searched, second, searching.
195
00:05:45,600 --> 00:05:48,600
That's actually finding the right content when someone asks a question.
196
00:05:48,600 --> 00:05:50,600
Let's go through the indexing process first.
197
00:05:50,600 --> 00:05:53,600
You start by pointing Azure AI search at your data source,
198
00:05:53,600 --> 00:05:56,600
which could be a blob storage container full of PDFs,
199
00:05:56,600 --> 00:06:00,600
a SQL database with customer records or any number of other sources.
200
00:06:00,600 --> 00:06:04,600
The service reads those documents, breaks them into smaller pieces called chunks,
201
00:06:04,600 --> 00:06:06,600
creates vector embeddings for each chunk,
202
00:06:06,600 --> 00:06:09,600
using that embedding model we talked about earlier,
203
00:06:09,600 --> 00:06:11,600
and stores everything in a search index.
204
00:06:11,600 --> 00:06:14,600
Think of the index as a giant, highly organized catalog.
205
00:06:14,600 --> 00:06:17,600
Every piece of content gets a unique ID, a chunk of text,
206
00:06:17,600 --> 00:06:20,600
and a set of coordinates on that map of meaning.
207
00:06:20,600 --> 00:06:21,600
Now the searching process.
208
00:06:21,600 --> 00:06:23,600
When a user asks a question,
209
00:06:23,600 --> 00:06:26,600
AI search converts that question into a vector using the same embedding model,
210
00:06:26,600 --> 00:06:31,600
then scans the index to find chunks whose coordinates are closest to the questions coordinates.
211
00:06:31,600 --> 00:06:35,600
It returns the top matches, usually the five to ten most relevant ones,
212
00:06:35,600 --> 00:06:37,600
and does all of this in milliseconds.
213
00:06:37,600 --> 00:06:39,600
You get a few strong options here.
214
00:06:39,600 --> 00:06:41,600
Pure vector search finds content by meaning,
215
00:06:41,600 --> 00:06:44,600
so similar concepts show up even if the exact words don't match.
216
00:06:44,600 --> 00:06:48,600
Keyword search works like a traditional search engine, finding exact word matches.
217
00:06:48,600 --> 00:06:51,600
Or you can use hybrid search, which combines both approaches,
218
00:06:51,600 --> 00:06:53,600
and that's usually the best choice.
219
00:06:53,600 --> 00:06:55,600
There's also something called semantic re-ranking,
220
00:06:55,600 --> 00:06:59,600
which takes those initial results and resorts them based on how well they actually answer the question.
221
00:06:59,600 --> 00:07:02,600
It's like a second pass that catches things the first one might have missed.
222
00:07:02,600 --> 00:07:04,600
One thing you should know.
223
00:07:04,600 --> 00:07:06,600
Azure AI search can get expensive.
224
00:07:06,600 --> 00:07:10,600
The basic tier starts around $75 a month and climbs from there.
225
00:07:10,600 --> 00:07:12,600
If you're just experimenting, use the free tier,
226
00:07:12,600 --> 00:07:15,600
or make sure you delete your resources when you're done.
227
00:07:15,600 --> 00:07:18,600
For production use though, the cost usually pays off.
228
00:07:18,600 --> 00:07:21,600
You're buying speed and reliability at scale.
229
00:07:21,600 --> 00:07:25,600
Azure Open AI plus AI Foundry, Generation and Orchestration.
230
00:07:25,600 --> 00:07:27,600
So now your data is indexed and ready for retrieval,
231
00:07:27,600 --> 00:07:31,600
but the LLM still needs to understand your question and generate a solid answer.
232
00:07:31,600 --> 00:07:34,600
That's where Azure Open AI and Azure AI Foundry come in.
233
00:07:34,600 --> 00:07:36,600
Azure Open AI provides the actual language model,
234
00:07:36,600 --> 00:07:40,600
GPT-40, GPT-4.1, or whatever model you choose.
235
00:07:40,600 --> 00:07:42,600
This is the Generation Engine.
236
00:07:42,600 --> 00:07:45,600
It receives a prompt that includes both the original question from the user
237
00:07:45,600 --> 00:07:48,600
and the context that Azure AI search retrieved.
238
00:07:48,600 --> 00:07:52,600
The model reads through that context and produces an answer based on what it finds.
239
00:07:52,600 --> 00:07:55,600
Now you can't just dump everything in and hope for the best.
240
00:07:55,600 --> 00:07:59,600
You need a system prompt, a set of instructions you give the model upfront about how to behave.
241
00:07:59,600 --> 00:08:03,600
You tell it to use only the provided context to cite its sources.
242
00:08:03,600 --> 00:08:08,600
And most importantly to say, I don't know if the relevant information isn't in the retrieve documents.
243
00:08:08,600 --> 00:08:10,600
That's what keeps the model honest.
244
00:08:10,600 --> 00:08:15,600
Without a good system prompt, the model might fall back on its training data or worse, make something up.
245
00:08:15,600 --> 00:08:17,600
So how do you actually wire all these pieces together?
246
00:08:17,600 --> 00:08:19,600
That's where Azure AI Foundry comes in.
247
00:08:19,600 --> 00:08:24,600
It's the portal where you create a project, deploy your models, connect your data sources and test your agent.
248
00:08:24,600 --> 00:08:27,600
Think of it as the control center for your entire rag setup.
249
00:08:27,600 --> 00:08:33,600
You can add your Azure AI search index as a knowledge source to your agent without writing custom code for the basic setup.
250
00:08:33,600 --> 00:08:37,600
You just create a connection, select your index and the agent can start querying it automatically.
251
00:08:37,600 --> 00:08:39,600
The portal handles the orchestration.
252
00:08:39,600 --> 00:08:47,600
When a user asks a question, the agent knows to call AI search, get the relevant chunks and pass them to the LLM along with the original question.
253
00:08:47,600 --> 00:08:48,600
Here's a concrete example.
254
00:08:48,600 --> 00:08:53,600
In one tutorial I came across, someone built a movie recommendation agent using a Netflix data set.
255
00:08:53,600 --> 00:09:01,600
They uploaded a CSV file with movie titles, descriptions and cast information to Azure Blob Storage, then used Azure AI search to index that data.
256
00:09:01,600 --> 00:09:06,600
In Azure AI Foundry, they created an agent with GPT-4O and connected the AI search index.
257
00:09:06,600 --> 00:09:14,600
When a user asked, "I like thrillers, give me some recommendations," the agent searched the index, found thriller movies from that specific data set, and returned recommendations.
258
00:09:14,600 --> 00:09:15,600
And here's the key.
259
00:09:15,600 --> 00:09:17,600
It only recommended movies that were actually in that data set.
260
00:09:17,600 --> 00:09:22,600
It didn't invent anything, didn't pull from its training data and worked exclusively from the content they provided.
261
00:09:22,600 --> 00:09:24,600
That's the power of Ragn.
262
00:09:24,600 --> 00:09:26,600
Classic vs. AgenteGrag.
263
00:09:26,600 --> 00:09:27,600
Now let's talk about setting this up.
264
00:09:27,600 --> 00:09:32,600
There are two main patterns, classic Ragn and AgenteGrag, which one you pick depends on what you're building.
265
00:09:32,600 --> 00:09:34,600
Classic Ragn is the straightforward approach.
266
00:09:34,600 --> 00:09:37,600
One query goes in, one search runs, and one answer comes out.
267
00:09:37,600 --> 00:09:39,600
Simple, fast, predictable.
268
00:09:39,600 --> 00:09:43,600
Think of it like walking up to the reception desk in an office building and asking for one specific file.
269
00:09:43,600 --> 00:09:46,600
The receptionist grabs it, hands it to you, done.
270
00:09:46,600 --> 00:09:49,600
Classic Ragn works great for factual Q&A on a single topic.
271
00:09:49,600 --> 00:09:55,600
Like, what's the company policy on remote work? Or show me the troubleshooting steps for this error code?
272
00:09:55,600 --> 00:09:59,600
It's reliable, cheap, and easy to fix when something goes wrong because the flow is linear.
273
00:09:59,600 --> 00:10:03,600
AgenteGrag is different. The AI agent decides how to retrieve information.
274
00:10:03,600 --> 00:10:10,600
It can ask multiple subquestions, search different indexes, iterate on results, and synthesize across sources before answering.
275
00:10:10,600 --> 00:10:14,600
It's like having a building manager who coordinates with different departments to gather all the pieces.
276
00:10:14,600 --> 00:10:16,600
They don't just grab one file.
277
00:10:16,600 --> 00:10:21,600
They talk to HR, check records, cross-reference policies, and then come back with a complete answer.
278
00:10:21,600 --> 00:10:26,600
Microsoft ran their own tests and found AgenteGrag gives about a 40% improvement in answer relevance for complex questions.
279
00:10:26,600 --> 00:10:30,600
That's significant, but it uses 3 to 10 times more tokens and adds latency.
280
00:10:30,600 --> 00:10:33,600
For simple questions, that extra work is wasted.
281
00:10:33,600 --> 00:10:35,600
Classic Ragn handles those just fine.
282
00:10:35,600 --> 00:10:36,600
So when do you use each?
283
00:10:36,600 --> 00:10:40,600
Classic for routine, high volume queries where speed and cost matter most.
284
00:10:40,600 --> 00:10:45,600
AgenteG for multi-step ambiguous or cross-domain questions where accuracy is worth the extra compute.
285
00:10:45,600 --> 00:10:50,600
Connection section end to end example. Let's see how all these pieces fit together with a real example.
286
00:10:50,600 --> 00:10:55,600
Imagine you're building a customer support agent for a travel agency called Marygold Travels.
287
00:10:55,600 --> 00:11:00,600
You have trip brochures, hotel reviews, and policy documents scattered across your organization.
288
00:11:00,600 --> 00:11:06,600
You want customers to ask questions and get accurate answers based on your actual content, not generic guesses.
289
00:11:06,600 --> 00:11:09,600
Step 1. Store all that content as your blob storage.
290
00:11:09,600 --> 00:11:14,600
Brochures in one folder, reviews in another, policies in a third, just files in a container.
291
00:11:14,600 --> 00:11:19,600
Step 2. Use Azure AI search to index that data by running the import data wizard,
292
00:11:19,600 --> 00:11:22,600
pointing it at your blob storage and letting it work.
293
00:11:22,600 --> 00:11:28,600
The service reads every document, chunks them into pieces, creates vector embeddings, and builds a searchable index.
294
00:11:28,600 --> 00:11:29,600
The whole process takes a few minutes.
295
00:11:29,600 --> 00:11:32,600
Step 3. Go into Azure AI Foundry and create an agent.
296
00:11:32,600 --> 00:11:39,600
Deploy GPT-4.0 as your model, write a system prompt that tells it to use only the provided context and site its sources,
297
00:11:39,600 --> 00:11:42,600
then connect your Azure AI search index as a knowledge source.
298
00:11:42,600 --> 00:11:44,600
A few clicks and it's done.
299
00:11:44,600 --> 00:11:48,600
Now a customer asks, what's the best review hotel in Dubai under $300?
300
00:11:48,600 --> 00:11:51,600
The agent takes that question and sends it to Azure AI search.
301
00:11:51,600 --> 00:11:56,600
Azure AI search converts it to a vector and finds the most relevant chunks from your reviews and brochures.
302
00:11:56,600 --> 00:11:59,600
Those chunks get passed to GPT-4.0 along with the original question.
303
00:11:59,600 --> 00:12:05,600
The model reads through them, identifies the top match, and generates a response with citations pointing back to your source documents.
304
00:12:05,600 --> 00:12:08,600
Here's the thing, the LLM never memorizes that data.
305
00:12:08,600 --> 00:12:15,600
It never trained on your hotel reviews, it works from your specific content every single time, add new reviews tomorrow and the answers update automatically.
306
00:12:15,600 --> 00:12:18,600
Remove an outdated brochure and it stops showing up.
307
00:12:18,600 --> 00:12:21,600
The model doesn't need retraining, it just needs the right retrieval layer.
308
00:12:21,600 --> 00:12:23,600
So how do you actually start building Ragnar Azure?
309
00:12:23,600 --> 00:12:25,600
Let's break it down into three steps.
310
00:12:25,600 --> 00:12:27,600
First things first, know your data.
311
00:12:27,600 --> 00:12:33,600
Before you even pick a service, figure out where your content lives, how it's organized, and who should have access.
312
00:12:33,600 --> 00:12:35,600
Ragnar lives or dies on the quality of your data.
313
00:12:35,600 --> 00:12:38,600
Next, keep it simple, don't overcomplicate the setup.
314
00:12:38,600 --> 00:12:42,600
Use the import data wizard in Azure AI search to index just one set of documents.
315
00:12:42,600 --> 00:12:47,600
Test with basic queries. Make sure your retrieval works perfectly before you add the LLM.
316
00:12:47,600 --> 00:12:51,600
Most Ragn failures aren't model problems, they're retrieval problems.
317
00:12:51,600 --> 00:12:56,600
Last step, add the LLM. Once your search is returning the right results, then you layer in generation.
318
00:12:56,600 --> 00:12:59,600
That's when you connect as you open AI and write your system prompt.
319
00:12:59,600 --> 00:13:03,600
Here's the thing, Ragnar isn't something you buy, it's a pattern you build.
320
00:13:03,600 --> 00:13:08,600
Azure gives you the pieces, storage, search, models, and the glue to connect them.
321
00:13:08,600 --> 00:13:11,600
You just need to put it all together the right way.
322
00:13:11,600 --> 00:13:15,600
If this helped you see the big picture, hit subscribe for more play in English breakdowns.
323
00:13:15,600 --> 00:13:17,600
Drop a comment with your Ragnar use case.
324
00:13:17,600 --> 00:13:19,600
I'd love to know what you're building.
