The Copilot Assumption That's Holding You Back — Agentic RAG

1
00:00:00,000 --> 00:00:02,260
Before we get into it, quick ask, if you're new here,

2
00:00:02,260 --> 00:00:04,100
subscribe to M365FM.

3
00:00:04,100 --> 00:00:06,600
We break down Microsoft 365 like this every week,

4
00:00:06,600 --> 00:00:09,800
no fluff, just the stuff that actually changes how you work.

5
00:00:09,800 --> 00:00:11,600
Okay, here's the assumption.

6
00:00:11,600 --> 00:00:14,200
Most people think co-pilot's job is to answer questions.

7
00:00:14,200 --> 00:00:16,320
You type something in, it goes and finds a document,

8
00:00:16,320 --> 00:00:17,680
it gives you back an answer.

9
00:00:17,680 --> 00:00:18,500
Simple.

10
00:00:18,500 --> 00:00:20,000
Clean, that's what everyone assumes it does

11
00:00:20,000 --> 00:00:21,500
because that's what it looks like it does.

12
00:00:21,500 --> 00:00:22,960
That assumption is broken.

13
00:00:22,960 --> 00:00:25,200
And it's costing you the real value of the platform

14
00:00:25,200 --> 00:00:27,100
because co-pilot isn't a search box anymore.

15
00:00:27,100 --> 00:00:29,200
It's becoming something closer to an orchestrator,

16
00:00:29,200 --> 00:00:31,500
a system that decides where to look, how many times to look,

17
00:00:31,500 --> 00:00:33,000
and what to do once it finds something.

18
00:00:33,000 --> 00:00:35,200
Most people using co-pilot right now have no idea

19
00:00:35,200 --> 00:00:36,600
that shift already happened.

20
00:00:36,600 --> 00:00:37,600
By the end of this episode,

21
00:00:37,600 --> 00:00:39,800
you're going to understand a gentec rag well enough

22
00:00:39,800 --> 00:00:41,200
to spot it when you see it.

23
00:00:41,200 --> 00:00:43,100
Question it when someone pitches it to you

24
00:00:43,100 --> 00:00:45,600
and actually use it right instead of just turning it on

25
00:00:45,600 --> 00:00:46,800
and hoping.

26
00:00:46,800 --> 00:00:49,800
Let's get into why the old assumption held for so long.

27
00:00:49,800 --> 00:00:52,400
Why everyone thinks co-pilot is just search?

28
00:00:52,400 --> 00:00:54,700
Here's the mental model almost everyone walks in with.

29
00:00:54,700 --> 00:00:55,600
It comes from Google.

30
00:00:55,600 --> 00:00:56,900
It comes from SharePoint Search.

31
00:00:56,900 --> 00:00:58,000
You type a question,

32
00:00:58,000 --> 00:00:59,700
something goes and fetches the document

33
00:00:59,700 --> 00:01:00,900
and you get an answer back.

34
00:01:00,900 --> 00:01:02,700
Done. Transaction complete.

35
00:01:02,700 --> 00:01:04,700
And look, that model isn't wrong exactly.

36
00:01:04,700 --> 00:01:06,200
It's just incomplete.

37
00:01:06,200 --> 00:01:08,500
It's describing one specific architecture

38
00:01:08,500 --> 00:01:10,400
and that architecture has a name,

39
00:01:10,400 --> 00:01:12,100
retrieval augmented generation.

40
00:01:12,100 --> 00:01:13,800
Traditional rag, one shot, one answer.

41
00:01:13,800 --> 00:01:14,800
Let me walk you through the loop

42
00:01:14,800 --> 00:01:17,100
because once you see it, you'll recognize it everywhere.

43
00:01:17,100 --> 00:01:19,600
You ask a question that query gets converted into something

44
00:01:19,600 --> 00:01:20,800
called an embedding,

45
00:01:20,800 --> 00:01:23,100
basically a mathematical fingerprint of what you're asking.

46
00:01:23,100 --> 00:01:25,400
That fingerprint goes and searches a vector database

47
00:01:25,400 --> 00:01:27,600
which is just a specialized store of your documents

48
00:01:27,600 --> 00:01:29,300
chunked up in index the same way.

49
00:01:29,300 --> 00:01:31,400
Whatever chunks come back gets stuffed into a prompt

50
00:01:31,400 --> 00:01:32,700
along with your original question.

51
00:01:32,700 --> 00:01:35,000
That whole package gets sent to the language model

52
00:01:35,000 --> 00:01:36,900
and the model does exactly one thing with it.

53
00:01:36,900 --> 00:01:38,200
It generates an answer.

54
00:01:38,200 --> 00:01:40,300
That's it. That's the entire loop query,

55
00:01:40,300 --> 00:01:43,300
vector database context prompt, LLM output,

56
00:01:43,300 --> 00:01:45,200
one pass, front to back and you're done.

57
00:01:45,200 --> 00:01:46,100
And here's the thing.

58
00:01:46,100 --> 00:01:49,500
This actually works great for a huge chunk of what people ask.

59
00:01:49,500 --> 00:01:50,700
What's our PTO policy?

60
00:01:50,700 --> 00:01:52,800
What's the Wi-Fi password for the Denver office?

61
00:01:52,800 --> 00:01:54,300
Where do I submit an expense report?

62
00:01:54,300 --> 00:01:55,800
These are FAQ style questions.

63
00:01:55,800 --> 00:01:56,600
One clear question.

64
00:01:56,600 --> 00:01:57,800
One document that has the answer,

65
00:01:57,800 --> 00:01:59,400
no ambiguity about where to look.

66
00:01:59,400 --> 00:02:01,200
Traditional rag handles this beautifully.

67
00:02:01,200 --> 00:02:03,700
Fast, cheap, predictable.

68
00:02:03,700 --> 00:02:05,900
But here's where most people miss something important.

69
00:02:05,900 --> 00:02:08,700
In that entire loop, the language model only gets used once

70
00:02:08,700 --> 00:02:11,300
and only for one job, summarizing what got retrieved.

71
00:02:11,300 --> 00:02:12,500
It's not deciding anything.

72
00:02:12,500 --> 00:02:14,400
It's not choosing which database to check.

73
00:02:14,400 --> 00:02:16,500
It's not looking at the first result and thinking,

74
00:02:16,500 --> 00:02:18,700
"This isn't enough. Let me try again."

75
00:02:18,700 --> 00:02:21,300
It's a passive tool sitting at the end of a pipeline,

76
00:02:21,300 --> 00:02:22,900
waiting to be handed context,

77
00:02:22,900 --> 00:02:24,400
so it can write a nice paragraph.

78
00:02:24,400 --> 00:02:25,900
Think about what that actually means.

79
00:02:25,900 --> 00:02:27,400
All the intelligence in the system,

80
00:02:27,400 --> 00:02:29,000
the reasoning, the language understanding,

81
00:02:29,000 --> 00:02:31,200
the thing that makes an LLM, an LLM,

82
00:02:31,200 --> 00:02:33,700
gets used for exactly one narrow task.

83
00:02:33,700 --> 00:02:35,100
Summarization.

84
00:02:35,100 --> 00:02:36,700
Everything before that point,

85
00:02:36,700 --> 00:02:39,100
the actual decision of what to retrieve and where,

86
00:02:39,100 --> 00:02:40,100
is hard-coded.

87
00:02:40,100 --> 00:02:42,200
A developer wrote that logic in Python months ago

88
00:02:42,200 --> 00:02:43,700
and it hasn't changed since.

89
00:02:43,700 --> 00:02:46,500
That's fine when the question is simple in the source is obvious.

90
00:02:46,500 --> 00:02:47,700
But that setup has a ceiling,

91
00:02:47,700 --> 00:02:50,100
and the moment a question doesn't fit neatly into one query,

92
00:02:50,100 --> 00:02:52,500
one document, the whole thing starts to wobble.

93
00:02:52,500 --> 00:02:54,000
The moment rag breaks.

94
00:02:54,000 --> 00:02:56,200
So let's test that ceiling with two questions.

95
00:02:56,200 --> 00:02:58,400
First one, what's our remote work policy?

96
00:02:58,400 --> 00:02:59,000
Easy.

97
00:02:59,000 --> 00:03:00,700
There's a document somewhere in SharePoint

98
00:03:00,700 --> 00:03:02,300
called Remote Work Policy,

99
00:03:02,300 --> 00:03:04,300
or it's a section in the Employee Handbook.

100
00:03:04,300 --> 00:03:06,000
One query, one source, one clean answer.

101
00:03:06,000 --> 00:03:08,300
Traditional rag nails this every time.

102
00:03:08,300 --> 00:03:09,200
Now try this one.

103
00:03:09,200 --> 00:03:11,500
Why did my invoice go up this month?

104
00:03:11,500 --> 00:03:14,700
Stop and actually think about what has to happen to answer that well.

105
00:03:14,700 --> 00:03:15,900
You need the current invoice.

106
00:03:15,900 --> 00:03:18,800
You need last month's invoice to know what changed.

107
00:03:18,800 --> 00:03:21,300
You probably need usage data because maybe the increases

108
00:03:21,300 --> 00:03:23,400
from higher consumption, not a price change.

109
00:03:23,400 --> 00:03:27,600
You might need the contract terms in case there's a rate adjustment clause that kicked in.

110
00:03:27,600 --> 00:03:29,100
That's four different sources minimum

111
00:03:29,100 --> 00:03:30,900
and none of them live in the same place.

112
00:03:30,900 --> 00:03:32,400
Traditional rag can't do that.

113
00:03:32,400 --> 00:03:35,100
Not because the model isn't smart enough to reason through it,

114
00:03:35,100 --> 00:03:37,600
but because the architecture never gives it the chance.

115
00:03:37,600 --> 00:03:41,200
Remember the loop from a minute ago query goes to the vector database.

116
00:03:41,200 --> 00:03:43,700
Whatever comes back is what the model gets end of story.

117
00:03:43,700 --> 00:03:45,800
There's no step where the system pauses and asks,

118
00:03:45,800 --> 00:03:48,400
wait, do I actually have everything I need here?

119
00:03:48,400 --> 00:03:50,000
It just looks where it's told to look,

120
00:03:50,000 --> 00:03:52,300
gets one batch of chunks and hands them off.

121
00:03:52,300 --> 00:03:55,600
This is the actual structural floor and it's worth saying plainly.

122
00:03:55,600 --> 00:03:59,900
Single-shot retrieval assumes the question is simple and assumes the source is obvious.

123
00:03:59,900 --> 00:04:01,800
That assumption holds for PTO policies.

124
00:04:01,800 --> 00:04:06,000
It falls apart completely for anything that requires connecting pieces across systems.

125
00:04:06,000 --> 00:04:07,600
And here's what that costs you in practice.

126
00:04:07,600 --> 00:04:10,500
It's not just that copilot gets a slightly worse answer.

127
00:04:10,500 --> 00:04:13,400
It's one of three things and none of them are good, wrong answer,

128
00:04:13,400 --> 00:04:16,000
where it retrieves the invoice but not the usage data

129
00:04:16,000 --> 00:04:19,000
and gives you a confident explanation that's flat out incorrect.

130
00:04:19,000 --> 00:04:22,300
In complete answer where it grabs part of the picture and presents it like the whole picture

131
00:04:22,300 --> 00:04:25,300
so you walk away thinking you understand what happened when you don't

132
00:04:25,300 --> 00:04:28,600
or the one that actually damages trust the fastest.

133
00:04:28,600 --> 00:04:29,800
I don't know.

134
00:04:29,800 --> 00:04:32,000
Copilot tells you it can't find the information

135
00:04:32,000 --> 00:04:34,200
even though the answer exists somewhere in your tenant.

136
00:04:34,200 --> 00:04:35,800
It's just spread across sources.

137
00:04:35,800 --> 00:04:37,800
The single-shot pipeline was never built to check.

138
00:04:37,800 --> 00:04:40,800
That last one is the killer because people don't blame the architecture.

139
00:04:40,800 --> 00:04:41,800
They blame the tool.

140
00:04:41,800 --> 00:04:43,300
They say copilot's not good enough.

141
00:04:43,300 --> 00:04:45,100
Copilot doesn't have access to our data.

142
00:04:45,100 --> 00:04:46,800
Copilot isn't ready for real work,

143
00:04:46,800 --> 00:04:48,300
but the data was there the whole time.

144
00:04:48,300 --> 00:04:51,400
The retrieval step just never went looking for it because looking again

145
00:04:51,400 --> 00:04:52,500
wasn't part of the design.

146
00:04:52,500 --> 00:04:54,500
So here's the question that actually matters.

147
00:04:54,500 --> 00:04:59,000
What if the model wasn't locked into look once, wherever you're told and report back?

148
00:04:59,000 --> 00:05:01,300
What if instead it could look at the question,

149
00:05:01,300 --> 00:05:03,400
decide for itself what it actually needs,

150
00:05:03,400 --> 00:05:06,100
go get the first piece, evaluate whether that's enough

151
00:05:06,100 --> 00:05:08,600
and if it's not, go look again somewhere else.

152
00:05:08,600 --> 00:05:09,800
That's not a hypothetical.

153
00:05:09,800 --> 00:05:14,200
That's a different architecture entirely and it's what we're covering next.

154
00:05:14,200 --> 00:05:16,100
What a genetic rag actually is.

155
00:05:16,100 --> 00:05:18,000
Here's the definition, plain and simple.

156
00:05:18,000 --> 00:05:21,200
A genetic rag turns the LLM from a responder into a decision maker.

157
00:05:21,200 --> 00:05:22,900
That's the whole shift right there.

158
00:05:22,900 --> 00:05:26,300
In the old model, the language model waits for context and writes a summary.

159
00:05:26,300 --> 00:05:30,100
In this model, the language model decides what context it even needs.

160
00:05:30,100 --> 00:05:32,200
It chooses which data source to query first.

161
00:05:32,200 --> 00:05:34,800
It looks at what comes back and decides whether that's enough

162
00:05:34,800 --> 00:05:38,300
or whether it needs to go query again somewhere else with a different question.

163
00:05:38,300 --> 00:05:39,700
It's not a passive endpoint anymore.

164
00:05:39,700 --> 00:05:40,800
It's running the show.

165
00:05:40,800 --> 00:05:43,100
There's a name for how this actually works.

166
00:05:43,100 --> 00:05:45,700
And it's worth knowing because you'll see it referenced constantly

167
00:05:45,700 --> 00:05:47,500
if you read anything technical about this.

168
00:05:47,500 --> 00:05:48,500
The React pattern.

169
00:05:48,500 --> 00:05:51,400
Reason, then Act, then Reason again.

170
00:05:51,400 --> 00:05:52,900
The model thinks through what it needs.

171
00:05:52,900 --> 00:05:55,200
Takes an action, an actual retrieval or a tool call.

172
00:05:55,200 --> 00:05:58,900
Looks at the result and reasons about that result before deciding its next move.

173
00:05:58,900 --> 00:06:02,700
Reason, Act, Reason, Act in a loop until it decides it has enough to answer.

174
00:06:02,700 --> 00:06:05,700
Put the two models side by side and the difference is stark.

175
00:06:05,700 --> 00:06:07,500
Old model, one shot look up.

176
00:06:07,500 --> 00:06:11,200
Ask, retrieve, generate, done, no matter what.

177
00:06:11,200 --> 00:06:14,000
New model, an iterative, self-correcting loop

178
00:06:14,000 --> 00:06:16,600
where the system can recognize this isn't enough

179
00:06:16,600 --> 00:06:20,500
and go back for more on its own without a developer having to hard code

180
00:06:20,500 --> 00:06:23,000
that specific path in advance.

181
00:06:23,000 --> 00:06:26,600
Now here's why this matters so much specifically for Microsoft 365

182
00:06:26,600 --> 00:06:28,800
and not just as an abstract AI concept.

183
00:06:28,800 --> 00:06:31,400
Think about where your organization's actual knowledge lives,

184
00:06:31,400 --> 00:06:33,400
policies and project docs in SharePoint,

185
00:06:33,400 --> 00:06:36,000
conversations and decisions buried in Teams threads,

186
00:06:36,000 --> 00:06:38,100
commitments and context sitting in Outlook,

187
00:06:38,100 --> 00:06:40,000
structured business data in Dataverse,

188
00:06:40,000 --> 00:06:41,800
maybe a CRM, maybe a ticketing system.

189
00:06:41,800 --> 00:06:43,800
None of these were designed to be searched the same way.

190
00:06:43,800 --> 00:06:45,400
SharePoint is document-centric.

191
00:06:45,400 --> 00:06:47,400
Teams is conversational and time-ordered.

192
00:06:47,400 --> 00:06:49,200
Outlook is threaded and personal.

193
00:06:49,200 --> 00:06:52,400
Dataverse is structured, relational, nothing like a document at all.

194
00:06:52,400 --> 00:06:55,400
A single-shot retrieval system has to pick one of these in advance,

195
00:06:55,400 --> 00:06:57,800
hard-coded and hope the answer lives there.

196
00:06:57,800 --> 00:07:00,300
That's the whole problem we walked through with the invoice question.

197
00:07:00,300 --> 00:07:01,400
The answer wasn't missing.

198
00:07:01,400 --> 00:07:05,400
It was just scattered across systems that don't speak the same retrieval language.

199
00:07:05,400 --> 00:07:08,300
An agentic system doesn't need that upfront guess.

200
00:07:08,300 --> 00:07:11,200
It can look at your question, reason about what kind of information

201
00:07:11,200 --> 00:07:13,800
would actually answer it and go check SharePoint first.

202
00:07:13,800 --> 00:07:15,800
And if that's thin, pivot to Dataverse.

203
00:07:15,800 --> 00:07:18,800
And if it still needs more, pull recent Teams context too.

204
00:07:18,800 --> 00:07:21,300
Multiple sources evaluated in sequence,

205
00:07:21,300 --> 00:07:23,800
not a single role of the dice on one database.

206
00:07:23,800 --> 00:07:27,400
This is the actual architectural leap that Microsoft has been building toward.

207
00:07:27,400 --> 00:07:29,600
It's not co-pilot got smarter at writing.

208
00:07:29,600 --> 00:07:33,600
It's co-pilot getting the ability to make retrieval decisions the way a person would.

209
00:07:33,600 --> 00:07:37,800
If you asked a coworker to go find something and they knew which system to check first

210
00:07:37,800 --> 00:07:40,800
and knew to keep looking if the first place came up empty.

211
00:07:40,800 --> 00:07:45,300
So now the natural question is, what does this actually look like in practice step by step

212
00:07:45,300 --> 00:07:47,100
inside a real co-pilot session?

213
00:07:47,100 --> 00:07:51,100
Because the model makes decisions is easy to say and much less obvious to picture.

214
00:07:51,100 --> 00:07:54,400
Let's walk through the orchestration loop itself piece by piece

215
00:07:54,400 --> 00:07:57,800
and see exactly where the reasoning happens and where the action happens.

216
00:07:57,800 --> 00:08:01,100
The orchestration loop, step by step,

217
00:08:01,100 --> 00:08:03,100
picture a real question hitting co-pilot,

218
00:08:03,100 --> 00:08:07,600
something like, pull together everything we know about the meridian account before my call in an hour.

219
00:08:07,600 --> 00:08:11,600
The first thing that happens isn't retrieval, it's query understanding, the system passing

220
00:08:11,600 --> 00:08:15,600
what you're actually asking for, not just the words but the intent behind them.

221
00:08:15,600 --> 00:08:18,600
Everything we know isn't a search term, it's a scope.

222
00:08:18,600 --> 00:08:22,600
Co-pilot has to figure out that scope means recent emails and you open documents

223
00:08:22,600 --> 00:08:24,600
maybe a contract maybe notes from the last team's call,

224
00:08:24,600 --> 00:08:28,100
that's already more interpretation than a traditional rag system ever attempts.

225
00:08:28,100 --> 00:08:31,600
Then comes the part that didn't exist before, planning.

226
00:08:31,600 --> 00:08:34,600
This is where the model decides what it needs before it goes looking.

227
00:08:34,600 --> 00:08:37,600
Not after, before it's building something like a mental checklist,

228
00:08:37,600 --> 00:08:39,600
contract terms probably in SharePoint,

229
00:08:39,600 --> 00:08:41,600
recent conversation history, probably an outlook or teams,

230
00:08:41,600 --> 00:08:44,600
account status maybe in a connected CRM.

231
00:08:44,600 --> 00:08:49,600
The planning step is the model committing to a strategy instead of just firing off one query and hoping.

232
00:08:49,600 --> 00:08:52,600
Once that plan exists, you get multi source retrieval

233
00:08:52,600 --> 00:08:56,600
and this is where it starts looking genuinely different from anything traditional rag could do.

234
00:08:56,600 --> 00:09:00,600
Co-pilot might hit SharePoint first, pull the contract, check it against what it's looking for,

235
00:09:00,600 --> 00:09:01,600
then it goes to Graph.

236
00:09:01,600 --> 00:09:05,600
Microsoft's connective layer that ties together your emails, calendar, chats and files

237
00:09:05,600 --> 00:09:08,600
to pull recent interactions tied to that account.

238
00:09:08,600 --> 00:09:13,600
If there's a connector, set up to a line of business system, maybe a CRM holding deal status or support tickets,

239
00:09:13,600 --> 00:09:14,600
it goes there next.

240
00:09:14,600 --> 00:09:18,600
Three separate systems, three separate retrieval calls, all in service of one question.

241
00:09:18,600 --> 00:09:21,600
Now here's the part that actually matters most, the loop.

242
00:09:21,600 --> 00:09:24,600
Say the first pass through SharePoint only turns up an old version of the contract.

243
00:09:24,600 --> 00:09:28,600
A traditional system would just hand that over and call it done, wrong or not.

244
00:09:28,600 --> 00:09:31,600
An agentex system evaluates what it got back.

245
00:09:31,600 --> 00:09:34,600
It asks essentially, does this actually answer what I was asked?

246
00:09:34,600 --> 00:09:37,600
If the answer is no or even not fully, it doesn't stop.

247
00:09:37,600 --> 00:09:43,600
It refines the query, maybe narrows the date range, maybe searches for an amendment instead of the original contract and tries again.

248
00:09:43,600 --> 00:09:47,600
It can do this multiple times, adjusting each pass based on what the last one revealed.

249
00:09:47,600 --> 00:09:49,600
That loop is the entire ball game.

250
00:09:49,600 --> 00:09:52,600
It's fundamentally different from a single vector database lookup,

251
00:09:52,600 --> 00:09:55,600
because a single lookup has no concept of good enough.

252
00:09:55,600 --> 00:09:58,600
It just returns whatever is closest to the query embedding and moves on.

253
00:09:58,600 --> 00:10:01,600
There's no evaluation step, no second attempt, no course correction.

254
00:10:01,600 --> 00:10:03,600
One shot, whatever you get is what you get.

255
00:10:03,600 --> 00:10:08,600
After the retrieval loop settles, once the system decides it actually has what it needs, you hit tool execution.

256
00:10:08,600 --> 00:10:14,600
This might mean running a calculation, checking a status field, formatting data from a structured source into something readable.

257
00:10:14,600 --> 00:10:18,600
Then summarization, where the model finally does what people assume it's been doing the whole time,

258
00:10:18,600 --> 00:10:21,600
turning everything gathered into a clear readable answer.

259
00:10:21,600 --> 00:10:26,600
And last, safety checks verifying the response doesn't expose something it shouldn't, respects permissions,

260
00:10:26,600 --> 00:10:29,600
and stays inside whatever governance rules apply to that data.

261
00:10:29,600 --> 00:10:35,600
Query understanding, planning, multi source retrieval, tool execution, summarization, safety checks, six stages, not two.

262
00:10:35,600 --> 00:10:38,600
This is the react pattern we named earlier actually running.

263
00:10:38,600 --> 00:10:41,600
Reason about what's needed, act by retrieving from a source.

264
00:10:41,600 --> 00:10:44,600
Reason about whether that's enough, act again if it's not.

265
00:10:44,600 --> 00:10:48,600
Every step in that pipeline is either a reasoning moment or an action moment alternating.

266
00:10:48,600 --> 00:10:53,600
Until the loop resolves itself, that's the mechanical difference between the two architectures laid out step by step.

267
00:10:53,600 --> 00:10:59,600
And once you can see the loop like this, you start to understand why this isn't just a faster or smarter version of search.

268
00:10:59,600 --> 00:11:05,600
It's a different category of system entirely, and that difference has consequences way beyond how fast you get an answer.

269
00:11:05,600 --> 00:11:07,600
Why this isn't just a technical detail.

270
00:11:07,600 --> 00:11:13,600
Now step back from the mechanics for a second, because if you're the person making budget decisions or signing off on rollout, none of what we just walk through matters.

271
00:11:13,600 --> 00:11:16,600
If you can't answer one question, so what?

272
00:11:16,600 --> 00:11:22,600
Here's the so what? This isn't an engineering nuance. It's not a detail for the IT team to sort out while everyone else keeps working the same way.

273
00:11:22,600 --> 00:11:25,600
This changes what co-pilot can actually be trusted to do.

274
00:11:25,600 --> 00:11:29,600
Under the old model, co-pilot answers questions. That's the entire job description.

275
00:11:29,600 --> 00:11:33,600
You ask, it retrieves, it summarizes, you read the output and decide what to do with it.

276
00:11:33,600 --> 00:11:37,600
The human stays in the loop for every single action that follows.

277
00:11:37,600 --> 00:11:42,600
Co-pilot hands you information and you're the one who acts on it. Under the new model, co-pilot completes tasks.

278
00:11:42,600 --> 00:11:55,600
That helps you find the information you'd need to complete the task. Complets it, pulls the contract, checks the account history, cross references the CRM, drafts the summary, maybe even roots it to the right person, all without you touching each individual step.

279
00:11:55,600 --> 00:12:00,600
That's not a faster question answering tool. That's a different category of system, which is really the shift happening here.

280
00:12:00,600 --> 00:12:07,600
Co-pilot is moving from a Q&A tool towards something a lot closer to a digital employee, not in some sci-fi sense. Nothing mysterious about it just practically.

281
00:12:07,600 --> 00:12:11,600
An employee doesn't wait for you to specify exactly which database to check.

282
00:12:11,600 --> 00:12:16,600
You hand them a goal and they figure out the path. That's what the orchestration loop from the last section is actually doing.

283
00:12:16,600 --> 00:12:25,600
It's the difference between a tool you operate and a system you delegate to and that word delegate is exactly where this gets uncomfortable for anyone responsible for governance.

284
00:12:25,600 --> 00:12:32,600
Because the moment co-pilot can act instead of just answer, a new question shows up that never existed with traditional reg, who's accountable for what it does.

285
00:12:32,600 --> 00:12:36,600
If co-pilot answers a question wrong, that's a bad answer, annoying but contained.

286
00:12:36,600 --> 00:12:44,600
If co-pilot completes a task wrong, sends the wrong file, pulls data it shouldn't have touched, roots an approval to the wrong person, that's not a bad answer anymore.

287
00:12:44,600 --> 00:12:47,600
That's an action with consequences and somebody has to own it.

288
00:12:47,600 --> 00:12:53,600
Not the model, a person, a policy, a framework that was actually thought through before the agent started acting on its own.

289
00:12:53,600 --> 00:12:58,600
We're going to come back to that governance question directly because it deserves its own real conversation, not a footnote.

290
00:12:58,600 --> 00:13:04,600
But before we get there, it's worth grounding all of this in something harder than architecture diagrams, actual performance numbers.

291
00:13:04,600 --> 00:13:10,600
Because right now this whole conversation has been conceptual and conceptual arguments are exactly the kind of thing that fall apart.

292
00:13:10,600 --> 00:13:13,600
The second someone asks, "Okay, but does it actually work better?"

293
00:13:13,600 --> 00:13:15,600
Let's look at what the data says.

294
00:13:15,600 --> 00:13:18,600
The performance data nobody is talking about.

295
00:13:18,600 --> 00:13:21,600
Let's start with the number that gets people's attention fastest.

296
00:13:21,600 --> 00:13:30,600
Agenetic Rags shows 2.8 times higher task completion on long horizon tasks, meaning anything requiring more than 10 steps to resolve, not a marginal improvement nearly triple.

297
00:13:30,600 --> 00:13:41,600
Think about what that means in practice. A task like the Meridian account example from earlier, pulling contract terms, cross referencing conversation history, checking CRM status, that's a long horizon task by definition.

298
00:13:41,600 --> 00:13:43,600
It's not one lookup, it's a chain of them.

299
00:13:43,600 --> 00:13:49,600
And on that kind of work, traditional single-shot retrieval just falls over far more often than the agentic loop does.

300
00:13:49,600 --> 00:13:52,600
Nearly three times more often, according to the benchmark data.

301
00:13:52,600 --> 00:13:55,600
Then there's the number that matters even more if you care about trust.

302
00:13:55,600 --> 00:14:01,600
Hallucination Reduction - Agenetic Rags cuts hallucinations by 47% compared to traditional rag.

303
00:14:01,600 --> 00:14:06,600
Almost half, that's not a small footnote. Hallucination is the thing that actually kills adoption inside organizations.

304
00:14:06,600 --> 00:14:11,600
Because one confidently wrong answer does more damage than 10 honest, I don't know responses.

305
00:14:11,600 --> 00:14:17,600
And the reason agentic architectures cut that number nearly in half comes straight from the loop we walked through earlier.

306
00:14:17,600 --> 00:14:26,600
The evaluation step, where the model checks whether what it retrieved actually answers the question, catches a huge share of the errors that a single-pass system would have just shipped straight to you.

307
00:14:26,600 --> 00:14:28,600
So far this sounds like a clean win, it isn't.

308
00:14:28,600 --> 00:14:32,600
Here's the trade-off. And it needs to be named immediately, not buried three sections later.

309
00:14:32,600 --> 00:14:36,600
Agenetic Rags is 2-10 times more expensive and noticeably slower.

310
00:14:36,600 --> 00:14:43,600
That's a real range and it's wide for a reason because the cost depends entirely on how many reasoning loops a given task actually requires.

311
00:14:43,600 --> 00:14:49,600
But even at the low end you're paying double for every query that goes through this pipeline instead of the traditional one.

312
00:14:49,600 --> 00:14:52,600
One benchmark makes this trade-off concrete in a way that's worth sitting with.

313
00:14:52,600 --> 00:14:59,600
Iterative Agent Rag, the fully looping, self-correcting version, came in roughly 10 times more expensive and four times slower than traditional retrieval.

314
00:14:59,600 --> 00:15:03,600
But it also cut the hallucination rate from 23% down to 8%.

315
00:15:03,600 --> 00:15:08,600
Same task, same underlying data, wildly different cost and wildly different reliability.

316
00:15:08,600 --> 00:15:13,600
So here's what that trade-off actually forces you to confront. It's not a question of which architecture is better, full stop.

317
00:15:13,600 --> 00:15:19,600
That framing doesn't survive contact with the numbers. A system that's 10 times more expensive isn't better.

318
00:15:19,600 --> 00:15:23,600
It's better for something specific at a cost that has to be worth paying.

319
00:15:23,600 --> 00:15:27,600
The real question isn't which is better. It's which task deserves which architecture.

320
00:15:27,600 --> 00:15:35,600
Because if you're answering a PTO question, paying 10 times more and waiting four times longer for a marginal accuracy gain is a bad trade every time.

321
00:15:35,600 --> 00:15:43,600
You're making a decision that involves real money or real risk and getting a wrong cost you a client or a compliance violation that same trade-off flips completely.

322
00:15:43,600 --> 00:15:51,600
The extra cost isn't overhead anymore. It's insurance. That reframing architecture matched to task, not architecture chosen by default is the thing most people skip.

323
00:15:51,600 --> 00:15:58,600
And it's exactly why the next place this breaks down isn't in the data. It's in the assumption that a GENTIK is simply the upgrade.

324
00:15:58,600 --> 00:16:07,600
When traditional rags still wins. So let's push back on the hype for a second because everything we've covered so far could easily be read as "agentic rag wins" always end of story.

325
00:16:07,600 --> 00:16:14,600
It doesn't and treating it that way is exactly how organizations waste money. Take a simple factoid query. What's the guest Wi-Fi password?

326
00:16:14,600 --> 00:16:22,600
What time does the Denver office close on Fridays? Run that through an agentic pipeline and you get what? A planning step deciding it needs to check a source.

327
00:16:22,600 --> 00:16:31,600
A retrieval call. An evaluation step asking whether the result is sufficient. All of that reasoning overhead for a question with one obvious answer sitting in one obvious document.

328
00:16:31,600 --> 00:16:34,600
Not only does that add cost for no reason, it can actually make things worse.

329
00:16:34,600 --> 00:16:42,600
Slower. For the exact same answer you would have gotten from a single look up in a fraction of the time. The agentic loop doesn't know the question was simple.

330
00:16:42,600 --> 00:16:50,600
It runs its full process anyway unless something upstream tells it not to. So you end up paying the latency tax and the compute tax on a question that never needed either.

331
00:16:50,600 --> 00:17:05,600
And here's where it gets expensive fast, not slowly. Multiply that overhead by volume. If every single query coming into your co-pilot deployment triggers a full reasoning loop, planning, multi source retrieval, evaluation, possible requering, your costs aren't going up by some fixed percentage.

332
00:17:05,600 --> 00:17:11,600
They're multiplying because each step in that loop is its own model call and model calls are what you're paying for.

333
00:17:11,600 --> 00:17:23,600
A help desk, fielding a thousand basic questions a day, doesn't need a thousand reasoning loops. It needs a thousand fast, cheap lookups. So the actual skill here isn't implement a genetic rag. That's not a skill that's a checkbox.

334
00:17:23,600 --> 00:17:30,600
The real skill is knowing inside your own organization, which questions are simple lookups and which are genuine multi-step reasoning problems.

335
00:17:30,600 --> 00:17:37,600
That's a triage job and it's harder than it sounds because the surface wording of a question doesn't always tell you which category it belongs to.

336
00:17:37,600 --> 00:17:47,600
What's my pto balance is simple, why did my request get denied might not be get that triage wrong in one direction and you're running expensive reasoning loops on questions that never needed them.

337
00:17:47,600 --> 00:17:56,600
Get it wrong in the other direction and you're back to the invoice problem from earlier where a single short system gives you a wrong or incomplete answer because the question actually needed to reach into three different sources.

338
00:17:56,600 --> 00:18:05,600
This is exactly why almost nobody builds a system that's purely one or purely the other, which is why most companies whether they say it out loud or not, land on a hybrid approach.

339
00:18:05,600 --> 00:18:11,600
Hybrid rag, the quiet enterprise default. Here's something that doesn't get said enough in conversations about this.

340
00:18:11,600 --> 00:18:21,600
Most production systems running today aren't fully agentic. They're hybrid, quietly without much fanfare. Hybrid rag has become the default architecture almost everywhere. This is deployed seriously.

341
00:18:21,600 --> 00:18:30,600
The logic is straight forward once you see it. Simple queries get routed to fast, cheap, traditional retrieval, one embedding, one vector lookup, one pass through the model for summarization.

342
00:18:30,600 --> 00:18:42,600
No planning step, no evaluation loop, no multiple retrieval attempts, just the classic pipeline, doing what it's always done well, complex or ambiguous queries get routed somewhere else entirely. The agentic path with full planning and iteration.

343
00:18:42,600 --> 00:18:49,600
That's where the reasoning loop we walked through earlier actually kicks in checking multiple sources, evaluating whether results are sufficient, requeering when they're not.

344
00:18:49,600 --> 00:18:56,600
What makes this work isn't luck. It's that the rooting decision itself becomes something someone actually designed, not something that happens by accident.

345
00:18:56,600 --> 00:19:03,600
Somewhere in the system there's a classification step, a moment where the incoming query gets assessed for complexity before it ever reaches a retrieval pipeline.

346
00:19:03,600 --> 00:19:10,600
Is this a single hop factoid or does it look like it needs reasoning across sources? That classification determines which path the query takes.

347
00:19:10,600 --> 00:19:21,600
Get that routing logic right and you capture the speed and cost benefits of traditional rag on the majority of queries, while still getting the accuracy gains of a gentic rag on the smaller share of questions that actually justify it.

348
00:19:21,600 --> 00:19:26,600
Now tie that back to copilot specifically because this isn't an abstract enterprise pattern happening somewhere else.

349
00:19:26,600 --> 00:19:38,600
It's effectively the direction Microsoft is building toward with wave three. Instead of treating every query the same way, running everything through one fixed pipeline, regardless of complexity, the architecture is shifting toward tiered response strategies.

350
00:19:38,600 --> 00:19:47,600
Simple asks get handled quickly. Anything that looks like it needs multi-step reasoning gets escalated into the fuller orchestration loop with planning, multi source retrieval and iteration.

351
00:19:47,600 --> 00:19:55,600
That's not a compromise between two competing architectures. It's the recognition that neither one on its own is the right answer for everything copilot gets asked to do.

352
00:19:55,600 --> 00:20:04,600
Copilot wave three, what actually changed? So let's actually name what's happening here because wave three sounds like marketing language until you look at what it's describing underneath.

353
00:20:04,600 --> 00:20:13,600
Copilot started as a chat assistant, you typed, it responded, the conversation stayed inside a single box on your screen. That's the version most people still picture when they hear the name.

354
00:20:13,600 --> 00:20:26,600
But that version has been quietly replaced by something else, an autonomous multi-step executor, not a chat window anymore, a system that can take a goal, break it into steps and carry those steps out across multiple apps without unirating each one.

355
00:20:26,600 --> 00:20:33,600
That's the actual shift wave three signals and it's worth putting in plain words because Microsoft's own language tends to stay pretty abstract about it.

356
00:20:33,600 --> 00:20:36,600
Less ask and receive, more delegate and verify.

357
00:20:36,600 --> 00:20:48,600
You're not typing a question and waiting for a paragraph back. You're handing off a task and checking the result afterward. The work happens in between and you're not watching it happen step by step anymore. That's a genuinely different relationship with the tool.

358
00:20:48,600 --> 00:20:54,600
Ask and receive keeps you in the loop constantly. Every query, every answer, every next question you type based on what came back.

359
00:20:54,600 --> 00:21:06,600
You get and verify pulls you out of the middle of the process and puts you at the start and the end of it instead. You set the goal, you check the outcome. Everything in between is the orchestration loop we spent the last few sections walking through.

360
00:21:06,600 --> 00:21:18,600
Now, none of this is theoretical or early stage anymore and that's the part worth sitting with for a second. Microsoft 365 Copilot has more than 20 million paid enterprise users, more than 60% of the Fortune 500 are already running it.

361
00:21:18,600 --> 00:21:34,600
This is deployed at scale inside the majority of the largest companies in the world. Whatever architecture is running under the hood right now is already touching millions of actual work days, which brings up something subtle that's easy to miss if you're not paying close attention to how this gets framed.

362
00:21:34,600 --> 00:21:44,600
Copilot agents are increasingly described both by Microsoft and by the IT teams deploying them as digital employees, not features, not add-ons bolted onto word or outlook, employees.

363
00:21:44,600 --> 00:21:58,600
That word choice isn't accidental and it isn't just branding either. A feature does one thing the same way every time you click it and employee gets handed a goal and figures out a path to it using judgment along the way. That's exactly the distinction we've been building toward this whole episode.

364
00:21:58,600 --> 00:22:06,600
The move from a tool you operate to a system you delegate to calling it a digital employee is just naming out loud what the architecture already does.

365
00:22:06,600 --> 00:22:17,600
But here's the thing about employees, digital or otherwise, you don't just hand someone a badge and unlimited access and walk away, you manage them, you define what they can touch, what they can't and who's responsible when something goes wrong.

366
00:22:17,600 --> 00:22:26,600
If Copilot is being treated like an employee at 20 million users and rising, it needs to be managed like one, not eventually. Now agents as digital employees.

367
00:22:26,600 --> 00:22:40,600
What is managing an agent actually look like in practice? It starts with identity and Microsoft's answer to that is building it directly into Entra. Here's how it works. Each agent gets its own identity, not a shared service account, not a generic API key sitting in some config file.

368
00:22:40,600 --> 00:22:44,600
An actual identity, the same category of object Entra already uses to manage people.

369
00:22:44,600 --> 00:22:56,600
That identity carries access controls, specific defined permissions about what this agent can see and touch and it carries an ordered trail, a record of what the agent actually did, when and with what data. Think about what that replaces.

370
00:22:56,600 --> 00:23:04,600
Before this, agent access in most organizations meant a service account with broad permissions, set up once, rarely revisited and almost never ordered it line by line.

371
00:23:04,600 --> 00:23:15,600
Now the agent shows up in the same identity system as your employees. It can see it, it can restrict it, it can pull its history the same way they'd pull an access log for a person. Compliance and oversight run through Microsoft purview.

372
00:23:15,600 --> 00:23:31,600
Purview is what tracks what these agents are doing with your data, whether that's flagging when an agent touches sensitive content, enforcing data loss prevention policies against agent actions, the same way it does against human actions, or giving compliance teams a place to actually review agent behavior instead of taking it on faith.

373
00:23:31,600 --> 00:23:38,600
Purview doesn't care whether the thing querying a SharePoint library is a person or an agent, it applies the same policy layer either way.

374
00:23:38,600 --> 00:23:45,600
And here's why that matters more than it might sound like on the surface. An agent with tenant wide access and no accountability isn't a productivity win.

375
00:23:45,600 --> 00:23:48,600
It's a liability sitting quietly until it isn't quiet anymore.

376
00:23:48,600 --> 00:23:59,600
Given autonomous system broad reach into your SharePoint, your outlook, your data verse without identity, without access controls, without an ordered trail, and you've built something that can move fast and touch a lot.

377
00:23:59,600 --> 00:24:15,600
With no way to reconstruct what happened if something goes wrong, that's not hypothetical risk, that's the exact kind of gap regulators and security teams have spent years closing for human accounts, and it doesn't disappear just because the thing accessing the data isn't a person, which gets at the actual shift underneath all of this.

378
00:24:15,600 --> 00:24:22,600
The old model treated permissions as a people problem, who has access to what, reviewed on a schedule tied to a person's role.

379
00:24:22,600 --> 00:24:31,600
The new model has to treat permissions as a people and agents problem together in the same system under the same rules. An agent isn't a separate category you handle differently.

380
00:24:31,600 --> 00:24:36,600
It's another identity in the same governance framework, just one that happens to act autonomously instead of clicking through menus itself.

381
00:24:36,600 --> 00:24:46,600
And this is exactly the layer most organization skip. It's not that they ignore it on purpose. It's that agente features are the exciting part, the demo that gets applause in a meeting.

382
00:24:46,600 --> 00:25:00,600
Identity, access controls, audit trails, purview policies, none of that demos well. So it gets pushed to, we'll set that up later, while the agents already running in production already touching data already accumulating in access history nobody's watching.

383
00:25:00,600 --> 00:25:10,600
The real world workflows already running. So let's leave the governance conversation for a second and look at what's actually running right now today inside real Microsoft 365 tenants.

384
00:25:10,600 --> 00:25:25,600
Because everything we've covered so far could still sound theoretical if we stopped at architecture diagrams. It isn't theoretical. It's already live. Take email triage. There's a workflow agent pattern running inside outlook right now that watches your inbox checks whether a new message contains a question directed at you.

385
00:25:25,600 --> 00:25:35,600
And if it does goes and researches the answer using whatever knowledge sources it has access to not just pulling a can reply actually drafting a full response formatted with the question addressed directly.

386
00:25:35,600 --> 00:25:54,600
And it sends that draft back to you for review before anything goes to the original sender on top of that it posts the summary in teams telling you who emailed what they asked and what it's proposing to say that's not a single look up that's passing intent retrieving from more than one place drafting content and routing the results to two different destinations you and teams before a human even steps in.

387
00:25:54,600 --> 00:26:06,600
Then there's the personal productivity digest same underlying pattern different shape instead of reacting to one email the agent runs on a schedule scanning across teams email and documents and pulls together what actually matters.

388
00:26:06,600 --> 00:26:19,600
An AI news digest that checks the source every morning and delivers a summary to your inbox and teams a scan of your planet tasks that flags anything due today and pings you before you have noticed yourself a rundown of important emails from the last 24 hours.

389
00:26:19,600 --> 00:26:32,600
You're not the one hunting through your inbox to figure out what needs attention none of that is one query against one source it's continuous it's multi source and it's proactive instead of waiting for you to ask calendar time blocking works the same way just with a different trigger.

390
00:26:32,600 --> 00:26:46,600
This isn't just copilot looking at your existing meetings and finding gaps it's reasoning from your task list add something to a time block list in Microsoft to do something like booking a room for next Friday and the agent has to actually interpret that what does next Friday mean relative to today.

391
00:26:46,600 --> 00:26:58,600
What does the task title imply about duration then it writes an actual event into your calendar based on that interpretation that's language understanding feeding a decision feeding an action not a static rule that just checks for open slots.

392
00:26:58,600 --> 00:27:09,600
And then there's the one that looks the most like a traditional business process on the surface but isn't underneath form driven approval workflows someone submits a request a trading inquiry and expense exception whatever the form is for.

393
00:27:09,600 --> 00:27:17,600
And instead of rooting on one fixed rule the workflow pulls context from three separate systems before it decides where that request goes.

394
00:27:17,600 --> 00:27:37,600
Maybe it checks the request is history in one system current volume thresholds in another and account status in the third the rooting decision isn't hard coded to one field on the form it's assembled from context gathered across systems in real time before the workflow branches notice what all four of these actually have in common none of them are single question hitting a single source and coming back with an answer.

395
00:27:37,600 --> 00:27:50,600
Every one of them requires the model to check more than one place decide whether what it found is enough and in some cases take an action based on what it concludes that's the orchestration loop we walked through earlier running quietly inside tools people already use every day.

396
00:27:50,600 --> 00:28:03,600
And that's really the point worth sitting with here these aren't edge cases anymore some clever demo built to show off what's technically possible they're becoming the baseline the default shape of what a co pilot workflow looks like once you stop assuming it's just answering questions.

397
00:28:03,600 --> 00:28:23,600
Industry specific applications step outside Microsoft 365 for a second because this pattern isn't unique to email and calendars the same orchestration loop shows up in industries where getting the answer wrong actually hurt someone and looking at those cases makes the stakes a lot more concrete health care first picture an agent feeling a patient question about a new medication.

398
00:28:23,600 --> 00:28:49,600
A single shot system would grab whatever document mentions that drug and summarize it that's not good enough here and everyone in that field knows it a patient asking about side effects might actually be asking whether it's safe given a condition mentioned in their chart or whether it conflicts with something in an active clinical trial they're enrolled in answering that well means cross referencing the patient's own records against trial protocols and current treatment guidelines three sources not one evaluated together before anything gets set back to that patient.

399
00:28:49,600 --> 00:29:18,600
Finance works the same way just with a different kind of cost for getting it wrong fraud monitoring built on single anomaly flagging catches the obvious stuff one weird transaction one out of patent login but actual fraud rarely looks like one anomaly it looks like a pattern a small transaction here a login from a new location there a change to account details a few days later none of which trips in alarm on its own an agent that reasons across transaction history instead of flagging isolated events can connect that sequence the way a human investigator would if that investigator had time to check every account personally.

400
00:29:18,600 --> 00:29:47,600
It doesn't have that time the agent does manufacturing is maybe the clearest version of this because the sources are so obviously different from each other sensor data tells you a machine's temperature is drifting slightly that alone means almost nothing sensors drift constantly but cross reference that against maintenance logs has this part failed before under similar conditions and against technician notes did someone flag something unusual on the last inspection that never made it into a formal report and suddenly a small sensor deviation turns into an actual actionable prediction.

401
00:29:47,600 --> 00:30:16,600
None of those three sources alone gets you there notice the thread running through all three of these it's not the industry that matters here it's the shape of the problem health care finance manufacturing completely different domains completely different data and yet every single one of them needs multiple sources reconciled in real time before good answer or good prediction is even possible that's not a coincidence that's exactly the class of problem agentic rag exists to solve the one traditional single shot retrieval was never built to touch but here's the catch nobody glasses over for the first time.

402
00:30:16,600 --> 00:30:37,600
Nobody glasses over for long none of this works if the agent has no way to actually reach the systems sensor feeds clinical databases transaction logs technician notes those don't just sit in a share point library waiting to be indexed getting an agent from reasoning about what it needs to actually pulling it from a live system is a different problem entirely and it's the one we're covering next.

403
00:30:37,600 --> 00:31:06,600
The model context protocol shift so here's the actual answer to that reach problem and it has a name mcp model context protocol it's emerging as the standard way for agents to connect to enterprise systems whether that's a clinical database a transaction log or a maintenance tracker running on some line of business platform nobody outside IT has ever heard of think about why a standard even matters here without one every single connection between an agent in a data source gets built from scratch custom code custom authentication custom error handling for every system every time that's not just.

404
00:31:06,600 --> 00:31:35,600
It's fragile once it exists because it breaks the moment the underlying system changes something on its end and nobody notices until an agent starts failing silently and it's expensive to maintain because someone has to own that custom connection forever patching it every time either side updates mcp replaces that one off approach with something closer to a shared language instead of building a bespoke bridge to every system an agent might need you build to one protocol and any agent that speaks mcp can use it that's the difference between wiring a new cable for every device your own versus everything just work.

405
00:31:35,600 --> 00:31:56,600
There's a phrase that captures this shift better than any technical description api design is the new embedding strategy sit with that for a second because it's a real reframe for years the whole game and rag was about documents how you chunk them how you embed them how you make them searchable that matter because the thing being retrieved was text sitting still waiting to be found but agents don't just need

406
00:31:56,600 --> 00:32:25,600
searchable documents anymore they need calable systems things they can query trigger and get structured answers back from if your api isn't designed for that clearly described predictable discoverable it doesn't matter how good your embeddings are the agent can't reach it now none of this means every organization needs a team of engineers hand building mcp connections one at a time that's exactly where copilot studio and the workflow agent come in they give business users a way to define these connections through configuration and natural language instructions not custom code someone who understands the process not the plumbing can be a little bit more difficult to do.

407
00:32:25,600 --> 00:32:54,600
Not the plumbing can set up what an agent should connect to and when without needing to be the person who wrote the integration layer underneath it but here's where it gets harder not easier building the connection is only half the problem the protocol gets the agent to the data it says nothing about whether what comes back is actually correct actually current actually safe to act on you can have a perfectly engineered pipe running between your agent and a clinical database and still have no idea whether the answer flowing through it deserves to be trusted that's the real problem waiting on the other side of this not reaching the system.

408
00:32:54,600 --> 00:33:13,600
Trusting what it hands you once you do why grounding still matters more than ever let's talk about the word that's been sitting quietly underneath this entire episode grounding because everything we've walked through the planning the multisource retrieval the loop that checks and re checks itself none of it means anything if the answer at the end isn't actually tied to something real.

409
00:33:13,600 --> 00:33:33,600
Grounding just means this response comes from your actual enterprise content your contracts your emails your CRM records not from whatever the model happens to remember from its training data that distinction sounds small until you picture the alternative a model answering a question about your remote work policy using general knowledge about remote work policies instead of the actual document sitting in your share point

410
00:33:33,600 --> 00:33:44,600
isn't wrong exactly it's just not answering about you it's answering about some average version of every company like yours and passing that off as your policy we already looked at the number that shows why this matters so much.

411
00:33:44,600 --> 00:34:01,600
Agentech rag cuts hallucinations by 47% compared to traditional rag that game doesn't come from the model getting smarter in some general sense it comes from grounding getting enforced more aggressively at every step of the loop the evaluation stage checking whether a retrieved document actually supports the claim being made the requerying that happens when it doesn't.

412
00:34:01,600 --> 00:34:16,600
Here's the part worth sitting with though because it's a little counter intuitive you think more autonomy means less need to check the model's work it's the opposite more autonomy means more chances for the agent to go off track a single shot system can only be wrong once on one query in one place

413
00:34:16,600 --> 00:34:29,600
and a genetic system is making judgment calls at every stage of a multi-step loop deciding what counts as sufficient deciding when to stop looking deciding how to interpret an ambiguous result every one of those judgment calls is a place where drift can creep in.

414
00:34:29,600 --> 00:34:45,600
So the more freedom you give the system to reason on its own the more grounding has to be enforced not less it's not a trade off you get to relax on just because the system got more capable this is exactly why safety checks sit at the end of the orchestration pipeline and why they're not optional polish tacked on at the finish.

415
00:34:45,600 --> 00:35:14,600
They exist specifically to catch the moment an agents conclusion outpaces what it sources actually support before anything reaches you something in that pipeline is asking whether the final answer can actually be traced back to a real document a real record a real piece of content the agent was authorized to see if it can't that's the failure point those checks are built to catch and this is where the employee framing earns its keep again you don't let an employee act on a hunch and presented as fact not without asking where that came from you expect them to point to the file the email the meeting note that backs up what they're telling you.

416
00:35:14,600 --> 00:35:26,600
And agent deserves exactly the same expectation more autonomy doesn't earn it more trust by default it earns it more scrutiny because the more it's allowed to decide on its own the more it needs a way to show its work.

417
00:35:26,600 --> 00:35:43,600
The adoption curve reality check so with all of that in mind where does the rest of the market actually stand right now because it's easy to listen to everything we've covered and assume this is already the norm everywhere it isn't not yet the research is direct about this mainstream enterprise adoption of a gentle drag is projected for 20 20 20 20

418
00:35:43,600 --> 00:35:58,600
28 not now not next quarter a couple of years out at least right now most organizations sit in the innovator or early adopter phase not the mainstream phase that matters because those two phases behave completely differently innovators tolerate rough edges they expect to troubleshoot they expect some failures

419
00:35:58,600 --> 00:36:12,600
and they treat that as the cost of getting ahead of the curve mainstream adoption assume something closer to a solved problem something you deploy without needing a dedicated team babysitting it we're not there most companies experimenting with this right now are whether they admit it or not still in the tolerate

420
00:36:12,600 --> 00:36:41,600
rough edges phase and there's a reason grounding the exact thing we just spent this whole section on matters so much at this specific point in the adoption curve 47% of organizations using generative AI have already had at least one negative consequence from it almost half that's not a hypothetical risk sitting somewhere in the future that's already happened broadly across the market which is exactly why grounded approaches aren't a nice to have feature right now they're the thing standing between early adopter with a rough process and cautionary tail in next quarters board meeting

421
00:36:41,600 --> 00:36:58,600
here's the number that should really slow you down though only about 5% of enterprise agent systems that get built average production 5% 95% of what gets built demoed funded staffed never actually makes it to the point where real employees are using it on real work every day that gap between demo deployment is where most

422
00:36:58,600 --> 00:37:18,600
agentech rack projects actually die not in the pitch meeting not in the proof of concept where everything looks impressive because someone hand picked the test cases they die in the space between this work when we showed it to leadership and this needs to handle every edge case every day at scale without someone standing next to it that gap is enormous and it's exactly the gap we're going to look at next because

423
00:37:18,600 --> 00:37:47,600
understanding why projects die there is the difference between building something real and building another entry in that 95% why most agentech projects never reach production so let's actually break down what's sitting inside that gap because it's hard to operationalize isn't an answer it's a shrug the blockers have names and they show up in roughly the same order every time latency cost reliability complexity operational overhead 5 separate problems and a team can solve 4 of them and still get taken out by the 5th

424
00:37:47,600 --> 00:38:06,600
operational overhead is the one that catches people off guard because it doesn't show up until after launch an agent making autonomous decisions doesn't run itself once it's live someone has to watch what it's doing keep it working when an upstream system changes something without warning and figure out why it made a specific call when it doesn't match what a person would have done that's not a one time set up cost

425
00:38:06,600 --> 00:38:31,600
and it's a job most teams never actually budget it for because the project plan stopped at deploy not staff the person who babysits this thing every day going forward then there's the complexity trap and this one's almost a right of passage at this point a team build something genuinely impressive shows it off gets the funding gets the go ahead then it hits real usage real usage means edge cases nobody thought to test because the demo only ever ran the cases someone hand picked to make it look good

426
00:38:31,600 --> 00:39:00,600
the question phrase slightly differently than expected a document that doesn't quite match the form at the agent was built around a source that's momentarily down the demo never had to survive any of that production does every single day and that's exactly where a lot of these projects quietly stall out now layer something else on top of that something that makes the whole situation worse instead of just harder only 27% of generative AI users review all outputs before use roughly a quarter which means the other three quarters are shipping acting on or forwarding outputs they haven't actually checked think about what that means once an age

427
00:39:00,600 --> 00:39:25,600
not a person is the one generating those outputs most organizations are trusting decisions and answers they've never verified from a system that's making autonomous calls about what to check what to trust and when to stop looking nobody's watching most of it and the review rate tells you that plainly that's the actual accountability gap and it's exactly what governance frameworks the identity models the purview oversight the ordered trails we walked through earlier are built to close

428
00:39:25,600 --> 00:39:53,600
not because autonomy is inherently unsafe but because autonomy without anyone checking the work is a different kind of risk than a person making the same mistake a person you can ask a system running unreviewed you find out about after the fact none of this is an argument against building these systems it's an argument against building them carelessly the organizations that make it past that 95% failure point aren't the ones that skipped the hard parts they are the ones that planned for the monitoring budgeted for the edge cases and built review into the process instead of assuming the output would just be right

429
00:39:53,600 --> 00:39:58,300
build it deliberately not fast. The ROI timeline nobody advertises. Let's talk money,

430
00:39:58,300 --> 00:40:01,920
because everything so far has been architecture and governance and at some point

431
00:40:01,920 --> 00:40:05,120
somebody in finance is going to ask when this pays for itself. Here's the

432
00:40:05,120 --> 00:40:08,520
finding stated plainly. Enterprise is implementing agent-based, rag

433
00:40:08,520 --> 00:40:12,040
architectures, often see ROI within three to six months. That's a real number and

434
00:40:12,040 --> 00:40:15,600
it's faster than a lot of enterprise software ever pays back. But there's a

435
00:40:15,600 --> 00:40:19,320
qualifier attached to it that gets dropped constantly when people repeat this

436
00:40:19,320 --> 00:40:23,480
stat in a pitch deck. High volume operations, not every use case, not any use case

437
00:40:23,480 --> 00:40:27,080
high volume operation specifically. That qualifier is doing almost all the work in

438
00:40:27,080 --> 00:40:30,440
that sentence and it's worth understanding why volume is what makes the payback

439
00:40:30,440 --> 00:40:34,040
math actually function. Building an orchestration loop, the planning step, the

440
00:40:34,040 --> 00:40:38,200
multi-source retrieval, the evaluation and requiring isn't free. Someone has to

441
00:40:38,200 --> 00:40:42,640
design it connected to the right systems, test it against edge cases and keep it

442
00:40:42,640 --> 00:40:46,800
running. That's a fixed cost paid once upfront, regardless of how often the

443
00:40:46,800 --> 00:40:50,440
thing gets used. Afterward, now run that fixed cost against two different

444
00:40:50,440 --> 00:40:54,960
scenarios. In the first, that orchestration runs a few dozen times a day. In the

445
00:40:54,960 --> 00:40:58,640
second, it runs a few thousand times a day. Same build cost in both cases. Completely

446
00:40:58,640 --> 00:41:01,960
different payback timeline because the value generated each time it runs has to

447
00:41:01,960 --> 00:41:05,600
divide against that same fixed cost and dividing it across a few dozen uses a

448
00:41:05,600 --> 00:41:10,280
day gets you nowhere fast. Dividing it across a few thousand gets you to that three

449
00:41:10,280 --> 00:41:13,560
to six month window. The research is actually describing this is exactly where

450
00:41:13,560 --> 00:41:16,800
companies get it wrong and it's a mistake that's easy to make because it

451
00:41:16,800 --> 00:41:19,960
doesn't look like a mistake going in. A team picks a process that feels

452
00:41:19,960 --> 00:41:23,360
important. Maybe it's genuinely complex. Maybe it genuinely needs multi-step

453
00:41:23,360 --> 00:41:27,480
reasoning across sources and builds the full agente pipeline for it. Then it turns

454
00:41:27,480 --> 00:41:31,120
out that process only gets triggered 20 or 30 times a day. The complexity was

455
00:41:31,120 --> 00:41:34,720
real. The volume wasn't. Six months later, someone's asking why the ROI never

456
00:41:34,720 --> 00:41:38,920
materialized and the answer isn't that agente.rag failed. It's that agente.rag was

457
00:41:38,920 --> 00:41:42,160
aimed at a target that could never generate enough repetitions to earn back

458
00:41:42,160 --> 00:41:45,400
what it costs to build, which is exactly why the decision of where to deploy

459
00:41:45,400 --> 00:41:48,600
agente.rag matters as much as how you build it. You can get the architecture,

460
00:41:48,600 --> 00:41:52,560
the react loop, the grounding, the governance, all completely right and still lose

461
00:41:52,560 --> 00:41:56,200
the ROI argument because you pointed all of that engineering at a process that

462
00:41:56,200 --> 00:42:01,200
runs 30 times a day instead of 30,000 efficiency techniques making this

463
00:42:01,200 --> 00:42:05,440
viable. Now, everything we've said about cost and latency so far has been treated

464
00:42:05,440 --> 00:42:09,600
as a fixed tax, something you pay because that's what agente reasoning costs.

465
00:42:09,600 --> 00:42:12,640
That's starting to shift and it's worth understanding how because it changes

466
00:42:12,640 --> 00:42:16,520
the volume math we just walked through. Take this adapt. What it does is adapt

467
00:42:16,520 --> 00:42:20,400
how much reasoning the model actually applies based on how hard the question is

468
00:42:20,400 --> 00:42:24,560
instead of running the same heavy process on every query regardless of difficulty.

469
00:42:24,560 --> 00:42:29,440
The result up to 22.4% token reduction while keeping accuracy comparable.

470
00:42:29,440 --> 00:42:32,680
That's not a rounding error. That's cost control without sacrificing the output

471
00:42:32,680 --> 00:42:35,960
quality you were getting before. Then there's focused chain of thought reasoning,

472
00:42:35,960 --> 00:42:39,920
sometimes shortened to F-coating. Standard chain of thought has the model reason

473
00:42:39,920 --> 00:42:44,360
through a problem in a fairly loose exploratory way which works but burns time

474
00:42:44,360 --> 00:42:48,200
and tokens on paths that don't end up mattering. Focused chain of thought structures

475
00:42:48,200 --> 00:42:51,840
that reasoning process more tightly and the result is inference running two to

476
00:42:51,840 --> 00:42:55,160
three times faster than standard chain of thought again without giving up

477
00:42:55,160 --> 00:42:58,840
accuracy to get there. Here's why this actually matters if you're the one making

478
00:42:58,840 --> 00:43:02,160
the build or don't build call. Everything we covered earlier that two to ten

479
00:43:02,160 --> 00:43:05,280
X cost multiplier, the multi second latency gap between traditional and

480
00:43:05,280 --> 00:43:08,640
agente pipelines, all of that was measured against today's baseline. That

481
00:43:08,640 --> 00:43:11,920
baseline is not fixed. It's shrinking as techniques like these mature and get

482
00:43:11,920 --> 00:43:15,240
folded into how these systems run by default and this isn't some academic

483
00:43:15,240 --> 00:43:19,000
footnote sitting in a research paper nobody outside a lab will ever read.

484
00:43:19,000 --> 00:43:22,640
These are the kinds of optimizations vendors are already building into production

485
00:43:22,640 --> 00:43:26,480
co-pilot systems right now quietly under the hood without necessarily calling

486
00:43:26,480 --> 00:43:29,360
attention to it which means the tradeoff calculation we walked through a few

487
00:43:29,360 --> 00:43:33,440
sections back weighing whether a process has enough volume or complexity to

488
00:43:33,440 --> 00:43:37,640
justify the agente overhead gets easier to justify with every one of these

489
00:43:37,640 --> 00:43:41,640
efficiency gains that ships. But token savings and faster inference only address

490
00:43:41,640 --> 00:43:46,040
one axis of this they make a single agents reasoning loop cheaper and quicker

491
00:43:46,040 --> 00:43:49,440
they don't touch what happens when the problem itself outgrows what one agent

492
00:43:49,440 --> 00:43:52,640
reasoning alone can actually handle that's a different kind of scaling question

493
00:43:52,640 --> 00:43:56,880
and it's the one waiting on the other side of this where this is headed multi

494
00:43:56,880 --> 00:44:01,960
agent orchestration so one agent choosing between data sources reasoning in a loop

495
00:44:01,960 --> 00:44:05,400
checking its own work that's everything we've covered so far but that's not

496
00:44:05,400 --> 00:44:08,800
where this stops the next layer isn't a smarter single agent its multiple

497
00:44:08,800 --> 00:44:13,120
agents coordinating with each other picture it this way instead of one system

498
00:44:13,120 --> 00:44:17,840
that plans retrieves and verifies you get separate agents built for separate

499
00:44:17,840 --> 00:44:23,240
jobs one that specializes in retrieval one that specializes in verification one

500
00:44:23,240 --> 00:44:27,520
that handles a specific line of business system and they hand work back and forth

501
00:44:27,520 --> 00:44:31,480
instead of one model trying to do all of it alone that's a different shape of

502
00:44:31,480 --> 00:44:34,920
problem than anything we've walked through this episode because now the question

503
00:44:34,920 --> 00:44:39,480
isn't just did this agent make a good decision it's did these agents coordinate

504
00:44:39,480 --> 00:44:43,800
well enough to avoid stepping on each other there's early research into decentralized

505
00:44:43,800 --> 00:44:47,680
versions of this agents working as a network rather than reporting up to one

506
00:44:47,680 --> 00:44:51,920
central decision maker no single orchestrator sitting on top directing traffic

507
00:44:51,920 --> 00:44:55,640
instead agents that discover each other specialize over time and collaborate

508
00:44:55,640 --> 00:44:59,400
directly closer to a network than a hierarchy it's early and it's worth

509
00:44:59,400 --> 00:45:03,040
naming without overselling it but it's the direction a meaningful chunk of the

510
00:45:03,040 --> 00:45:06,400
research is pointing here's the part that actually changes what retrieval even

511
00:45:06,400 --> 00:45:10,600
means going forward applications themselves are becoming calable resources not

512
00:45:10,600 --> 00:45:13,960
just data sources sitting there waiting to be searched that's a real shift from

513
00:45:13,960 --> 00:45:17,880
everything we covered with mcp a few sections back it's not just that an agent

514
00:45:17,880 --> 00:45:21,520
can reach into a system and pull data out it's that agents can trigger other

515
00:45:21,520 --> 00:45:25,680
agents or trigger entire services directly as part of completing a task the

516
00:45:25,680 --> 00:45:29,280
line between look something up and make something happen starts to disappear

517
00:45:29,280 --> 00:45:33,360
because both are just actions the agent can take at any point in its reasoning now

518
00:45:33,360 --> 00:45:36,560
tie that back to co-pilot specifically because this isn't some distant

519
00:45:36,560 --> 00:45:40,280
hypothetical sitting in a research paper workflow agent and co-pilot studio are

520
00:45:40,280 --> 00:45:44,080
already pointing exactly this direction we already walked through agents that

521
00:45:44,080 --> 00:45:48,320
can call workflows as tools and workflows that can call agents as steps that's

522
00:45:48,320 --> 00:45:51,520
the early shape of this it's not agents talking to a database it's agents

523
00:45:51,520 --> 00:45:55,840
talking to other agents and to other workflows deciding at runtime which one to

524
00:45:55,840 --> 00:45:59,440
invoke based on what the task actually needs which gives us the through line for

525
00:45:59,440 --> 00:46:03,760
the entire episode laid out plainly retrieval one document one answer then

526
00:46:03,760 --> 00:46:08,160
reasoning a model that plans chooses iterates then autonomous action a system

527
00:46:08,160 --> 00:46:12,200
that doesn't just answer but does and now coordinated systems of action multiple

528
00:46:12,200 --> 00:46:15,560
agents working together on pieces of a problem too large for any one of them to

529
00:46:15,560 --> 00:46:18,600
hold alone that's the actual arc underneath everything we've covered and it's

530
00:46:18,600 --> 00:46:23,360
still being written what this means for your co-pilot strategy so pull back up

531
00:46:23,360 --> 00:46:26,720
to the level where decisions actually get made because none of this matters if it

532
00:46:26,720 --> 00:46:29,840
doesn't change what you do Monday morning this isn't about chasing every new

533
00:46:29,840 --> 00:46:34,000
feature Microsoft chips it's tempting to treat each wave each new capability

534
00:46:34,000 --> 00:46:38,000
announcement as something you need to adopt immediately just because it exists

535
00:46:38,000 --> 00:46:41,680
that's not a strategy that's just reacting to a release calendar and reacting to

536
00:46:41,680 --> 00:46:45,920
a release calendar is exactly how you end up with a kind of unmonetored

537
00:46:45,920 --> 00:46:50,720
ungoverned agent sprawl we spent half this episode warning against the real

538
00:46:50,720 --> 00:46:54,480
strategic question is narrower and it's the one worth actually sitting with which

539
00:46:54,480 --> 00:46:58,640
processes in your organization are complex enough to justify a gentick

540
00:46:58,640 --> 00:47:02,400
or a ag and which are better served by simple retrieval we've spent this whole

541
00:47:02,400 --> 00:47:06,400
episode building the case for both sides of that question some of your workflows

542
00:47:06,400 --> 00:47:10,320
are genuinely multi-step multi-source and worth the cost of orchestration a lot of

543
00:47:10,320 --> 00:47:14,680
them aren't and forcing a gentick architecture onto a simple lookup just burns

544
00:47:14,680 --> 00:47:18,280
budget for no gain governance isn't optional overhead sitting on top of the fun

545
00:47:18,280 --> 00:47:21,720
part it's the difference between a copilot rollout that scales and one that

546
00:47:21,720 --> 00:47:26,520
gets shut down after an incident identity access controls audit trails none of

547
00:47:26,520 --> 00:47:30,040
that is bureaucracy standing in the way of progress it's the thing that let's

548
00:47:30,040 --> 00:47:33,400
progress keep happening after the first mistake instead of getting frozen the

549
00:47:33,400 --> 00:47:36,760
moment something goes wrong and nobody can explain why here's what's actually

550
00:47:36,760 --> 00:47:40,920
separating the organizations getting real value from this right now it's not speed

551
00:47:40,920 --> 00:47:44,440
the organization's winning with this aren't the ones moving fastest they're the ones

552
00:47:44,440 --> 00:47:48,680
matching the architecture to the actual task deploying a gentick rag where the volume

553
00:47:48,680 --> 00:47:53,320
and complexity earn it back and leaving simple retrieval alone everywhere else

554
00:47:53,320 --> 00:47:56,760
which brings us right back to where we started the assumption we opened with was

555
00:47:56,760 --> 00:48:00,520
simple copilot's job is to answer questions type something in get something

556
00:48:00,520 --> 00:48:04,120
back done that assumption held up fine right up until a gentick rag entered the

557
00:48:04,120 --> 00:48:08,280
picture and the moment it did copilot stopped being a question answering tool it

558
00:48:08,280 --> 00:48:13,000
became something that plans checks multiple sources decides whether what it

559
00:48:13,000 --> 00:48:16,840
found is good enough and in a growing number of cases actually does something with the

560
00:48:16,840 --> 00:48:20,280
answer instead of just handing it to you so here's what to do with that

561
00:48:20,280 --> 00:48:24,680
starting now not someday pick one copilot workflow you're running today anyone

562
00:48:24,680 --> 00:48:28,760
and ask it a single question does this actually need retrieval or does it need reasoning

563
00:48:28,760 --> 00:48:33,000
if it's a simple lookup one source one clear answer leave it alone simple retrieval is cheap

564
00:48:33,000 --> 00:48:38,440
and it works but if you find yourself explaining why the answer came back incomplete or wrong

565
00:48:38,440 --> 00:48:44,040
or missing entirely there's a decent chance you're asking a single short system to do a multi-step job

566
00:48:44,040 --> 00:48:48,440
that's worth auditing this week not next quarter if this changed how you think about copilot's

567
00:48:48,440 --> 00:48:52,520
architecture leave a review it's the fastest way to help someone else find this before they

568
00:48:52,520 --> 00:48:57,080
build the wrong thing at scale and if you want to keep pushing on this connect with me mercopieters

569
00:48:57,080 --> 00:49:01,800
on LinkedIn tell me what you're running into tell me what you want covered next this show gets

570
00:49:01,800 --> 00:49:03,800
let's build from those conversations.