Your AI Landing Zone Is A Liability

1
00:00:00,000 --> 00:00:02,400
Azure OpenAI looks like just another API endpoint,

2
00:00:02,400 --> 00:00:04,120
you deploy a resource, you get an endpoint,

3
00:00:04,120 --> 00:00:06,160
you give it a key, you move on.

4
00:00:06,160 --> 00:00:08,360
This assumption, it's the liability.

5
00:00:08,360 --> 00:00:09,600
Here's what actually happens.

6
00:00:09,600 --> 00:00:12,920
You deploy Azure OpenAI without thinking about the perimeter.

7
00:00:12,920 --> 00:00:15,000
No private endpoints, no managed identities,

8
00:00:15,000 --> 00:00:17,700
no governance layer, just the model sitting there,

9
00:00:17,700 --> 00:00:18,960
waiting to be called.

10
00:00:18,960 --> 00:00:20,560
And for a proof of concept, it works fine,

11
00:00:20,560 --> 00:00:22,560
but the moment you scale past one team,

12
00:00:22,560 --> 00:00:25,040
the moment you try to audit what's connected to what.

13
00:00:25,040 --> 00:00:27,440
The moment leadership asks for compliance evidence,

14
00:00:27,440 --> 00:00:29,000
everything breaks, enterprise AI

15
00:00:29,000 --> 00:00:31,440
isn't about the model, it's about the perimeter.

16
00:00:31,440 --> 00:00:33,040
The perimeter is the structural boundary

17
00:00:33,040 --> 00:00:35,760
that keeps your data safe, your costs visible,

18
00:00:35,760 --> 00:00:37,040
and your reasoning auditable.

19
00:00:37,040 --> 00:00:39,040
Without it, you don't have an AI platform,

20
00:00:39,040 --> 00:00:40,040
you have a liability.

21
00:00:40,040 --> 00:00:41,840
Manual deployments create audit failures,

22
00:00:41,840 --> 00:00:43,960
data leaks, governance chaos.

23
00:00:43,960 --> 00:00:45,720
When an auditor walks into your organization

24
00:00:45,720 --> 00:00:48,680
and asks to see the code that deployed your LLM infrastructure,

25
00:00:48,680 --> 00:00:52,360
if your answer is, we did it in the portal, you've already lost.

26
00:00:52,360 --> 00:00:54,760
Bicep isn't optional, it's the enforcement mechanism

27
00:00:54,760 --> 00:00:56,280
that makes the perimeter real.

28
00:00:56,280 --> 00:00:59,400
This masterclass shows you why the old model fails,

29
00:00:59,400 --> 00:01:00,960
and how to build the hardened perimeter,

30
00:01:00,960 --> 00:01:02,920
the integrated structure of identity network,

31
00:01:02,920 --> 00:01:04,880
governance, observability, and orchestration

32
00:01:04,880 --> 00:01:06,480
that transforms AI infrastructure

33
00:01:06,480 --> 00:01:09,360
from a security hole into a controlled, auditable,

34
00:01:09,360 --> 00:01:11,640
scalable platform.

35
00:01:11,640 --> 00:01:13,920
Why standard landing zones fail with AI?

36
00:01:13,920 --> 00:01:16,040
As your landing zones were built for a specific type

37
00:01:16,040 --> 00:01:18,760
of workload, static resources with predictable access

38
00:01:18,760 --> 00:01:20,560
patterns, you deploy a web app.

39
00:01:20,560 --> 00:01:22,520
You know it needs network access to a database,

40
00:01:22,520 --> 00:01:25,360
you know which teams operated, you know which data it touches.

41
00:01:25,360 --> 00:01:27,080
These patterns are stable, they don't change,

42
00:01:27,080 --> 00:01:29,920
so landing zones were designed around this stability.

43
00:01:29,920 --> 00:01:32,960
Management groups organize subscriptions by business function.

44
00:01:32,960 --> 00:01:35,440
Azure Policy and Forces configuration baselines,

45
00:01:35,440 --> 00:01:37,760
are back as science permissions to known roles.

46
00:01:37,760 --> 00:01:40,240
Everything assumes that once you've deployed a resource,

47
00:01:40,240 --> 00:01:41,840
it stays the same.

48
00:01:41,840 --> 00:01:44,440
Then AI arrived, LLM's don't follow static patterns,

49
00:01:44,440 --> 00:01:46,720
and LLM consumes data from multiple sources,

50
00:01:46,720 --> 00:01:50,680
SharePoint, SQL databases, Cosmos DB, external APIs.

51
00:01:50,680 --> 00:01:52,480
The access patterns are unpredictable.

52
00:01:52,480 --> 00:01:55,080
The data flowing through the model changes constantly,

53
00:01:55,080 --> 00:01:56,960
the reasoning chains adapt based on input.

54
00:01:56,960 --> 00:01:58,640
This violates the fundamental assumption

55
00:01:58,640 --> 00:02:01,560
that landing zones are built on predictability.

56
00:02:01,560 --> 00:02:03,080
Here's what happens when you treat AI

57
00:02:03,080 --> 00:02:04,480
like traditional workloads.

58
00:02:04,480 --> 00:02:06,840
Teams deploy Azure Open AI in the portal.

59
00:02:06,840 --> 00:02:09,040
No infrastructure as code, no governance layer,

60
00:02:09,040 --> 00:02:11,240
they connect it to a SharePoint site manually,

61
00:02:11,240 --> 00:02:13,480
they add a managed identity with broad permissions,

62
00:02:13,480 --> 00:02:15,000
it works, so they move on.

63
00:02:15,000 --> 00:02:17,320
Then another team does the same thing differently.

64
00:02:17,320 --> 00:02:20,040
They use API keys instead of managed identities.

65
00:02:20,040 --> 00:02:21,680
They enable public network access,

66
00:02:21,680 --> 00:02:23,000
they skip the private endpoints,

67
00:02:23,000 --> 00:02:25,520
configuration drifts spreads across the organization.

68
00:02:25,520 --> 00:02:28,120
Now you have three ways to deploy the same resource.

69
00:02:28,120 --> 00:02:30,400
This is the creation of pet infrastructure,

70
00:02:30,400 --> 00:02:32,640
resources that can't be audited, can't be replicated,

71
00:02:32,640 --> 00:02:33,800
can't be secured at scale.

72
00:02:33,800 --> 00:02:35,880
When a third team tries to replicate the pattern,

73
00:02:35,880 --> 00:02:36,880
they miss something.

74
00:02:36,880 --> 00:02:38,560
Maybe they forget the role assignment,

75
00:02:38,560 --> 00:02:40,480
maybe they enable public access by default,

76
00:02:40,480 --> 00:02:42,400
maybe they skip the diagnostic logging.

77
00:02:42,400 --> 00:02:44,320
Each manual deployment is a copy of a copy,

78
00:02:44,320 --> 00:02:46,680
and with each iteration, fidelity is lost.

79
00:02:46,680 --> 00:02:49,520
Governance layers, Azure policy, management groups,

80
00:02:49,520 --> 00:02:52,360
they were designed to enforce rules about regions, SKUs,

81
00:02:52,360 --> 00:02:54,560
and tagging, they don't understand reasoning chains,

82
00:02:54,560 --> 00:02:57,520
they don't know that when an LLM accesses your knowledge base,

83
00:02:57,520 --> 00:02:59,200
that connection needs to stay private.

84
00:02:59,200 --> 00:03:00,920
They don't account for the fact that token spend

85
00:03:00,920 --> 00:03:02,680
is invisible until the bill arrives.

86
00:03:02,680 --> 00:03:04,440
The result is shadow AI,

87
00:03:04,440 --> 00:03:06,920
untract deployments, untract token consumption,

88
00:03:06,920 --> 00:03:08,880
compliance violations that nobody knew existed

89
00:03:08,880 --> 00:03:11,520
until the audit began, and then the auditor arrives.

90
00:03:11,520 --> 00:03:14,280
She asks, show me the code that deployed this resource.

91
00:03:14,280 --> 00:03:17,800
Silence, she asks, who has access to this endpoint?

92
00:03:17,800 --> 00:03:19,800
The answer is scattered across the portal,

93
00:03:19,800 --> 00:03:21,800
across different teams, across spreadsheets

94
00:03:21,800 --> 00:03:24,520
that don't match, she asks, how is data flowing

95
00:03:24,520 --> 00:03:26,200
from your knowledge base to the model?

96
00:03:26,200 --> 00:03:28,640
Without private endpoints defined in code,

97
00:03:28,640 --> 00:03:31,200
the answer reveals public internet exposure.

98
00:03:31,200 --> 00:03:34,320
This is what auditors find, evidence of control failure.

99
00:03:34,320 --> 00:03:36,120
Not because you're trying to do something wrong,

100
00:03:36,120 --> 00:03:38,840
but because you treated AI like traditional infrastructure

101
00:03:38,840 --> 00:03:40,800
when the model demands something different,

102
00:03:40,800 --> 00:03:42,800
that difference is what we're going to fix.

103
00:03:42,800 --> 00:03:44,640
The cost of manual AI deployment,

104
00:03:44,640 --> 00:03:46,360
the liability isn't just architectural,

105
00:03:46,360 --> 00:03:47,720
it's financial and legal,

106
00:03:47,720 --> 00:03:49,760
when you deploy manually cost compounds

107
00:03:49,760 --> 00:03:52,160
in ways that traditional infrastructure never does.

108
00:03:52,160 --> 00:03:54,560
Not because the individual deployments are expensive,

109
00:03:54,560 --> 00:03:57,640
but because each one introduces a new source of divergence.

110
00:03:57,640 --> 00:03:59,120
Let me be specific about what happens.

111
00:03:59,120 --> 00:04:00,880
Your first team deploys Azure OpenAI

112
00:04:00,880 --> 00:04:03,600
with all the security controls, private endpoints,

113
00:04:03,600 --> 00:04:06,000
managed identities, diagnostic logging,

114
00:04:06,000 --> 00:04:08,520
the deployment works, the infrastructure is locked down,

115
00:04:08,520 --> 00:04:09,480
they're compliant.

116
00:04:09,480 --> 00:04:11,000
Your second team is under time pressure,

117
00:04:11,000 --> 00:04:13,680
they deploy Azure OpenAI without the private endpoints,

118
00:04:13,680 --> 00:04:15,520
it's faster, they know they can add them later,

119
00:04:15,520 --> 00:04:17,080
they deploy with an API key,

120
00:04:17,080 --> 00:04:19,600
instead of a managed identity, same reasoning,

121
00:04:19,600 --> 00:04:21,600
faster now, secure later.

122
00:04:21,600 --> 00:04:23,520
Your third team uses a different pattern,

123
00:04:23,520 --> 00:04:26,200
they enable private endpoints for Azure OpenAI,

124
00:04:26,200 --> 00:04:29,120
but they forget to apply the same pattern to Azure AI search.

125
00:04:29,120 --> 00:04:31,720
So now you have three deployments of the same architecture

126
00:04:31,720 --> 00:04:33,400
with three different security models,

127
00:04:33,400 --> 00:04:34,760
this is configuration drift,

128
00:04:34,760 --> 00:04:36,760
it happens the moment you stop deploying through code

129
00:04:36,760 --> 00:04:38,040
and start using the portal.

130
00:04:38,040 --> 00:04:40,120
Here's the problem with configuration drift

131
00:04:40,120 --> 00:04:41,560
in AI infrastructure,

132
00:04:41,560 --> 00:04:43,960
it's invisible until it costs you.

133
00:04:43,960 --> 00:04:46,240
Your first team's deployment is tracking token usage

134
00:04:46,240 --> 00:04:47,800
through application insights,

135
00:04:47,800 --> 00:04:49,440
your second team's deployment sends logs

136
00:04:49,440 --> 00:04:50,680
to a different workspace,

137
00:04:50,680 --> 00:04:53,400
your third team never enable diagnostic logging at all.

138
00:04:53,400 --> 00:04:55,320
Now when the monthly Azure Bill arrives,

139
00:04:55,320 --> 00:04:57,760
you can't tell which team consumed which tokens,

140
00:04:57,760 --> 00:04:59,640
you can't attribute cost to which project

141
00:04:59,640 --> 00:05:01,320
use the most expensive model,

142
00:05:01,320 --> 00:05:03,360
you can't see where spending is accelerating.

143
00:05:03,360 --> 00:05:06,160
Token spend becomes a mystery until the invoice appears,

144
00:05:06,160 --> 00:05:07,880
by then it's too late to optimize,

145
00:05:07,880 --> 00:05:10,000
you're paying for decisions you can't see,

146
00:05:10,000 --> 00:05:11,400
and here's where it gets worse,

147
00:05:11,400 --> 00:05:13,200
managed identities that were bypassed

148
00:05:13,200 --> 00:05:14,960
for faster deployments.

149
00:05:14,960 --> 00:05:17,440
One team configured managed identities correctly,

150
00:05:17,440 --> 00:05:20,000
another team used an API key stored in a key vault,

151
00:05:20,000 --> 00:05:22,560
a third team stored the key in environment variables,

152
00:05:22,560 --> 00:05:24,880
all three approaches work, none of them are the same.

153
00:05:24,880 --> 00:05:26,560
When an auditor arrives and asks

154
00:05:26,560 --> 00:05:28,800
who has access to this LLM endpoint,

155
00:05:28,800 --> 00:05:30,760
you're not giving one answer, you're giving three,

156
00:05:30,760 --> 00:05:34,040
and they don't align, data lineage becomes impossible to trace.

157
00:05:34,040 --> 00:05:34,960
In manual deployments,

158
00:05:34,960 --> 00:05:37,680
the connection from your SharePoint content to the LLM,

159
00:05:37,680 --> 00:05:39,120
to the application output,

160
00:05:39,120 --> 00:05:41,040
is scattered across different configurations,

161
00:05:41,040 --> 00:05:43,720
different identity models, different logging strategies.

162
00:05:43,720 --> 00:05:46,600
You can't draw a line from data into reasoning to data out,

163
00:05:46,600 --> 00:05:48,280
you can't prove that sensitive information

164
00:05:48,280 --> 00:05:49,680
stayed within your security boundary,

165
00:05:49,680 --> 00:05:51,280
you can't show which documents were used

166
00:05:51,280 --> 00:05:53,200
to ground a specific model response,

167
00:05:53,200 --> 00:05:54,320
this matters for compliance,

168
00:05:54,320 --> 00:05:55,800
it matters for data governance,

169
00:05:55,800 --> 00:05:57,200
it matters when you need to prove

170
00:05:57,200 --> 00:05:59,120
that you didn't expose confidential information

171
00:05:59,120 --> 00:06:00,400
to an external service.

172
00:06:00,400 --> 00:06:02,880
But the real cost comes at remediation time,

173
00:06:02,880 --> 00:06:06,040
if you realize six months in that your AI infrastructure

174
00:06:06,040 --> 00:06:09,200
doesn't comply with your governance policy, you need to fix it.

175
00:06:09,200 --> 00:06:11,920
But fixing manual deployments isn't about updating a module,

176
00:06:11,920 --> 00:06:14,000
it's about going back to each deployment,

177
00:06:14,000 --> 00:06:15,640
understanding how it was built,

178
00:06:15,640 --> 00:06:17,520
modifying it manually, testing it,

179
00:06:17,520 --> 00:06:20,120
and hoping you don't break something else in the process.

180
00:06:20,120 --> 00:06:22,040
I've seen organizations spend six months

181
00:06:22,040 --> 00:06:24,360
remediating manual deployments

182
00:06:24,360 --> 00:06:26,840
when they could have had the controls in place from the beginning.

183
00:06:26,840 --> 00:06:29,200
Remediation costs exponentially more than prevention,

184
00:06:29,200 --> 00:06:31,320
not because the fix is technically difficult,

185
00:06:31,320 --> 00:06:32,640
but because you're working blind,

186
00:06:32,640 --> 00:06:35,040
you're changing live infrastructure without knowing exactly

187
00:06:35,040 --> 00:06:36,240
what's connected to what,

188
00:06:36,240 --> 00:06:38,360
this is the hidden cost of manual deployment,

189
00:06:38,360 --> 00:06:39,840
not the infrastructure itself,

190
00:06:39,840 --> 00:06:41,640
but the cost of trying to govern it afterward,

191
00:06:41,640 --> 00:06:43,600
this is where bicep changes the model,

192
00:06:43,600 --> 00:06:44,600
the hidden assumption,

193
00:06:44,600 --> 00:06:46,120
here's where the thinking breaks down,

194
00:06:46,120 --> 00:06:49,120
the hidden assumption that drives manual AI deployments is this,

195
00:06:49,120 --> 00:06:51,480
will secure it after we prove the concept works,

196
00:06:51,480 --> 00:06:52,680
it sounds reasonable,

197
00:06:52,680 --> 00:06:55,240
you want to validate that the technology actually solves a problem

198
00:06:55,240 --> 00:06:56,600
before you invest in hardening,

199
00:06:56,600 --> 00:06:58,960
so you deploy fast, you cut corners,

200
00:06:58,960 --> 00:07:00,120
you get something running,

201
00:07:00,120 --> 00:07:01,400
and then once it's proven,

202
00:07:01,400 --> 00:07:03,560
you'll retrofit all the governance controls.

203
00:07:03,560 --> 00:07:05,320
This assumption is broken for AI

204
00:07:05,320 --> 00:07:07,920
in a way it never was for traditional infrastructure,

205
00:07:07,920 --> 00:07:11,240
security debt compounds faster with AI than any other workload type,

206
00:07:11,240 --> 00:07:14,320
when you manually deploy a web server without security controls,

207
00:07:14,320 --> 00:07:15,840
you can lock it down afterward,

208
00:07:15,840 --> 00:07:17,720
you can update the network configuration,

209
00:07:17,720 --> 00:07:19,920
you can adjust our back, you can add logging,

210
00:07:19,920 --> 00:07:22,280
these are overlays on top of existing infrastructure,

211
00:07:22,280 --> 00:07:25,560
but with AI, once an LLM has access to sensitive data,

212
00:07:25,560 --> 00:07:27,440
removing that access isn't an overlay,

213
00:07:27,440 --> 00:07:29,600
it requires redeployment.

214
00:07:29,600 --> 00:07:30,840
Think about what happens,

215
00:07:30,840 --> 00:07:33,200
you deploy an LLM with a managed identity

216
00:07:33,200 --> 00:07:35,760
that has broad access to your SharePoint content,

217
00:07:35,760 --> 00:07:36,560
the model works,

218
00:07:36,560 --> 00:07:38,880
it grounds reasoning on your proprietary information,

219
00:07:38,880 --> 00:07:40,080
the business sees value,

220
00:07:40,080 --> 00:07:42,680
then months later compliance reviews the deployment

221
00:07:42,680 --> 00:07:44,480
and flags the overly broad permissions.

222
00:07:44,480 --> 00:07:46,440
Now you need to remove that access.

223
00:07:46,440 --> 00:07:48,800
The obvious approach create a new managed identity

224
00:07:48,800 --> 00:07:50,240
with restricted permissions,

225
00:07:50,240 --> 00:07:53,280
but the LLM is already deployed with the old identity,

226
00:07:53,280 --> 00:07:55,880
changing the identity isn't a configuration update,

227
00:07:55,880 --> 00:07:57,760
it means redeploying the entire resource,

228
00:07:57,760 --> 00:07:59,280
and if you're redeploying manually,

229
00:07:59,280 --> 00:08:01,400
you're repeating all the decisions you made the first time.

230
00:08:01,400 --> 00:08:03,720
Did you remember to enable private endpoints this time,

231
00:08:03,720 --> 00:08:05,840
did you configure the diagnostic logging,

232
00:08:05,840 --> 00:08:07,840
did you set up the role assignments correctly?

233
00:08:07,840 --> 00:08:10,320
Without bicep, each redeployment is a new opportunity

234
00:08:10,320 --> 00:08:11,320
to miss something,

235
00:08:11,320 --> 00:08:12,760
each time you redeploy,

236
00:08:12,760 --> 00:08:14,640
you introduce new vulnerabilities.

237
00:08:14,640 --> 00:08:17,120
Enterprise governance requires audit trails.

238
00:08:17,120 --> 00:08:19,320
When you deploy manually, there's no trail,

239
00:08:19,320 --> 00:08:21,640
you make a decision in the portal, you click a button,

240
00:08:21,640 --> 00:08:24,480
nothing documents why you made that choice or what you intended.

241
00:08:24,480 --> 00:08:26,720
When the auditor asks why the LLM has access

242
00:08:26,720 --> 00:08:27,840
to that data source,

243
00:08:27,840 --> 00:08:29,400
you're guessing at your own reasoning.

244
00:08:29,400 --> 00:08:31,160
Manual deployments leave gaps,

245
00:08:31,160 --> 00:08:33,240
not because you're cutting corners intentionally,

246
00:08:33,240 --> 00:08:35,720
but because the portal doesn't enforce documentation,

247
00:08:35,720 --> 00:08:37,320
it doesn't ask you to justify decisions,

248
00:08:37,320 --> 00:08:39,560
it lets you deploy without explanation.

249
00:08:39,560 --> 00:08:42,520
The will fix it later approach becomes impossible at scale.

250
00:08:42,520 --> 00:08:44,800
One LLM, you can retrofit it, 10 LLMs,

251
00:08:44,800 --> 00:08:46,240
you can retrofit them all in a week,

252
00:08:46,240 --> 00:08:49,680
100 LLMs deployed across your organization by different teams,

253
00:08:49,680 --> 00:08:51,160
using different patterns,

254
00:08:51,160 --> 00:08:52,880
now you're looking at months of work

255
00:08:52,880 --> 00:08:55,400
and you're fixing problems you don't even know exist yet.

256
00:08:55,400 --> 00:08:57,600
By the time leadership asks for compliance evidence,

257
00:08:57,600 --> 00:08:58,920
it's too late to retrofit,

258
00:08:58,920 --> 00:09:00,880
you're not missing a few security controls,

259
00:09:00,880 --> 00:09:01,880
you're missing the evidence

260
00:09:01,880 --> 00:09:03,720
that you ever made a governance decision at all.

261
00:09:03,720 --> 00:09:05,520
There's no code, there's no documentation,

262
00:09:05,520 --> 00:09:06,600
there's no trail.

263
00:09:06,600 --> 00:09:08,080
This is when you realized the liability

264
00:09:08,080 --> 00:09:09,200
wasn't the initial deployment.

265
00:09:09,200 --> 00:09:11,520
It was the decision to defer security to later.

266
00:09:11,520 --> 00:09:13,040
Bicep forces a different model,

267
00:09:13,040 --> 00:09:14,240
you secure it from the beginning

268
00:09:14,240 --> 00:09:15,680
because the code demands it.

269
00:09:15,680 --> 00:09:18,520
What auditors actually look for?

270
00:09:18,520 --> 00:09:21,640
The audit begins with a single, deceptively simple question.

271
00:09:21,640 --> 00:09:24,080
Show me the code that deployed this resource.

272
00:09:24,080 --> 00:09:26,720
In organizations with bicep, the answer is clear.

273
00:09:26,720 --> 00:09:28,960
Here's the module, here's the parameter file,

274
00:09:28,960 --> 00:09:30,880
here's the commit history, here's what changed

275
00:09:30,880 --> 00:09:32,920
when it changed and who approved it.

276
00:09:32,920 --> 00:09:35,920
The entire decision trail is embedded in source control.

277
00:09:35,920 --> 00:09:37,760
In organizations with manual deployments,

278
00:09:37,760 --> 00:09:40,400
the answer is silence or worse,

279
00:09:40,400 --> 00:09:42,120
a screenshot of the portal from months ago

280
00:09:42,120 --> 00:09:43,760
when someone created the resource.

281
00:09:43,760 --> 00:09:45,480
That screenshot is now outdated,

282
00:09:45,480 --> 00:09:46,840
the configuration has drifted,

283
00:09:46,840 --> 00:09:49,080
the screenshot doesn't match what's actually running

284
00:09:49,080 --> 00:09:51,320
and even if it did, a screenshot isn't evidence,

285
00:09:51,320 --> 00:09:52,280
it's a picture.

286
00:09:52,280 --> 00:09:55,560
It proves nothing about intent or governance.

287
00:09:55,560 --> 00:09:57,440
The auditor moves to the next question,

288
00:09:57,440 --> 00:10:00,160
who has access to this LLM endpoint?

289
00:10:00,160 --> 00:10:02,480
With bicep, the answer lives in the R-back assignments

290
00:10:02,480 --> 00:10:03,560
defined in code.

291
00:10:03,560 --> 00:10:06,480
The module specifies which roles can read from the endpoint,

292
00:10:06,480 --> 00:10:08,560
which can write to it, which can manage it.

293
00:10:08,560 --> 00:10:10,160
The code is the source of truth.

294
00:10:10,160 --> 00:10:11,880
When someone grants a new team access

295
00:10:11,880 --> 00:10:13,600
that change goes through version control,

296
00:10:13,600 --> 00:10:15,280
it's reviewable, it's reversible.

297
00:10:15,280 --> 00:10:17,040
Without bicep, access is scattered.

298
00:10:17,040 --> 00:10:19,760
One person added the tenant to the Azure Open AI

299
00:10:19,760 --> 00:10:21,560
resource directly in the portal.

300
00:10:21,560 --> 00:10:23,280
Another person created a service principle

301
00:10:23,280 --> 00:10:25,360
in assigned permissions through the CLI.

302
00:10:25,360 --> 00:10:28,400
A third person shared an API key through a password manager.

303
00:10:28,400 --> 00:10:29,800
Nobody documented any of this.

304
00:10:29,800 --> 00:10:31,880
When the auditor asks who has access,

305
00:10:31,880 --> 00:10:35,080
the honest answer is we're not entirely sure.

306
00:10:35,080 --> 00:10:36,880
Then comes the question about Dataflow,

307
00:10:36,880 --> 00:10:39,680
how is data moving from your knowledge base to the model?

308
00:10:39,680 --> 00:10:41,880
This is where architecture reveals itself.

309
00:10:41,880 --> 00:10:45,000
If your knowledge base sits in a private Azure AI search instance,

310
00:10:45,000 --> 00:10:46,880
accessible only through a private endpoint

311
00:10:46,880 --> 00:10:49,280
with a managed identity handling authentication,

312
00:10:49,280 --> 00:10:50,600
then the answer is clean.

313
00:10:50,600 --> 00:10:52,200
All data movement is encrypted,

314
00:10:52,200 --> 00:10:53,880
on Microsoft's network backbone,

315
00:10:53,880 --> 00:10:55,480
never crossing the public internet.

316
00:10:55,480 --> 00:10:56,400
It's provable.

317
00:10:56,400 --> 00:10:58,160
It's in the code, but if the knowledge base

318
00:10:58,160 --> 00:11:00,680
was connected manually, the data path is unknown,

319
00:11:00,680 --> 00:11:02,080
is the search endpoint public?

320
00:11:02,080 --> 00:11:02,880
Probably.

321
00:11:02,880 --> 00:11:05,280
Is the connection authenticated through a managed identity

322
00:11:05,280 --> 00:11:06,680
or through an API key?

323
00:11:06,680 --> 00:11:07,480
Unclear.

324
00:11:07,480 --> 00:11:09,880
Did someone accidentally enable public network access

325
00:11:09,880 --> 00:11:10,880
on the storage account?

326
00:11:10,880 --> 00:11:11,680
Possibly.

327
00:11:11,680 --> 00:11:13,680
The auditor can't see a clear data flow

328
00:11:13,680 --> 00:11:15,880
because it was never designed as a unified system.

329
00:11:15,880 --> 00:11:17,880
The conversation shifts to economics.

330
00:11:17,880 --> 00:11:19,480
What's your cost attribution model?

331
00:11:19,480 --> 00:11:22,080
Which business units are paying for which tokens?

332
00:11:22,080 --> 00:11:24,280
Bicep deployments have built in cost tracking.

333
00:11:24,280 --> 00:11:27,480
API management sits in front of the LLM emitting metrics.

334
00:11:27,480 --> 00:11:29,280
Each request carries metadata,

335
00:11:29,280 --> 00:11:31,480
tenant, feature, user, cost center.

336
00:11:31,480 --> 00:11:34,080
Tokens consumed by the marketing teams co-pilot

337
00:11:34,080 --> 00:11:35,880
are tracked separately from tokens consumed

338
00:11:35,880 --> 00:11:37,480
by the support teams assistant.

339
00:11:37,480 --> 00:11:38,880
You can see where money is going.

340
00:11:38,880 --> 00:11:39,680
You can explain it.

341
00:11:39,680 --> 00:11:40,880
Manual deployments.

342
00:11:40,880 --> 00:11:43,080
Token usage is invisible until the bill arrives.

343
00:11:43,080 --> 00:11:44,560
There's no cost attribution.

344
00:11:44,560 --> 00:11:46,360
No way to tie spend back to decisions.

345
00:11:46,360 --> 00:11:47,760
No way to answer the question,

346
00:11:47,760 --> 00:11:50,160
what value did those tokens generate?

347
00:11:50,160 --> 00:11:53,760
Finally, the auditor asks the question that matters most.

348
00:11:53,760 --> 00:11:56,600
Can you deploy this exact same setup in another region?

349
00:11:56,600 --> 00:11:58,000
Can you replicate this infrastructure

350
00:11:58,000 --> 00:12:00,600
at the same level of security and governance?

351
00:12:00,600 --> 00:12:02,600
The answer from bicep organizations is yes.

352
00:12:02,600 --> 00:12:03,800
The module is portable.

353
00:12:03,800 --> 00:12:05,200
It carries all the logic.

354
00:12:05,200 --> 00:12:07,480
Deploy it to another region, another subscription,

355
00:12:07,480 --> 00:12:09,040
and you get identical architecture,

356
00:12:09,040 --> 00:12:11,200
identical governance, identical security.

357
00:12:11,200 --> 00:12:13,280
The answer from manual deployments is no.

358
00:12:13,280 --> 00:12:14,880
Because the knowledge doesn't live in code,

359
00:12:14,880 --> 00:12:17,120
it lives in the heads of the people who built it.

360
00:12:17,120 --> 00:12:19,400
And they're no longer sure they remember all the decisions

361
00:12:19,400 --> 00:12:22,800
they made that no is what the auditor was looking for all along.

362
00:12:22,800 --> 00:12:24,240
It means you can't scale.

363
00:12:24,240 --> 00:12:26,080
It means you can't prove repeatability.

364
00:12:26,080 --> 00:12:29,120
It means the structural weakness has been exposed.

365
00:12:29,120 --> 00:12:30,880
bicep as the control plane.

366
00:12:30,880 --> 00:12:33,240
Now we move from diagnosis to structure.

367
00:12:33,240 --> 00:12:35,640
The question shifts from what went wrong.

368
00:12:35,640 --> 00:12:36,960
To how do we build it right?

369
00:12:36,960 --> 00:12:38,360
bicep isn't just a tool.

370
00:12:38,360 --> 00:12:40,360
When you're deploying AI infrastructure,

371
00:12:40,360 --> 00:12:42,400
bicep becomes the control plane,

372
00:12:42,400 --> 00:12:44,720
the declarative system that encodes governance

373
00:12:44,720 --> 00:12:46,280
into the deployment itself.

374
00:12:46,280 --> 00:12:47,640
Here's the shift in thinking.

375
00:12:47,640 --> 00:12:50,000
A tool is something you use to accomplish a task.

376
00:12:50,000 --> 00:12:51,880
You pick it up, use it, put it down.

377
00:12:51,880 --> 00:12:53,040
bicep is different.

378
00:12:53,040 --> 00:12:55,360
It's a language that doesn't just describe what you're deploying.

379
00:12:55,360 --> 00:12:56,840
It describes how governance flows

380
00:12:56,840 --> 00:12:58,920
through every resource you create.

381
00:12:58,920 --> 00:13:01,960
It's the difference between using a hammer and building a house.

382
00:13:01,960 --> 00:13:04,640
The moment you shift to bicep, something fundamental changes.

383
00:13:04,640 --> 00:13:07,200
You're not making individual decisions in the portal anymore.

384
00:13:07,200 --> 00:13:09,320
You're writing code that makes the decisions for you.

385
00:13:09,320 --> 00:13:11,680
Consistently, every single time the module runs.

386
00:13:11,680 --> 00:13:13,360
Let me show you what this means in practice.

387
00:13:13,360 --> 00:13:15,840
In the portal, you create an Azure OpenAI resource.

388
00:13:15,840 --> 00:13:17,000
It's a resource with properties.

389
00:13:17,000 --> 00:13:18,800
You set the SQ, you set the region,

390
00:13:18,800 --> 00:13:21,680
you click a checkbox for diagnostic logging, then you leave.

391
00:13:21,680 --> 00:13:23,520
That's one deployment, one snapshot.

392
00:13:23,520 --> 00:13:25,760
The next time someone needs to deploy Azure OpenAI,

393
00:13:25,760 --> 00:13:27,000
they start from scratch.

394
00:13:27,000 --> 00:13:28,760
They remember some of the decisions you made.

395
00:13:28,760 --> 00:13:29,760
They forget others.

396
00:13:29,760 --> 00:13:32,440
They might make different choices for reasons they don't write down.

397
00:13:32,440 --> 00:13:34,840
In bicep, you write a module called OpenAI.

398
00:13:34,840 --> 00:13:37,280
bicep, that module doesn't just create the resource.

399
00:13:37,280 --> 00:13:39,320
It encodes decisions.

400
00:13:39,320 --> 00:13:41,320
Every resource created through that module

401
00:13:41,320 --> 00:13:43,640
carries metadata embedded in the code.

402
00:13:43,640 --> 00:13:46,760
Ownership, compliance tier, data classification.

403
00:13:46,760 --> 00:13:48,560
That metadata flows through automatically.

404
00:13:48,560 --> 00:13:51,400
You don't remember to add it, the code adds it, always.

405
00:13:51,400 --> 00:13:53,040
The structure of your bicep modules

406
00:13:53,040 --> 00:13:54,880
should mirror the reasoning architecture,

407
00:13:54,880 --> 00:13:56,400
not the Azure Service catalog.

408
00:13:56,400 --> 00:13:57,760
This is critical.

409
00:13:57,760 --> 00:14:00,200
Most infrastructure code follows the Cloud providers

410
00:14:00,200 --> 00:14:02,360
organizational structure, networking modules,

411
00:14:02,360 --> 00:14:04,320
compute modules, storage modules.

412
00:14:04,320 --> 00:14:06,480
But AI infrastructure isn't organized that way.

413
00:14:06,480 --> 00:14:08,800
Your intelligence flows from data ingest

414
00:14:08,800 --> 00:14:10,600
through vectorization into retrieval,

415
00:14:10,600 --> 00:14:12,080
into reasoning, into output.

416
00:14:12,080 --> 00:14:13,880
Your code structure should reflect that flow,

417
00:14:13,880 --> 00:14:15,080
not Azure's catalog.

418
00:14:15,080 --> 00:14:17,400
When your code mirrors your reasoning architecture,

419
00:14:17,400 --> 00:14:18,320
something clicks.

420
00:14:18,320 --> 00:14:20,240
You're not deploying a collection of services.

421
00:14:20,240 --> 00:14:21,720
You're deploying a reasoning system.

422
00:14:21,720 --> 00:14:23,640
The network module isn't about network security

423
00:14:23,640 --> 00:14:24,640
in the abstract.

424
00:14:24,640 --> 00:14:26,520
It's about keeping your knowledge-based private

425
00:14:26,520 --> 00:14:27,960
so the reasoning is grounded.

426
00:14:27,960 --> 00:14:30,640
The identity module isn't about RBIAC in general.

427
00:14:30,640 --> 00:14:33,760
It's about ensuring that when the model retrieves context,

428
00:14:33,760 --> 00:14:36,720
it only accesses what the current user is authorized to see.

429
00:14:36,720 --> 00:14:38,680
The observability module isn't about dashboards.

430
00:14:38,680 --> 00:14:40,440
It's about tracking which reasoning decisions

431
00:14:40,440 --> 00:14:41,880
consumed which tokens.

432
00:14:41,880 --> 00:14:43,840
This alignment matters because it forces you

433
00:14:43,840 --> 00:14:45,320
to think architecturally.

434
00:14:45,320 --> 00:14:47,720
The moment your code structure matches your reasoning flow,

435
00:14:47,720 --> 00:14:48,760
gaps become obvious.

436
00:14:48,760 --> 00:14:50,360
If you're deploying a retrieval module,

437
00:14:50,360 --> 00:14:52,600
but there's no corresponding network isolation,

438
00:14:52,600 --> 00:14:54,520
the module itself highlights the gap.

439
00:14:54,520 --> 00:14:56,320
The code becomes a structural checklist.

440
00:14:56,320 --> 00:14:58,360
Byset modules become reusable blueprints

441
00:14:58,360 --> 00:15:00,160
that enforce security by default.

442
00:15:00,160 --> 00:15:02,240
You don't have to remember to add private endpoints.

443
00:15:02,240 --> 00:15:03,400
The module includes them.

444
00:15:03,400 --> 00:15:05,680
You don't have to remember to enable diagnostic logging.

445
00:15:05,680 --> 00:15:06,640
The module enables it.

446
00:15:06,640 --> 00:15:09,000
You don't have to remember to create managed identities

447
00:15:09,000 --> 00:15:09,960
and assign roles.

448
00:15:09,960 --> 00:15:11,280
The module creates them.

449
00:15:11,280 --> 00:15:13,920
This is the power of encoding governance into code.

450
00:15:13,920 --> 00:15:16,240
The first time you deploy a pattern, it's documented.

451
00:15:16,240 --> 00:15:17,760
Every decision lives in the module.

452
00:15:17,760 --> 00:15:19,600
There's a reason the private endpoint is created

453
00:15:19,600 --> 00:15:20,840
in that specific subnet.

454
00:15:20,840 --> 00:15:22,360
There's a reason the managed identity

455
00:15:22,360 --> 00:15:24,080
has those particular role assignments.

456
00:15:24,080 --> 00:15:25,800
There's a reason the diagnostic setting

457
00:15:25,800 --> 00:15:27,360
roots to log analytics.

458
00:15:27,360 --> 00:15:28,240
It's all written down.

459
00:15:28,240 --> 00:15:30,520
The hundredth time you deploy the same module,

460
00:15:30,520 --> 00:15:32,160
it's identical to the first time.

461
00:15:32,160 --> 00:15:33,920
No drift, no forgotten steps.

462
00:15:33,920 --> 00:15:35,880
No, we'll add this later.

463
00:15:35,880 --> 00:15:37,960
Every deployment carries the same governance,

464
00:15:37,960 --> 00:15:39,960
the same security, the same structure.

465
00:15:39,960 --> 00:15:42,680
This consistency is what transforms AI infrastructure

466
00:15:42,680 --> 00:15:45,120
from a liability into a controlled system.

467
00:15:45,120 --> 00:15:47,040
When you deploy the same pattern a hundred times

468
00:15:47,040 --> 00:15:48,520
and it's identical every time,

469
00:15:48,520 --> 00:15:51,280
you've eliminated the source of most governance failures,

470
00:15:51,280 --> 00:15:52,440
human variation.

471
00:15:52,440 --> 00:15:54,720
The control plane isn't forcing you to follow rules.

472
00:15:54,720 --> 00:15:56,560
It's making the right decisions automatic.

473
00:15:56,560 --> 00:15:58,280
It's the difference between an auditor asking,

474
00:15:58,280 --> 00:16:00,080
why didn't you do this?

475
00:16:00,080 --> 00:16:03,880
And an auditor asking, how did you ensure this was done every time?

476
00:16:03,880 --> 00:16:06,040
The three layers of the hardened perimeter

477
00:16:06,040 --> 00:16:08,400
that consistency, that automatic governance,

478
00:16:08,400 --> 00:16:09,520
it comes from structure.

479
00:16:09,520 --> 00:16:11,520
The control plane isn't a single decision.

480
00:16:11,520 --> 00:16:13,040
It's three layers working together,

481
00:16:13,040 --> 00:16:14,880
each meaningless without the others.

482
00:16:14,880 --> 00:16:17,040
The first layer is the identity perimeter.

483
00:16:17,040 --> 00:16:19,680
This is where secrets stop existing in your infrastructure.

484
00:16:19,680 --> 00:16:22,480
Not because you've eliminated the problem of authentication,

485
00:16:22,480 --> 00:16:25,040
but because you've eliminated the human from the problem.

486
00:16:25,040 --> 00:16:26,360
When you deploy without BICEP,

487
00:16:26,360 --> 00:16:29,440
you create a managed identity or you generate an API key,

488
00:16:29,440 --> 00:16:32,280
then you store that key somewhere in a configuration file,

489
00:16:32,280 --> 00:16:34,160
in a key vault, in an environment variable,

490
00:16:34,160 --> 00:16:36,880
and immediately you've created a secret that can leak.

491
00:16:36,880 --> 00:16:38,320
Not because you are careless,

492
00:16:38,320 --> 00:16:41,400
but because secrets have a gravitational pull to what exposure,

493
00:16:41,400 --> 00:16:42,960
they get copied, they get shared,

494
00:16:42,960 --> 00:16:44,880
they get stored in places they shouldn't be.

495
00:16:44,880 --> 00:16:47,080
BICEP flips this, you don't create secrets,

496
00:16:47,080 --> 00:16:48,320
you create identities.

497
00:16:48,320 --> 00:16:51,240
The module defines a managed identity on your compute resource.

498
00:16:51,240 --> 00:16:53,480
It assigns that identity specific roles.

499
00:16:53,480 --> 00:16:56,240
It grants that identity access to specific services.

500
00:16:56,240 --> 00:16:58,920
No keys change hands, no human ever sees the secret,

501
00:16:58,920 --> 00:17:00,520
as your manages it cryptographically.

502
00:17:00,520 --> 00:17:03,000
This isn't optional governance, this is structural.

503
00:17:03,000 --> 00:17:04,800
The moment you deploy through a BICEP module

504
00:17:04,800 --> 00:17:07,520
that creates managed identities as the default,

505
00:17:07,520 --> 00:17:10,040
keys become the exception, not the rule.

506
00:17:10,040 --> 00:17:11,240
And exceptions get noticed.

507
00:17:11,240 --> 00:17:13,440
They trigger policy violations, they show up in audits,

508
00:17:13,440 --> 00:17:14,680
the RBAC layer sits on top,

509
00:17:14,680 --> 00:17:16,760
role assignments are defined in the same code

510
00:17:16,760 --> 00:17:18,320
that creates the identity.

511
00:17:18,320 --> 00:17:20,920
The identity exists, its permissions are defined alongside

512
00:17:20,920 --> 00:17:23,360
its creation, when you update one, you update the other,

513
00:17:23,360 --> 00:17:25,640
they don't drift apart, they can't get out of sync.

514
00:17:25,640 --> 00:17:27,280
The second layer is the network perimeter,

515
00:17:27,280 --> 00:17:29,160
this is about where your data actually moves,

516
00:17:29,160 --> 00:17:30,600
not theoretically but physically,

517
00:17:30,600 --> 00:17:33,120
through which networks, over which connections,

518
00:17:33,120 --> 00:17:34,440
with what exposure.

519
00:17:34,440 --> 00:17:36,040
Private endpoints are the mechanism.

520
00:17:36,040 --> 00:17:38,680
They create a private IP address inside your vnet

521
00:17:38,680 --> 00:17:40,440
that maps to an Azure service,

522
00:17:40,440 --> 00:17:44,320
Azure OpenAI, AI search, storage, key vault.

523
00:17:44,320 --> 00:17:46,480
Traffic destined for that service no longer roots

524
00:17:46,480 --> 00:17:48,120
across the public internet.

525
00:17:48,120 --> 00:17:50,120
It stays on Microsoft's network backbone,

526
00:17:50,120 --> 00:17:52,000
it never leaves your security boundary,

527
00:17:52,000 --> 00:17:53,720
but private endpoints are only useful

528
00:17:53,720 --> 00:17:55,240
if they're consistently deployed.

529
00:17:55,240 --> 00:17:57,760
If one team creates them and another doesn't,

530
00:17:57,760 --> 00:17:59,640
you have an architecture where some data movement

531
00:17:59,640 --> 00:18:01,120
is protected and some isn't.

532
00:18:01,120 --> 00:18:02,600
The network perimeter is broken,

533
00:18:02,600 --> 00:18:04,480
bicep makes this decision once.

534
00:18:04,480 --> 00:18:06,160
The module for deploying Azure OpenAI

535
00:18:06,160 --> 00:18:07,520
doesn't just create the service,

536
00:18:07,520 --> 00:18:09,640
it creates the private endpoint in the correct subnet,

537
00:18:09,640 --> 00:18:11,360
it links the private DNS zone,

538
00:18:11,360 --> 00:18:14,040
it enforces NSG rules that restrict traffic,

539
00:18:14,040 --> 00:18:16,040
it validates that public access is disabled

540
00:18:16,040 --> 00:18:17,240
on the target service.

541
00:18:17,240 --> 00:18:18,320
This layering is critical,

542
00:18:18,320 --> 00:18:20,360
the private endpoint is useless without DNS,

543
00:18:20,360 --> 00:18:22,680
the DNS is useless without the NSG rules,

544
00:18:22,680 --> 00:18:25,120
and all of it is useless if someone can still access

545
00:18:25,120 --> 00:18:27,280
the service through its public endpoint.

546
00:18:27,280 --> 00:18:30,000
bicep ensures that all these pieces deploy together.

547
00:18:30,000 --> 00:18:30,840
It's structural,

548
00:18:30,840 --> 00:18:33,160
the perimeter exists because the code makes it impossible

549
00:18:33,160 --> 00:18:34,200
to deploy without it.

550
00:18:34,200 --> 00:18:36,160
The third layer is the reasoning perimeter.

551
00:18:36,160 --> 00:18:37,680
This is where AI actually happens.

552
00:18:37,680 --> 00:18:39,440
This is where as your AI foundry sits

553
00:18:39,440 --> 00:18:41,080
as the governance boundary that defines

554
00:18:41,080 --> 00:18:42,360
which models are available,

555
00:18:42,360 --> 00:18:44,000
which data sources can be accessed,

556
00:18:44,000 --> 00:18:45,520
which tools can be invoked.

557
00:18:45,520 --> 00:18:48,040
Alongside it API management becomes the gateway

558
00:18:48,040 --> 00:18:50,480
that controls every request flowing to the LLM,

559
00:18:50,480 --> 00:18:52,440
and observability becomes the visibility layer

560
00:18:52,440 --> 00:18:54,520
that tracks every token flowing through.

561
00:18:54,520 --> 00:18:57,640
These three components, foundry, API management,

562
00:18:57,640 --> 00:19:00,040
observability, they're only powerful together.

563
00:19:00,040 --> 00:19:01,960
Foundry without API management means

564
00:19:01,960 --> 00:19:04,360
you can't enforce rate limits or attribution.

565
00:19:04,360 --> 00:19:06,360
API management without foundry means

566
00:19:06,360 --> 00:19:08,800
your controlling traffic to a governance free system.

567
00:19:08,800 --> 00:19:10,120
Observability without either means

568
00:19:10,120 --> 00:19:12,720
you can see tokens flowing, but you can't control them.

569
00:19:12,720 --> 00:19:14,440
bicep wires them together as a unit.

570
00:19:14,440 --> 00:19:15,840
When you deploy the reasoning perimeter

571
00:19:15,840 --> 00:19:17,160
through a single module set,

572
00:19:17,160 --> 00:19:18,720
they're automatically connected.

573
00:19:18,720 --> 00:19:21,760
The API management backend points to the foundry project.

574
00:19:21,760 --> 00:19:24,640
The diagnostics from both flow into the observability layer,

575
00:19:24,640 --> 00:19:26,040
they're not three separate decisions.

576
00:19:26,040 --> 00:19:27,760
They're one integrated architecture.

577
00:19:27,760 --> 00:19:29,760
Each layer is meaningless without the others.

578
00:19:29,760 --> 00:19:30,680
That's the design.

579
00:19:30,680 --> 00:19:32,360
Identity without network means secrets

580
00:19:32,360 --> 00:19:34,000
can still move across the internet.

581
00:19:34,000 --> 00:19:35,360
Network without reasoning means

582
00:19:35,360 --> 00:19:36,840
you've secured the infrastructure,

583
00:19:36,840 --> 00:19:38,560
but not the intelligence.

584
00:19:38,560 --> 00:19:40,560
Reasoning without observability means

585
00:19:40,560 --> 00:19:42,400
governance is happening in the dark.

586
00:19:42,400 --> 00:19:44,720
bicep ensures all three are deployed together

587
00:19:44,720 --> 00:19:47,000
consistently across every environment.

588
00:19:47,000 --> 00:19:49,520
Manual deployments inevitably skip layers.

589
00:19:49,520 --> 00:19:51,080
Someone forgets the private endpoint.

590
00:19:51,080 --> 00:19:52,920
Someone skip setting up API management.

591
00:19:52,920 --> 00:19:54,200
Someone deploys observability,

592
00:19:54,200 --> 00:19:55,840
but doesn't wire the metrics correctly.

593
00:19:55,840 --> 00:19:57,600
The perimeter only works if it's complete.

594
00:19:57,600 --> 00:20:00,520
Modular bicep for AI search and vectorization.

595
00:20:00,520 --> 00:20:02,440
Let's look at the first concrete pattern.

596
00:20:02,440 --> 00:20:05,040
How bicep structures the vector data path?

597
00:20:05,040 --> 00:20:07,120
Azure AI search isn't just a search service.

598
00:20:07,120 --> 00:20:08,600
That's the old way of thinking about it.

599
00:20:08,600 --> 00:20:10,200
You deploy search as a tool.

600
00:20:10,200 --> 00:20:11,240
You index documents.

601
00:20:11,240 --> 00:20:12,120
You query them.

602
00:20:12,120 --> 00:20:14,520
It's a database with better search capabilities.

603
00:20:14,520 --> 00:20:16,360
In the AI context, search becomes

604
00:20:16,360 --> 00:20:18,040
something fundamentally different.

605
00:20:18,040 --> 00:20:19,320
It's the grounding mechanism.

606
00:20:19,320 --> 00:20:20,800
It's how you prevent hallucinations.

607
00:20:20,800 --> 00:20:22,560
When an LLM generates text,

608
00:20:22,560 --> 00:20:25,040
you want it grounded in facts from your knowledge base,

609
00:20:25,040 --> 00:20:27,120
not from patterns in its training data.

610
00:20:27,120 --> 00:20:28,800
Search is what makes that possible.

611
00:20:28,800 --> 00:20:30,080
It retrieves relevant context.

612
00:20:30,080 --> 00:20:32,200
The LLM uses that context to reason.

613
00:20:32,200 --> 00:20:34,520
Without it, you're asking the model to make things up.

614
00:20:34,520 --> 00:20:37,280
This reframing changes how you deploy it through bicep.

615
00:20:37,280 --> 00:20:40,440
In the portal, someone creates an Azure AI search service.

616
00:20:40,440 --> 00:20:41,440
They index documents.

617
00:20:41,440 --> 00:20:42,280
They're done.

618
00:20:42,280 --> 00:20:43,800
It works in isolation, but when you structure it

619
00:20:43,800 --> 00:20:45,280
through bicep for AI reasoning,

620
00:20:45,280 --> 00:20:47,800
the service exists as part of a reasoning pipeline.

621
00:20:47,800 --> 00:20:49,760
The module doesn't just create the search service.

622
00:20:49,760 --> 00:20:51,600
It creates the service, the private endpoints,

623
00:20:51,600 --> 00:20:54,000
the managed identities, the vector configuration,

624
00:20:54,000 --> 00:20:55,920
and the integration with the embedding model.

625
00:20:55,920 --> 00:20:57,760
All of it deployed as a single unit.

626
00:20:57,760 --> 00:20:59,360
Here's what that module must handle.

627
00:20:59,360 --> 00:21:01,040
First, the search service itself.

628
00:21:01,040 --> 00:21:03,120
It needs to exist with public access disabled,

629
00:21:03,120 --> 00:21:05,120
not disabled later as a follow-up task,

630
00:21:05,120 --> 00:21:06,960
disabled from the moment it deploys.

631
00:21:06,960 --> 00:21:08,720
The bicep module creates the service

632
00:21:08,720 --> 00:21:10,560
with that configuration baked in.

633
00:21:10,560 --> 00:21:12,600
Anyone using the module gets a search service

634
00:21:12,600 --> 00:21:14,320
that's locked down by default.

635
00:21:14,320 --> 00:21:16,560
Second, private endpoints for client access.

636
00:21:16,560 --> 00:21:19,120
The module creates the endpoint in the correct subnet.

637
00:21:19,120 --> 00:21:21,200
It links private DNS zones so clients

638
00:21:21,200 --> 00:21:24,040
can resolve the search service name to the private IP.

639
00:21:24,040 --> 00:21:27,120
Without this, any application trying to query the search service

640
00:21:27,120 --> 00:21:29,120
would either need to configure complex DNS

641
00:21:29,120 --> 00:21:31,040
or root across the public internet.

642
00:21:31,040 --> 00:21:32,960
The module eliminates that complexity.

643
00:21:32,960 --> 00:21:35,720
Third, managed identities for indexer access to data sources.

644
00:21:35,720 --> 00:21:36,680
This is critical.

645
00:21:36,680 --> 00:21:38,720
When search indexes documents from SharePoint

646
00:21:38,720 --> 00:21:41,360
or a SQL database, it needs to authenticate.

647
00:21:41,360 --> 00:21:43,280
The old way is to store credentials.

648
00:21:43,280 --> 00:21:45,920
The bicep way is to create a managed identity,

649
00:21:45,920 --> 00:21:48,080
assign it permissions on the data source,

650
00:21:48,080 --> 00:21:50,000
and let the indexer use that identity.

651
00:21:50,000 --> 00:21:52,880
No credentials stored, no secrets to leak.

652
00:21:52,880 --> 00:21:56,280
Fourth, vector profiles and embedding model configuration.

653
00:21:56,280 --> 00:21:57,920
This is where the reasoning happens.

654
00:21:57,920 --> 00:22:00,000
The module specifies which embedding model

655
00:22:00,000 --> 00:22:02,000
should generate vectors for documents.

656
00:22:02,000 --> 00:22:04,120
It defines the vector profile,

657
00:22:04,120 --> 00:22:06,680
the algorithm that will be used for similarity search,

658
00:22:06,680 --> 00:22:08,960
the dimensions of the vectors, the distance metric.

659
00:22:08,960 --> 00:22:11,280
This configuration ties search directly to your embedding

660
00:22:11,280 --> 00:22:11,880
service.

661
00:22:11,880 --> 00:22:12,640
They're not separate.

662
00:22:12,640 --> 00:22:14,200
There is single unit.

663
00:22:14,200 --> 00:22:16,320
Fifth, integrated vectorization.

664
00:22:16,320 --> 00:22:18,120
This is the pipeline that chunks documents,

665
00:22:18,120 --> 00:22:20,880
generates embeddings and indexes them automatically.

666
00:22:20,880 --> 00:22:23,760
The module wires this up so that documents flowing into search

667
00:22:23,760 --> 00:22:25,760
are chunked according to your strategy,

668
00:22:25,760 --> 00:22:27,640
embedded using your embedding model,

669
00:22:27,640 --> 00:22:29,320
and indexed into vector fields.

670
00:22:29,320 --> 00:22:31,800
It's one pipeline, not three separate operations.

671
00:22:31,800 --> 00:22:33,360
The module structure separates concerns,

672
00:22:33,360 --> 00:22:34,800
but they all deploy together.

673
00:22:34,800 --> 00:22:37,240
The network concern, private endpoints and DNS

674
00:22:37,240 --> 00:22:38,600
is isolated in code.

675
00:22:38,600 --> 00:22:41,120
The identity concern, managed identities and role assignments,

676
00:22:41,120 --> 00:22:41,840
is clear.

677
00:22:41,840 --> 00:22:43,640
The search configuration is distinct.

678
00:22:43,640 --> 00:22:45,440
The vectorization pipeline is explicit,

679
00:22:45,440 --> 00:22:47,840
but when you deploy the module, they're wired together.

680
00:22:47,840 --> 00:22:49,040
They're a system.

681
00:22:49,040 --> 00:22:50,960
This modularity creates flexibility

682
00:22:50,960 --> 00:22:52,800
that manual deployments never have.

683
00:22:52,800 --> 00:22:55,040
Six months in, you realize your chunking strategy

684
00:22:55,040 --> 00:22:55,840
isn't optimal.

685
00:22:55,840 --> 00:22:57,720
Documents are being split at sentence boundaries,

686
00:22:57,720 --> 00:23:00,040
but your use case needs paragraph boundaries.

687
00:23:00,040 --> 00:23:02,040
In a manual deployment, fixing this means

688
00:23:02,040 --> 00:23:03,800
going back to the indexer definition,

689
00:23:03,800 --> 00:23:05,400
changing the chunking configuration

690
00:23:05,400 --> 00:23:07,680
and redeploying everything connected to search.

691
00:23:07,680 --> 00:23:09,320
With Bicep, you update the module.

692
00:23:09,320 --> 00:23:12,280
You change one parameter that controls chunking strategy.

693
00:23:12,280 --> 00:23:13,160
You redeploy.

694
00:23:13,160 --> 00:23:14,600
The indexer is updated.

695
00:23:14,600 --> 00:23:16,680
Documents are re-indexed with the new strategy.

696
00:23:16,680 --> 00:23:18,080
Everything else stays the same.

697
00:23:18,080 --> 00:23:19,960
No risk of accidentally breaking something else

698
00:23:19,960 --> 00:23:22,480
because you weren't manually configuring every piece.

699
00:23:22,480 --> 00:23:24,480
Without this structure, you're changing infrastructure

700
00:23:24,480 --> 00:23:25,240
in the dark.

701
00:23:25,240 --> 00:23:29,240
With Bicep, you're updating a known, tested, reusable pattern.

702
00:23:29,240 --> 00:23:30,920
This is how you scale reasoning.

703
00:23:30,920 --> 00:23:32,480
Not by deploying search once,

704
00:23:32,480 --> 00:23:34,160
but by having a module, you can deploy

705
00:23:34,160 --> 00:23:36,560
50 times in 50 different contexts.

706
00:23:36,560 --> 00:23:38,960
Knowing every deployment is identical, compliant,

707
00:23:38,960 --> 00:23:41,800
and grounded in the right vector strategy.

708
00:23:41,800 --> 00:23:43,320
Private endpoints is code.

709
00:23:43,320 --> 00:23:45,640
The network parameter is where most deployments fail.

710
00:23:45,640 --> 00:23:46,960
Here's how Bicep fixes it.

711
00:23:46,960 --> 00:23:49,200
Private endpoints aren't optional for enterprise AI.

712
00:23:49,200 --> 00:23:50,320
They're not an add-on.

713
00:23:50,320 --> 00:23:52,440
They're not something you implement if you have time.

714
00:23:52,440 --> 00:23:53,960
They're the foundational mechanism

715
00:23:53,960 --> 00:23:56,440
that keeps LLM traffic off the public internet

716
00:23:56,440 --> 00:23:58,960
and without them, you're accepting data exposure

717
00:23:58,960 --> 00:24:00,240
as a cost of doing business.

718
00:24:00,240 --> 00:24:02,480
Here's what most organizations don't understand.

719
00:24:02,480 --> 00:24:04,920
When you connect an LLM to your knowledge base,

720
00:24:04,920 --> 00:24:06,760
that connection is happening somewhere.

721
00:24:06,760 --> 00:24:09,160
Either it's happening across a private network backbone

722
00:24:09,160 --> 00:24:10,320
that Azure controls,

723
00:24:10,320 --> 00:24:12,320
or it's happening across the public internet.

724
00:24:12,320 --> 00:24:13,520
There's no middle ground.

725
00:24:13,520 --> 00:24:16,160
The question isn't whether the traffic has an attack surface.

726
00:24:16,160 --> 00:24:18,120
The question is how large that surface is.

727
00:24:18,120 --> 00:24:21,080
Manual private endpoint creation invites inconsistency.

728
00:24:21,080 --> 00:24:24,240
One team creates a private endpoint for Azure OpenAI,

729
00:24:24,240 --> 00:24:25,760
configures the DNS correctly,

730
00:24:25,760 --> 00:24:27,200
and sets up the NSG rules.

731
00:24:27,200 --> 00:24:29,960
Another team creates a private endpoint for AI search,

732
00:24:29,960 --> 00:24:31,680
but forgets to disable public access

733
00:24:31,680 --> 00:24:32,880
on the search service itself.

734
00:24:32,880 --> 00:24:34,920
Now the endpoint exists, but it's useless,

735
00:24:34,920 --> 00:24:37,680
because traffic can still flow to the public endpoint.

736
00:24:37,680 --> 00:24:40,240
A third team creates a private endpoint for storage,

737
00:24:40,240 --> 00:24:43,200
links it to the wrong DNS zone, so name resolution fails.

738
00:24:43,200 --> 00:24:44,520
Applications can't connect

739
00:24:44,520 --> 00:24:47,240
even though the infrastructure is technically in place.

740
00:24:47,240 --> 00:24:50,360
Bicet modules solve this by encoding the complete pattern in code.

741
00:24:50,360 --> 00:24:52,200
The module doesn't just create the endpoint.

742
00:24:52,200 --> 00:24:54,320
It creates the endpoint in the correct subnet.

743
00:24:54,320 --> 00:24:57,160
It links the private DNS zone for name resolution.

744
00:24:57,160 --> 00:25:00,320
It enforces NSG rules that restrict traffic to that endpoint.

745
00:25:00,320 --> 00:25:01,920
It validates that the target service

746
00:25:01,920 --> 00:25:03,760
has public access disabled.

747
00:25:03,760 --> 00:25:05,680
All of this happens in a single deployment.

748
00:25:05,680 --> 00:25:08,160
All of it is documented, all of it can be audited.

749
00:25:08,160 --> 00:25:10,080
The critical insight is reusability.

750
00:25:10,080 --> 00:25:11,960
A single bicep module for private endpoints

751
00:25:11,960 --> 00:25:14,920
can be reused for Azure OpenAI, AI search, storage,

752
00:25:14,920 --> 00:25:16,720
key vault, SQL databases,

753
00:25:16,720 --> 00:25:19,280
and any other Azure service that supports private link.

754
00:25:19,280 --> 00:25:21,600
You're not writing five different modules.

755
00:25:21,600 --> 00:25:23,520
You're writing one module that's parameterized

756
00:25:23,520 --> 00:25:24,920
for different services.

757
00:25:24,920 --> 00:25:27,440
This matters because consistency becomes automatic.

758
00:25:27,440 --> 00:25:29,000
When you deploy Azure OpenAI

759
00:25:29,000 --> 00:25:30,560
through the same private endpoint module

760
00:25:30,560 --> 00:25:33,720
you used for storage, they follow identical security patterns.

761
00:25:33,720 --> 00:25:35,760
The endpoint is created in the same subnet.

762
00:25:35,760 --> 00:25:37,520
The DNS zone linking is identical.

763
00:25:37,520 --> 00:25:39,240
The NSG rules follow the same logic.

764
00:25:39,240 --> 00:25:41,960
You're not hoping that all your private endpoints are secure.

765
00:25:41,960 --> 00:25:44,920
You know they are because they all came from the same template.

766
00:25:44,920 --> 00:25:47,600
The module handles specifics that vary by service.

767
00:25:47,600 --> 00:25:51,320
The group IDs differ between Azure OpenAI and AI search, for example.

768
00:25:51,320 --> 00:25:53,320
But the structural pattern is identical.

769
00:25:53,320 --> 00:25:55,840
The module abstracts away the service-specific details

770
00:25:55,840 --> 00:25:57,960
and enforces the universal principles.

771
00:25:57,960 --> 00:26:00,720
Private network first, name resolution configured,

772
00:26:00,720 --> 00:26:03,320
traffic restricted, public access disabled.

773
00:26:03,320 --> 00:26:07,040
Manual private endpoint creation leaves gaps, not intentionally.

774
00:26:07,040 --> 00:26:10,120
But because the process requires multiple steps and humans missteps

775
00:26:10,120 --> 00:26:11,120
you create the endpoint.

776
00:26:11,120 --> 00:26:12,680
Did you link the DNS zone?

777
00:26:12,680 --> 00:26:14,200
Probably, did you link it to the right zone?

778
00:26:14,200 --> 00:26:15,000
Maybe.

779
00:26:15,000 --> 00:26:18,280
Did you verify that the target service has public access disabled?

780
00:26:18,280 --> 00:26:19,760
Possibly, but it's easy to forget.

781
00:26:19,760 --> 00:26:23,040
Did you document why you chose that specific subnet for the endpoint?

782
00:26:23,040 --> 00:26:25,280
No, the next person deploying a similar endpoint

783
00:26:25,280 --> 00:26:26,880
has to guess at your decisions.

784
00:26:26,880 --> 00:26:28,640
Bicep modules eliminate the gaps

785
00:26:28,640 --> 00:26:30,480
because the module is the documentation.

786
00:26:30,480 --> 00:26:32,400
When someone wants to know why a private endpoint

787
00:26:32,400 --> 00:26:35,440
was created in that specific subnet, the answer is in the code.

788
00:26:35,440 --> 00:26:37,440
When they want to understand the naming convention

789
00:26:37,440 --> 00:26:39,880
for the DNS zone link, it's in the module.

790
00:26:39,880 --> 00:26:43,080
When they need to replicate the pattern, the module is the blueprint.

791
00:26:43,080 --> 00:26:46,840
No gaps, no guessing, no forgotten steps.

792
00:26:46,840 --> 00:26:49,120
This is how you scale the network perimeter,

793
00:26:49,120 --> 00:26:52,360
not by hoping that every team configures private endpoints correctly,

794
00:26:52,360 --> 00:26:56,320
but by making the configuration automatic, reusable, and identical

795
00:26:56,320 --> 00:26:57,720
across every deployment.

796
00:26:57,720 --> 00:27:00,240
The network perimeter isn't something you check after the fact.

797
00:27:00,240 --> 00:27:02,200
It's something that can't be deployed incorrectly

798
00:27:02,200 --> 00:27:04,760
because the code enforces correctness.

799
00:27:04,760 --> 00:27:07,240
Managed identities and the identity perimeter.

800
00:27:07,240 --> 00:27:10,640
The identity layer is where governance either succeeds or collapses.

801
00:27:10,640 --> 00:27:13,480
This is where the structural decision you make in bicep determines

802
00:27:13,480 --> 00:27:16,880
whether your AI infrastructure is auditable or opaque.

803
00:27:16,880 --> 00:27:19,440
Managed identities are the only way to eliminate secrets

804
00:27:19,440 --> 00:27:20,520
from your infrastructure.

805
00:27:20,520 --> 00:27:22,080
Not reduce them, not hide them better.

806
00:27:22,080 --> 00:27:25,880
Eliminate them because once a secret exists in a configuration file

807
00:27:25,880 --> 00:27:28,800
in environment variable or a connection string, it exists.

808
00:27:28,800 --> 00:27:30,520
Someone will find it, someone will copy it,

809
00:27:30,520 --> 00:27:32,320
someone will put it somewhere it shouldn't be.

810
00:27:32,320 --> 00:27:35,640
The life cycle of a secret once created is inevitably toward exposure.

811
00:27:35,640 --> 00:27:38,280
Think about what happens without bicep in this layer.

812
00:27:38,280 --> 00:27:41,760
A developer needs to authenticate an application to Azure Open AI.

813
00:27:41,760 --> 00:27:44,040
The easy path is to create an API key, copy it,

814
00:27:44,040 --> 00:27:46,240
paste it into a just net configuration file,

815
00:27:46,240 --> 00:27:48,200
or into an environment variable on the VM

816
00:27:48,200 --> 00:27:50,000
or into a parameter on the function app.

817
00:27:50,000 --> 00:27:52,680
The key is now in multiple places, multiple people have access to it.

818
00:27:52,680 --> 00:27:56,400
It's inversion control history probably, it's been shared in Slack maybe.

819
00:27:56,400 --> 00:27:58,720
The likelihood that it stays secret approaches zero.

820
00:27:58,720 --> 00:28:00,720
Managed identities invert this dynamic.

821
00:28:00,720 --> 00:28:03,400
There are cryptographic identities that Azure manages for you.

822
00:28:03,400 --> 00:28:05,080
You don't create them, you don't see them.

823
00:28:05,080 --> 00:28:06,320
You don't copy them anywhere.

824
00:28:06,320 --> 00:28:09,920
The infrastructure declares that a service needs to authenticate as itself

825
00:28:09,920 --> 00:28:11,400
and Azure handles the mechanics.

826
00:28:11,400 --> 00:28:13,360
No human ever touches the credential.

827
00:28:13,360 --> 00:28:15,200
Bicep is where this declaration happens

828
00:28:15,200 --> 00:28:18,600
and it's where the identity parameter becomes structural or becomes a wish.

829
00:28:18,600 --> 00:28:21,920
The module must define system assigned identities on compute resources.

830
00:28:21,920 --> 00:28:25,520
When you deploy a VM that hosts your reasoning orchestration layer

831
00:28:25,520 --> 00:28:31,120
or a function app that coordinates LLM calls or an AKS cluster running AI agents

832
00:28:31,120 --> 00:28:33,680
that resource needs a system assigned identity.

833
00:28:33,680 --> 00:28:36,480
This identity belongs to the resource, it can't be shared.

834
00:28:36,480 --> 00:28:39,680
It can't be compromised through key exposure because there's no key.

835
00:28:39,680 --> 00:28:43,200
Azure issues a cryptographic token when the resource needs to authenticate.

836
00:28:43,200 --> 00:28:45,600
The resource uses the token, the token expires.

837
00:28:45,600 --> 00:28:48,200
A new one is issued no human interaction.

838
00:28:48,200 --> 00:28:52,640
The module must also define user assigned identities for shared services.

839
00:28:52,640 --> 00:28:55,680
Some services need to act on behalf of multiple resources.

840
00:28:55,680 --> 00:28:58,480
An indexer might need to read from multiple data sources.

841
00:28:58,480 --> 00:29:02,040
An agent might need to access different services for different types of queries.

842
00:29:02,040 --> 00:29:04,960
A user assigned identity can be attached to multiple resources,

843
00:29:04,960 --> 00:29:09,120
letting them all authenticate as that identity without each needing its own key.

844
00:29:09,120 --> 00:29:11,520
But here's where most manual deployments break.

845
00:29:11,520 --> 00:29:12,760
RBAC assignments.

846
00:29:12,760 --> 00:29:16,480
You can create the identity but if you don't assign it the right roles, it's useless.

847
00:29:16,480 --> 00:29:17,600
It has no permissions.

848
00:29:17,600 --> 00:29:21,240
When the application tries to authenticate and access a resource, it fails.

849
00:29:21,240 --> 00:29:23,720
In manual deployments, this is a separate step.

850
00:29:23,720 --> 00:29:27,000
You create the identity then you remember maybe to assign it roles.

851
00:29:27,000 --> 00:29:28,280
Maybe you assign the right roles.

852
00:29:28,280 --> 00:29:31,200
Maybe you assign too many roles to make the application work,

853
00:29:31,200 --> 00:29:34,160
violating least privilege, bicep handles this as a unit.

854
00:29:34,160 --> 00:29:37,440
The module creates the identity and assigns the roles in the same declaration.

855
00:29:37,440 --> 00:29:38,560
They're not separate decisions.

856
00:29:38,560 --> 00:29:40,440
They're part of the same infrastructure definition.

857
00:29:40,440 --> 00:29:43,200
The identity exists with exactly the permissions it needs.

858
00:29:43,200 --> 00:29:44,880
Not more, not less.

859
00:29:44,880 --> 00:29:47,560
The module must also handle key vault access policies.

860
00:29:47,560 --> 00:29:51,040
Some services legitimately need secrets, database passwords,

861
00:29:51,040 --> 00:29:54,320
encryption keys, API credentials for external systems.

862
00:29:54,320 --> 00:29:55,920
These secrets live in key vault.

863
00:29:55,920 --> 00:29:57,560
But the service can't just read any secret.

864
00:29:57,560 --> 00:30:01,440
It can only read the secrets it's authorized to read through a specific access policy

865
00:30:01,440 --> 00:30:02,960
tied to its managed identity.

866
00:30:02,960 --> 00:30:04,720
Again, bicep makes the structural.

867
00:30:04,720 --> 00:30:06,720
When you deploy a service through the module,

868
00:30:06,720 --> 00:30:09,840
it's granted access to the specific secrets it needs from key vault.

869
00:30:09,840 --> 00:30:11,360
Not access to the vault in general.

870
00:30:11,360 --> 00:30:12,960
Not access to all secrets.

871
00:30:12,960 --> 00:30:14,560
Access to these specific secrets.

872
00:30:14,560 --> 00:30:16,280
The policy is defined in code,

873
00:30:16,280 --> 00:30:18,240
auditable and reviewable.

874
00:30:18,240 --> 00:30:22,560
The identity perimeter fails if any component uses a key instead of an identity.

875
00:30:22,560 --> 00:30:26,400
A single API key embedded in configuration breaks the entire model.

876
00:30:26,400 --> 00:30:30,480
bicep enforces this by making identities the default and keys the exception.

877
00:30:30,480 --> 00:30:32,800
The module structure makes identities automatic.

878
00:30:32,800 --> 00:30:36,800
Creating a key requires a deliberate override, which stands out in code review.

879
00:30:36,800 --> 00:30:38,240
This is structural governance.

880
00:30:38,240 --> 00:30:40,080
Not a policy you hope teams follow.

881
00:30:40,080 --> 00:30:44,080
A code pattern that makes the right choice the path of least resistance.

882
00:30:44,080 --> 00:30:46,320
As you are policy as the reasoning framework,

883
00:30:46,320 --> 00:30:48,080
bicep deploys the infrastructure,

884
00:30:48,080 --> 00:30:50,320
but deployment is only half the equation.

885
00:30:50,320 --> 00:30:52,720
Once it's deployed, it needs to stay compliant.

886
00:30:52,720 --> 00:30:54,320
And that's where Azure Policy enters.

887
00:30:54,320 --> 00:30:58,480
Azure Policy is the control plane for everything that comes after infrastructure exists.

888
00:30:58,480 --> 00:31:00,000
It's the enforcement layer that says,

889
00:31:00,000 --> 00:31:00,960
these are the rules.

890
00:31:00,960 --> 00:31:03,280
Every resource deployed in this subscription

891
00:31:03,280 --> 00:31:04,800
must conform to these rules.

892
00:31:04,800 --> 00:31:06,720
If it doesn't, we block it.

893
00:31:06,720 --> 00:31:08,320
Think of what happens without policy.

894
00:31:08,320 --> 00:31:09,760
You deploy your bicep modules.

895
00:31:09,760 --> 00:31:11,200
The identity perimeter is correct.

896
00:31:11,200 --> 00:31:12,800
The network perimeter is locked down.

897
00:31:12,800 --> 00:31:15,600
But then someone else in the organization working independently

898
00:31:15,600 --> 00:31:17,840
deploys an AI service directly in the portal.

899
00:31:17,840 --> 00:31:19,200
They skip the private endpoint.

900
00:31:19,200 --> 00:31:21,200
They enable public access to save time.

901
00:31:21,200 --> 00:31:22,960
They justify it as temporary.

902
00:31:22,960 --> 00:31:26,160
Three months later, that temporary deployment is still running.

903
00:31:26,160 --> 00:31:29,200
Still public facing, still violating your governance baseline.

904
00:31:29,200 --> 00:31:32,560
Azure Policy prevents this by treating governance as a technical enforcement,

905
00:31:32,560 --> 00:31:34,000
not a policy document.

906
00:31:34,000 --> 00:31:36,240
Policy definitions are rules written as code.

907
00:31:36,240 --> 00:31:39,840
They can restrict which models are deployed from the Azure AI Foundry catalog.

908
00:31:39,840 --> 00:31:42,560
They can enforce private endpoints on all AI services,

909
00:31:42,560 --> 00:31:46,400
making it technically impossible to deploy an Azure Open AI service without one.

910
00:31:46,400 --> 00:31:48,720
They can require specific regions for data residency.

911
00:31:48,720 --> 00:31:51,520
If you're in a jurisdiction that mandates where data physically lives,

912
00:31:51,520 --> 00:31:53,680
they can mandate encryption at rest and in transit,

913
00:31:53,680 --> 00:31:56,960
ensuring that every deployment meets security baselines automatically.

914
00:31:56,960 --> 00:31:58,400
The key insight is this.

915
00:31:58,400 --> 00:32:01,120
Policy should be written in code, not enforced manually.

916
00:32:01,120 --> 00:32:02,880
When policy lives only in documents,

917
00:32:02,880 --> 00:32:05,280
governance frameworks printed on shared drives,

918
00:32:05,280 --> 00:32:08,560
compliance checklists scattered across wikis, it's aspirational.

919
00:32:08,560 --> 00:32:10,480
It describes what should happen.

920
00:32:10,480 --> 00:32:13,040
Azure Policy makes it happen automatically.

921
00:32:13,040 --> 00:32:15,920
Bicep and Azure Policy are designed to work as a pair.

922
00:32:15,920 --> 00:32:19,520
Bicep deploys the infrastructure according to the architecture you've defined.

923
00:32:19,520 --> 00:32:23,840
Azure Policy enforces that the infrastructure conforms to governance rules you've written.

924
00:32:23,840 --> 00:32:27,600
The moment someone tries to deploy something that violates policy,

925
00:32:27,600 --> 00:32:28,800
Azure doesn't deploy it.

926
00:32:28,800 --> 00:32:29,520
There's no debate.

927
00:32:29,520 --> 00:32:31,200
There's no exception process.

928
00:32:31,200 --> 00:32:32,480
The deployment fails.

929
00:32:32,480 --> 00:32:35,600
Azure Policy returns an error explaining which rule was violated

930
00:32:35,600 --> 00:32:37,520
and why the resource can't be created.

931
00:32:37,520 --> 00:32:40,320
This is governance by design, not governance by exception.

932
00:32:40,320 --> 00:32:42,880
Most organizations run governance by exception.

933
00:32:42,880 --> 00:32:43,840
You have a rule.

934
00:32:43,840 --> 00:32:44,640
Someone breaks it.

935
00:32:44,640 --> 00:32:45,680
You grant an exception.

936
00:32:45,680 --> 00:32:47,120
The exception becomes permanent.

937
00:32:47,120 --> 00:32:48,640
Three exceptions become a new baseline

938
00:32:48,640 --> 00:32:51,280
before long nobody remembers what the original rule was.

939
00:32:51,280 --> 00:32:52,880
Governance by design inverts this.

940
00:32:52,880 --> 00:32:54,000
The rule is enforcement.

941
00:32:54,000 --> 00:32:57,280
Exceptions require explicit override which creates an audit trail.

942
00:32:57,280 --> 00:32:59,840
If someone needs to deploy a resource outside policy,

943
00:32:59,840 --> 00:33:01,200
they can't do it silently.

944
00:33:01,200 --> 00:33:02,880
They have to declare the exception,

945
00:33:02,880 --> 00:33:04,160
document why it's needed,

946
00:33:04,160 --> 00:33:05,840
and explain when it expires.

947
00:33:05,840 --> 00:33:07,200
The override is visible.

948
00:33:07,200 --> 00:33:07,920
It's trackable.

949
00:33:09,440 --> 00:33:12,160
When you combine bicep modules with Azure Policy,

950
00:33:12,160 --> 00:33:13,600
something powerful happens.

951
00:33:13,600 --> 00:33:16,480
Teams deploy using the modules because that's the fastest path.

952
00:33:16,480 --> 00:33:19,680
The modules create resources that conform to policy by default,

953
00:33:19,680 --> 00:33:22,960
even if someone tries to deploy a non-compliant resource directly,

954
00:33:22,960 --> 00:33:24,000
policy blocks it.

955
00:33:24,000 --> 00:33:27,040
You've created a system where compliance is easier than non-compliance.

956
00:33:27,040 --> 00:33:28,240
For AI specifically,

957
00:33:28,240 --> 00:33:30,080
this changes governance dramatically.

958
00:33:30,080 --> 00:33:32,640
Policy can restrict which models teams are permitted to deploy

959
00:33:32,640 --> 00:33:33,920
from the Foundry catalog.

960
00:33:33,920 --> 00:33:35,600
This isn't about preventing innovation.

961
00:33:35,600 --> 00:33:37,280
It's about preventing chaos.

962
00:33:37,280 --> 00:33:40,160
If your organization has decided that only models that have passed

963
00:33:40,160 --> 00:33:42,560
internal red teaming are approved for production,

964
00:33:42,560 --> 00:33:43,840
policy enforces that.

965
00:33:43,840 --> 00:33:46,480
When a team tries to deploy an unapproved model,

966
00:33:46,480 --> 00:33:47,040
they can't.

967
00:33:47,040 --> 00:33:49,360
They don't discover this limitation after deployment.

968
00:33:49,360 --> 00:33:52,080
They discover it during development when they can still pivot

969
00:33:52,080 --> 00:33:53,760
to an approved alternative.

970
00:33:53,760 --> 00:33:57,120
Policy can mandate that all AI services have private endpoints,

971
00:33:57,120 --> 00:33:58,400
not as a recommendation.

972
00:33:58,400 --> 00:34:00,880
As a requirement, the moment someone tries to deploy

973
00:34:00,880 --> 00:34:02,640
as you open AI with public access,

974
00:34:02,640 --> 00:34:03,440
policy blocks it.

975
00:34:03,440 --> 00:34:04,560
There's no negotiation.

976
00:34:04,560 --> 00:34:06,800
There's no temporary exemption that becomes permanent.

977
00:34:06,800 --> 00:34:08,560
The only way to deploy is the secure way.

978
00:34:08,560 --> 00:34:10,960
This is where the control plane concept becomes real.

979
00:34:10,960 --> 00:34:12,480
Bicep describes the infrastructure.

980
00:34:12,480 --> 00:34:15,680
Policy describes the rules that infrastructure must follow.

981
00:34:15,680 --> 00:34:18,800
Together, they create a system where correct deployment is automated

982
00:34:18,800 --> 00:34:20,880
and incorrect deployment is prevented,

983
00:34:20,880 --> 00:34:21,920
not just detected.

984
00:34:21,920 --> 00:34:23,920
When a deployment violates policy,

985
00:34:23,920 --> 00:34:25,440
as your blocks it automatically,

986
00:34:25,440 --> 00:34:27,440
the failure message tells you why.

987
00:34:27,440 --> 00:34:30,400
You update your infrastructure to conform and deployment succeeds.

988
00:34:30,400 --> 00:34:31,920
This creates a feedback loop

989
00:34:31,920 --> 00:34:33,920
where teams learn the governance rules

990
00:34:33,920 --> 00:34:34,960
through interaction,

991
00:34:34,960 --> 00:34:36,640
not through reading documentation.

992
00:34:36,640 --> 00:34:38,560
This is how you scale governance in an enterprise,

993
00:34:38,560 --> 00:34:40,160
not by hoping teams follow rules,

994
00:34:40,160 --> 00:34:42,160
but by making the rules technical requirements

995
00:34:42,160 --> 00:34:43,360
that the platform enforces.

996
00:34:43,360 --> 00:34:46,480
Management groups and the organizational model

997
00:34:46,480 --> 00:34:48,160
bicep deploys resources,

998
00:34:48,160 --> 00:34:49,680
management groups organize them.

999
00:34:49,680 --> 00:34:51,600
Together, they create accountability.

1000
00:34:51,600 --> 00:34:54,160
Here's the distinction that most organizations miss.

1001
00:34:54,160 --> 00:34:56,400
Azure subscriptions are the billing boundary.

1002
00:34:56,400 --> 00:34:58,240
They're where cost gets tracked and charged

1003
00:34:58,240 --> 00:35:00,080
to a department or business unit.

1004
00:35:00,080 --> 00:35:02,160
But subscriptions aren't where governance lives.

1005
00:35:02,160 --> 00:35:04,000
Governance lives in management groups.

1006
00:35:04,000 --> 00:35:06,320
The hierarchy that sits above subscriptions

1007
00:35:06,320 --> 00:35:09,280
and determines which rules apply to which workloads.

1008
00:35:09,280 --> 00:35:12,080
This hierarchy is critical because it's how you organize

1009
00:35:12,080 --> 00:35:14,880
your entire cloud estate as a coherent system,

1010
00:35:14,880 --> 00:35:16,240
not as isolated silos.

1011
00:35:16,240 --> 00:35:19,120
Management groups are the organizational structure for policies.

1012
00:35:19,120 --> 00:35:21,280
When you assign a policy to a management group,

1013
00:35:21,280 --> 00:35:23,120
every subscription under that management group

1014
00:35:23,120 --> 00:35:24,480
inherits that policy.

1015
00:35:24,480 --> 00:35:26,640
When you assign a policy at a subscription level,

1016
00:35:26,640 --> 00:35:28,160
only that subscription gets it.

1017
00:35:28,160 --> 00:35:30,080
The hierarchy determines scope.

1018
00:35:30,080 --> 00:35:31,760
For AI workloads specifically,

1019
00:35:31,760 --> 00:35:34,960
the structure typically separates concerns by workload class.

1020
00:35:35,200 --> 00:35:36,800
This separation isn't arbitrary.

1021
00:35:36,800 --> 00:35:38,960
It reflects different security, compliance,

1022
00:35:38,960 --> 00:35:40,560
and operational requirements.

1023
00:35:40,560 --> 00:35:41,920
The first category is COREP.

1024
00:35:41,920 --> 00:35:44,320
This is internal AI, systems that employees

1025
00:35:44,320 --> 00:35:46,000
use to improve their productivity

1026
00:35:46,000 --> 00:35:47,680
or that the organization uses internally

1027
00:35:47,680 --> 00:35:48,800
to automate processes.

1028
00:35:48,800 --> 00:35:50,560
A co-pilot for sales teams.

1029
00:35:50,560 --> 00:35:53,200
An agent that summarizes internal documentation.

1030
00:35:53,200 --> 00:35:55,200
A system that analyzes your own data.

1031
00:35:55,200 --> 00:35:57,040
These systems touch proprietary information

1032
00:35:57,040 --> 00:35:59,280
but they're not exposed to external users.

1033
00:35:59,280 --> 00:36:00,800
The security baseline is high.

1034
00:36:00,800 --> 00:36:02,640
Data classification is strict.

1035
00:36:02,640 --> 00:36:04,400
But the approval process can be faster

1036
00:36:04,400 --> 00:36:06,320
than systems exposed to the public.

1037
00:36:06,320 --> 00:36:07,920
The second category is online.

1038
00:36:07,920 --> 00:36:09,840
These are customer-facing AI systems,

1039
00:36:09,840 --> 00:36:11,760
public chatbots, external APIs,

1040
00:36:11,760 --> 00:36:13,600
systems that third parties can interact with.

1041
00:36:13,600 --> 00:36:14,880
Security is still critical,

1042
00:36:14,880 --> 00:36:16,560
but the threat model is different.

1043
00:36:16,560 --> 00:36:18,800
You're protecting against external attackers,

1044
00:36:18,800 --> 00:36:20,640
not just internal data leakage.

1045
00:36:20,640 --> 00:36:22,240
The data flowing through these systems

1046
00:36:22,240 --> 00:36:23,920
is typically less sensitive.

1047
00:36:23,920 --> 00:36:26,480
Information you've already decided to publish publicly

1048
00:36:26,480 --> 00:36:29,840
or information your customers provide to the system explicitly.

1049
00:36:29,840 --> 00:36:31,360
The third category is confidential.

1050
00:36:31,360 --> 00:36:32,720
These are regulated workloads.

1051
00:36:32,720 --> 00:36:35,120
AI systems that handle financial data,

1052
00:36:35,120 --> 00:36:37,200
healthcare information, personal data,

1053
00:36:37,200 --> 00:36:40,240
subject to GDPR or any other data classification

1054
00:36:40,240 --> 00:36:42,000
that demands encryption in use,

1055
00:36:42,000 --> 00:36:43,280
advanced audit trails,

1056
00:36:43,280 --> 00:36:45,200
and intense compliance scrutiny.

1057
00:36:45,200 --> 00:36:46,720
These deployments move slower.

1058
00:36:46,720 --> 00:36:48,480
They require additional approval steps.

1059
00:36:48,480 --> 00:36:50,480
They demand stronger evidence of governance,

1060
00:36:50,480 --> 00:36:52,720
but they're organized in their own management group

1061
00:36:52,720 --> 00:36:54,240
so that the policies governing them

1062
00:36:54,240 --> 00:36:56,240
don't inadvertently restrict innovation

1063
00:36:56,240 --> 00:36:58,240
in the corp and online categories.

1064
00:36:58,240 --> 00:37:01,040
Each management group inherits policies from its parent.

1065
00:37:01,040 --> 00:37:02,640
This inheritance is structural.

1066
00:37:02,640 --> 00:37:04,880
When you create a policy at the root level,

1067
00:37:04,880 --> 00:37:08,400
all AI resources must have diagnostic logging enabled.

1068
00:37:08,400 --> 00:37:11,360
That policy applies to corp, online, and confidential.

1069
00:37:11,360 --> 00:37:12,560
They all inherited.

1070
00:37:12,560 --> 00:37:14,240
This creates a universal baseline.

1071
00:37:14,240 --> 00:37:16,640
No AI workload, regardless of classification,

1072
00:37:16,640 --> 00:37:18,240
can be deployed without logging,

1073
00:37:18,240 --> 00:37:21,040
but policies can also be assigned at intermediate levels.

1074
00:37:21,040 --> 00:37:24,480
A policy saying data residency must be within the EU

1075
00:37:24,480 --> 00:37:27,440
applies only to subscriptions in a Europe-specific management group,

1076
00:37:27,440 --> 00:37:29,680
not to subscriptions in North America.

1077
00:37:29,680 --> 00:37:31,600
The hierarchy creates fine-grained control

1078
00:37:31,600 --> 00:37:33,440
while maintaining baseline consistency.

1079
00:37:33,440 --> 00:37:35,600
This inheritance is critical because it means

1080
00:37:35,600 --> 00:37:37,920
security baselines are applied automatically

1081
00:37:37,920 --> 00:37:39,120
to every new subscription.

1082
00:37:39,120 --> 00:37:40,640
When a new team creates a subscription

1083
00:37:40,640 --> 00:37:42,080
under the corp management group,

1084
00:37:42,080 --> 00:37:44,000
the policies governing the corp hierarchy

1085
00:37:44,000 --> 00:37:45,200
are automatically active.

1086
00:37:45,200 --> 00:37:47,360
That team doesn't have to know about the policies.

1087
00:37:47,360 --> 00:37:49,040
They don't have to manually assign them.

1088
00:37:49,040 --> 00:37:51,200
The policies are there because of the structure.

1089
00:37:51,200 --> 00:37:52,960
Bicet modules should be designed to work

1090
00:37:52,960 --> 00:37:55,200
within a specific management group context.

1091
00:37:55,200 --> 00:37:56,800
This doesn't mean the modules are aware

1092
00:37:56,800 --> 00:37:58,400
of the management group directly.

1093
00:37:58,400 --> 00:38:00,400
It means the modules are designed with the assumption

1094
00:38:00,400 --> 00:38:02,560
that certain policies are already in place.

1095
00:38:02,560 --> 00:38:04,640
A bicep module for deploying an AI workload

1096
00:38:04,640 --> 00:38:06,320
in the corp hierarchy can assume

1097
00:38:06,320 --> 00:38:08,720
that policies enforcing private endpoints are active.

1098
00:38:08,720 --> 00:38:10,400
The module doesn't need to include workarounds

1099
00:38:10,400 --> 00:38:11,200
for public access.

1100
00:38:11,200 --> 00:38:14,080
It builds infrastructure, assuming the policy enforced baseline

1101
00:38:14,080 --> 00:38:14,880
exists.

1102
00:38:14,880 --> 00:38:16,400
When you deploy a bicep module,

1103
00:38:16,400 --> 00:38:19,600
it inherits the policies of its management group automatically.

1104
00:38:19,600 --> 00:38:21,200
The module creates the resource.

1105
00:38:21,200 --> 00:38:23,600
The policy evaluates whether the resource conforms

1106
00:38:23,600 --> 00:38:25,120
if it does deployment succeeds,

1107
00:38:25,120 --> 00:38:26,800
if it doesn't, deployment fails.

1108
00:38:26,800 --> 00:38:29,200
This removes the need for manual policy assignment

1109
00:38:29,200 --> 00:38:31,680
because the policy is inherited through the hierarchy.

1110
00:38:31,680 --> 00:38:33,360
This creates a scaling dynamic

1111
00:38:33,360 --> 00:38:35,120
that manual systems can't achieve.

1112
00:38:35,120 --> 00:38:37,520
You're not assigning policies to individual resources

1113
00:38:37,520 --> 00:38:39,280
or individual subscriptions.

1114
00:38:39,280 --> 00:38:41,280
You're defining a management group hierarchy

1115
00:38:41,280 --> 00:38:43,520
that reflects your organizational structure,

1116
00:38:43,520 --> 00:38:45,840
assigning policies at appropriate levels,

1117
00:38:45,840 --> 00:38:47,920
and then letting that structure govern deployments

1118
00:38:47,920 --> 00:38:48,800
automatically.

1119
00:38:48,800 --> 00:38:51,680
The result is accountability that flows from structure,

1120
00:38:51,680 --> 00:38:52,480
not from hope.

1121
00:38:52,480 --> 00:38:54,640
Bicep and Azure AI Foundry Integration.

1122
00:38:54,640 --> 00:38:56,880
Now we connect bicep to the actual AI platform

1123
00:38:56,880 --> 00:38:57,840
as your AI Foundry.

1124
00:38:57,840 --> 00:38:59,520
If everything we've discussed so far,

1125
00:38:59,520 --> 00:39:01,520
the identity parameter, the network parameter,

1126
00:39:01,520 --> 00:39:03,840
the policy enforcement is the foundation.

1127
00:39:03,840 --> 00:39:06,320
Then Azure AI Foundry is the orchestration layer

1128
00:39:06,320 --> 00:39:08,240
where AI governance actually happens.

1129
00:39:08,240 --> 00:39:09,600
This is where models get selected.

1130
00:39:09,600 --> 00:39:11,120
Where data sources get connected,

1131
00:39:11,120 --> 00:39:12,400
where agents get deployed,

1132
00:39:12,400 --> 00:39:13,680
where reasoning happens.

1133
00:39:13,680 --> 00:39:15,200
Foundry is the unified control plane

1134
00:39:15,200 --> 00:39:17,520
for AI model deployment, evaluation, and governance.

1135
00:39:17,520 --> 00:39:18,640
It's not another service.

1136
00:39:18,640 --> 00:39:20,960
It's the workspace where intelligence gets assembled.

1137
00:39:20,960 --> 00:39:22,480
And the moment you understand that,

1138
00:39:22,480 --> 00:39:24,880
you understand why bicep becomes essential here.

1139
00:39:24,880 --> 00:39:27,200
In the portal, someone creates a Foundry Hub manually.

1140
00:39:27,200 --> 00:39:28,160
They select a region.

1141
00:39:28,160 --> 00:39:28,960
They pick a SKU.

1142
00:39:28,960 --> 00:39:30,400
They create a project underneath it.

1143
00:39:30,400 --> 00:39:32,240
They connect Azure OpenI to the project.

1144
00:39:32,240 --> 00:39:34,400
They link Azure AI Search for Retrieval.

1145
00:39:34,400 --> 00:39:35,840
They attach a data source.

1146
00:39:35,840 --> 00:39:37,760
Each connection is a separate decision point.

1147
00:39:37,760 --> 00:39:39,680
Each one is a place where something can go wrong

1148
00:39:39,680 --> 00:39:40,800
or get forgotten.

1149
00:39:40,800 --> 00:39:42,640
With bicep, the hub and all its connections

1150
00:39:42,640 --> 00:39:44,400
deploy as a unified system.

1151
00:39:44,400 --> 00:39:46,400
The module creates the hub with managed identities

1152
00:39:46,400 --> 00:39:47,280
already configured.

1153
00:39:47,280 --> 00:39:49,120
It creates projects with data classification

1154
00:39:49,120 --> 00:39:50,560
and compliance tags embedded.

1155
00:39:50,560 --> 00:39:52,560
It connects to search with the correct identity

1156
00:39:52,560 --> 00:39:53,600
and network settings.

1157
00:39:53,600 --> 00:39:55,600
That integrates OpenAI with the right endpoint

1158
00:39:55,600 --> 00:39:56,560
and access controls.

1159
00:39:56,560 --> 00:39:58,080
It attaches log analytics

1160
00:39:58,080 --> 00:40:00,560
so that every reasoning decision gets logged.

1161
00:40:00,560 --> 00:40:02,160
These aren't five separate deployments.

1162
00:40:02,160 --> 00:40:04,720
They're one coherent architecture expressed in code.

1163
00:40:04,720 --> 00:40:06,240
The hub becomes the governance boundary.

1164
00:40:06,240 --> 00:40:07,680
This is the critical distinction.

1165
00:40:07,680 --> 00:40:10,240
The hub defines what's available to teams working within it.

1166
00:40:10,240 --> 00:40:11,520
Which models can they access?

1167
00:40:11,520 --> 00:40:13,920
Only the ones you've added to the hub's model catalog.

1168
00:40:13,920 --> 00:40:15,520
Which data sources are available?

1169
00:40:15,520 --> 00:40:16,880
Only the ones you've connected.

1170
00:40:16,880 --> 00:40:18,720
Which tools can agents invoke?

1171
00:40:18,720 --> 00:40:20,080
Only the ones you've registered.

1172
00:40:20,080 --> 00:40:21,760
The hub isn't just a container.

1173
00:40:21,760 --> 00:40:23,120
It's a governance enforcement point.

1174
00:40:23,120 --> 00:40:26,160
Without foundry as a unified boundary, governance fragments.

1175
00:40:26,160 --> 00:40:28,800
One team connects a data source directly to an agent.

1176
00:40:28,800 --> 00:40:30,720
Another team uses the hub's connection.

1177
00:40:30,720 --> 00:40:32,240
A third team uses neither

1178
00:40:32,240 --> 00:40:33,760
and imports data into their agent

1179
00:40:33,760 --> 00:40:34,880
through a different mechanism.

1180
00:40:34,880 --> 00:40:37,200
Now you have three different data governance models

1181
00:40:37,200 --> 00:40:38,800
within the same organization.

1182
00:40:38,800 --> 00:40:41,760
When an auditor asks how data flows to your LLMs,

1183
00:40:41,760 --> 00:40:43,200
the answer is inconsistent.

1184
00:40:43,200 --> 00:40:44,560
With foundry as the boundary,

1185
00:40:44,560 --> 00:40:46,960
data flows through a single orchestration layer.

1186
00:40:46,960 --> 00:40:48,160
Teams don't bypass it.

1187
00:40:48,160 --> 00:40:48,880
They can't.

1188
00:40:48,880 --> 00:40:52,240
The hub is the only way to access approved models and data sources.

1189
00:40:52,240 --> 00:40:55,120
Governance becomes enforced by architecture, not by hope.

1190
00:40:55,120 --> 00:40:57,840
But here's where bicep matters most.

1191
00:40:57,840 --> 00:41:00,800
The hub is only a boundary if it's deployed consistently.

1192
00:41:00,800 --> 00:41:03,760
If one team creates a hub with strong arbare controls

1193
00:41:03,760 --> 00:41:05,440
and another creates a hub without them,

1194
00:41:05,440 --> 00:41:07,200
you have two different governance models

1195
00:41:07,200 --> 00:41:08,720
within the same organization.

1196
00:41:08,720 --> 00:41:12,000
Bicep ensures that every hub, every project, every connection

1197
00:41:12,000 --> 00:41:13,680
follows the same pattern.

1198
00:41:13,680 --> 00:41:15,680
The bicep module for foundry deployment

1199
00:41:15,680 --> 00:41:17,520
must handle a specific set of concerns.

1200
00:41:18,240 --> 00:41:20,960
Hub creation with managed identities in arbiac.

1201
00:41:20,960 --> 00:41:23,920
The identity perimeter extends into foundry.

1202
00:41:23,920 --> 00:41:27,280
Project creation with data classification and compliance tagging.

1203
00:41:27,280 --> 00:41:30,240
The governance tags that flow through your entire infrastructure

1204
00:41:30,240 --> 00:41:31,840
are embedded at the project level.

1205
00:41:31,840 --> 00:41:35,360
Connection to Azure AI Search for grounding.

1206
00:41:35,360 --> 00:41:37,840
The knowledge base is linked through managed identity

1207
00:41:37,840 --> 00:41:40,400
ensuring that only authenticated access happens.

1208
00:41:40,400 --> 00:41:43,280
Integration with Azure OpenAI for inference.

1209
00:41:43,280 --> 00:41:45,520
The reasoning engine is wired with the correct endpoint

1210
00:41:45,520 --> 00:41:46,240
and throttling.

1211
00:41:46,240 --> 00:41:48,640
Attachment to log analytics for observability.

1212
00:41:48,640 --> 00:41:50,560
Every agent decision, every model call,

1213
00:41:50,560 --> 00:41:52,960
every token consumption flows into unified logging.

1214
00:41:52,960 --> 00:41:54,960
These aren't optional capabilities you add later.

1215
00:41:54,960 --> 00:41:56,400
They're the default structure.

1216
00:41:56,400 --> 00:41:58,080
The module creates nothing without them.

1217
00:41:58,080 --> 00:41:59,680
No hub exists without arbac.

1218
00:41:59,680 --> 00:42:01,440
No project exists without tagging.

1219
00:42:01,440 --> 00:42:03,840
No data source connects without proper identity.

1220
00:42:03,840 --> 00:42:06,560
Without bicep, these connections are manual and inconsistent.

1221
00:42:06,560 --> 00:42:07,760
You get partial governance.

1222
00:42:07,760 --> 00:42:08,800
You get gaps.

1223
00:42:08,800 --> 00:42:11,520
You get the assumption that everyone will implement best practices

1224
00:42:11,520 --> 00:42:13,040
and the disappointment when they don't.

1225
00:42:13,040 --> 00:42:14,960
With bicep, the connections are automated,

1226
00:42:14,960 --> 00:42:16,240
repeatable and auditable.

1227
00:42:16,240 --> 00:42:18,160
The module is the governance enforcement.

1228
00:42:18,160 --> 00:42:21,600
Deploy twice and you get identical architecture both times.

1229
00:42:21,600 --> 00:42:22,880
No variation.

1230
00:42:22,880 --> 00:42:23,840
No gaps.

1231
00:42:23,840 --> 00:42:26,880
No prayer that this deployment is more secure than the last one.

1232
00:42:26,880 --> 00:42:29,680
The reasoning perimeter where foundry sits,

1233
00:42:29,680 --> 00:42:32,240
where orchestration happens, where governance becomes real,

1234
00:42:32,240 --> 00:42:33,680
only works if it's structured

1235
00:42:33,680 --> 00:42:35,760
and structure comes from code, not from hope.

1236
00:42:35,760 --> 00:42:38,160
Policy-driven model governance.

1237
00:42:38,160 --> 00:42:40,640
Here's where the reasoning layer enforces control.

1238
00:42:40,640 --> 00:42:41,600
Model selection.

1239
00:42:41,600 --> 00:42:42,960
The infrastructure is in place.

1240
00:42:42,960 --> 00:42:44,960
The identity perimeter secures access.

1241
00:42:44,960 --> 00:42:46,880
The network perimeter isolates traffic.

1242
00:42:46,880 --> 00:42:48,960
The foundry hub orchestrates connectivity.

1243
00:42:48,960 --> 00:42:50,640
But none of that matters if you can't control

1244
00:42:50,640 --> 00:42:52,320
which models actually get deployed.

1245
00:42:52,320 --> 00:42:54,400
A developer could choose an unapproved vendor.

1246
00:42:54,400 --> 00:42:56,560
Could select a model that hasn't been validated?

1247
00:42:56,560 --> 00:42:58,800
Could deploy reasoning engines that your organization

1248
00:42:58,800 --> 00:43:00,800
has deliberately decided not to use.

1249
00:43:00,800 --> 00:43:03,840
Without enforcement, infrastructure governance is meaningless

1250
00:43:03,840 --> 00:43:06,640
because the intelligence running on top of it is uncontrolled.

1251
00:43:06,640 --> 00:43:08,080
Azure Policy solves this.

1252
00:43:08,080 --> 00:43:11,360
By restricting which models can be deployed from the foundry catalog.

1253
00:43:11,360 --> 00:43:13,760
This matters because enterprises face real constraints

1254
00:43:13,760 --> 00:43:15,280
that models must satisfy.

1255
00:43:15,280 --> 00:43:16,640
Vendor risk is one of them.

1256
00:43:16,640 --> 00:43:18,240
If your organization has decided

1257
00:43:18,240 --> 00:43:21,040
that only Microsoft hosted models are acceptable in production

1258
00:43:21,040 --> 00:43:23,840
because you've negotiated specific data handling terms

1259
00:43:23,840 --> 00:43:26,080
or because you've completed due diligence on Microsoft

1260
00:43:26,080 --> 00:43:28,080
but not on OpenAI or Anthropic,

1261
00:43:28,080 --> 00:43:30,560
then a policy can encode that decision.

1262
00:43:30,560 --> 00:43:32,960
When a developer tries to deploy a non-Microsoft model,

1263
00:43:32,960 --> 00:43:36,160
the policy blocks it, no negotiation, no exception process.

1264
00:43:36,160 --> 00:43:39,120
The deployment fails and the developer learns the boundary immediately.

1265
00:43:39,120 --> 00:43:41,120
Licensing is another constraint.

1266
00:43:41,120 --> 00:43:44,720
Some organizations license certain models for specific use cases

1267
00:43:44,720 --> 00:43:47,840
and the licensing terms don't permit unlimited deployment.

1268
00:43:47,840 --> 00:43:49,760
Policy can enforce licensing boundaries.

1269
00:43:49,760 --> 00:43:51,920
A model might be approved for internal use

1270
00:43:51,920 --> 00:43:54,080
but not for customer facing applications.

1271
00:43:54,080 --> 00:43:55,920
Policy can encode that distinction,

1272
00:43:55,920 --> 00:43:58,720
blocking the model from deployment in production subscriptions

1273
00:43:58,720 --> 00:44:00,320
while allowing it in development.

1274
00:44:00,320 --> 00:44:01,920
Compliance is the third constraint.

1275
00:44:01,920 --> 00:44:03,520
Your organization might have decided

1276
00:44:03,520 --> 00:44:06,320
that only models that have passed internal red teaming

1277
00:44:06,320 --> 00:44:09,120
adversarial testing to detect safety flaws

1278
00:44:09,120 --> 00:44:10,720
are deployable to production.

1279
00:44:10,720 --> 00:44:13,520
Or only models that meet specific accuracy standards.

1280
00:44:13,520 --> 00:44:16,720
Or only models that have been evaluated for bias or fairness.

1281
00:44:16,720 --> 00:44:19,520
Policy turns those decisions into technical requirements.

1282
00:44:19,520 --> 00:44:21,920
When the evaluation team completes testing on a new model

1283
00:44:21,920 --> 00:44:24,320
and approves it, the policy gets updated.

1284
00:44:24,320 --> 00:44:25,520
The model becomes deployable.

1285
00:44:25,520 --> 00:44:28,720
When testing is incomplete, the policy blocks deployment.

1286
00:44:28,720 --> 00:44:31,920
The specific rules look simple in code but powerful in practice.

1287
00:44:31,920 --> 00:44:33,120
A policy that says,

1288
00:44:33,120 --> 00:44:35,920
"Only Microsoft hosted models are allowed in production"

1289
00:44:35,920 --> 00:44:38,720
means every production foundry hub inherits that constraint.

1290
00:44:38,720 --> 00:44:42,720
A developer working in a production project opens the model catalog.

1291
00:44:42,720 --> 00:44:44,320
They see only Microsoft models.

1292
00:44:44,320 --> 00:44:45,920
Non-Microsoft options aren't visible.

1293
00:44:45,920 --> 00:44:47,320
This isn't hidden complexity.

1294
00:44:47,320 --> 00:44:49,120
It's visibility working in your favor.

1295
00:44:49,120 --> 00:44:50,720
The developer knows what's available.

1296
00:44:50,720 --> 00:44:53,920
They don't waste time requesting a model you've already decided not to use.

1297
00:44:53,920 --> 00:44:55,120
A policy that says,

1298
00:44:55,120 --> 00:44:57,920
"Only models that have passed red teaming are deployable"

1299
00:44:57,920 --> 00:45:00,720
ties model governance to your evaluation process.

1300
00:45:00,720 --> 00:45:02,120
Before a model can be used,

1301
00:45:02,120 --> 00:45:05,520
it must be tagged as red teamed in your foundry model registry.

1302
00:45:05,520 --> 00:45:06,920
The policy checks for that tag.

1303
00:45:06,920 --> 00:45:08,920
If the tag is missing, deployment fails.

1304
00:45:08,920 --> 00:45:10,920
This creates accountability on both sides.

1305
00:45:10,920 --> 00:45:14,120
Evaluation teams know their work feeds directly into availability.

1306
00:45:14,120 --> 00:45:17,720
Development teams know that available models have been validated.

1307
00:45:17,720 --> 00:45:18,720
A policy that says,

1308
00:45:18,720 --> 00:45:20,920
"Only models in approved regions can be used"

1309
00:45:20,920 --> 00:45:23,720
ensures compliance with data residency requirements.

1310
00:45:23,720 --> 00:45:26,720
If your organization operates in Europe and data residency laws

1311
00:45:26,720 --> 00:45:28,920
demand that LLM inference happen within Europe,

1312
00:45:28,920 --> 00:45:30,520
a policy can enforce this.

1313
00:45:30,520 --> 00:45:32,520
Non-European models become unavailable.

1314
00:45:32,520 --> 00:45:35,320
The policy doesn't require developers to understand the regulation.

1315
00:45:35,320 --> 00:45:37,320
It bakes the regulation into the platform.

1316
00:45:37,320 --> 00:45:40,520
When a developer tries to deploy a model that violates policy

1317
00:45:40,520 --> 00:45:42,120
as you're blocks it automatically,

1318
00:45:42,120 --> 00:45:44,920
the error message explains which policy was violated.

1319
00:45:44,920 --> 00:45:46,720
The developer updates their deployment.

1320
00:45:46,720 --> 00:45:48,120
Everything else stays the same.

1321
00:45:48,120 --> 00:45:49,720
This is governance without friction.

1322
00:45:49,720 --> 00:45:51,320
Developers stay productive.

1323
00:45:51,320 --> 00:45:52,520
Teams move fast,

1324
00:45:52,520 --> 00:45:56,120
but they move within guardrails that the organization has deliberately chosen.

1325
00:45:56,120 --> 00:45:57,720
Bicep doesn't write these policies,

1326
00:45:57,720 --> 00:46:00,520
but it deploys the infrastructure that Azure Policy governs.

1327
00:46:00,520 --> 00:46:04,520
Together, they create a system where compliance isn't something you hope happens.

1328
00:46:04,520 --> 00:46:06,520
It's something the platform enforces.

1329
00:46:06,520 --> 00:46:08,920
Token level observability as governance.

1330
00:46:08,920 --> 00:46:11,320
Deployment and governance are only half the picture.

1331
00:46:11,320 --> 00:46:12,920
The other half is visibility.

1332
00:46:12,920 --> 00:46:14,520
You can build the hardened perimeter.

1333
00:46:14,520 --> 00:46:15,720
You can enforce policy.

1334
00:46:15,720 --> 00:46:17,720
You can lock down identity and network.

1335
00:46:17,720 --> 00:46:20,120
But if you can't see what's happening inside your reasoning system,

1336
00:46:20,120 --> 00:46:21,320
governance is theoretical.

1337
00:46:21,320 --> 00:46:22,520
Your controlling infrastructure.

1338
00:46:22,520 --> 00:46:24,120
You're not controlling intelligence.

1339
00:46:24,120 --> 00:46:27,320
Token consumption is the fundamental unit of AI economics.

1340
00:46:27,320 --> 00:46:29,320
Every prompt contains input tokens.

1341
00:46:29,320 --> 00:46:31,120
Every response contains output tokens.

1342
00:46:31,120 --> 00:46:32,920
Cost flows directly from token usage.

1343
00:46:32,920 --> 00:46:34,120
You can't separate them.

1344
00:46:34,120 --> 00:46:38,920
A thousand token calls to GPT 3.5 costs less than a hundred token calls to GPT 4.

1345
00:46:38,920 --> 00:46:40,120
The difference is exponential.

1346
00:46:40,120 --> 00:46:41,720
This isn't an implementation detail.

1347
00:46:41,720 --> 00:46:43,320
This is business economics.

1348
00:46:43,320 --> 00:46:44,920
Observability isn't about dashboards.

1349
00:46:44,920 --> 00:46:45,720
It's about governance.

1350
00:46:45,720 --> 00:46:47,320
You can't control what you can't measure.

1351
00:46:47,320 --> 00:46:49,320
And you can't measure what you don't instrument.

1352
00:46:49,320 --> 00:46:51,120
The moment you decide observability matters,

1353
00:46:51,120 --> 00:46:53,920
you're deciding that token level visibility is non-negotiable.

1354
00:46:53,920 --> 00:46:54,920
It's foundational.

1355
00:46:54,920 --> 00:46:58,520
Token level observability means tracking specific things.

1356
00:46:58,520 --> 00:47:00,520
Input tokens per prompt component.

1357
00:47:00,520 --> 00:47:02,520
System messages, user input,

1358
00:47:02,520 --> 00:47:04,520
retrieved context from your knowledge base,

1359
00:47:04,520 --> 00:47:05,720
all count separately.

1360
00:47:05,720 --> 00:47:07,320
Output tokens per response.

1361
00:47:07,320 --> 00:47:09,320
Cost per token for each model you're using

1362
00:47:09,320 --> 00:47:11,720
because different models have different pricing tiers.

1363
00:47:11,720 --> 00:47:14,320
Cost attribution by feature, team, or business units

1364
00:47:14,320 --> 00:47:16,320
so you know which parts of your organization are driving,

1365
00:47:16,320 --> 00:47:17,120
which costs.

1366
00:47:17,120 --> 00:47:18,320
This isn't metadata.

1367
00:47:18,320 --> 00:47:22,320
This is the data that determines whether your AI strategy is economically sustainable.

1368
00:47:22,320 --> 00:47:24,120
Without token level visibility,

1369
00:47:24,120 --> 00:47:25,920
several things become impossible.

1370
00:47:25,920 --> 00:47:28,520
You can't detect runaway costs before the bill arrives.

1371
00:47:28,520 --> 00:47:30,720
You think you're spending five thousand a month on AI.

1372
00:47:30,720 --> 00:47:31,920
The bill shows fifty thousand dollars.

1373
00:47:31,920 --> 00:47:32,920
The spike wasn't gradual.

1374
00:47:32,920 --> 00:47:34,720
It wasn't visible until the invoice arrived.

1375
00:47:34,720 --> 00:47:36,920
You can't optimize prompts and retrieval strategies

1376
00:47:36,920 --> 00:47:39,920
because you don't know which prompts are consuming the most tokens.

1377
00:47:39,920 --> 00:47:42,320
A prompt that seems like it's working efficiently

1378
00:47:42,320 --> 00:47:45,120
might be retrieving ten times more context than necessary,

1379
00:47:45,120 --> 00:47:46,520
wasting tokens and money.

1380
00:47:46,520 --> 00:47:48,920
You can't justify AI spending to finance teams

1381
00:47:48,920 --> 00:47:51,520
because you have no evidence of value relative to cost.

1382
00:47:51,520 --> 00:47:53,120
You can't prove compliance to auditors

1383
00:47:53,120 --> 00:47:56,120
because you have no way to show which data flowed through which models

1384
00:47:56,120 --> 00:47:57,720
or how many times it was processed.

1385
00:47:57,720 --> 00:47:59,920
This is why observability becomes governance.

1386
00:47:59,920 --> 00:48:01,920
The moment you instrument token consumption,

1387
00:48:01,920 --> 00:48:04,320
you gain control over it, you can set budgets.

1388
00:48:04,320 --> 00:48:05,920
You can alert when spending spikes.

1389
00:48:05,920 --> 00:48:07,920
You can attribute cost back to decisions.

1390
00:48:07,920 --> 00:48:12,120
You can make informed choices about model selection based on actual usage patterns,

1391
00:48:12,120 --> 00:48:13,120
not guesses.

1392
00:48:13,120 --> 00:48:16,720
Bicep must deploy the monitoring infrastructure that makes this visibility possible.

1393
00:48:16,720 --> 00:48:17,720
This isn't optional.

1394
00:48:17,720 --> 00:48:18,720
It's structural.

1395
00:48:18,720 --> 00:48:21,920
The moment you create a bicep module for deploying an AI workload,

1396
00:48:21,920 --> 00:48:24,320
that module includes observability by default.

1397
00:48:24,320 --> 00:48:26,720
It deploys application insights to capture telemetry.

1398
00:48:26,720 --> 00:48:31,920
It configures diagnostic settings to root logs from all AI services into log analytics.

1399
00:48:31,920 --> 00:48:35,920
It wires up alert rules that trigger when token consumption exceeds thresholds.

1400
00:48:35,920 --> 00:48:40,320
It creates the data structures needed to aggregate tokens by feature team and business unit.

1401
00:48:40,320 --> 00:48:42,920
The module doesn't ask whether observability is wanted.

1402
00:48:42,920 --> 00:48:47,120
It doesn't provide an option to skip it to save money on monitoring costs.

1403
00:48:47,120 --> 00:48:48,920
Observability is the default.

1404
00:48:48,920 --> 00:48:51,920
Removing it requires deliberate override which creates an audit trail.

1405
00:48:51,920 --> 00:48:53,920
Why would a team disable monitoring?

1406
00:48:53,920 --> 00:48:55,520
The question itself becomes suspicious.

1407
00:48:55,520 --> 00:48:57,120
Here's what this means in practice.

1408
00:48:57,120 --> 00:48:59,920
When you deploy Azure OpenAI through a bicep module,

1409
00:48:59,920 --> 00:49:01,920
the deployment doesn't just create the service.

1410
00:49:01,920 --> 00:49:04,920
It creates the service, configures diagnostic logging,

1411
00:49:04,920 --> 00:49:07,120
sets up alert rules for token thresholds,

1412
00:49:07,120 --> 00:49:10,520
creates the namespace in application insights where token metrics will flow

1413
00:49:10,520 --> 00:49:12,520
and wires everything to log analytics.

1414
00:49:12,520 --> 00:49:14,720
All of this happens in a single deployment.

1415
00:49:14,720 --> 00:49:17,120
There's no separate step to enable monitoring later.

1416
00:49:17,120 --> 00:49:20,520
There's no forgetting because all the components deploy together.

1417
00:49:20,520 --> 00:49:23,920
Without the structural integration, observability becomes fragmented.

1418
00:49:23,920 --> 00:49:26,720
You have token metrics in one place, cost data in another,

1419
00:49:26,720 --> 00:49:28,320
and no connection between them.

1420
00:49:28,320 --> 00:49:31,520
You can see tokens consumed but not which feature consumed them.

1421
00:49:31,520 --> 00:49:34,720
You can see costs on the bill but not which decisions drove the costs.

1422
00:49:34,720 --> 00:49:37,920
You have visibility into components but not into the system as a whole.

1423
00:49:37,920 --> 00:49:40,520
With bicep enforcing observability by default,

1424
00:49:40,520 --> 00:49:42,520
token level visibility becomes automatic.

1425
00:49:42,520 --> 00:49:43,720
Governance gains teeth.

1426
00:49:43,720 --> 00:49:45,720
You're not hoping teams report their usage.

1427
00:49:45,720 --> 00:49:46,920
You're measuring it.

1428
00:49:46,920 --> 00:49:49,120
You're not asking whether costs are being tracked.

1429
00:49:49,120 --> 00:49:50,320
You're aggregating them.

1430
00:49:50,320 --> 00:49:53,320
You're not wondering whether you can answer auditor questions about token flow.

1431
00:49:53,320 --> 00:49:57,120
You have the data because the data collection was built into the foundation.

1432
00:49:57,120 --> 00:49:59,120
Bicep and FinOps integration.

1433
00:49:59,120 --> 00:50:02,520
Observability connects to cost governance through FinOps patterns.

1434
00:50:02,520 --> 00:50:04,920
This is where measurement becomes accountability.

1435
00:50:04,920 --> 00:50:08,920
FinOps is the discipline of managing cloud costs through visibility and accountability.

1436
00:50:08,920 --> 00:50:10,520
It's not about spending less.

1437
00:50:10,520 --> 00:50:12,120
It's about spending intentionally.

1438
00:50:12,120 --> 00:50:14,520
You make conscious decisions about where your money goes.

1439
00:50:14,520 --> 00:50:16,120
Based on data, not on hope.

1440
00:50:16,120 --> 00:50:19,120
You allocate budget to the features that matter most.

1441
00:50:19,120 --> 00:50:21,120
You eliminate waste in the features that don't.

1442
00:50:21,120 --> 00:50:23,920
And you hold teams accountable for the costs they incur.

1443
00:50:23,920 --> 00:50:25,920
For this to work, you need three things.

1444
00:50:25,920 --> 00:50:29,920
Visibility into spending, attribution of spending to decisions and automation

1445
00:50:29,920 --> 00:50:31,520
that makes governance repeatable.

1446
00:50:31,520 --> 00:50:34,320
For AI workloads, FinOps translates into token tracking,

1447
00:50:34,320 --> 00:50:36,720
cost attribution and budget enforcement.

1448
00:50:36,720 --> 00:50:37,720
Tokens are the currency.

1449
00:50:37,720 --> 00:50:38,920
Each token costs money.

1450
00:50:38,920 --> 00:50:40,920
The cost varies by model, by region,

1451
00:50:40,920 --> 00:50:43,120
by whether you're using input or output tokens.

1452
00:50:43,120 --> 00:50:47,320
FinOps for AI means converting that token consumption into business accountability.

1453
00:50:47,320 --> 00:50:48,520
You track tokens.

1454
00:50:48,520 --> 00:50:50,720
You attribute them to features or teams.

1455
00:50:50,720 --> 00:50:53,520
You set budgets based on expected token usage.

1456
00:50:53,520 --> 00:50:55,520
You enforce those budgets through automation,

1457
00:50:55,520 --> 00:50:58,920
not through manual reviews that happen months after the damage is done.

1458
00:50:58,920 --> 00:51:03,920
Microsoft provides a FinOps toolkit that includes bicep modules for cost ingestion and reporting.

1459
00:51:03,920 --> 00:51:04,920
This isn't theoretical.

1460
00:51:04,920 --> 00:51:09,520
This is a concrete set of deployable components that turns FinOps from a concept into infrastructure.

1461
00:51:09,520 --> 00:51:13,120
The toolkit includes modules that pull cost data from Azure Cost Management,

1462
00:51:13,120 --> 00:51:15,320
normalize it according to the focus standard,

1463
00:51:15,320 --> 00:51:17,320
a vendor neutral cost data schema,

1464
00:51:17,320 --> 00:51:20,120
and feed it into Power BI for reporting.

1465
00:51:20,120 --> 00:51:23,920
This normalization matters because it means your cost data looks the same,

1466
00:51:23,920 --> 00:51:27,320
whether you're analyzing Azure OpenAI or storage or compute.

1467
00:51:27,320 --> 00:51:31,520
It's one unified language for talking about spending across your entire cloud estate.

1468
00:51:31,520 --> 00:51:35,320
The bicep modules deploy cost management exports that run on a schedule,

1469
00:51:35,320 --> 00:51:38,520
pulling detailed cost and usage data from your Azure subscriptions.

1470
00:51:38,520 --> 00:51:42,720
They deploy log analytics workspaces that aggregate token metrics we discussed earlier.

1471
00:51:42,720 --> 00:51:46,720
They deploy Power BI dashboards that visualize cost trends in real time.

1472
00:51:46,720 --> 00:51:50,720
They deploy alerts and budgets that trigger when spending exceeds thresholds.

1473
00:51:50,720 --> 00:51:52,520
All of this happens through code.

1474
00:51:52,520 --> 00:51:56,120
All of it can be versioned, reviewed, and replicated across environments.

1475
00:51:56,120 --> 00:51:59,920
Here's the critical difference between FinOps with bicep and FinOps without it.

1476
00:51:59,920 --> 00:52:02,320
Without bicep, you set up cost governance once.

1477
00:52:02,320 --> 00:52:05,720
Someone deploys the cost ingestion infrastructure in one subscription.

1478
00:52:05,720 --> 00:52:07,720
Someone else configures the Power BI dashboard.

1479
00:52:07,720 --> 00:52:09,520
A third person sets up the alerts.

1480
00:52:09,520 --> 00:52:12,720
Their ad hoc decisions made by different people at different times.

1481
00:52:12,720 --> 00:52:15,120
If you need the same cost governance in another subscription,

1482
00:52:15,120 --> 00:52:16,920
you have to repeat all those steps manually.

1483
00:52:16,920 --> 00:52:19,720
Or you don't, and that subscription has no cost visibility.

1484
00:52:19,720 --> 00:52:23,320
You end up with some subscriptions that have cost tracking and some that don't.

1485
00:52:23,320 --> 00:52:25,920
Your finance team has partial visibility into the landscape.

1486
00:52:25,920 --> 00:52:28,320
They can see costs in some areas, but not others.

1487
00:52:28,320 --> 00:52:31,520
With bicep, you build the cost governance once and replicated.

1488
00:52:31,520 --> 00:52:34,920
The bicep modules that deploy cost ingestion, log analytics, dashboards,

1489
00:52:34,920 --> 00:52:37,120
and alerts become your organizational standard.

1490
00:52:37,120 --> 00:52:40,520
Every subscription that hosts AI workloads gets the same cost governance.

1491
00:52:40,520 --> 00:52:42,520
Not because you manually deployed it to each one,

1492
00:52:42,520 --> 00:52:44,720
but because you deployed the bicep module to each one,

1493
00:52:44,720 --> 00:52:46,520
the output is identical every time.

1494
00:52:46,520 --> 00:52:50,520
You don't have to remember which power BI workspace to connect to which log analytics instance.

1495
00:52:50,520 --> 00:52:51,520
The module handles it.

1496
00:52:51,520 --> 00:52:53,920
You don't have to manually configure alert thresholds.

1497
00:52:53,920 --> 00:52:55,120
The module includes them.

1498
00:52:55,120 --> 00:52:58,920
Bicep makes Finops repeatable in a way that manual infrastructure never can.

1499
00:52:58,920 --> 00:53:01,720
Without it, cost governance is fragmented and inconsistent.

1500
00:53:01,720 --> 00:53:04,920
You have visibility into some areas of spending, blindness in others.

1501
00:53:04,920 --> 00:53:07,520
You enforce budget controls and some projects, but not others.

1502
00:53:07,520 --> 00:53:09,520
Some teams can see their token consumption.

1503
00:53:09,520 --> 00:53:12,920
Others have no idea how much they're spending until the invoice arrives.

1504
00:53:12,920 --> 00:53:17,120
This is how organizations discover six months later that a single AI feature

1505
00:53:17,120 --> 00:53:20,320
has burned through hundreds of thousands of dollars that nobody was tracking.

1506
00:53:20,320 --> 00:53:23,720
With bicep, cost visibility is automatic and standardized.

1507
00:53:23,720 --> 00:53:26,120
Every AI deployment includes cost governance

1508
00:53:26,120 --> 00:53:29,920
because every AI module pulls the cost governance infrastructure in with it.

1509
00:53:29,920 --> 00:53:31,720
You don't have to remember to enable monitoring.

1510
00:53:31,720 --> 00:53:33,720
You don't have to manually wire up alerts.

1511
00:53:33,720 --> 00:53:34,720
The tokens are measured.

1512
00:53:34,720 --> 00:53:35,720
The costs are attributed.

1513
00:53:35,720 --> 00:53:39,720
The budgets are enforced, all because the infrastructure was built that way from the start.

1514
00:53:39,720 --> 00:53:44,720
Azure Monitor and application insights for AI token metrics flow into Azure Monitor

1515
00:53:44,720 --> 00:53:46,720
where they become actionable intelligence.

1516
00:53:46,720 --> 00:53:51,720
But the infrastructure that captures those metrics is more complex than simply turning on logging.

1517
00:53:51,720 --> 00:53:54,720
Azure Monitor is the observability platform for every Azure resource.

1518
00:53:54,720 --> 00:53:59,720
It's the system that sits behind the scenes collecting signals about what's happening across your infrastructure.

1519
00:53:59,720 --> 00:54:04,720
For traditional workloads, VMs, databases, storage accounts, the signals are straightforward.

1520
00:54:04,720 --> 00:54:08,720
CPU utilization, memory consumption, disk IO network throughput.

1521
00:54:08,720 --> 00:54:11,720
Straight forward because the metrics map directly to operational health.

1522
00:54:11,720 --> 00:54:13,720
AI workloads are different.

1523
00:54:13,720 --> 00:54:14,720
The operational metrics matter.

1524
00:54:14,720 --> 00:54:18,720
Latency error rates availability, but they're not the full story.

1525
00:54:18,720 --> 00:54:20,720
You need to know what's actually happening inside the reasoning pipeline.

1526
00:54:20,720 --> 00:54:22,720
Token usage per request tells you consumption.

1527
00:54:22,720 --> 00:54:25,720
But you also need latency broken down by component.

1528
00:54:25,720 --> 00:54:27,720
How long did retrieval take?

1529
00:54:27,720 --> 00:54:28,720
How long did the model think?

1530
00:54:28,720 --> 00:54:31,720
How long did post processing consume error rates matter?

1531
00:54:31,720 --> 00:54:33,720
But you need to distinguish between transient errors?

1532
00:54:33,720 --> 00:54:36,720
A temporary network issue that the system will retry

1533
00:54:36,720 --> 00:54:39,720
and systematic failures that indicate architectural problems.

1534
00:54:39,720 --> 00:54:42,720
Model performance and quality metrics matter most.

1535
00:54:42,720 --> 00:54:44,720
Is the model actually answering questions correctly?

1536
00:54:44,720 --> 00:54:46,720
Are retrieved documents actually relevant?

1537
00:54:46,720 --> 00:54:50,720
Is the safety filtering too aggressive rejecting legitimate requests?

1538
00:54:50,720 --> 00:54:53,720
Application insights is where these signal layers come together.

1539
00:54:53,720 --> 00:54:58,720
It's the application level observability layer that understands your code and your business logic,

1540
00:54:58,720 --> 00:55:00,720
not just the infrastructure underneath.

1541
00:55:00,720 --> 00:55:02,720
When your code makes a call to Azure OpenAI,

1542
00:55:02,720 --> 00:55:05,720
application insights captures that moment.

1543
00:55:05,720 --> 00:55:08,720
It records custom properties, which feature triggered this call,

1544
00:55:08,720 --> 00:55:11,720
which user made the request, which tenant owns the user.

1545
00:55:11,720 --> 00:55:13,720
It captures prompt and completion metadata,

1546
00:55:13,720 --> 00:55:17,720
the length of the prompt, the model used, the temperature, and max token settings.

1547
00:55:17,720 --> 00:55:20,720
It records evaluation scores that your code calculated.

1548
00:55:20,720 --> 00:55:23,720
Did a secondary model judge this response as hallucinated?

1549
00:55:23,720 --> 00:55:24,720
Did a safety check flag it?

1550
00:55:24,720 --> 00:55:29,720
Did a grounding validator confirm that the response was supported by retrieved context?

1551
00:55:29,720 --> 00:55:32,720
This layering matters because no single tool tells the complete story.

1552
00:55:32,720 --> 00:55:34,720
Azure Monitor shows you metrics.

1553
00:55:34,720 --> 00:55:36,720
Application insights shows you traces.

1554
00:55:36,720 --> 00:55:39,720
Together, they create a unified view of what's happening in your AI system.

1555
00:55:39,720 --> 00:55:41,720
You're not jumping between tools to answer questions.

1556
00:55:41,720 --> 00:55:44,720
You're querying a unified observability platform.

1557
00:55:44,720 --> 00:55:46,720
Bicep handles the wiring.

1558
00:55:46,720 --> 00:55:50,720
The module must deploy diagnostic settings that root logs from all AI services.

1559
00:55:50,720 --> 00:55:54,720
Azure OpenAI, AI Search, Storage, Log Analytics itself,

1560
00:55:54,720 --> 00:55:56,720
into a central log analytics workspace.

1561
00:55:56,720 --> 00:56:01,720
The module must deploy application insights instances with the correct instrumentation keys

1562
00:56:01,720 --> 00:56:06,720
so that when your applications run, they automatically emit telemetry to the right place.

1563
00:56:06,720 --> 00:56:11,720
The module must deploy alert rules that trigger when costs, spikes, exceed expected thresholds,

1564
00:56:11,720 --> 00:56:15,720
when latency anomalies occur, or when error rates climb above acceptable levels.

1565
00:56:15,720 --> 00:56:17,720
The integration is critical.

1566
00:56:17,720 --> 00:56:21,720
When you deploy an AI workload through bicep, the observability infrastructure deploys with it.

1567
00:56:21,720 --> 00:56:24,720
Azure OpenAI sends its diagnostic logs to log analytics.

1568
00:56:24,720 --> 00:56:29,720
Application insights captures trace data from your orchestration layer, both flow into the same workspace.

1569
00:56:29,720 --> 00:56:30,720
You can query across both.

1570
00:56:30,720 --> 00:56:34,720
You can create alerts that fire when metrics from different layers cross thresholds simultaneously.

1571
00:56:34,720 --> 00:56:39,720
High token usage, combined with high latency, for example, suggesting a retrieval problem.

1572
00:56:39,720 --> 00:56:42,720
Without this unified deployment, observability fragments.

1573
00:56:42,720 --> 00:56:44,720
Your token metrics live in one tool.

1574
00:56:44,720 --> 00:56:46,720
Your latency data lives in another.

1575
00:56:46,720 --> 00:56:48,720
Cost attribution happens in a third place.

1576
00:56:48,720 --> 00:56:49,720
You're not observing your system.

1577
00:56:49,720 --> 00:56:53,720
You're observing its shadows fragmented across multiple platforms that don't talk to each other.

1578
00:56:53,720 --> 00:56:58,720
With bicep enforcing unified observability as a default, visibility becomes coherent.

1579
00:56:58,720 --> 00:57:00,720
The moment you deploy, you can see what's happening.

1580
00:57:00,720 --> 00:57:04,720
You understand token flow, you understand cost, you understand where latency is accumulating,

1581
00:57:04,720 --> 00:57:11,720
you understand whether your safety controls are working, you understand whether your retrieval is actually grounding the model or just adding noise.

1582
00:57:11,720 --> 00:57:14,720
This visibility doesn't happen because you're remembered to configure it.

1583
00:57:14,720 --> 00:57:17,720
It happens because the infrastructure was designed that way from the start.

1584
00:57:17,720 --> 00:57:19,720
Cost attribution and chargeback.

1585
00:57:19,720 --> 00:57:22,720
Once you can measure token usage, you can attribute cost to who caused it.

1586
00:57:22,720 --> 00:57:28,720
This is where observability transforms from a technical capability into a business accountability mechanism.

1587
00:57:28,720 --> 00:57:31,720
Cost attribution is critical for enterprise accountability.

1588
00:57:31,720 --> 00:57:34,720
And without it, AI spend becomes a black box on the cloud bill.

1589
00:57:34,720 --> 00:57:39,720
Finance reviews the charges. They see a $150,000 monthly spike for cognitive services.

1590
00:57:39,720 --> 00:57:41,720
They have no idea which feature drove it.

1591
00:57:41,720 --> 00:57:46,720
They can't tell whether the cost came from legitimate production workloads or from experimentation that should have stopped months ago.

1592
00:57:46,720 --> 00:57:49,720
They can't justify the spend to their leadership.

1593
00:57:49,720 --> 00:57:54,720
They can't forecast whether AI spending will become a runaway line item in the budget.

1594
00:57:54,720 --> 00:57:56,720
When you have cost attribution, everything changes.

1595
00:57:56,720 --> 00:57:59,720
You know which business units are consuming the AI infrastructure.

1596
00:57:59,720 --> 00:58:03,720
You can see that the sales department's co-pilot is costing $40,000 monthly.

1597
00:58:03,720 --> 00:58:10,720
The customer service department's bot is running $30,000 and internal productivity tools are burning through another $50,000.

1598
00:58:10,720 --> 00:58:12,720
You see which features are consuming the most tokens.

1599
00:58:12,720 --> 00:58:16,720
You discover that one specific feature, maybe an experimental agent that didn't work out,

1600
00:58:16,720 --> 00:58:21,720
is consuming 40% of your token budget, despite driving almost no business value.

1601
00:58:21,720 --> 00:58:24,720
You identify which teams are deploying inefficient prompts.

1602
00:58:24,720 --> 00:58:28,720
One team is using a 50-page system prompt when a 5-page system prompt works just as well.

1603
00:58:28,720 --> 00:58:31,720
Another team is retrieving 20 documents from your knowledge base

1604
00:58:31,720 --> 00:58:34,720
when the top three contain 95% of the relevant information.

1605
00:58:34,720 --> 00:58:37,720
These aren't architectural problems. They're implementation problems.

1606
00:58:37,720 --> 00:58:39,720
But you can't see them if cost is invisible.

1607
00:58:39,720 --> 00:58:42,720
With attribution, you can see them, call them out and fix them.

1608
00:58:42,720 --> 00:58:47,720
Bicep enables cost attribution by encoding the infrastructure that makes attribution possible.

1609
00:58:47,720 --> 00:58:49,720
The first requirement is consistent tagging.

1610
00:58:49,720 --> 00:58:52,720
Every resource created through bicep carries tags.

1611
00:58:52,720 --> 00:58:54,720
Costscenter, business unit, environment owner.

1612
00:58:54,720 --> 00:58:56,720
These tags aren't suggestions. They're enforced by the module.

1613
00:58:56,720 --> 00:59:00,720
If you want to deploy an AI resource through bicep, that resource gets tagged.

1614
00:59:00,720 --> 00:59:02,720
The tags flow into cost management.

1615
00:59:02,720 --> 00:59:06,720
Finance can filter spending by tag. They can group by cost center and see what each department spent.

1616
00:59:06,720 --> 00:59:11,720
They can filter by environment and see how much is flowing to production versus development.

1617
00:59:11,720 --> 00:59:13,720
The second requirement is tracking at the gateway level.

1618
00:59:13,720 --> 00:59:16,720
API management sits between clients and your LLM endpoints.

1619
00:59:16,720 --> 00:59:21,720
Bicep deploys APM with the infrastructure to track usage per client or per subscription.

1620
00:59:21,720 --> 00:59:25,720
Every request flowing through APM is logged with context about who made the request.

1621
00:59:25,720 --> 00:59:27,720
That context is stored in log analytics.

1622
00:59:27,720 --> 00:59:30,720
Later you can query that context and map requests back to cost.

1623
00:59:30,720 --> 00:59:32,720
Feature X made 10,000 requests last month.

1624
00:59:32,720 --> 00:59:35,720
Feature X consumed an average of 250 tokens per request.

1625
00:59:35,720 --> 00:59:37,720
That's 2.5 million tokens of cost.

1626
00:59:37,720 --> 00:59:40,720
You can attribute that cost back to the team that owns Feature X.

1627
00:59:40,720 --> 00:59:45,720
The third requirement is wiring token metrics into log analytics with cost attribution dimensions.

1628
00:59:45,720 --> 00:59:48,720
When you log a token consumption event, you don't just log the count.

1629
00:59:48,720 --> 00:59:49,720
You log it with context.

1630
00:59:49,720 --> 00:59:50,720
Who triggered this?

1631
00:59:50,720 --> 00:59:51,720
Which feature?

1632
00:59:51,720 --> 00:59:52,720
Which tenant?

1633
00:59:52,720 --> 00:59:53,720
Which business unit?

1634
00:59:53,720 --> 00:59:55,720
That context becomes a dimension in your reporting.

1635
00:59:55,720 --> 00:59:56,720
You create pivot tables.

1636
00:59:56,720 --> 01:00:01,720
You slice cost by feature by team, by business unit, by any dimension that matters for accountability.

1637
01:00:01,720 --> 01:00:05,720
The final requirement is Power BI reporting that breaks down spend by tag.

1638
01:00:05,720 --> 01:00:08,720
Finance shouldn't have to write KQL queries in log analytics.

1639
01:00:08,720 --> 01:00:11,720
They should open a Power BI dashboard and see.

1640
01:00:11,720 --> 01:00:13,720
Here's monthly spending by business unit.

1641
01:00:13,720 --> 01:00:15,720
Here's which features consume the most tokens.

1642
01:00:15,720 --> 01:00:16,720
Here's the trend.

1643
01:00:16,720 --> 01:00:18,720
Is AI spending growing or shrinking?

1644
01:00:18,720 --> 01:00:20,720
That dashboard is built by someone once.

1645
01:00:20,720 --> 01:00:22,720
The bicep module deploys the data infrastructure.

1646
01:00:22,720 --> 01:00:24,720
Power BI connects to log analytics.

1647
01:00:24,720 --> 01:00:26,720
The dashboard queries the data automatically.

1648
01:00:26,720 --> 01:00:29,720
Every month, new cost data flows in and the dashboard updates.

1649
01:00:29,720 --> 01:00:31,720
The key insight is this.

1650
01:00:31,720 --> 01:00:35,720
Cost attribution is only possible if your infrastructure is designed for it from day one.

1651
01:00:35,720 --> 01:00:39,720
You can't retrofit attribution onto a system that wasn't designed for it.

1652
01:00:39,720 --> 01:00:41,720
Manual deployments don't have this design.

1653
01:00:41,720 --> 01:00:44,720
Resources get deployed without tags because tagging is optional.

1654
01:00:44,720 --> 01:00:48,720
API calls happen without logging because logging wasn't built in.

1655
01:00:48,720 --> 01:00:51,720
Token consumption gets recorded without context about who triggered it.

1656
01:00:51,720 --> 01:00:55,720
By the time you realize you need attribution, the infrastructure doesn't support it.

1657
01:00:55,720 --> 01:00:59,720
Bicep deployments can be architected with cost attribution as a first-class concern.

1658
01:00:59,720 --> 01:01:00,720
The module includes tagging.

1659
01:01:00,720 --> 01:01:01,720
It includes IPM wiring.

1660
01:01:01,720 --> 01:01:03,720
It includes log analytics dimensionality.

1661
01:01:03,720 --> 01:01:06,720
It includes the data structures that Power BI needs.

1662
01:01:06,720 --> 01:01:08,720
You don't add attribution as an afterthought.

1663
01:01:08,720 --> 01:01:09,720
You deploy it.

1664
01:01:09,720 --> 01:01:11,720
API management as the AI gateway.

1665
01:01:11,720 --> 01:01:12,720
The perimeter is in place.

1666
01:01:12,720 --> 01:01:15,720
Now we need a gateway that controls traffic through it.

1667
01:01:15,720 --> 01:01:17,720
Everything we've built so far creates a boundary.

1668
01:01:17,720 --> 01:01:20,720
Identity parameter, network parameter, reasoning parameter.

1669
01:01:20,720 --> 01:01:23,720
But a boundary only works if traffic actually flows through it.

1670
01:01:23,720 --> 01:01:26,720
The moment you have an AI endpoint, you have a critical decision.

1671
01:01:26,720 --> 01:01:28,720
Will clients connect directly to that endpoint?

1672
01:01:28,720 --> 01:01:31,720
Or will they connect through a gateway that controls their access?

1673
01:01:31,720 --> 01:01:33,720
Direct connection is the easy path.

1674
01:01:33,720 --> 01:01:35,720
Developers get the endpoint URL.

1675
01:01:35,720 --> 01:01:36,720
They get the API key.

1676
01:01:36,720 --> 01:01:38,720
They call it directly from their application.

1677
01:01:38,720 --> 01:01:41,720
For a proof of concept or an internal experiment, this works.

1678
01:01:41,720 --> 01:01:44,720
For an enterprise, it breaks immediately.

1679
01:01:44,720 --> 01:01:48,720
Because the moment clients connect directly to your LLM endpoints, governance disappears.

1680
01:01:48,720 --> 01:01:50,720
API management is the gateway that changes this.

1681
01:01:50,720 --> 01:01:56,720
It sits between clients and Azure OpenAI, and everything that flows between them passes through APIM first.

1682
01:01:56,720 --> 01:01:59,720
This matters because APIM enforces layers of control that exist nowhere else.

1683
01:01:59,720 --> 01:02:02,720
Authentication and authorization is the first layer.

1684
01:02:02,720 --> 01:02:05,720
Clients don't authenticate directly to Azure OpenAI.

1685
01:02:05,720 --> 01:02:06,720
They authenticate to APIM.

1686
01:02:06,720 --> 01:02:08,720
APIM validates the credential.

1687
01:02:08,720 --> 01:02:11,720
If the client is authorized to make this request, APIM forwards it.

1688
01:02:11,720 --> 01:02:12,720
If not, APIM rejects it.

1689
01:02:12,720 --> 01:02:15,720
This seems like a small difference, but it's structural.

1690
01:02:15,720 --> 01:02:18,720
Azure OpenAI has no visibility into who's really making the request.

1691
01:02:18,720 --> 01:02:19,720
APIM does.

1692
01:02:19,720 --> 01:02:20,720
APIM knows the client identity.

1693
01:02:20,720 --> 01:02:21,720
It knows the tenant.

1694
01:02:21,720 --> 01:02:22,720
It knows the subscription.

1695
01:02:22,720 --> 01:02:25,720
It can make sophisticated authorization decisions based on that context.

1696
01:02:25,720 --> 01:02:28,720
Rate limiting and quotas is the second layer.

1697
01:02:28,720 --> 01:02:33,720
Without APIM, a single misbehaving client can consume your entire token budget in an afternoon.

1698
01:02:33,720 --> 01:02:35,720
With APIM, you set rate limits per client.

1699
01:02:35,720 --> 01:02:38,720
Maybe the sales team's copilot gets 1 million tokens per day.

1700
01:02:38,720 --> 01:02:40,720
The customer service bot gets 500,000.

1701
01:02:40,720 --> 01:02:42,720
Experimentation tools get 100,000.

1702
01:02:42,720 --> 01:02:45,720
When any of those clients hits their limit, APIM throttles them.

1703
01:02:45,720 --> 01:02:47,720
They get a 4 or 29 response.

1704
01:02:47,720 --> 01:02:48,720
They have to back off.

1705
01:02:48,720 --> 01:02:51,720
Your infrastructure doesn't get drowned by a single runaway workload.

1706
01:02:51,720 --> 01:02:53,720
Request and response validation is the third layer.

1707
01:02:53,720 --> 01:02:59,720
APIM can inspect requests before they reach Azure OpenAI and responses before they reach clients.

1708
01:02:59,720 --> 01:03:02,720
You can validate that requests conform to your standards.

1709
01:03:02,720 --> 01:03:05,720
Maybe you've decided that the maximum prompt length is 100,000 tokens.

1710
01:03:05,720 --> 01:03:10,720
APIM can check the incoming request, count tokens, and reject the request if it exceeds that limit.

1711
01:03:10,720 --> 01:03:15,720
On the response side, you can validate that responses conform to your safety requirements.

1712
01:03:15,720 --> 01:03:20,720
If a response triggers a safety concern, APIM can block it before it reaches the client.

1713
01:03:20,720 --> 01:03:22,720
Cost tracking and billing is the fourth layer.

1714
01:03:22,720 --> 01:03:27,720
When a request flows through APIM, you can extract metadata about that request and log it.

1715
01:03:27,720 --> 01:03:28,720
Who made the request?

1716
01:03:28,720 --> 01:03:29,720
Which feature triggered it?

1717
01:03:29,720 --> 01:03:31,720
How many tokens did it consume?

1718
01:03:31,720 --> 01:03:33,720
APIM can emit that data to application insights.

1719
01:03:33,720 --> 01:03:39,720
Later, you can reconstruct the entire cost history of your AI infrastructure by tenant, by feature, by client.

1720
01:03:39,720 --> 01:03:42,720
You have the granularity that manual deployments never achieve.

1721
01:03:42,720 --> 01:03:45,720
Byset must deploy APIM with all of this wired together.

1722
01:03:45,720 --> 01:03:48,720
The module doesn't just create the API management instance.

1723
01:03:48,720 --> 01:03:53,720
It creates the instance with policies embedded that emit token metrics to application insights.

1724
01:03:53,720 --> 01:03:58,720
It configures backends that point to private Azure OpenAI endpoints, not the public endpoints.

1725
01:03:58,720 --> 01:04:00,720
The traffic never touches the public internet.

1726
01:04:00,720 --> 01:04:04,720
It defines products and subscriptions that map to business units or teams.

1727
01:04:04,720 --> 01:04:06,720
Each product has its own rate limit policy.

1728
01:04:06,720 --> 01:04:12,720
The module includes the rate limit policies themselves, not as configuration you add later, but as part of the infrastructure definition.

1729
01:04:12,720 --> 01:04:15,720
When you deploy APIM through Byset, you get a functioning gateway immediately.

1730
01:04:15,720 --> 01:04:19,720
Clients have endpoints to call. Those endpoints root through APIM.

1731
01:04:19,720 --> 01:04:22,720
APIM enforces authentication, rate limits, validation and logging.

1732
01:04:22,720 --> 01:04:27,720
The moment you start using the gateway, you start accumulating visibility into who's doing what.

1733
01:04:27,720 --> 01:04:32,720
Without APIM, clients connect directly to Azure OpenAI, bypassing all governance.

1734
01:04:32,720 --> 01:04:35,720
With APIM, every request is visible, controlled and attributed.

1735
01:04:35,720 --> 01:04:38,720
APIM becomes the single point of governance for all LLM traffic.

1736
01:04:38,720 --> 01:04:41,720
This is where the hardened perimeter closes.

1737
01:04:41,720 --> 01:04:44,720
The identity, network and reasoning layers create structure.

1738
01:04:44,720 --> 01:04:46,720
APIM is where that structure enforces behavior.

1739
01:04:46,720 --> 01:04:50,720
It's the enforcement point that makes governance real instead of theoretical.

1740
01:04:50,720 --> 01:04:53,720
Orchestration and reasoning chains, the gateway controls traffic.

1741
01:04:53,720 --> 01:04:56,720
Now we need to control how that traffic flows through reasoning.

1742
01:04:56,720 --> 01:04:59,720
Enterprise AI workloads aren't single model calls.

1743
01:04:59,720 --> 01:05:03,720
A user makes a request. The system doesn't just send that request to Azure OpenAI and return the response.

1744
01:05:03,720 --> 01:05:05,720
What actually happens is far more complex.

1745
01:05:05,720 --> 01:05:08,720
The system retrieves relevant context from your knowledge base.

1746
01:05:08,720 --> 01:05:10,720
It structures that context for the model.

1747
01:05:10,720 --> 01:05:13,720
It calls the model with the context and the user's question, the model response.

1748
01:05:13,720 --> 01:05:17,720
The system validates that response, checking whether it's grounded in the retrieved context,

1749
01:05:17,720 --> 01:05:22,720
checking whether it violates safety policies, checking whether it makes sense.

1750
01:05:22,720 --> 01:05:26,720
If validation fails, the system might call the model again with different parameters.

1751
01:05:26,720 --> 01:05:31,720
If it passes, the system stores the result, logs the interaction and returns the response to the user.

1752
01:05:31,720 --> 01:05:36,720
That's one complete reasoning chain, a single interaction that flows through multiple steps,

1753
01:05:36,720 --> 01:05:38,720
multiple services, multiple decision points.

1754
01:05:38,720 --> 01:05:41,720
Most enterprises have dozens of these chains running simultaneously.

1755
01:05:41,720 --> 01:05:43,720
Some are simple, retrieve an answer.

1756
01:05:43,720 --> 01:05:49,720
Some are complex retrieve, analyze, retrieve again, call external APIs, aggregate results, validate, respond.

1757
01:05:49,720 --> 01:05:52,720
The complexity grows as your organization scales AI usage.

1758
01:05:52,720 --> 01:05:57,720
As your AI foundry agents are the orchestration layer where these chains get defined and controlled.

1759
01:05:57,720 --> 01:06:01,720
An agent isn't just a model. It's a system that coordinates reasoning.

1760
01:06:01,720 --> 01:06:03,720
You can define multiple agents with different roles.

1761
01:06:03,720 --> 01:06:07,720
A retrieval agent focuses on finding relevant information in your knowledge base.

1762
01:06:07,720 --> 01:06:12,720
An analysis agent takes that information and performs calculations or transformations on it.

1763
01:06:12,720 --> 01:06:15,720
An action agent takes the analysis results and makes changes in other systems,

1764
01:06:15,720 --> 01:06:18,720
creating records, sending notifications, triggering workflows.

1765
01:06:18,720 --> 01:06:21,720
Bicep must deploy these agents as a coherent system.

1766
01:06:21,720 --> 01:06:23,720
The module doesn't just create an agent.

1767
01:06:23,720 --> 01:06:25,720
It creates the agent with specific roles defined.

1768
01:06:25,720 --> 01:06:31,720
It registers the tools that agents can invoke, connections to your APIs, your databases, your external services.

1769
01:06:31,720 --> 01:06:36,720
It sets up evaluation flows that test agent behavior before it reaches production.

1770
01:06:36,720 --> 01:06:40,720
It configures monitoring that tracks how agents are performing once they're live.

1771
01:06:40,720 --> 01:06:43,720
This is where governance becomes architectural rather than aspirational.

1772
01:06:43,720 --> 01:06:48,720
Most governance failures in AI orchestration happen because agents have too much autonomy.

1773
01:06:48,720 --> 01:06:52,720
An agent is created, it's given access to a data source and some external API.

1774
01:06:52,720 --> 01:06:55,720
Nobody explicitly controls what the agent can do with those tools.

1775
01:06:55,720 --> 01:06:57,720
The agent tries something unexpected.

1776
01:06:57,720 --> 01:06:59,720
Data flows somewhere it shouldn't.

1777
01:06:59,720 --> 01:07:02,720
An API gets called with parameters that break something.

1778
01:07:02,720 --> 01:07:05,720
The organization discovers the failure when it's already caused damage.

1779
01:07:05,720 --> 01:07:08,720
Bicep prevents this by enforcing governance at the orchestration layer.

1780
01:07:08,720 --> 01:07:12,720
When you define an agent through bicep, you restrict which data sources the agent can access.

1781
01:07:12,720 --> 01:07:15,720
You don't give the agent access to your entire knowledge base.

1782
01:07:15,720 --> 01:07:18,720
You give it access to specific indexes, specific document sets.

1783
01:07:18,720 --> 01:07:21,720
If the agent needs to read customer data, you define that permission explicitly.

1784
01:07:21,720 --> 01:07:25,720
If the agent shouldn't be able to modify data, only read it.

1785
01:07:25,720 --> 01:07:27,720
You enforce read only access.

1786
01:07:27,720 --> 01:07:31,720
The permissions aren't something the agent requests and gets because nobody set up controls.

1787
01:07:31,720 --> 01:07:34,720
The permissions are defined in code before the agent is created.

1788
01:07:34,720 --> 01:07:37,720
Roll-based permissions for agent actions follow the same pattern.

1789
01:07:37,720 --> 01:07:41,720
An agent might be able to invoke certain APIs, but not others.

1790
01:07:41,720 --> 01:07:45,720
The analysis agent can call your internal calculations API, but not your payment API.

1791
01:07:45,720 --> 01:07:49,720
The action agent can update records in one database, but not another.

1792
01:07:49,720 --> 01:07:55,720
These permissions are enforced by bicep configuration, not by hope that the agent developer will implement them correctly.

1793
01:07:55,720 --> 01:07:57,720
High-risk operations require human approval.

1794
01:07:57,720 --> 01:08:02,720
If an agent is about to delete a record or transfer money or send a message to thousands of customers,

1795
01:08:02,720 --> 01:08:07,720
the bicep configuration can require explicit human approval before the operation proceeds.

1796
01:08:07,720 --> 01:08:08,720
The agent doesn't just do it.

1797
01:08:08,720 --> 01:08:09,720
The orchestration layer pauses.

1798
01:08:09,720 --> 01:08:11,720
It alerts a human reviewer.

1799
01:08:11,720 --> 01:08:12,720
The human confirms the action.

1800
01:08:12,720 --> 01:08:14,720
Only then does the agent proceed.

1801
01:08:14,720 --> 01:08:16,720
Every agent decision gets logged for audit purposes.

1802
01:08:16,720 --> 01:08:19,720
Not just the final result, but the reasoning chain itself.

1803
01:08:19,720 --> 01:08:21,720
Which data sources did the agent query?

1804
01:08:21,720 --> 01:08:22,720
Which tools did it invoke?

1805
01:08:22,720 --> 01:08:24,720
What parameters did it pass?

1806
01:08:24,720 --> 01:08:25,720
What was the response?

1807
01:08:25,720 --> 01:08:28,720
Each step flows into your observability layer.

1808
01:08:28,720 --> 01:08:32,720
An auditor can reconstruct the entire chain of reasoning that led to a decision.

1809
01:08:32,720 --> 01:08:35,720
The orchestration layer is where reasoning actually happens.

1810
01:08:35,720 --> 01:08:38,720
It's also the place where governance enforcement becomes possible.

1811
01:08:38,720 --> 01:08:41,720
Bicep structures that enforcements or governance isn't added later.

1812
01:08:41,720 --> 01:08:43,720
It's built in from the start.

1813
01:08:43,720 --> 01:08:45,720
Data governance and rag patterns.

1814
01:08:45,720 --> 01:08:46,720
Agents need knowledge to reason.

1815
01:08:46,720 --> 01:08:48,720
That knowledge must be governed.

1816
01:08:48,720 --> 01:08:52,720
Retrieval augmented generation is how enterprises ground LLMs in proprietary data.

1817
01:08:52,720 --> 01:08:54,720
The pattern is straightforward conceptually.

1818
01:08:54,720 --> 01:08:56,720
Before the model generates a response,

1819
01:08:56,720 --> 01:08:59,720
the system retrieves relevant documents from your knowledge base

1820
01:08:59,720 --> 01:09:02,720
and passes them to the model along with the user's question.

1821
01:09:02,720 --> 01:09:05,720
The model reads the retrieved context and answers based on it.

1822
01:09:05,720 --> 01:09:07,720
This solves a critical problem.

1823
01:09:07,720 --> 01:09:09,720
It prevents the model from hallucinating answers

1824
01:09:09,720 --> 01:09:13,720
and keeps responses tied to sources your organization actually owns.

1825
01:09:13,720 --> 01:09:15,720
But rag introduces a new governance challenge

1826
01:09:15,720 --> 01:09:17,720
that orchestration patterns don't fully address.

1827
01:09:17,720 --> 01:09:19,720
When you retrieve documents to pass into an LLM,

1828
01:09:19,720 --> 01:09:21,720
you're not just selecting information.

1829
01:09:21,720 --> 01:09:24,720
You're exposing that information to the model's reasoning process.

1830
01:09:24,720 --> 01:09:29,720
If those documents contain sensitive data, customer information, financial records,

1831
01:09:29,720 --> 01:09:33,720
proprietary research, you're now allowing the model to process that data.

1832
01:09:33,720 --> 01:09:36,720
The question becomes, who should the model be allowed to see?

1833
01:09:36,720 --> 01:09:40,720
If user A asks a question and user B's confidential data appears in the retrieved context,

1834
01:09:40,720 --> 01:09:42,720
has the model just leaked information?

1835
01:09:42,720 --> 01:09:45,720
This is where data governance becomes architectural rather than aspirational.

1836
01:09:45,720 --> 01:09:49,720
When documents enter your knowledge base, they must be classified by sensitivity level.

1837
01:09:49,720 --> 01:09:52,720
Public data can appear in retrieved context for any user.

1838
01:09:52,720 --> 01:09:55,720
Internal data should only appear for employees of your organization.

1839
01:09:55,720 --> 01:09:58,720
Confidential data should only appear for people with explicit clearance.

1840
01:09:58,720 --> 01:10:01,720
Restricted data shouldn't appear in LLM contexts at all.

1841
01:10:01,720 --> 01:10:04,720
It's too sensitive to risk exposure, even to a model,

1842
01:10:04,720 --> 01:10:07,720
but classification alone doesn't solve the problem.

1843
01:10:07,720 --> 01:10:09,720
You also need filtering based on user permissions.

1844
01:10:09,720 --> 01:10:13,720
A document might be classified as internal, but that doesn't mean every employee should see it.

1845
01:10:13,720 --> 01:10:19,720
Maybe it's internal finance data, only finance team members should get that document in their retrieved context.

1846
01:10:19,720 --> 01:10:23,720
Maybe it's HR records, only HR and authorised managers should access it.

1847
01:10:23,720 --> 01:10:26,720
The retrieval system needs to know who's asking the question, check their permissions,

1848
01:10:26,720 --> 01:10:29,720
and only include documents they're authorised to see.

1849
01:10:29,720 --> 01:10:32,720
This filtering happens at query time, not at index time.

1850
01:10:32,720 --> 01:10:38,720
The same document might appear in retrieved context for one user and be filtered out for another asking nearly identical questions.

1851
01:10:38,720 --> 01:10:40,720
Audit trails must track what got retrieved and used.

1852
01:10:40,720 --> 01:10:43,720
When an LLM generates a response based on retrieved context,

1853
01:10:43,720 --> 01:10:45,720
you need to know which documents were included.

1854
01:10:45,720 --> 01:10:50,720
If something goes wrong later, a user claims they received incorrect information or an auditor questions how a decision was made,

1855
01:10:50,720 --> 01:10:55,720
you can reconstruct the exact context the model was working with by-seps structures this entire pattern.

1856
01:10:55,720 --> 01:10:59,720
The module deploys Azure AI search with private endpoints and managed identities.

1857
01:10:59,720 --> 01:11:02,720
Traffic doesn't leak onto the public internet.

1858
01:11:02,720 --> 01:11:08,720
The search service only accepts connections from authorised services using cryptographic identity, not API keys.

1859
01:11:08,720 --> 01:11:12,720
The module creates data source connections, to share point where your documents live,

1860
01:11:12,720 --> 01:11:17,720
to SQL where structured data sits, to cause most where your knowledge graph might be stored.

1861
01:11:17,720 --> 01:11:19,720
These connections use least privileged access.

1862
01:11:19,720 --> 01:11:22,720
The indexer that reads from share point doesn't have permissions to write.

1863
01:11:22,720 --> 01:11:27,720
It doesn't have access to anything except the specific document library you've designated as the knowledge base.

1864
01:11:27,720 --> 01:11:30,720
The indexing pipelines respect data classification.

1865
01:11:30,720 --> 01:11:34,720
As documents get indexed, metadata about sensitivity gets attached.

1866
01:11:34,720 --> 01:11:40,720
A skill in the indexing pipeline reads document properties, determines the classification level, and tags each index document.

1867
01:11:40,720 --> 01:11:45,720
Later when the model queries, the search service can filter based on those tags and the current user's permissions.

1868
01:11:45,720 --> 01:11:48,720
Query time filtering enforces user permissions.

1869
01:11:48,720 --> 01:11:51,720
When a user makes a request, the orchestration layer captures their identity.

1870
01:11:51,720 --> 01:11:53,720
That identity flows with the search query.

1871
01:11:53,720 --> 01:11:58,720
The search service filters results to include only documents the user has permissions to access.

1872
01:11:58,720 --> 01:12:03,720
The model receives only documents that have passed both the classification check and the user permission check.

1873
01:12:03,720 --> 01:12:06,720
Without governance baked into the infrastructure, Ragsystems leak.

1874
01:12:06,720 --> 01:12:08,720
Documents get indexed without classification.

1875
01:12:08,720 --> 01:12:10,720
Permissions don't get checked.

1876
01:12:10,720 --> 01:12:15,720
The same query returns different results for different users or returns results that shouldn't be visible to any user.

1877
01:12:15,720 --> 01:12:20,720
An auditor asks which documents were used in a specific response and discovers that audit logging was never enabled.

1878
01:12:20,720 --> 01:12:22,720
Bicep deployments eliminate these gaps.

1879
01:12:22,720 --> 01:12:26,720
The infrastructure enforces classification, permissions, and audit trails from day one.

1880
01:12:26,720 --> 01:12:28,720
Bicep modules is organizational DNA.

1881
01:12:28,720 --> 01:12:30,720
We've covered the architecture.

1882
01:12:30,720 --> 01:12:32,720
Now let's talk about how organizations actually scale this.

1883
01:12:32,720 --> 01:12:37,720
The moment you build your first bicep module, you've created something that looks like infrastructure code.

1884
01:12:37,720 --> 01:12:40,720
But what you've actually built is institutional memory.

1885
01:12:40,720 --> 01:12:43,720
A well designed module encodes years of lessons learned.

1886
01:12:43,720 --> 01:12:48,720
It captures decisions about how identity should work, how data should flow, how governance should enforce itself.

1887
01:12:48,720 --> 01:12:52,720
When a team deploys that module, they inherit all of that knowledge automatically.

1888
01:12:52,720 --> 01:12:54,720
They don't have to rediscover what works.

1889
01:12:54,720 --> 01:12:57,720
They don't have to make the same mistakes you already made and learned from.

1890
01:12:57,720 --> 01:12:59,720
This is what we mean by DNA.

1891
01:12:59,720 --> 01:13:01,720
Not the biological metaphor, but the structural sense.

1892
01:13:01,720 --> 01:13:04,720
DNA is how an organism passes its patterns forward.

1893
01:13:04,720 --> 01:13:07,720
It's how complexity reproduces consistently.

1894
01:13:07,720 --> 01:13:09,720
A bicep module is the same thing for your infrastructure.

1895
01:13:09,720 --> 01:13:11,720
It's how the organization passes its patterns forward.

1896
01:13:11,720 --> 01:13:16,720
Deploy it once and the pattern is documented, deploy it a hundred times, and it's identical every time.

1897
01:13:16,720 --> 01:13:18,720
No variation, no drift.

1898
01:13:18,720 --> 01:13:22,720
The organizational knowledge doesn't live in a slack conversation or a wiki page that gets outdated.

1899
01:13:22,720 --> 01:13:24,720
It lives in code that gets executed.

1900
01:13:24,720 --> 01:13:29,720
Enterprise organizations typically build a library of modules arranged by function.

1901
01:13:29,720 --> 01:13:35,720
There's a core AI infrastructure module that deploys the perimeter we've spent the last few sections discussing.

1902
01:13:35,720 --> 01:13:40,720
It creates the hub, sets up the network isolation, configures the identity layer, attaches observability.

1903
01:13:40,720 --> 01:13:44,720
One module, one deployment, everything we've talked about gets instantiated.

1904
01:13:44,720 --> 01:13:47,720
Teams don't build this themselves. They deploy the module.

1905
01:13:47,720 --> 01:13:50,720
On the first day they use it, they have the hardened perimeter.

1906
01:13:50,720 --> 01:13:52,720
On the hundredth deployment, they have it again.

1907
01:13:52,720 --> 01:13:55,720
No learning curve required beyond understanding how to invoke the module.

1908
01:13:55,720 --> 01:14:00,720
But one core module isn't enough for scale. Organizations build specialized modules for specific patterns.

1909
01:14:00,720 --> 01:14:05,720
There's a rag module that deploys AI search with all the data governance patterns we discussed.

1910
01:14:05,720 --> 01:14:09,720
There's an agent orchestration module that sets up agents with roll-based permissions and audit logging.

1911
01:14:09,720 --> 01:14:13,720
There's a fine tuning module that handles the infrastructure for custom model training.

1912
01:14:13,720 --> 01:14:15,720
Each module specializes.

1913
01:14:15,720 --> 01:14:17,720
Each one handles a specific type of workload.

1914
01:14:17,720 --> 01:14:20,720
But they all follow the same patterns established in the core module.

1915
01:14:20,720 --> 01:14:23,720
They all include identity perimeter, network perimeter, observability.

1916
01:14:23,720 --> 01:14:27,720
They just specialize the reasoning layer for their specific purpose.

1917
01:14:27,720 --> 01:14:29,720
Governance modules live in this library too.

1918
01:14:29,720 --> 01:14:33,720
There's a module that deploys as your policy assignments for AI workloads.

1919
01:14:33,720 --> 01:14:38,720
It defines which models are allowed, which regions are permitted, which tagging standards are enforced.

1920
01:14:38,720 --> 01:14:43,720
The module takes parameters, your organization's specific policies, and wires them in.

1921
01:14:43,720 --> 01:14:45,720
Different business units might have different policies.

1922
01:14:45,720 --> 01:14:50,720
The module accommodates that through parameterization, not through manual policy assignment.

1923
01:14:50,720 --> 01:14:53,720
Observability modules complete the picture.

1924
01:14:53,720 --> 01:15:02,720
There's a module that deploys the entire monitoring stack, log analytics workspace, application insights, alert rules, power BI dashboards, all pre-configured and wired together.

1925
01:15:02,720 --> 01:15:05,720
Teams don't design their own observability. They deploy the module.

1926
01:15:05,720 --> 01:15:09,720
They immediately have visibility into tokens, cost, latency, errors.

1927
01:15:09,720 --> 01:15:10,720
The module does the wiring.

1928
01:15:10,720 --> 01:15:11,720
Humans do the analysis.

1929
01:15:11,720 --> 01:15:12,720
Here's the scaling dynamic.

1930
01:15:12,720 --> 01:15:14,720
Teams don't write their own infrastructure.

1931
01:15:14,720 --> 01:15:15,720
They consume modules.

1932
01:15:15,720 --> 01:15:17,720
This ensures consistency across the organization.

1933
01:15:17,720 --> 01:15:21,720
You don't have one team implementing identity one way and another team implementing it differently.

1934
01:15:21,720 --> 01:15:24,720
Every deployment of the core module creates the same identity structure.

1935
01:15:24,720 --> 01:15:26,720
This reduces the surface area for mistakes.

1936
01:15:26,720 --> 01:15:30,720
A team can't accidentally skip the observability layer because the module includes it.

1937
01:15:30,720 --> 01:15:33,720
They can't deploy without private endpoints because the module enforces them.

1938
01:15:33,720 --> 01:15:37,720
Over time, the module library becomes the organization's institutional knowledge.

1939
01:15:37,720 --> 01:15:41,720
New teams onboard faster because the patterns are already proven.

1940
01:15:41,720 --> 01:15:43,720
A team wants to build an AI feature.

1941
01:15:43,720 --> 01:15:44,720
They find the rag module.

1942
01:15:44,720 --> 01:15:45,720
They deploy it.

1943
01:15:45,720 --> 01:15:48,720
They inherit years of learning about how to structure vector search,

1944
01:15:48,720 --> 01:15:51,720
how to handle data governance, how to track retrieval quality.

1945
01:15:51,720 --> 01:15:52,720
They don't have to invent this.

1946
01:15:52,720 --> 01:15:54,720
It's already encoded in the module.

1947
01:15:54,720 --> 01:15:56,720
The library also creates a feedback loop.

1948
01:15:56,720 --> 01:15:59,720
When a team discovers a better way to handle something, the module gets updated.

1949
01:15:59,720 --> 01:16:01,720
Every subsequent deployment includes that improvement.

1950
01:16:01,720 --> 01:16:05,720
The organizational learning propagates forward instead of getting siloed.

1951
01:16:05,720 --> 01:16:09,720
A security team discovers a better way to configure managed identities.

1952
01:16:09,720 --> 01:16:10,720
The core module gets updated.

1953
01:16:10,720 --> 01:16:13,720
The next 100 teams that deploy automatically benefit from that discovery.

1954
01:16:13,720 --> 01:16:19,720
This is how enterprises scale from one AI pilot to hundreds of production workloads without creating chaos.

1955
01:16:19,720 --> 01:16:21,720
Not by hoping teams will follow best practices.

1956
01:16:21,720 --> 01:16:23,720
Not by writing documentation that becomes obsolete.

1957
01:16:23,720 --> 01:16:29,720
But by encoding the practices into modules and making the modules the default way infrastructure gets built.

1958
01:16:29,720 --> 01:16:31,720
CICD and infrastructure validation.

1959
01:16:31,720 --> 01:16:34,720
Modules are deployed through pipelines, not manually.

1960
01:16:34,720 --> 01:16:36,720
And that distinction is structural.

1961
01:16:36,720 --> 01:16:40,720
The moment you have a bicep module, you have something that can be validated before it reaches production.

1962
01:16:40,720 --> 01:16:45,720
This is where infrastructure as code stops being just a convenience and becomes a governance mechanism.

1963
01:16:45,720 --> 01:16:48,720
Manual infrastructure gets deployed first and discovered to be broken later.

1964
01:16:48,720 --> 01:16:51,720
Bicep infrastructure gets validated before it ever touches your environment.

1965
01:16:51,720 --> 01:16:54,720
The validation happens in CICD pipelines.

1966
01:16:54,720 --> 01:16:58,720
When a developer pushes a bicep module to source control the pipeline wakes up.

1967
01:16:58,720 --> 01:17:00,720
It doesn't immediately deploy. It runs a series of checks.

1968
01:17:00,720 --> 01:17:02,720
Each check is automated.

1969
01:17:02,720 --> 01:17:04,720
Each one prevents a category of mistakes from reaching production.

1970
01:17:04,720 --> 01:17:06,720
The first check is linting.

1971
01:17:06,720 --> 01:17:12,720
Bicep has a linter that checks code for syntax errors, missing parameters, unused variables, formatting inconsistencies.

1972
01:17:12,720 --> 01:17:14,720
It catches the obvious mistakes.

1973
01:17:14,720 --> 01:17:18,720
Typoes in resource names, incorrect property types, logic errors.

1974
01:17:18,720 --> 01:17:20,720
A module that fails linting doesn't move forward.

1975
01:17:20,720 --> 01:17:22,720
The developer gets immediate feedback.

1976
01:17:22,720 --> 01:17:25,720
They fix the issue locally. They push again, the pipeline runs again.

1977
01:17:25,720 --> 01:17:29,720
This catches problems at the speed of development, not at the speed of discovery after deployment.

1978
01:17:29,720 --> 01:17:32,720
The second check validates that the module complies with Azure Policy.

1979
01:17:32,720 --> 01:17:34,720
This is critical and non-obvious.

1980
01:17:34,720 --> 01:17:38,720
You can have syntactically correct bicep that violates your organization's policies.

1981
01:17:38,720 --> 01:17:42,720
Maybe the module tries to deploy an AI service in an unapproved region.

1982
01:17:42,720 --> 01:17:44,720
Maybe it creates resources without the required tagging.

1983
01:17:44,720 --> 01:17:48,720
Maybe it doesn't include private endpoints where your policy mandates them.

1984
01:17:48,720 --> 01:17:52,720
The validation check queries your Azure Policy definitions and tests the module against them.

1985
01:17:52,720 --> 01:17:55,720
Will this module when deployed violate any policies?

1986
01:17:55,720 --> 01:18:00,720
The pipeline determines that before you ever hit Azure, if it would violate policy, the deployment fails.

1987
01:18:00,720 --> 01:18:04,720
The developer is forced to fix the module to comply with policy before it can proceed.

1988
01:18:04,720 --> 01:18:08,720
The third check tests that the module actually deploys without errors.

1989
01:18:08,720 --> 01:18:10,720
This uses a what if deployment?

1990
01:18:10,720 --> 01:18:14,720
The pipeline spins up a temporary environment, runs the bicep module against it,

1991
01:18:14,720 --> 01:18:17,720
and checks whether all resources get created successfully.

1992
01:18:17,720 --> 01:18:20,720
The what if deployment doesn't actually create the resources.

1993
01:18:20,720 --> 01:18:23,720
It simulates the deployment and reports what would happen.

1994
01:18:23,720 --> 01:18:26,720
If any part of the deployment would fail, the pipeline catches it.

1995
01:18:26,720 --> 01:18:29,720
Maybe the module tries to reference a resource that doesn't exist.

1996
01:18:29,720 --> 01:18:33,720
Maybe it tries to create a child resource on a parent that doesn't support children.

1997
01:18:33,720 --> 01:18:37,720
The what if deployment finds these issues before they become production incidents.

1998
01:18:37,720 --> 01:18:41,720
The fourth check verifies that deployed resources have the correct configuration.

1999
01:18:41,720 --> 01:18:44,720
After the what if deployment runs, the pipeline queries, the simulated environment,

2000
01:18:44,720 --> 01:18:47,720
and checks that resources have the expected properties.

2001
01:18:47,720 --> 01:18:49,720
The RBAC assignment is configured correctly?

2002
01:18:49,720 --> 01:18:50,720
Yes.

2003
01:18:50,720 --> 01:18:52,720
The private endpoint is linked to the right subnet.

2004
01:18:52,720 --> 01:18:54,720
Yes, the managed identity has been created.

2005
01:18:54,720 --> 01:18:58,720
Yes, any mismatch between what the module intended and what actually got created triggers a failure.

2006
01:18:58,720 --> 01:19:00,720
Again, the developer fixes the module.

2007
01:19:00,720 --> 01:19:02,720
They push again, the pipeline validates again.

2008
01:19:02,720 --> 01:19:05,720
The fifth check runs security scans to detect misconfigurations.

2009
01:19:05,720 --> 01:19:09,720
There are tools that look at your infrastructure and ask, could this be hacked?

2010
01:19:09,720 --> 01:19:11,720
Are there ways that an attacker could exploit this?

2011
01:19:11,720 --> 01:19:13,720
Does this resource expose secrets?

2012
01:19:13,720 --> 01:19:15,720
Is there a path to privilege escalation?

2013
01:19:15,720 --> 01:19:18,720
These tools scan the what if deployment results and report security issues.

2014
01:19:18,720 --> 01:19:22,720
If critical issues are found, the pipeline fails, the developer remediates.

2015
01:19:22,720 --> 01:19:23,720
The pipeline tries again.

2016
01:19:23,720 --> 01:19:27,720
Only after all of these validations pass, should the module be deployed to actual Azure?

2017
01:19:27,720 --> 01:19:33,720
Not because humans are afraid to trust other humans, but because validation is faster than remediation,

2018
01:19:33,720 --> 01:19:39,720
catching a mistake during CI is a 30 second fix, catching a mistake after its deployed to production is a three hour incident.

2019
01:19:39,720 --> 01:19:41,720
Preventing the mistake entirely is better than both.

2020
01:19:41,720 --> 01:19:44,720
This automation prevents specific categories of failure.

2021
01:19:44,720 --> 01:19:48,720
Configuration drift happens when resources change over time, in ways the code doesn't reflect.

2022
01:19:48,720 --> 01:19:51,720
A human logs into Azure and manually updates something.

2023
01:19:51,720 --> 01:19:54,720
The bicep module still exists, but it no longer represents reality.

2024
01:19:54,720 --> 01:19:59,720
With CI, CD validation enforcing that modules deploy consistently, drift becomes obvious.

2025
01:19:59,720 --> 01:20:01,720
You update the module, the pipeline validates.

2026
01:20:01,720 --> 01:20:02,720
You deploy.

2027
01:20:02,720 --> 01:20:04,720
The infrastructure stays aligned with code.

2028
01:20:04,720 --> 01:20:06,720
Policy violations get prevented.

2029
01:20:06,720 --> 01:20:11,720
Without pipeline validation, developers could deploy resources that violate organizational policy.

2030
01:20:11,720 --> 01:20:12,720
With validation, they can't.

2031
01:20:12,720 --> 01:20:15,720
Policy gets enforced before deployment, not after.

2032
01:20:15,720 --> 01:20:17,720
Security misconfigurations get caught.

2033
01:20:17,720 --> 01:20:20,720
Resources exposed to the internet when they shouldn't be encryption not enabled.

2034
01:20:20,720 --> 01:20:21,720
Reac authentication.

2035
01:20:21,720 --> 01:20:24,720
The security scans find these and block deployment.

2036
01:20:24,720 --> 01:20:26,720
Audit failures become impossible.

2037
01:20:26,720 --> 01:20:28,720
Every deployment goes through the same validated pipeline.

2038
01:20:28,720 --> 01:20:32,720
Every module gets linted, policy checked, tested and scanned.

2039
01:20:32,720 --> 01:20:34,720
Every deployment creates an audit trail.

2040
01:20:34,720 --> 01:20:37,720
You can show an auditor exactly what validation happened and that it passed.

2041
01:20:37,720 --> 01:20:41,720
CI/CD for infrastructure is non-negotiable for enterprise AI.

2042
01:20:41,720 --> 01:20:45,720
It's the difference between hoping deployments are correct and knowing they are.

2043
01:20:45,720 --> 01:20:50,720
From liability to asset, your AI landing zone is a liability if it's built manually.

2044
01:20:50,720 --> 01:20:55,720
Governance, observability and security end up as afterthoughts, not part of the foundation.

2045
01:20:55,720 --> 01:20:57,720
Bicep transforms it into an asset.

2046
01:20:57,720 --> 01:20:59,720
Automated, auditable, scalable.

2047
01:20:59,720 --> 01:21:01,720
The hardened perimeter isn't a single control.

2048
01:21:01,720 --> 01:21:04,720
It's the integration of five structural layers deployed together.

2049
01:21:04,720 --> 01:21:06,720
Identity through managed identities in RBAC.

2050
01:21:06,720 --> 01:21:08,720
Network through private endpoints and v-nets.

2051
01:21:08,720 --> 01:21:11,720
Governance through Azure Policy and Management Groups.

2052
01:21:11,720 --> 01:21:14,720
Observability through token tracking and cost attribution.

2053
01:21:14,720 --> 01:21:17,720
Orchestration through AI Foundry and API Management.

2054
01:21:17,720 --> 01:21:22,720
When these layers deploy together through bicep, they create a system where compliance is automatic.

2055
01:21:22,720 --> 01:21:26,720
Cost is visible, data is protected and reasoning is auditable.

2056
01:21:26,720 --> 01:21:29,720
This is how enterprises scale AI without chaos.

2057
01:21:29,720 --> 01:21:33,720
The alternative is shadow AI, audit failures and uncontrolled costs.

2058
01:21:33,720 --> 01:21:34,720
Your choice is clear.

2059
01:21:34,720 --> 01:21:37,720
Build the perimeter with bicep or inherit the liability.