The Silent Crash: Why Your Platform is Rotting from the Inside

1
00:00:00,000 --> 00:00:07,360
It's O347 UTC. Most of the IT team is asleep, but somewhere in the cloud logs a pattern is forming.

2
00:00:07,360 --> 00:00:12,720
It starts small, three new SharePoint lists named "Test" created within four minutes.

3
00:00:12,720 --> 00:00:16,240
Then a sudden spike in permission changes without a ticket.

4
00:00:16,240 --> 00:00:20,000
Finally, a series of power automate flows fail concurrently.

5
00:00:20,000 --> 00:00:25,680
And the phone doesn't ring. That's the scary part. The CIO doesn't get a call because the system isn't technically down.

6
00:00:25,680 --> 00:00:32,080
The server lights are still green, but if you look closely at that pattern, what you're seeing isn't an outage. It's entropy.

7
00:00:32,080 --> 00:00:36,320
Entropy. That gradual decline into disorder.

8
00:00:36,320 --> 00:00:42,160
It feels like we focus so much on the loud failures, the ransomware, the DDoS attacks,

9
00:00:42,160 --> 00:00:50,320
that we ignore this quiet rot. We're talking about the death by a thousand cuts in the Microsoft 365 ecosystem today, aren't we?

10
00:00:50,320 --> 00:00:55,120
Exactly. We are talking about how platform systems fail quietly long before they fail loudly

11
00:00:55,120 --> 00:00:59,680
and more importantly, how to stop that drift with discipline instead of heroics.

12
00:00:59,680 --> 00:01:02,800
Because right now most organizations are running on heroics.

13
00:01:02,800 --> 00:01:06,640
They have that one power platform guy who fixes the flow when it breaks at midnight.

14
00:01:06,640 --> 00:01:08,960
That's not a strategy, that's a liability.

15
00:01:08,960 --> 00:01:11,360
So let's break down this entropy.

16
00:01:11,360 --> 00:01:15,760
The source material we're looking at today describes it as a compounding instability.

17
00:01:15,760 --> 00:01:19,520
It starts with the substrate itself. SharePoint.

18
00:01:19,520 --> 00:01:23,360
Right, SharePoint. The Swiss Army knife that everyone tries to use as a hammer,

19
00:01:23,360 --> 00:01:26,000
a saw and a relational database all at once.

20
00:01:26,000 --> 00:01:29,520
The core problem is that SharePoint is a collaboration substrate.

21
00:01:29,520 --> 00:01:33,280
It is designed for documents and lists of data that humans read.

22
00:01:33,280 --> 00:01:35,600
But people try to treat it like SQL Server.

23
00:01:35,600 --> 00:01:36,960
I've seen this a hundred times.

24
00:01:36,960 --> 00:01:40,240
You get a request for a simple tracker.

25
00:01:40,240 --> 00:01:44,400
And suddenly you have a list with 50 columns, five different look-up chains,

26
00:01:44,400 --> 00:01:48,080
and a permission structure that requires a PhD to understand.

27
00:01:48,080 --> 00:01:49,520
And that's where the rot starts.

28
00:01:49,520 --> 00:01:51,760
The document calls it schema discipline.

29
00:01:51,760 --> 00:01:53,600
It sounds boring, but it's critical.

30
00:01:53,600 --> 00:01:55,200
For example, naming lists.

31
00:01:55,200 --> 00:01:58,960
You see lists named Genstrial Tracker or Project X-Temp.

32
00:01:58,960 --> 00:02:00,960
That's fine for a sandbox, but in production?

33
00:02:00,960 --> 00:02:02,320
No.

34
00:02:02,320 --> 00:02:03,920
Name the list after the business noun.

35
00:02:03,920 --> 00:02:06,160
It's an incident, not Genstacker.

36
00:02:06,160 --> 00:02:09,680
And technical debt accumulates in the column names too, right?

37
00:02:09,680 --> 00:02:14,320
You name a column status today, then change the display name to current state.

38
00:02:14,320 --> 00:02:16,720
But the API still sees status.

39
00:02:16,720 --> 00:02:18,720
Then you delete it and make status too.

40
00:02:19,360 --> 00:02:22,160
Status too. Status final. Status real.

41
00:02:22,160 --> 00:02:25,040
It's a nightmare for anyone trying to build an app on top of it.

42
00:02:25,040 --> 00:02:26,000
The rule is simple.

43
00:02:26,000 --> 00:02:27,840
Internal names never change.

44
00:02:27,840 --> 00:02:30,640
Labels are for the user interface, not the API.

45
00:02:30,640 --> 00:02:31,760
If you don't enforce this,

46
00:02:31,760 --> 00:02:33,840
you're building a skyscraper on a swamp.

47
00:02:33,840 --> 00:02:35,840
And then there's the issue of relationships.

48
00:02:35,840 --> 00:02:37,840
Oh, the dreaded look-up column?

49
00:02:37,840 --> 00:02:38,880
Look-ups are seductive.

50
00:02:38,880 --> 00:02:40,720
They feel like a relational join.

51
00:02:40,720 --> 00:02:42,960
But SharePoint isn't a relational database.

52
00:02:42,960 --> 00:02:44,720
The guidance here is strict.

53
00:02:44,720 --> 00:02:46,880
One look-up per critical view.

54
00:02:46,880 --> 00:02:48,640
Maybe two if you're lucky.

55
00:02:48,640 --> 00:02:51,600
If you are trying to do multi-hop relationships,

56
00:02:51,600 --> 00:02:54,800
like filtering a list based on a look-up that looks up another list,

57
00:02:54,800 --> 00:02:55,760
you need to stop.

58
00:02:55,760 --> 00:02:56,560
So where do you go?

59
00:02:56,560 --> 00:02:59,440
If SharePoint can't handle that complexity,

60
00:02:59,440 --> 00:03:00,800
do you just give up?

61
00:03:00,800 --> 00:03:01,840
You move the data.

62
00:03:01,840 --> 00:03:04,320
That's a dataverse conversation or a SQL conversation.

63
00:03:04,320 --> 00:03:07,280
If you need transactional integrity and complex joins,

64
00:03:07,280 --> 00:03:08,960
you're abusing SharePoint.

65
00:03:08,960 --> 00:03:10,480
And the system will punish you for it.

66
00:03:10,480 --> 00:03:12,640
Not today, maybe not tomorrow,

67
00:03:12,640 --> 00:03:15,440
but eventually, a query will hit a threshold.

68
00:03:15,440 --> 00:03:18,160
Delegation will break and your app will show blank screens.

69
00:03:18,960 --> 00:03:20,560
Speaking of breaking things,

70
00:03:20,560 --> 00:03:22,320
let's talk about permissions.

71
00:03:22,320 --> 00:03:24,240
This is a huge pain point.

72
00:03:24,240 --> 00:03:26,400
The item-level permission trap.

73
00:03:26,400 --> 00:03:30,000
It's the classic I want only the creator to see their own items request.

74
00:03:30,000 --> 00:03:33,440
It sounds reasonable, but technically, it's a treadmill.

75
00:03:33,440 --> 00:03:37,600
If you break inheritance on every single item in a list of 50,000 items,

76
00:03:37,600 --> 00:03:40,160
the server has to check the access control list

77
00:03:40,160 --> 00:03:42,880
for every single row every time a user queries it.

78
00:03:42,880 --> 00:03:44,320
It destroys performance.

79
00:03:44,320 --> 00:03:47,120
So the fix is to break inheritance at the site level,

80
00:03:47,120 --> 00:03:48,480
not the item-level.

81
00:03:48,480 --> 00:03:50,480
Always, if you need to secure rows differently,

82
00:03:50,480 --> 00:03:53,040
you need different lists or a different data source.

83
00:03:53,040 --> 00:03:54,240
And please use groups.

84
00:03:54,240 --> 00:03:56,400
Never assign permissions to an individual person.

85
00:03:56,400 --> 00:03:58,160
People leave, groups remain.

86
00:03:58,160 --> 00:04:00,960
It seems like the theme here is "Stop improvising",

87
00:04:00,960 --> 00:04:04,800
which leads us perfectly into the second pillar, "Power Apps".

88
00:04:04,800 --> 00:04:07,760
The text describes "Power Apps" as either "deterministic"

89
00:04:07,760 --> 00:04:09,200
or "KeyOS" with a "Save" button.

90
00:04:09,200 --> 00:04:11,760
That's a pretty stark dichotomy.

91
00:04:11,760 --> 00:04:12,960
It is, but it's accurate.

92
00:04:12,960 --> 00:04:14,960
A canvas app feels like a blank canvas.

93
00:04:14,960 --> 00:04:16,240
You can paint whatever you want.

94
00:04:16,240 --> 00:04:17,360
That's the danger.

95
00:04:17,360 --> 00:04:21,040
A, "deterministic" app means "You the developer control the state".

96
00:04:21,040 --> 00:04:23,200
You know exactly what data is loaded

97
00:04:23,200 --> 00:04:26,000
when it's submitted and what happens if it fails.

98
00:04:26,000 --> 00:04:27,360
Versus the "KeyOS" approach,

99
00:04:27,360 --> 00:04:29,680
which I assume is just throwing controls on a screen

100
00:04:29,680 --> 00:04:32,240
and hoping the submit form may function works?

101
00:04:32,240 --> 00:04:33,360
Precisely.

102
00:04:33,360 --> 00:04:35,840
"KeyOS" is relying on implicit behaviors.

103
00:04:35,840 --> 00:04:38,880
It's putting complex logic inside a button's on-select property

104
00:04:38,880 --> 00:04:41,120
that patches three different data sources,

105
00:04:41,120 --> 00:04:42,160
sends an email,

106
00:04:42,160 --> 00:04:43,920
and navigates to a new screen,

107
00:04:43,920 --> 00:04:45,920
all without any error handling.

108
00:04:45,920 --> 00:04:47,840
If the internet blinks in the middle of that,

109
00:04:47,840 --> 00:04:49,200
your data is corrupted.

110
00:04:49,200 --> 00:04:50,800
So how do we make it deterministic?

111
00:04:50,800 --> 00:04:53,440
You centralize your reads and rights.

112
00:04:53,440 --> 00:04:56,400
Your controls shouldn't be queering the database directly.

113
00:04:56,400 --> 00:04:58,000
They should be looking at local collections

114
00:04:58,000 --> 00:04:59,840
or variables where appropriate,

115
00:04:59,840 --> 00:05:03,040
and you need to treat submit form as an atomic operation.

116
00:05:03,040 --> 00:05:04,400
If you use patch,

117
00:05:04,400 --> 00:05:06,160
you have to handle concurrency.

118
00:05:06,160 --> 00:05:08,000
Concurrency is a big one.

119
00:05:08,000 --> 00:05:10,880
Two people editing the same record at the same time.

120
00:05:10,880 --> 00:05:12,640
If you don't handle it explicitly,

121
00:05:12,640 --> 00:05:14,240
the last person to save wins

122
00:05:14,240 --> 00:05:16,640
and the other person's work is overwritten without them knowing.

123
00:05:16,640 --> 00:05:18,720
That's that quiet failure again.

124
00:05:18,720 --> 00:05:21,200
The system didn't crash, but the data is wrong.

125
00:05:21,200 --> 00:05:24,240
The text also mentions operational discipline in apps,

126
00:05:24,240 --> 00:05:25,360
logging what the app did,

127
00:05:25,360 --> 00:05:26,960
not just what it hoped to do.

128
00:05:26,960 --> 00:05:29,360
That's the difference between a toy and a tool.

129
00:05:29,360 --> 00:05:31,840
A professional app generates telemetry.

130
00:05:31,840 --> 00:05:34,320
User X navigated to screen Y.

131
00:05:34,320 --> 00:05:38,240
User Z attempted to submit form failed with error code 133.

132
00:05:38,240 --> 00:05:39,520
If you don't have that logs,

133
00:05:39,520 --> 00:05:41,120
you're debugging with guesses.

134
00:05:41,120 --> 00:05:42,960
Let's shift gears to the engine room,

135
00:05:42,960 --> 00:05:44,080
power automate.

136
00:05:44,080 --> 00:05:45,840
The description here is vivid.

137
00:05:45,840 --> 00:05:47,680
Time bombs diffused with protocols.

138
00:05:47,680 --> 00:05:50,400
Flows are terrifying because they run in the background.

139
00:05:50,400 --> 00:05:52,320
You can't see them failing unless you're looking

140
00:05:52,320 --> 00:05:54,560
and they fail just like buildings burn.

141
00:05:54,560 --> 00:05:56,400
Quietly at first, then all at once.

142
00:05:56,400 --> 00:05:58,960
What's the number one cause of that fire?

143
00:05:58,960 --> 00:06:01,600
A lack of defensive design, specifically triggers.

144
00:06:01,600 --> 00:06:03,360
If you have a flow that triggers on

145
00:06:03,360 --> 00:06:04,960
when an item is modified,

146
00:06:04,960 --> 00:06:06,800
and you don't set a trigger condition,

147
00:06:06,800 --> 00:06:10,240
that flow runs every single time anyone touches that item,

148
00:06:10,240 --> 00:06:11,920
even if they just fix a typo.

149
00:06:11,920 --> 00:06:15,840
So you have thousands of unnecessary runs clogging up your API limits.

150
00:06:15,840 --> 00:06:17,520
Exactly. You need scope triggers.

151
00:06:17,520 --> 00:06:20,160
Only run this flow if the status changes to approved,

152
00:06:20,160 --> 00:06:22,960
that one change can reduce your load by 90%.

153
00:06:22,960 --> 00:06:24,240
And then there's concurrency.

154
00:06:24,240 --> 00:06:25,040
Right.

155
00:06:25,040 --> 00:06:29,600
By default, power automate tries to run everything in parallel to be helpful.

156
00:06:29,600 --> 00:06:30,800
Which is a disaster.

157
00:06:30,800 --> 00:06:34,000
If you are trying to update the same list item or increment a counter,

158
00:06:34,000 --> 00:06:35,200
you get race conditions,

159
00:06:35,200 --> 00:06:36,800
you have to set concurrency limits.

160
00:06:36,800 --> 00:06:39,760
Sometimes you need to force it to run sequentially one at a time

161
00:06:39,760 --> 00:06:41,520
to ensure data integrity.

162
00:06:41,520 --> 00:06:43,040
And error handling?

163
00:06:43,040 --> 00:06:45,840
The try catch pattern isn't native to the visual designer

164
00:06:45,840 --> 00:06:48,000
in the same way it is in code, isn't it?

165
00:06:48,000 --> 00:06:49,040
You have to build it.

166
00:06:49,040 --> 00:06:51,280
You create a scope block for your main actions.

167
00:06:51,280 --> 00:06:52,320
That's your try.

168
00:06:52,320 --> 00:06:55,280
And then a parallel block that only runs if the first one fails.

169
00:06:55,280 --> 00:06:56,560
That's your catch.

170
00:06:56,560 --> 00:06:58,480
If you don't have a catch block that alerts you,

171
00:06:58,480 --> 00:07:00,000
a flow can fail for weeks.

172
00:07:00,000 --> 00:07:01,840
And you'll never know until a user asks,

173
00:07:01,840 --> 00:07:03,360
"Hey, where's my approval?"

174
00:07:03,360 --> 00:07:05,280
That silence is the enemy.

175
00:07:05,280 --> 00:07:09,680
It seems like the goal of all these protocols is to turn quiet failures

176
00:07:09,680 --> 00:07:12,080
into loud-contained failures.

177
00:07:12,080 --> 00:07:15,280
Yes, I want the system to scream at me when something breaks

178
00:07:15,280 --> 00:07:16,560
so I can fix it immediately.

179
00:07:16,560 --> 00:07:20,160
I don't want it to whisper while it corrupts the database for six months.

180
00:07:20,160 --> 00:07:22,320
Now we can't talk about the future of this platform

181
00:07:22,320 --> 00:07:24,640
without talking about AI.

182
00:07:24,640 --> 00:07:26,400
Copylates, AI Builder.

183
00:07:26,400 --> 00:07:29,280
Everyone wants to turn them on.

184
00:07:29,280 --> 00:07:31,040
But the text warns,

185
00:07:31,040 --> 00:07:33,120
the spike didn't start with code.

186
00:07:33,120 --> 00:07:34,640
It started with a PDF.

187
00:07:34,640 --> 00:07:38,080
AI is the ultimate accelerator.

188
00:07:38,080 --> 00:07:41,200
If your governance is bad, AI will make it bad faster.

189
00:07:41,200 --> 00:07:45,200
The danger with AI, specifically large language models or copylates,

190
00:07:45,200 --> 00:07:47,520
is hallucination and over-access.

191
00:07:47,520 --> 00:07:49,360
Librarian, not Oracle.

192
00:07:49,360 --> 00:07:51,680
I liked that line.

193
00:07:51,680 --> 00:07:53,040
It's the perfect analogy.

194
00:07:53,040 --> 00:07:54,960
An Oracle Invents Truth.

195
00:07:54,960 --> 00:07:57,600
A librarian finds truth that already exists.

196
00:07:57,600 --> 00:08:00,080
We want our corporate AI to be a librarian.

197
00:08:00,080 --> 00:08:03,280
It should only answer based on whitelisted knowledge sources.

198
00:08:03,280 --> 00:08:05,600
If it can't find the document, it should say,

199
00:08:05,600 --> 00:08:07,760
"I don't know, not invent a policy."

200
00:08:07,760 --> 00:08:09,440
And how do we enforce that technically?

201
00:08:09,440 --> 00:08:11,120
You need boundaries.

202
00:08:11,120 --> 00:08:12,720
For AI Builder processing documents,

203
00:08:12,720 --> 00:08:15,360
you need a human in the loop for low-confident scores.

204
00:08:15,360 --> 00:08:19,280
If the AI isn't 90% sure that invoice is for $500,

205
00:08:19,280 --> 00:08:20,480
a human needs to review it.

206
00:08:20,480 --> 00:08:22,560
You don't just let it write to the ERP.

207
00:08:22,560 --> 00:08:24,720
And for the chatbots, the copylates?

208
00:08:24,720 --> 00:08:26,640
Citations are mandatory.

209
00:08:26,640 --> 00:08:29,120
If the bot gives an answer, it must link to the source document.

210
00:08:29,120 --> 00:08:31,040
No citation, no output.

211
00:08:31,040 --> 00:08:33,680
And critically, no direct data mutation.

212
00:08:33,680 --> 00:08:36,960
You don't let a chatbot directly edit your SQL database.

213
00:08:36,960 --> 00:08:40,160
The chatbot should trigger a strictly governed power automate flow

214
00:08:40,160 --> 00:08:41,280
to make the change.

215
00:08:41,280 --> 00:08:42,960
The flow is the gatekeeper.

216
00:08:42,960 --> 00:08:45,200
So the AI is just the interface.

217
00:08:45,200 --> 00:08:47,520
The governance logic still lives in the rigid,

218
00:08:47,520 --> 00:08:49,520
deterministic layers we discussed earlier.

219
00:08:49,520 --> 00:08:50,960
Exactly.

220
00:08:50,960 --> 00:08:53,120
The AI is the soft fuzzy front end.

221
00:08:53,120 --> 00:08:55,840
The back end must remain cold, hard, and logical.

222
00:08:55,840 --> 00:08:57,280
We've covered the pillars.

223
00:08:57,280 --> 00:08:59,760
SharePoint, Apps, Automate, AI.

224
00:08:59,760 --> 00:09:05,040
But none of this works without what the text calls the "governance spine."

225
00:09:05,040 --> 00:09:06,720
The controls that don't blink.

226
00:09:06,720 --> 00:09:09,760
This is the unsexy stuff that saves your job.

227
00:09:09,760 --> 00:09:12,960
DLP, DataLoss Prevention Policies, Environment Strategies.

228
00:09:12,960 --> 00:09:15,040
You cannot have people building production apps

229
00:09:15,040 --> 00:09:16,320
in the default environment.

230
00:09:16,320 --> 00:09:18,560
The default environment is the Wild West.

231
00:09:18,560 --> 00:09:20,000
It's where governance goes to die.

232
00:09:20,000 --> 00:09:21,760
You need a dev test-prote separation.

233
00:09:21,760 --> 00:09:22,880
You need managed solutions.

234
00:09:22,880 --> 00:09:25,440
If you are moving raw zip files of unmanaged solutions

235
00:09:25,440 --> 00:09:27,680
into production, you are asking for trouble.

236
00:09:27,680 --> 00:09:29,280
And service principles.

237
00:09:29,280 --> 00:09:32,240
Stop running enterprise flows under Steve's account.

238
00:09:32,240 --> 00:09:34,720
Because when Steve wins the lottery in quits,

239
00:09:34,720 --> 00:09:37,120
the entire accounts payable process shuts down.

240
00:09:37,120 --> 00:09:39,360
User service principle, a non-human identity.

241
00:09:39,360 --> 00:09:41,040
So the system relies on the architecture

242
00:09:41,040 --> 00:09:42,960
and not the employee roster.

243
00:09:42,960 --> 00:09:45,200
It all comes down to a choice, doesn't it?

244
00:09:45,200 --> 00:09:46,880
The conclusion lays it out.

245
00:09:46,880 --> 00:09:48,640
Alignment or entropy.

246
00:09:48,640 --> 00:09:49,600
That's the binary choice.

247
00:09:49,600 --> 00:09:50,480
There is no middle ground.

248
00:09:50,480 --> 00:09:52,480
You either enforce structure early,

249
00:09:52,480 --> 00:09:55,520
which feels restrictive and slows you down initially.

250
00:09:55,520 --> 00:09:57,360
Or you pay for reconstruction later.

251
00:09:57,360 --> 00:10:00,160
And reconstruction is always ten times more expensive.

252
00:10:00,160 --> 00:10:01,360
It's interesting.

253
00:10:01,360 --> 00:10:03,440
We started by talking about technical settings.

254
00:10:03,440 --> 00:10:05,680
Indexed columns, concurrency limits.

255
00:10:05,680 --> 00:10:07,440
But this is really a philosophical stance

256
00:10:07,440 --> 00:10:09,520
on how technology should be managed.

257
00:10:09,520 --> 00:10:11,680
It is. It's about respecting the tool.

258
00:10:11,680 --> 00:10:14,240
SharePoint, Power Platform, AI.

259
00:10:14,240 --> 00:10:16,080
These are incredibly powerful tools.

260
00:10:16,080 --> 00:10:17,360
But they are not magic ones.

261
00:10:17,360 --> 00:10:18,720
They require a schema.

262
00:10:18,720 --> 00:10:20,320
They require protocols.

263
00:10:20,320 --> 00:10:22,080
If you treat them with discipline, they scale.

264
00:10:22,080 --> 00:10:23,600
If you treat them as a playground,

265
00:10:23,600 --> 00:10:25,920
they will become a graveyard for your data.

266
00:10:25,920 --> 00:10:28,720
Everything else is entropy with a friendly UI.

267
00:10:28,720 --> 00:10:31,040
That's a haunting thought.

268
00:10:31,040 --> 00:10:31,920
It should be.

269
00:10:31,920 --> 00:10:34,800
Because when you log off today, those flows are still running.

270
00:10:34,800 --> 00:10:36,800
The entropy is still pushing against your walls.

271
00:10:36,800 --> 00:10:39,600
The question is, did you build the walls strong enough?

272
00:10:39,600 --> 00:10:42,160
So for our listeners, the call to action is clear.

273
00:10:42,160 --> 00:10:44,080
Don't wait for the outage.

274
00:10:44,080 --> 00:10:45,680
Look for the signals.

275
00:10:45,680 --> 00:10:47,200
Look for the test lists.

276
00:10:47,200 --> 00:10:48,480
The failed flows.

277
00:10:48,480 --> 00:10:50,080
The permission drifts.

278
00:10:50,080 --> 00:10:51,520
Go check your environments.

279
00:10:51,520 --> 00:10:52,960
Implement the DLP.

280
00:10:52,960 --> 00:10:54,320
Separate your dev and prod.

281
00:10:54,320 --> 00:10:55,760
And for the love of data,

282
00:10:55,760 --> 00:10:57,440
go index your SharePoint columns.

283
00:10:57,440 --> 00:11:00,320
Simple, actionable and absolutely critical.

284
00:11:00,320 --> 00:11:01,920
If you ignore it, well,

285
00:11:01,920 --> 00:11:03,120
if you ignore it, I'll be back.

286
00:11:03,120 --> 00:11:05,120
But next time we won't be talking about prevention.

287
00:11:05,120 --> 00:11:07,200
We'll be talking about disaster recovery.

288
00:11:07,200 --> 00:11:10,000
On that cheerful note, thank you for listening.

289
00:11:10,000 --> 00:11:13,680
This has been a deep dive into the mechanics of platform governance.

290
00:11:13,680 --> 00:11:14,720
Stay disciplined.

291
00:11:14,720 --> 00:11:16,960
And we'll catch you in the next episode.

292
00:11:16,960 --> 00:11:17,680
Goodbye.