Beyond Copilot: Building Enterprise AI Agents That Actually Work with Copilot Studio with Manpreet Singh [MVP-MCT]

1
00:00:00,000 --> 00:00:04,560
Welcome everybody back to a new episode of the MC65FM podcast.

2
00:00:04,560 --> 00:00:11,440
For the last few years, Enterprise AI has largely been about one thing, ask AI a question

3
00:00:11,440 --> 00:00:18,120
and get answer, but we are now entering a very different phase.

4
00:00:18,120 --> 00:00:22,800
AI is moving from answering questions to understanding business context, connecting

5
00:00:22,800 --> 00:00:28,920
to enterprise systems or casting workflows and increasingly taking action on behalf of

6
00:00:28,920 --> 00:00:29,920
users.

7
00:00:29,920 --> 00:00:35,920
In other words, we are moving from co-pilot to agents, but building an impressive

8
00:00:35,920 --> 00:00:42,120
agent demo, it's relatively easy, build an agent that works reliable inside a global enterprise

9
00:00:42,120 --> 00:00:48,080
with real users, real business processes, sensitive data, security requirements, governance,

10
00:00:48,080 --> 00:00:54,880
integrations and measurable business outcomes, it's something different, definitely.

11
00:00:54,880 --> 00:01:01,120
My guest today is Manfred Sein, Microsoft MVP and NCT and leader of the Modern Workplace

12
00:01:01,120 --> 00:01:04,000
AI Solutions team at Cochneycent.

13
00:01:04,000 --> 00:01:11,160
Modern works with global enterprise across industries, including financial service, healthcare,

14
00:01:11,160 --> 00:01:17,120
media and entertainment, designing Microsoft-based AI solutions and the agenteic AI strategies.

15
00:01:17,120 --> 00:01:23,580
He also shared his technical knowledge with millions of readers and organized the boot

16
00:01:23,580 --> 00:01:29,640
camps, hackathons, conferences and other community initiatives around AI and Microsoft

17
00:01:29,640 --> 00:01:30,640
technologies.

18
00:01:30,640 --> 00:01:34,440
Today we are going beyond the co-pilot type.

19
00:01:34,440 --> 00:01:40,980
We are going deep into Microsoft co-pilot studio enterprise agent, autonomous AI, multi-agent

20
00:01:40,980 --> 00:01:43,460
architecture, governance and so on.

21
00:01:43,460 --> 00:01:47,580
So yeah, welcome on Fritusha, thank you so much for being here.

22
00:01:47,580 --> 00:01:49,380
Thank you, well, thank you for the invite.

23
00:01:49,380 --> 00:01:52,980
I'm so excited to be here and being part of your amazing podcast.

24
00:01:52,980 --> 00:01:55,540
Thank you, thank you.

25
00:01:55,540 --> 00:02:03,060
Before we deep dive into the agent world, so tell us a little bit about your journey into

26
00:02:03,060 --> 00:02:04,300
the Microsoft technology.

27
00:02:04,300 --> 00:02:06,340
Oh, yeah, yeah, of course.

28
00:02:06,340 --> 00:02:11,940
So I think I spent my whole career life with Microsoft tech and Blackforks, started my career

29
00:02:11,940 --> 00:02:20,700
with .NET and then went to SharePoint, SharePoint, more, 32 on 7, then 2030, 2060, SharePoint,

30
00:02:20,700 --> 00:02:26,180
online power platform, power automated, and now I saw about AI, right?

31
00:02:26,180 --> 00:02:32,180
That's where I came in and now I have been supporting multiple world-years, multiple clients

32
00:02:32,180 --> 00:02:38,220
where there are insurance, healthcare, life size, bank, anywhere, helping them and sustaining

33
00:02:38,220 --> 00:02:41,700
and building these AI solutions for them.

34
00:02:41,700 --> 00:02:43,940
Yeah, cool.

35
00:02:43,940 --> 00:02:55,020
So yeah, direct deep dive in the world, traditional co-pilot, interacts of like summarise this, write

36
00:02:55,020 --> 00:03:02,100
this, find this information, what change when we move towards solve this problem for me?

37
00:03:02,100 --> 00:03:03,100
Right.

38
00:03:03,100 --> 00:03:10,700
So I think the earlier before the world AI hit, we were all working on SaaS products, whether

39
00:03:10,700 --> 00:03:16,660
it's Microsoft Suite 365, power platform, and we were very happy with the world, right?

40
00:03:16,660 --> 00:03:22,100
And then all of a sudden, charge-gbd announced, I think, I was one of those fun boys, this is

41
00:03:22,100 --> 00:03:26,500
what it can do, it's so very, it was really something way beyond, right?

42
00:03:26,500 --> 00:03:31,700
We all remember charge-gbd through that process, adding a little step time or something.

43
00:03:31,700 --> 00:03:38,980
And then slowly we started moving towards a braiding agent, a pirate agent, personal agent,

44
00:03:38,980 --> 00:03:44,140
organization agents, companies started coming in, they like, one could any 10 agents, one for

45
00:03:44,140 --> 00:03:47,700
each department, and then started building the catalog of agents.

46
00:03:47,700 --> 00:03:52,540
And then slowly it became multi agent, where companies are like, I just need one agent and

47
00:03:52,540 --> 00:03:54,540
a child agent, right?

48
00:03:54,540 --> 00:04:01,500
So and it kept evolving, even now there are MCPs, the prototypes, configurations, which

49
00:04:01,500 --> 00:04:07,540
people are going with multi agent, multi LLM models coming in picture, they started evolving.

50
00:04:07,540 --> 00:04:14,180
That's where now our job, we come in and like me and you, we tell them, we talk about AI

51
00:04:14,180 --> 00:04:19,460
and how AI is coming in, helping in, go, we start here, we're talking about how it can

52
00:04:19,460 --> 00:04:25,140
make your life, the end their life easier, your employee experience as an end user is much

53
00:04:25,140 --> 00:04:30,940
better, it's not just depending on SaaS product, where you're opening portals and doing things.

54
00:04:30,940 --> 00:04:36,380
Now agent is doing some of the things for you and building that whole end-to-end solution,

55
00:04:36,380 --> 00:04:39,220
that's what AI is capable of.

56
00:04:39,220 --> 00:04:44,500
Every day something or the other, whether it's loud or jam, or even Microsoft, they have

57
00:04:44,500 --> 00:04:49,740
so many announcements, something or the other is coming in, organizations are immediately

58
00:04:49,740 --> 00:04:52,420
adopting to it and scaling with it.

59
00:04:52,420 --> 00:04:56,380
That's the future of AI and end-up solutions today.

60
00:04:56,380 --> 00:04:57,380
Yeah.

61
00:04:57,380 --> 00:05:02,140
Let's design an AI agent.

62
00:05:02,140 --> 00:05:07,900
How are the major architectural components of production and enterprise agent?

63
00:05:07,900 --> 00:05:08,900
Right.

64
00:05:08,900 --> 00:05:14,860
Let's take a client, for example, the client who was in Microsoft XTAR, my M365.

65
00:05:14,860 --> 00:05:21,140
He or she will reach out here like, "Makpith, I want to start a journey in agent, right?

66
00:05:21,140 --> 00:05:28,380
And I am an insurance company who deals with mortgage insurance auto came home, okay?

67
00:05:28,380 --> 00:05:30,180
So how do we start from there, right?

68
00:05:30,180 --> 00:05:33,100
So first thing would be, hey, what's your preferred XTAR?

69
00:05:33,100 --> 00:05:36,700
And of course, we are talking about Microsoft, so they are like, okay, Microsoft would expect

70
00:05:36,700 --> 00:05:43,500
preferred XTAR with the use cases, whether it's through between Microsoft 365, co-pilot,

71
00:05:43,500 --> 00:05:47,180
which is the agent builder, no code, no code.

72
00:05:47,180 --> 00:05:52,140
And co-pilot studio, which has become much more advanced with multi-agent solutions and

73
00:05:52,140 --> 00:05:54,860
multi-agent multi-aller providers.

74
00:05:54,860 --> 00:05:59,820
And then you have Azure AI Foundry where you're building these models much more designing,

75
00:05:59,820 --> 00:06:03,020
much more in-depth, compared to other those two parts.

76
00:06:03,020 --> 00:06:06,060
So you divide the use case here to three.

77
00:06:06,060 --> 00:06:09,660
And the best part of Microsoft ecosystem is they talk to each other.

78
00:06:09,660 --> 00:06:15,020
So it's not like Microsoft co-pilot is a standard agent and then studio agents are standard

79
00:06:15,020 --> 00:06:16,980
on the all talk to each other.

80
00:06:16,980 --> 00:06:21,580
They're all connected through teams and multiple, you know, of API connector.

81
00:06:21,580 --> 00:06:24,500
So an example, as you talk about, right?

82
00:06:24,500 --> 00:06:28,100
They somebody company will come, hey, I want to build an agent for my company.

83
00:06:28,100 --> 00:06:30,260
You can call it as Jarvis.

84
00:06:30,260 --> 00:06:34,060
My employee should be able to come in and do sub transactions.

85
00:06:34,060 --> 00:06:40,460
Now, imagine the day when you have to apply for a leave before AI, you used to go to a

86
00:06:40,460 --> 00:06:45,500
voltage system or a people's office system, check your leave balance and then go and apply

87
00:06:45,500 --> 00:06:46,500
for a leave.

88
00:06:46,500 --> 00:06:50,860
It used to trigger an email to your manager where he or she needs to approve it.

89
00:06:50,860 --> 00:06:54,420
And then they will get a confirmation as your leave has been approved, right?

90
00:06:54,420 --> 00:06:55,860
Now that has changed.

91
00:06:55,860 --> 00:06:57,860
Now if I have to go for a leave on Monday, I just want to say, I'm going to do a leave

92
00:06:57,860 --> 00:06:58,860
on Monday.

93
00:06:58,860 --> 00:07:02,980
I just have to ask the agent, hey, agent, can you apply for a personal day off on Monday?

94
00:07:02,980 --> 00:07:04,820
It understands what Monday is.

95
00:07:04,820 --> 00:07:11,700
It understands what my personal day a leave is based on the categories of leaves I have.

96
00:07:11,700 --> 00:07:12,700
Check my balance.

97
00:07:12,700 --> 00:07:17,580
If I have balance immediately since a team's pop up to my manager, hey, if you any approve

98
00:07:17,580 --> 00:07:20,860
or decline or share your comment, he or she approves it.

99
00:07:20,860 --> 00:07:23,540
And I get a notification, hey, your leave is approved.

100
00:07:23,540 --> 00:07:27,820
And everything was updated in the people's soft system in the voltage system.

101
00:07:27,820 --> 00:07:29,820
So I didn't open any of those systems.

102
00:07:29,820 --> 00:07:34,140
I just went to my co-pilot, chat, I had to phrase and like, hey, apply for a leave, right?

103
00:07:34,140 --> 00:07:36,060
That's an example I gave.

104
00:07:36,060 --> 00:07:37,060
Next one I'll tell you.

105
00:07:37,060 --> 00:07:41,180
There are so many insurance calls that just contact centers, right?

106
00:07:41,180 --> 00:07:43,300
They get so calls.

107
00:07:43,300 --> 00:07:48,860
So if I call my insurance, hey, I want to see what my current interest rate is, right?

108
00:07:48,860 --> 00:07:52,900
So as soon as I call, my agent is also part of the conversation.

109
00:07:52,900 --> 00:07:57,620
And while I'm talking and I share my member ID, my member ID is 1001.

110
00:07:57,620 --> 00:08:02,220
And goes get all information of their member ID and populates in the screen of the contact

111
00:08:02,220 --> 00:08:08,140
center where a contact center usually will have to go search multiple systems in their

112
00:08:08,140 --> 00:08:12,380
G-R, the box, and check my profile and test and my profile.

113
00:08:12,380 --> 00:08:14,580
Now AI just gets all information.

114
00:08:14,580 --> 00:08:19,900
Some rises it and help you and suggest, hey, this is what the new person there is.

115
00:08:19,900 --> 00:08:21,740
And you can also give him a couple of more offers.

116
00:08:21,740 --> 00:08:24,100
So it became my life faster.

117
00:08:24,100 --> 00:08:29,260
The 10 minutes call now becomes a one minute to a minute call, you're saving eight minutes,

118
00:08:29,260 --> 00:08:30,260
right?

119
00:08:30,260 --> 00:08:31,260
That's the ROI.

120
00:08:31,260 --> 00:08:35,820
So these are some of the agents which you can build and bring it to your ecosystem from as

121
00:08:35,820 --> 00:08:41,500
simple as an HR policy agent or a leaf tracker system or service now, connect to the agent

122
00:08:41,500 --> 00:08:43,220
or the health of the fight.

123
00:08:43,220 --> 00:08:47,900
These small, small fixations, you start from co-pilot, co-pilot studio or Azure 8.

124
00:08:47,900 --> 00:08:56,340
Or bring your own model, right?

125
00:08:56,340 --> 00:09:01,740
You can extend a scale these solutions as such.

126
00:09:01,740 --> 00:09:02,740
Awesome.

127
00:09:02,740 --> 00:09:04,140
I think we have four or five things when we talk about the agent.

128
00:09:04,140 --> 00:09:09,940
We have the instructions, we have the knowledge topic, we have the action and tool topic,

129
00:09:09,940 --> 00:09:12,340
the reasoning topic.

130
00:09:12,340 --> 00:09:17,180
How do we do it all fit together?

131
00:09:17,180 --> 00:09:18,180
Sure, sure.

132
00:09:18,180 --> 00:09:21,740
So let's take the same example, the holiday tracker, right?

133
00:09:21,740 --> 00:09:23,980
I'm applying to leave.

134
00:09:23,980 --> 00:09:28,820
So when I'm applying, he is an API call for validating and checking my balance.

135
00:09:28,820 --> 00:09:35,660
So that's the simple API tool connector which I'm adding as a connector to work day,

136
00:09:35,660 --> 00:09:37,060
which is readily available.

137
00:09:37,060 --> 00:09:39,460
There's a plus part of Microsoft suite.

138
00:09:39,460 --> 00:09:43,980
And I just can't do my API service account and I'm all set right.

139
00:09:43,980 --> 00:09:45,980
Then second thing is the knowledge, right?

140
00:09:45,980 --> 00:09:49,380
Now again, my best part is ecosystem share point.

141
00:09:49,380 --> 00:09:53,740
We have all the HR lease policies, documents loaded in a share point size.

142
00:09:53,740 --> 00:10:00,620
So if I need to understand, hey, can I apply more leaves than I have for I'm traveling to

143
00:10:00,620 --> 00:10:03,180
Italy or some other, right?

144
00:10:03,180 --> 00:10:07,900
So my HR system, the documents which are there in share point, which has no metadata,

145
00:10:07,900 --> 00:10:11,860
but AI reached them and pro indexes started for questions.

146
00:10:11,860 --> 00:10:16,820
And if you're an outward, yes, you can apply and take some leave from the next year portal,

147
00:10:16,820 --> 00:10:17,820
right?

148
00:10:17,820 --> 00:10:19,540
So that's where the knowledge comes in.

149
00:10:19,540 --> 00:10:24,340
So you have the tools where you're applying reading access or editing the access to work

150
00:10:24,340 --> 00:10:27,180
hand system, knowledge is where you are doing this.

151
00:10:27,180 --> 00:10:29,500
And then the third part is the trigger, right?

152
00:10:29,500 --> 00:10:30,660
What should be the trigger for me?

153
00:10:30,660 --> 00:10:35,500
The trigger was I went to teams and I asked to apply for a leave for it.

154
00:10:35,500 --> 00:10:39,540
So for other people, it could be an email, where I sent an email to a mailbox and the

155
00:10:39,540 --> 00:10:45,420
mail boss understands the query, could kick off my agent and then apply for a leave, right?

156
00:10:45,420 --> 00:10:47,140
So there's all the normal days.

157
00:10:47,140 --> 00:10:53,380
And so is the these three pillars, the trigger, the tool set and the knowledge shows it could

158
00:10:53,380 --> 00:10:57,820
be share point, it could be data set, it could be files, if you have directly uploaded,

159
00:10:57,820 --> 00:11:03,700
all becomes an investor and your agent just need these three things to kick off and provide

160
00:11:03,700 --> 00:11:07,820
their solution to you.

161
00:11:07,820 --> 00:11:14,460
And I think especially when we talk about knowledge, most enterprise agents are accessing

162
00:11:14,460 --> 00:11:22,460
to, yeah, or organization, how do you approach the grounding?

163
00:11:22,460 --> 00:11:29,540
Of course, I tell the customer, please go and bring your 20 years of share point as an

164
00:11:29,540 --> 00:11:30,540
index.

165
00:11:30,540 --> 00:11:31,540
Please don't do that.

166
00:11:31,540 --> 00:11:35,020
And I keep, I want to get that in the park, that's it as well.

167
00:11:35,020 --> 00:11:38,860
What happens is like company, they like, hey, they'll, they'll build a co-parts, studio

168
00:11:38,860 --> 00:11:49,100
agent and they'll add HR or IT sites as knowledge source, which has data from 1995 to 2026, right?

169
00:11:49,100 --> 00:11:51,780
But the agent doesn't understand that, right?

170
00:11:51,780 --> 00:11:53,740
It's very, very important.

171
00:11:53,740 --> 00:11:59,620
As part of the governance, I tell, please clean up your data, make sure the data, which is

172
00:11:59,620 --> 00:12:05,740
really it set or the latest data is only index to the data, so to talk agents because

173
00:12:05,740 --> 00:12:11,300
always when they see those answers coming in, like, hey, when is the upcoming leave?

174
00:12:11,300 --> 00:12:17,380
It can provide you a data from September 22 or 26 because it's not a prondent.

175
00:12:17,380 --> 00:12:21,900
So you have to give a very clear restriction, provide me an upcoming leave in September,

176
00:12:21,900 --> 00:12:26,460
26 and to avoid that and avoid user frustration.

177
00:12:26,460 --> 00:12:29,580
Please make sure the data is cleaned up.

178
00:12:29,580 --> 00:12:33,300
You only have the data which you want to index to the agent.

179
00:12:33,300 --> 00:12:39,260
You will see how agent understands very quickly as part of the indexing data.

180
00:12:39,260 --> 00:12:40,260
Right?

181
00:12:40,260 --> 00:12:46,900
But please, please, please don't bring your heavy thoughts of 2020,000 files of taking a co-parts

182
00:12:46,900 --> 00:12:47,900
to the agent.

183
00:12:47,900 --> 00:12:48,900
It's born.

184
00:12:48,900 --> 00:12:49,900
Okay.

185
00:12:49,900 --> 00:12:55,740
So that's the, the part when the peer view comes into the game.

186
00:12:55,740 --> 00:13:03,180
Yeah, because it's very, just to be honest, the metadata and the rat quality is very important

187
00:13:03,180 --> 00:13:04,980
for you and fight.

188
00:13:04,980 --> 00:13:10,180
So if you have done been 500 unstructured PDFs, I do a knowledge shows with no tags, no

189
00:13:10,180 --> 00:13:14,700
description, nothing, no watch and control, you're just building a random answer generator.

190
00:13:14,700 --> 00:13:17,340
It's, it's not gonna help you, right?

191
00:13:17,340 --> 00:13:23,380
Until you tag them, make sure you retire or archive the old files and give them a new fresh

192
00:13:23,380 --> 00:13:24,380
copy.

193
00:13:24,380 --> 00:13:28,420
You should tell people why you're on a creative new library where you just put in your test

194
00:13:28,420 --> 00:13:34,220
hard work 200 files and then shared with an agent, you will see the real experience, how

195
00:13:34,220 --> 00:13:39,780
the differences between those 10,000 files, which is from 20 years, compared to your data

196
00:13:39,780 --> 00:13:42,660
of rack metadata tag index files.

197
00:13:42,660 --> 00:13:43,660
Awesome.

198
00:13:43,660 --> 00:13:44,660
Yeah.

199
00:13:44,660 --> 00:13:52,660
I think that when, when we build the agents and so on, but really, yeah, I think that,

200
00:13:52,660 --> 00:14:01,540
the action begins when, yeah, when the, when AI handled something for you, right?

201
00:14:01,540 --> 00:14:08,460
You say, knowledge is, is, is useful, but agents become much more interesting when they

202
00:14:08,460 --> 00:14:10,460
can do something.

203
00:14:10,460 --> 00:14:18,060
Like, what types of actions have you implementing, also, of the standard thing?

204
00:14:18,060 --> 00:14:26,180
Or a lot, like, so as part of enterprise, we have connected to Salesforce, SAP, World Day,

205
00:14:26,180 --> 00:14:32,580
people, software application, service, now, Gira, Confluence, a n number of providers,

206
00:14:32,580 --> 00:14:33,580
right?

207
00:14:33,580 --> 00:14:37,340
So right now, when you're building this enterprise agent, you don't want it to be only

208
00:14:37,340 --> 00:14:40,580
stuck with Microsoft knowledge, say, is right?

209
00:14:40,580 --> 00:14:44,260
You want them to give an enterprise agent, which works with all their systems, whether

210
00:14:44,260 --> 00:14:49,380
they're a custom system or their tools of other company, then indication endpoint is very

211
00:14:49,380 --> 00:14:50,380
important.

212
00:14:50,380 --> 00:14:54,260
Copied studio and foundry, there's a best part, they already have those integrations built

213
00:14:54,260 --> 00:14:59,260
from so many years, which is because of power, platform gateways and power, platform characters.

214
00:14:59,260 --> 00:15:02,780
You just into them, bring them as part of the ecosystem.

215
00:15:02,780 --> 00:15:08,140
And then when I as an end user, I'm conversing or doing an action, like applying a lead on

216
00:15:08,140 --> 00:15:13,740
World Day, everything is being done on my teams or my agent interface, which is deployed

217
00:15:13,740 --> 00:15:16,500
in a chain, trying to add a web app or whatever.

218
00:15:16,500 --> 00:15:22,220
And it's calling multi agents, multi systems, multi tools and getting me done in form itself,

219
00:15:22,220 --> 00:15:23,220
right?

220
00:15:23,220 --> 00:15:26,900
Recently, we were working with one of the client, they'll just give you an example, right?

221
00:15:26,900 --> 00:15:28,820
They have Salesforce, right?

222
00:15:28,820 --> 00:15:34,580
So what they do is, so there are sales people who are always on the reward, all right?

223
00:15:34,580 --> 00:15:40,300
And I like me and you met, like, hey, Merco is a nice person, and he works for this company.

224
00:15:40,300 --> 00:15:43,540
Let me require his contact details, right?

225
00:15:43,540 --> 00:15:48,300
Usually what will happen is you open Salesforce on your phone or an agent on, right?

226
00:15:48,300 --> 00:15:49,980
And you will make an entry.

227
00:15:49,980 --> 00:15:55,220
And now with the AI, the fast press AI, I just call, hey, I just met Merco.

228
00:15:55,220 --> 00:15:56,460
This is his email ID.

229
00:15:56,460 --> 00:16:00,420
He works as a CTO for this company, record, right?

230
00:16:00,420 --> 00:16:04,660
And that just gets added his knowledge based on my sales for the environment for my whole

231
00:16:04,660 --> 00:16:07,460
sales, seeing the normal work.

232
00:16:07,460 --> 00:16:12,140
It was pretty, I just had to make a voice note instead of going and opening a SaaS program,

233
00:16:12,140 --> 00:16:14,060
adding your information to that, right?

234
00:16:14,060 --> 00:16:19,820
So those systems, those integrated systems is what help you scale the solutions and bring

235
00:16:19,820 --> 00:16:22,220
that as part of your enterprise.

236
00:16:22,220 --> 00:16:32,340
Yeah, I think how build we, build we stuff that's stopping, say, here, say, Merco, here's

237
00:16:32,340 --> 00:16:38,860
what you add, but you add Tril2, I have every, I have done it for you.

238
00:16:38,860 --> 00:16:41,220
From an enterprise, I take trip, perspective.

239
00:16:41,220 --> 00:16:45,300
How can we do this?

240
00:16:45,300 --> 00:16:47,060
Yeah, to just go.

241
00:16:47,060 --> 00:16:49,820
Yeah, so first step is open up, right?

242
00:16:49,820 --> 00:16:54,380
And when I speak at conferences, I always give them homework, right?

243
00:16:54,380 --> 00:16:56,820
Sometimes we speak and sometimes it's too much for them.

244
00:16:56,820 --> 00:16:59,820
So I create really simple steps as homework.

245
00:16:59,820 --> 00:17:03,500
So homework number one, open co-piles studio, right?

246
00:17:03,500 --> 00:17:07,460
Very simple, open co-piles studio.microsoft.com.

247
00:17:07,460 --> 00:17:08,460
Just go ahead.

248
00:17:08,460 --> 00:17:12,300
And if you don't have a license, you will get a trial version for 30 days and then you

249
00:17:12,300 --> 00:17:14,420
can keep extending till 90 days.

250
00:17:14,420 --> 00:17:15,420
Open that up.

251
00:17:15,420 --> 00:17:19,020
First thing what you should do is create a blank agent, okay?

252
00:17:19,020 --> 00:17:20,340
It's really a blank agent.

253
00:17:20,340 --> 00:17:21,340
Call it Jarvis.

254
00:17:21,340 --> 00:17:22,340
I'm an Iron Man fan.

255
00:17:22,340 --> 00:17:23,860
All my agents are Jarvis, okay?

256
00:17:23,860 --> 00:17:26,180
You put agent name as Jarvis.

257
00:17:26,180 --> 00:17:28,180
Add a knowledge source.

258
00:17:28,180 --> 00:17:32,020
Now the knowledge source could be your share point, but sometimes the company don't allow

259
00:17:32,020 --> 00:17:34,300
you to do it if you don't have license.

260
00:17:34,300 --> 00:17:35,580
Don't worry about it.

261
00:17:35,580 --> 00:17:42,180
Add a web site from learn.microsoft.com or add your own company domain, whatever company

262
00:17:42,180 --> 00:17:45,540
you're working for, Microsoft.com, com is in.com.

263
00:17:45,540 --> 00:17:47,500
Add it as a knowledge source, okay?

264
00:17:47,500 --> 00:17:50,140
So first thing is your knowledge source is ready.

265
00:17:50,140 --> 00:17:55,460
Second thing is you want to set up a tool, like whenever I get an information from AI, send

266
00:17:55,460 --> 00:17:57,460
that to me in an email, right?

267
00:17:57,460 --> 00:17:58,460
Very easy.

268
00:17:58,460 --> 00:17:59,460
So that is your trigger.

269
00:17:59,460 --> 00:18:05,300
So whenever AI, agent, piss up your information, it's a trigger and it can record and send

270
00:18:05,300 --> 00:18:06,300
it to your email.

271
00:18:06,300 --> 00:18:07,300
That's the tool.

272
00:18:07,300 --> 00:18:08,940
That's your action you can build.

273
00:18:08,940 --> 00:18:13,980
If you have some time, you can also create an excel sheet like, hey, create this record and

274
00:18:13,980 --> 00:18:16,940
add it to my excel as a row item.

275
00:18:16,940 --> 00:18:18,260
Everything is there in Power Automate.

276
00:18:18,260 --> 00:18:19,260
Add a new row item.

277
00:18:19,260 --> 00:18:20,620
You choose that action.

278
00:18:20,620 --> 00:18:25,340
You added the now when you're chatting with an agent, you are getting your grounded data

279
00:18:25,340 --> 00:18:27,580
form learned out of my source.com.

280
00:18:27,580 --> 00:18:30,340
It can build it as an academy for your company.

281
00:18:30,340 --> 00:18:35,300
Or you can do these processes where you are recording data on an excel file or sending

282
00:18:35,300 --> 00:18:36,300
it as an email.

283
00:18:36,300 --> 00:18:41,860
So those action, those trigger and then knowledge source, anyone can start from day one.

284
00:18:41,860 --> 00:18:45,460
That's what Hoback's story is very easy to start with for 30 days, 60 days.

285
00:18:45,460 --> 00:18:47,860
And that's what we do at our workshop, right?

286
00:18:47,860 --> 00:18:48,860
We give you that slag.

287
00:18:48,860 --> 00:18:53,260
We give you that solution and you do step by step.

288
00:18:53,260 --> 00:18:57,900
The first agent is difficult and then the second, third, you'll be becoming a pro because

289
00:18:57,900 --> 00:19:03,820
these agent builder has become so simpler, so easy that you can connect these tools and

290
00:19:03,820 --> 00:19:08,220
number of tools and number of your hand device systems and build it as one.

291
00:19:08,220 --> 00:19:09,220
Awesome.

292
00:19:09,220 --> 00:19:10,220
Yeah.

293
00:19:10,220 --> 00:19:22,060
So what's with, we can do a lot of automatization, especially when we talk about agents, multi-agent

294
00:19:22,060 --> 00:19:34,220
architecture and now we say or a lot of people speak about autonomous multi-agent agents.

295
00:19:34,220 --> 00:19:39,100
So where did you see the human in the loop?

296
00:19:39,100 --> 00:19:40,100
Right.

297
00:19:40,100 --> 00:19:45,100
So human loop will always come when there's a cost based cycle.

298
00:19:45,100 --> 00:19:49,860
For example, if you're working for a financial crime and there's a claim processing, if it's

299
00:19:49,860 --> 00:19:56,540
a claim with AI justifies as a $50 or $30, I think most of the companies are allowing AI

300
00:19:56,540 --> 00:19:58,140
to take that decision.

301
00:19:58,140 --> 00:20:00,820
But if there's a claim, which is $10,000, right?

302
00:20:00,820 --> 00:20:05,780
So they would want a human in the loop to uproot or decline this, right?

303
00:20:05,780 --> 00:20:11,700
Even in, even in like a usual proposal, right, where there are invoices coming in, but there

304
00:20:11,700 --> 00:20:16,700
used to be a team of 10 people who used to monitor each invoice and then scan it and upload

305
00:20:16,700 --> 00:20:20,780
it in an IBM data cap or any other server.

306
00:20:20,780 --> 00:20:26,420
And then there used to be a process automation to extract data and digitize it.

307
00:20:26,420 --> 00:20:28,420
Right now, AI is doing that.

308
00:20:28,420 --> 00:20:34,180
And there's just one person sitting behind the screen who just validates the invoice because

309
00:20:34,180 --> 00:20:38,980
AI will, it's self-taile, if there is 100% accurate or 60%.

310
00:20:38,980 --> 00:20:44,380
You as a human in the loop, you activate that and you do that transaction, whether it's

311
00:20:44,380 --> 00:20:49,860
creating tickets, whether it's approving server requests or it's access requests, if you're

312
00:20:49,860 --> 00:20:52,700
getting all those you require human in the loop.

313
00:20:52,700 --> 00:20:55,780
And even keeping human in the loop has become so simpler.

314
00:20:55,780 --> 00:21:00,380
You are either prompting them on an email, you're either prompting them on a telephone,

315
00:21:00,380 --> 00:21:04,540
hey, you're giving them a call while the agent is processing an information or you're sending

316
00:21:04,540 --> 00:21:08,100
them a text message, hey, this is where I am at, like cloud code, right?

317
00:21:08,100 --> 00:21:12,220
For example, I have an habit, I keep dreaming.

318
00:21:12,220 --> 00:21:16,340
So I dream something, I put it on the cloud code and it starts working for me.

319
00:21:16,340 --> 00:21:20,540
As human in the loop, it keeps asking me, is it, okay, am I on the right direction?

320
00:21:20,540 --> 00:21:21,540
I'll, yes, you are.

321
00:21:21,540 --> 00:21:23,540
I'll, no, no, this is wrong.

322
00:21:23,540 --> 00:21:24,540
It fixed it, right?

323
00:21:24,540 --> 00:21:29,220
So that human in the loop will always be the quiet in AI because we are not replacing

324
00:21:29,220 --> 00:21:30,220
human.

325
00:21:30,220 --> 00:21:33,140
We are just making them more empowered at their work cycle.

326
00:21:33,140 --> 00:21:38,460
The time they used to do certain activity, if you should take an hour or 30 minutes,

327
00:21:38,460 --> 00:21:40,540
now it used, now it takes a minute or so.

328
00:21:40,540 --> 00:21:45,700
So me as a human, I can do other stuff much more efficiently at faster than reading on

329
00:21:45,700 --> 00:21:47,060
my lazy work.

330
00:21:47,060 --> 00:21:52,220
So with the AI's, we are not replacing people, we are just empowering them to do their job

331
00:21:52,220 --> 00:21:53,220
much better.

332
00:21:53,220 --> 00:21:54,220
Awesome.

333
00:21:54,220 --> 00:21:55,220
Yeah.

334
00:21:55,220 --> 00:21:56,220
Yeah.

335
00:21:56,220 --> 00:21:58,220
Let's, let's talk.

336
00:21:58,220 --> 00:22:07,340
We are the human in the loop, I think it's one security aspect from, yeah.

337
00:22:07,340 --> 00:22:10,100
But yeah, we have other things.

338
00:22:10,100 --> 00:22:19,780
So what changes when an AI system can access multiple enterprise applications?

339
00:22:19,780 --> 00:22:20,780
Yeah.

340
00:22:20,780 --> 00:22:27,980
So see, because governance for AI can't be designed in a vacuum, you need to make sure

341
00:22:27,980 --> 00:22:32,340
all of your systems are far out the same governance sector, right?

342
00:22:32,340 --> 00:22:38,140
You learn by setting up those guardrails across the clients I work with, right?

343
00:22:38,140 --> 00:22:43,780
Whether you're working for a solutions from Microsoft tech stack to Amazon tech stack to service

344
00:22:43,780 --> 00:22:50,260
now, okay, they all come under the same purview, Microsoft security purview or DLP policies

345
00:22:50,260 --> 00:22:54,300
are applied or the sensitive labor is applied across the system.

346
00:22:54,300 --> 00:22:58,860
Some clients are here, I don't want any PHI information to be loaded to do it, right?

347
00:22:58,860 --> 00:23:02,580
So you know them, all those as part of the CUE, you when you set up.

348
00:23:02,580 --> 00:23:04,380
So that's why it's a client.

349
00:23:04,380 --> 00:23:08,100
So you start buying agents from the market or start building agents.

350
00:23:08,100 --> 00:23:10,700
First is set up your governance, setup CUE.

351
00:23:10,700 --> 00:23:14,620
So the set of excellence for you to set up that governance is very important.

352
00:23:14,620 --> 00:23:20,140
Now, when you set up this governance, also make sure you don't make it so tough for each

353
00:23:20,140 --> 00:23:25,060
agent, like I'll give an example, somebody wanted to build an agent which was extracting data

354
00:23:25,060 --> 00:23:26,860
from one drive and teeth, right?

355
00:23:26,860 --> 00:23:33,300
So those two platform, but the DLP which was applied to them was from enterprise level,

356
00:23:33,300 --> 00:23:37,220
working service now, there's and that like too many.

357
00:23:37,220 --> 00:23:41,380
So the response was coming was like a minute, I was waiting for the response to come is coming

358
00:23:41,380 --> 00:23:48,420
in a minute like why you have to design your environment, your DLP policy as such based

359
00:23:48,420 --> 00:23:52,500
on what you're building, what AI it affidels your building is required.

360
00:23:52,500 --> 00:23:57,940
If I'm building an agent who just require one drive and team, lock down your DLP policy

361
00:23:57,940 --> 00:24:02,980
on that environment of that agent to just get me information from that.

362
00:24:02,980 --> 00:24:08,420
Is there a blocking everything like, hey, I want to be a secured engine and lock, it will

363
00:24:08,420 --> 00:24:09,580
not have you.

364
00:24:09,580 --> 00:24:15,220
Make sure your agents are deployed in multiple environments, make sure it all has specific

365
00:24:15,220 --> 00:24:20,180
DLP or per view or defenders connected to the AM and system.

366
00:24:20,180 --> 00:24:24,540
And then most of your system, your enterprise is some which only existing whether it service

367
00:24:24,540 --> 00:24:29,220
now, constantly, they're already in part of your enterprise governance.

368
00:24:29,220 --> 00:24:34,980
The API is only the key which you're connecting through or MCP, my favorite.

369
00:24:34,980 --> 00:24:37,140
I always love MCP to do it.

370
00:24:37,140 --> 00:24:39,660
It's already covered in skate.

371
00:24:39,660 --> 00:24:44,580
Just make sure your governance is set up before you bring these agents for your enterprise

372
00:24:44,580 --> 00:24:45,580
level.

373
00:24:45,580 --> 00:24:50,740
You don't want them to come and upload her passport copy or SS and copy.

374
00:24:50,740 --> 00:24:52,460
You want to make sure you block it.

375
00:24:52,460 --> 00:24:57,860
And there are so many tools which we have built where even when somebody uploads, it started

376
00:24:57,860 --> 00:25:03,020
to, you get a blocker, you're uploading a page at for person, hold up.

377
00:25:03,020 --> 00:25:08,860
So you can bring those practices as part of your day to day life cycle in your enterprise

378
00:25:08,860 --> 00:25:12,220
and you will see how agent building is much more secure.

379
00:25:12,220 --> 00:25:17,060
I come from a company, we have 350 K associates.

380
00:25:17,060 --> 00:25:23,220
It's a lot of people globally, lots of people working in people in Germany, Australia, China,

381
00:25:23,220 --> 00:25:24,220
UK.

382
00:25:24,220 --> 00:25:29,380
Everyone has their own environment, everyone has their own DLP for your policy set.

383
00:25:29,380 --> 00:25:33,300
So it's the governance is much more easier when you do all of that.

384
00:25:33,300 --> 00:25:38,780
So please set up a COE first and then start rolling out your agents.

385
00:25:38,780 --> 00:25:44,340
Yeah, I think when we talk about, we have multiple tools.

386
00:25:44,340 --> 00:25:54,100
We have the Azure policy, we have the defender, we have peer view and imagine much

387
00:25:54,100 --> 00:25:55,100
more.

388
00:25:55,100 --> 00:25:57,580
Yeah, and body and so on.

389
00:25:57,580 --> 00:26:03,780
How did all these tools fit into an enterprise ready, a architecture?

390
00:26:03,780 --> 00:26:09,460
Yeah, so Microsoft now is making sure it's locking down the entire ID with defender and

391
00:26:09,460 --> 00:26:10,460
purview.

392
00:26:10,460 --> 00:26:12,540
All the policies are in sync.

393
00:26:12,540 --> 00:26:16,700
So you don't, you're not creating multiple policy in purview, which is not reflecting

394
00:26:16,700 --> 00:26:22,740
on your defender or start reflecting to your entry or security groups are in sync or your

395
00:26:22,740 --> 00:26:25,260
active, your data, DLP, so in sync.

396
00:26:25,260 --> 00:26:30,420
So it the enterprise modules of these systems are already working and I said, it's a set

397
00:26:30,420 --> 00:26:32,820
in the being because Microsoft is a system.

398
00:26:32,820 --> 00:26:34,540
It makes your life easier.

399
00:26:34,540 --> 00:26:39,380
The same security groups, the same user groups that applied across your tenant, whereas

400
00:26:39,380 --> 00:26:45,940
in Friday, outlaw, the SharePoint and bring these policies across organization across

401
00:26:45,940 --> 00:26:46,940
regions.

402
00:26:46,940 --> 00:26:51,900
Like I know you have the different sex of policy compared to what US and China had.

403
00:26:51,900 --> 00:26:58,900
So it's very easy for seeking between all the security, operand and building the sustainable

404
00:26:58,900 --> 00:27:00,900
systems.

405
00:27:00,900 --> 00:27:02,700
Okay.

406
00:27:02,700 --> 00:27:11,820
I think I heard from companies like yours, the company, some companies have thousands of

407
00:27:11,820 --> 00:27:14,140
agents.

408
00:27:14,140 --> 00:27:20,100
What's will I say, I don't know if it's, it's worth it, which so exists, but what is with

409
00:27:20,100 --> 00:27:22,100
agent life cycle management?

410
00:27:22,100 --> 00:27:23,100
Yeah.

411
00:27:23,100 --> 00:27:24,900
Oh, that's the next, that's what we are doing.

412
00:27:24,900 --> 00:27:29,780
So taking off a dock, the hospital, right, and hospital has a little doctor.

413
00:27:29,780 --> 00:27:35,900
Someone is specialist in cardiology, somebody's specialist in here, knows ENT, somebody's for

414
00:27:35,900 --> 00:27:37,100
boards, right?

415
00:27:37,100 --> 00:27:41,380
So when, as I said, before the multi agent network came in, people with building agents,

416
00:27:41,380 --> 00:27:47,420
I have companies were built like, yes, hundreds of thousands of agents for just small purposes.

417
00:27:47,420 --> 00:27:51,980
But if they are building one agent for HR, it's trying to service no confidence and they

418
00:27:51,980 --> 00:27:57,540
are making building another agent for IT, which is also connecting the service of confidence,

419
00:27:57,540 --> 00:28:01,780
but they just want to call it as IT agent and HR agent.

420
00:28:01,780 --> 00:28:06,740
But when multi agent that became easier for them now, those thousands of agents are getting

421
00:28:06,740 --> 00:28:12,060
committed under a child relationship between the multiple agents.

422
00:28:12,060 --> 00:28:18,140
So one agent drives another audit, another agent checks the output, working as a policy,

423
00:28:18,140 --> 00:28:19,140
awesome.

424
00:28:19,140 --> 00:28:23,260
I don't know, the kind of one agent is doing agent to agent calling or one agent is doing agent

425
00:28:23,260 --> 00:28:25,900
to the system, the enterprise system, call it, right?

426
00:28:25,900 --> 00:28:27,500
So that has become lower.

427
00:28:27,500 --> 00:28:33,180
So now the multiple agents are rounding towards their platform and then you have eight

428
00:28:33,180 --> 00:28:39,700
and 365 as dashboards, which are coming for the security productivity with dashboard.

429
00:28:39,700 --> 00:28:42,420
What are the agents utilized?

430
00:28:42,420 --> 00:28:45,420
How many agents have connected to what data sources?

431
00:28:45,420 --> 00:28:48,460
How many agents are being used by external people?

432
00:28:48,460 --> 00:28:50,940
That has become already in tune too.

433
00:28:50,940 --> 00:28:55,620
And agent 365 is still there's a lot of potential is still just started off.

434
00:28:55,620 --> 00:28:58,020
I think it came a couple of months back.

435
00:28:58,020 --> 00:29:03,660
So we as consumers, we are bringing every single agent whether it's build on cloud, whether

436
00:29:03,660 --> 00:29:07,660
it's below Microsoft, Blacksteer, you build them in agent 365.

437
00:29:07,660 --> 00:29:13,900
That creates a whole loop for making sure there are no agents without owners, making sure

438
00:29:13,900 --> 00:29:20,780
your agents are not connected to connectors which require additional access or environment

439
00:29:20,780 --> 00:29:22,380
uprooled.

440
00:29:22,380 --> 00:29:24,780
So that dashboards have started coming up.

441
00:29:24,780 --> 00:29:29,940
Yes, it became a pain point and then Microsoft brought in a, this is the agent 365.

442
00:29:29,940 --> 00:29:34,260
The future would be yes, the agents are monitoring itself.

443
00:29:34,260 --> 00:29:39,740
There would be a security agent monitoring all the agents and hoping sure all the preview,

444
00:29:39,740 --> 00:29:42,020
all the DLP defenders are applied to it.

445
00:29:42,020 --> 00:29:43,860
But for now is a manual process.

446
00:29:43,860 --> 00:29:46,620
You are adding agent like entriety.

447
00:29:46,620 --> 00:29:48,540
We all have entriety principle.

448
00:29:48,540 --> 00:29:54,140
Id is a spack in unique IDs agents now have those agent IDs agent and dry.

449
00:29:54,140 --> 00:29:59,060
Where they get added to agent 365, they have a unique ID and you are able to monitor what

450
00:29:59,060 --> 00:30:06,060
that agent is doing.

451
00:30:06,060 --> 00:30:10,060
Yes, it is.

452
00:30:10,060 --> 00:30:17,460
So, I think a little bit step in this side, we have this security co-pilot.

453
00:30:17,460 --> 00:30:22,940
Yeah, but it's more monitoring, it's not doing something really from from, from, from,

454
00:30:22,940 --> 00:30:23,940
quite so.

455
00:30:23,940 --> 00:30:25,340
Yeah, yeah, that's a good feature.

456
00:30:25,340 --> 00:30:26,340
It's free and admin center.

457
00:30:26,340 --> 00:30:30,220
You can ask him, hey, check my license, check this.

458
00:30:30,220 --> 00:30:32,220
So it's doing a basic work yet.

459
00:30:32,220 --> 00:30:37,100
But yeah, I would want it to do everything in the security admin center.

460
00:30:37,100 --> 00:30:38,100
Yeah, yeah.

461
00:30:38,100 --> 00:30:42,620
I like, like, talk a little bit more about agent governance.

462
00:30:42,620 --> 00:30:45,100
I thought this topic was really interesting.

463
00:30:45,100 --> 00:30:52,980
So, I think for years we had shadow IT then shadows are then power platforms broad.

464
00:30:52,980 --> 00:31:00,540
How are we adding towards again, towards again agents broad?

465
00:31:00,540 --> 00:31:02,340
You, not yet.

466
00:31:02,340 --> 00:31:09,580
I would say it will because I think what happened last year, the, the, the number of agents

467
00:31:09,580 --> 00:31:14,220
started increasing in every single company, everyone was building some of the other purposes.

468
00:31:14,220 --> 00:31:18,940
But now, as I said, they are controlling these agents as parent child relationship.

469
00:31:18,940 --> 00:31:25,020
So, that, this making governance much more easier compared to those 10,000 agents now have

470
00:31:25,020 --> 00:31:28,300
fewer agents, fewer covered, closely covered agents.

471
00:31:28,300 --> 00:31:31,260
So, I don't think we are there yet.

472
00:31:31,260 --> 00:31:36,140
This year, I've seen companies adopting the security in governance very seriously.

473
00:31:36,140 --> 00:31:39,820
They're making sure, because last year, everyone was learning when they go, they were checking

474
00:31:39,820 --> 00:31:44,100
enterprise, they were checking, why is it so slow, slow because of the TLP.

475
00:31:44,100 --> 00:31:51,100
So, they revised the AI platform, the revised COE now and I feel the future would be much

476
00:31:51,100 --> 00:31:52,100
nicer.

477
00:31:52,100 --> 00:31:56,780
Yeah, however, there are open source agents which can do a lot of things like cloud and all

478
00:31:56,780 --> 00:32:02,420
right, the new fable is so powerful and people use it because people can use it for so

479
00:32:02,420 --> 00:32:03,420
many other multipurpose.

480
00:32:03,420 --> 00:32:09,620
But I think for public access or enterprise access, you'll be still under government control,

481
00:32:09,620 --> 00:32:14,020
whether if you are using Microsoft ecosystem, I'm not heard anything going on.

482
00:32:14,020 --> 00:32:16,020
So, should we be fine?

483
00:32:16,020 --> 00:32:17,020
Awesome.

484
00:32:17,020 --> 00:32:22,340
Yeah, I think this would become the future.

485
00:32:22,340 --> 00:32:27,340
I think, yeah.

486
00:32:27,340 --> 00:32:43,620
And when we look, or I have to think about my question, but the first jump a little bit,

487
00:32:43,620 --> 00:32:54,060
in three to another topic, it's often recommended by Microsoft show there be a central specialized

488
00:32:54,060 --> 00:32:57,460
AI center of excellence that did we do we need it?

489
00:32:57,460 --> 00:32:59,460
Oh, yeah, you need that, right?

490
00:32:59,460 --> 00:33:03,540
So, that's where this new agent 365, that would be a center of excellent.

491
00:33:03,540 --> 00:33:08,540
You are able to see all the tools, the agents, you're able to see all the workflows which

492
00:33:08,540 --> 00:33:12,540
are connected to again, workflows, you're able to see what are the data sources, people

493
00:33:12,540 --> 00:33:14,540
are consuming in the company.

494
00:33:14,540 --> 00:33:18,180
So, I can check, oh, yeah, people are going to share point, conference service, and there's

495
00:33:18,180 --> 00:33:21,180
only third party tools, like who brought this third party tool?

496
00:33:21,180 --> 00:33:22,180
Let's not approve, right?

497
00:33:22,180 --> 00:33:25,660
So, that kind of governance is very important, I think.

498
00:33:25,660 --> 00:33:32,580
As I said, I think agent 365 coming the picture, still the step one, I should do a lot more

499
00:33:32,580 --> 00:33:33,580
in the future.

500
00:33:33,580 --> 00:33:39,100
I think probably at the night, content, they'll announce more features into it as part of

501
00:33:39,100 --> 00:33:40,100
the ecosystem.

502
00:33:40,100 --> 00:33:42,100
But yes, it is very important.

503
00:33:42,100 --> 00:33:48,780
Me as an admin, I want to see a dashboard where my 350 K people are building thousands of

504
00:33:48,780 --> 00:33:53,940
agents, but I should be able to monitor who is building both and what are they connecting

505
00:33:53,940 --> 00:33:54,940
to, right?

506
00:33:54,940 --> 00:33:55,940
What actions are they building?

507
00:33:55,940 --> 00:33:58,420
What knowledge shows is they are building?

508
00:33:58,420 --> 00:33:59,900
What FCP is they are connecting?

509
00:33:59,900 --> 00:34:02,740
I would want to see that dashboard, yes.

510
00:34:02,740 --> 00:34:06,740
So, that sounds really interesting.

511
00:34:06,740 --> 00:34:13,580
Yeah, what would I have?

512
00:34:13,580 --> 00:34:20,580
Should agents have a kind of risk classifications?

513
00:34:20,580 --> 00:34:23,340
There are, again, when you build agents, right?

514
00:34:23,340 --> 00:34:30,020
You build agents for enterprise systems to modeling processes, right?

515
00:34:30,020 --> 00:34:36,860
Again, as I said, the agent processing is not very complicated as we suppose.

516
00:34:36,860 --> 00:34:41,500
I don't see any risk for enterprise system because you are a governing them.

517
00:34:41,500 --> 00:34:42,940
Your data is not going outside.

518
00:34:42,940 --> 00:34:45,940
Your data is locked, you're a subscription, right?

519
00:34:45,940 --> 00:34:54,900
So, you're not using public domain chat, GPD or co-pilot or Gemini, so it's all, you're

520
00:34:54,900 --> 00:34:55,900
using enterprise agents.

521
00:34:55,900 --> 00:35:00,860
You're already governed with your DLP policy, so I don't see any risk going on.

522
00:35:00,860 --> 00:35:06,780
Though I've seen people by the build something on the cloud court and like, or get up, right?

523
00:35:06,780 --> 00:35:12,420
They sometimes lose some data because they, they progn it in such a way.

524
00:35:12,420 --> 00:35:14,860
Other than that, and that's like in human loop, right?

525
00:35:14,860 --> 00:35:17,900
It's always there and it's getting fixed.

526
00:35:17,900 --> 00:35:20,300
I still don't feel there's any risk or growth.

527
00:35:20,300 --> 00:35:22,180
And it's part of the picture.

528
00:35:22,180 --> 00:35:26,500
If your data is governed by companies like Microsoft, they assure you, right?

529
00:35:26,500 --> 00:35:28,420
And then you prick story up.

530
00:35:28,420 --> 00:35:33,740
So when AI came in and I was trying to pitch this Microsoft co-pilot to one of the life

531
00:35:33,740 --> 00:35:40,780
sites, because medicine maker in the United States, they were, I want Microsoft lawyer to

532
00:35:40,780 --> 00:35:46,660
come to my office, sign me and I can be meant that my knowledge, my data will not go outside

533
00:35:46,660 --> 00:35:48,620
to train the model, right?

534
00:35:48,620 --> 00:35:53,020
And Microsoft sent the lawyer to confidently sign that document.

535
00:35:53,020 --> 00:35:58,500
Yes, your data is your data and it will not be used to train the other model.

536
00:35:58,500 --> 00:36:04,900
So that confidence is what we required a couple of years back and now comes in very much

537
00:36:04,900 --> 00:36:09,500
in a variation where they're fine AI as a rest job.

538
00:36:09,500 --> 00:36:15,380
At least on the governance side job does a different debate with me and you can have, but

539
00:36:15,380 --> 00:36:20,580
for enterprise, improving their experience is really doing a job.

540
00:36:20,580 --> 00:36:21,580
Awesome.

541
00:36:21,580 --> 00:36:22,580
Awesome.

542
00:36:22,580 --> 00:36:27,980
I have seen, I was linked in my research.

543
00:36:27,980 --> 00:36:32,660
You worked around the concept of a unified AI command center.

544
00:36:32,660 --> 00:36:33,660
Yeah.

545
00:36:33,660 --> 00:36:34,660
Yeah.

546
00:36:34,660 --> 00:36:37,020
And what exactly is an AI command center?

547
00:36:37,020 --> 00:36:42,880
So again, AI command center would be again, starting with agent 365 or environment set

548
00:36:42,880 --> 00:36:43,880
up by it.

549
00:36:43,880 --> 00:36:49,280
You build a portal where a request comes in the you're governing every step.

550
00:36:49,280 --> 00:36:54,700
Like for example, I'm in a company company X, I'm on a unified command center would be

551
00:36:54,700 --> 00:37:01,260
people requesting for an agent or requesting for a tool or requesting for an MCP right from

552
00:37:01,260 --> 00:37:04,080
my command center where the admin approves or decline.

553
00:37:04,080 --> 00:37:05,080
Yes.

554
00:37:05,080 --> 00:37:06,080
Okay.

555
00:37:06,080 --> 00:37:07,080
Go ahead.

556
00:37:07,080 --> 00:37:09,080
I'm approving your access to go and connect to service now.

557
00:37:09,080 --> 00:37:10,080
And this is the API.

558
00:37:10,080 --> 00:37:11,080
It is API client.

559
00:37:11,080 --> 00:37:12,080
I declined.

560
00:37:12,080 --> 00:37:13,080
Right.

561
00:37:13,080 --> 00:37:17,280
You automate that whole process where you are also doing auditing.

562
00:37:17,280 --> 00:37:18,780
You are also doing validating.

563
00:37:18,780 --> 00:37:23,480
You're also checking making sure everything is human in the loop.

564
00:37:23,480 --> 00:37:27,360
That's where the command center, I would say, it would comment right and the command center

565
00:37:27,360 --> 00:37:33,160
for you to is build an agent and then you see agent and where you are also able to monitor

566
00:37:33,160 --> 00:37:39,080
the agent, how age this working is providing relevant information or is just blabbering

567
00:37:39,080 --> 00:37:42,960
because there are 20,000 files which is connected to it.

568
00:37:42,960 --> 00:37:45,440
So that is another part of the monitoring you want to see.

569
00:37:45,440 --> 00:37:50,400
You want to see that the agent is being shared with right people right set.

570
00:37:50,400 --> 00:37:53,840
Nobody is of doing a Bitcoin farming there right.

571
00:37:53,840 --> 00:37:57,920
So you need to control that put some limit put some shared it.

572
00:37:57,920 --> 00:38:01,280
Hey, don't go above a thousand dollar a month right.

573
00:38:01,280 --> 00:38:02,680
For example, right.

574
00:38:02,680 --> 00:38:08,060
What if somebody brings in a Bitcoin farm and starts doing on AI service and you get

575
00:38:08,060 --> 00:38:13,580
a bill of 100 K. So you put limitations on each agent each environment on what how much

576
00:38:13,580 --> 00:38:14,900
crop should go.

577
00:38:14,900 --> 00:38:18,060
So that you can monitor when you get a flag, you reach $1000.

578
00:38:18,060 --> 00:38:19,460
Why did they reach $1000?

579
00:38:19,460 --> 00:38:22,940
My employees are just 100 who is doing what right.

580
00:38:22,940 --> 00:38:26,460
So you need to bring those as part of the command center.

581
00:38:26,460 --> 00:38:30,140
But again, as I said, COE is all part of your COE.

582
00:38:30,140 --> 00:38:36,900
You need to make sure you set it up and then it meets your life as an enterprise much easier.

583
00:38:36,900 --> 00:38:44,920
And I think from the admin perspective, it's also do is the or we have two roads.

584
00:38:44,920 --> 00:38:51,760
We have the normal user who will build this app and say, okay, please give me $100 for

585
00:38:51,760 --> 00:38:52,760
this.

586
00:38:52,760 --> 00:38:53,760
I don't know.

587
00:38:53,760 --> 00:38:54,960
And this access rights.

588
00:38:54,960 --> 00:38:56,820
And then we have the the admins.

589
00:38:56,820 --> 00:39:06,820
So is this also think for my idea is it's like an agent inventory tool for the company.

590
00:39:06,820 --> 00:39:07,820
Yeah.

591
00:39:07,820 --> 00:39:08,820
Yeah.

592
00:39:08,820 --> 00:39:09,820
It is.

593
00:39:09,820 --> 00:39:11,780
So yeah, we said the right thing.

594
00:39:11,780 --> 00:39:17,260
So when copilot was a now right, the infrastructure required an agent builder people started

595
00:39:17,260 --> 00:39:19,820
building thousands of agents, right.

596
00:39:19,820 --> 00:39:22,860
And the admins are like, who does not understand AI.

597
00:39:22,860 --> 00:39:24,580
They are good and administrator job, right.

598
00:39:24,580 --> 00:39:27,540
But what are these saying is, how do I control them?

599
00:39:27,540 --> 00:39:28,860
I don't know what they are building.

600
00:39:28,860 --> 00:39:31,900
I see 1000 in a day coming in.

601
00:39:31,900 --> 00:39:35,740
What the first step they did was they turned off the feature where you can build an agent.

602
00:39:35,740 --> 00:39:37,780
They're like, they're like, they're like, they're not.

603
00:39:37,780 --> 00:39:38,780
I don't know what to do.

604
00:39:38,780 --> 00:39:43,740
I don't understand who is building the seasons and then get the list of where they're

605
00:39:43,740 --> 00:39:44,740
going to do.

606
00:39:44,740 --> 00:39:47,740
So that inventory was a very important factor.

607
00:39:47,740 --> 00:39:49,740
Microsoft gave you power platform.

608
00:39:49,740 --> 00:39:54,180
It added an agent feature there that dashboard started coming in, right.

609
00:39:54,180 --> 00:39:55,180
Then it's still coming in.

610
00:39:55,180 --> 00:39:56,740
They're not remotely yet.

611
00:39:56,740 --> 00:39:58,740
So at least the admin was I, oh my God.

612
00:39:58,740 --> 00:39:59,740
Okay.

613
00:39:59,740 --> 00:40:00,740
I see all the agents in one place.

614
00:40:00,740 --> 00:40:01,740
Okay.

615
00:40:01,740 --> 00:40:02,740
Which is good.

616
00:40:02,740 --> 00:40:07,020
And the next step, how do I see the dollar source or the tools that connect me.

617
00:40:07,020 --> 00:40:10,340
So agent 365 now is doing the same purpose.

618
00:40:10,340 --> 00:40:12,020
What an admin would have a shout it.

619
00:40:12,020 --> 00:40:18,420
I've seen admin screaming at a conference to a person like, I want an admin center.

620
00:40:18,420 --> 00:40:20,780
I want to see an inventory of all the agents, right.

621
00:40:20,780 --> 00:40:21,780
That was missing.

622
00:40:21,780 --> 00:40:25,420
But now I think people are happy that they were able to see inventory.

623
00:40:25,420 --> 00:40:28,300
They're able to see understand all the dollars that tool set.

624
00:40:28,300 --> 00:40:31,260
So as admin is very important.

625
00:40:31,260 --> 00:40:35,700
They, they, they, they, they understand what is coming into the system.

626
00:40:35,700 --> 00:40:41,060
It could be a cyber security access issue or DLP issue and future.

627
00:40:41,060 --> 00:40:45,540
They need to make sure all API is at the world and controlled, which they're having doing

628
00:40:45,540 --> 00:40:49,620
with multiple system are also being consumed by the AI agents.

629
00:40:49,620 --> 00:40:50,620
Yeah.

630
00:40:50,620 --> 00:40:53,460
So yes, they wanted that inventory and now they have it.

631
00:40:53,460 --> 00:40:55,700
So they have very happy about that.

632
00:40:55,700 --> 00:40:56,700
Okay.

633
00:40:56,700 --> 00:40:57,700
Awesome.

634
00:40:57,700 --> 00:41:04,380
Also, and it's also having a part of, I think, yeah, it's a lot, not the nicest topic,

635
00:41:04,380 --> 00:41:09,780
but we are all interested about the token, getting more than experience.

636
00:41:09,780 --> 00:41:11,540
It's, it's a dead dynamic.

637
00:41:11,540 --> 00:41:12,540
What?

638
00:41:12,540 --> 00:41:17,340
I think this time was trying to show the US dollars and all in.

639
00:41:17,340 --> 00:41:19,540
And now we will pay for the token.

640
00:41:19,540 --> 00:41:23,460
It's also be possible to have a cost overview with this.

641
00:41:23,460 --> 00:41:24,460
Yeah.

642
00:41:24,460 --> 00:41:25,460
I know.

643
00:41:25,460 --> 00:41:26,460
It's a lot of cost.

644
00:41:26,460 --> 00:41:32,620
I even, even for me is like, do you, I'm paying $30 for Microsoft corporate and you want me

645
00:41:32,620 --> 00:41:35,780
to pay another four credits and corporate studio.

646
00:41:35,780 --> 00:41:38,860
And now I think there's a bit of hardness there also charging there.

647
00:41:38,860 --> 00:41:40,220
So this is a lot of quiet.

648
00:41:40,220 --> 00:41:44,220
I think even to do a click, you would have to pay something, right?

649
00:41:44,220 --> 00:41:49,900
But when it's, it's just because the AI is expensive right, the parking, the data centers

650
00:41:49,900 --> 00:41:52,460
and data, what they are expensive at least.

651
00:41:52,460 --> 00:41:58,620
So the more users use it, the possible radio, but the, and the usage is also increasing a lot

652
00:41:58,620 --> 00:41:59,620
of other factors.

653
00:41:59,620 --> 00:42:01,140
So those are the thing.

654
00:42:01,140 --> 00:42:06,340
So yeah, I think right now, this very limited authentication in admin center where you can

655
00:42:06,340 --> 00:42:12,660
control the credits that, hey, this agent should be only consuming these many credits.

656
00:42:12,660 --> 00:42:19,340
I would want to see that I'm able to control that per user also so that I, that would help

657
00:42:19,340 --> 00:42:20,340
me in that work.

658
00:42:20,340 --> 00:42:22,020
Like me and you are pro users.

659
00:42:22,020 --> 00:42:25,220
You can consume 10,000 prompts, right?

660
00:42:25,220 --> 00:42:29,900
And what if there's another user who just wanted to come and check something and because I

661
00:42:29,900 --> 00:42:32,340
consume everything of that agent.

662
00:42:32,340 --> 00:42:37,260
Now that they are, they are getting, so you don't have enough credit, right?

663
00:42:37,260 --> 00:42:39,020
Or enough corporate funds, right?

664
00:42:39,020 --> 00:42:43,740
So I think, yes, the companies will start now understanding because right now they were

665
00:42:43,740 --> 00:42:48,940
happy with the $30 corporate cost and then they were happy with corporate studio as being

666
00:42:48,940 --> 00:42:54,900
the interface of building agents, but when you charge them for AI builder credits, you charge

667
00:42:54,900 --> 00:42:58,100
them for power platform premium, panator.

668
00:42:58,100 --> 00:43:03,300
And yes, they will want to see how much have I spent every month, every quarter, even

669
00:43:03,300 --> 00:43:04,820
every day.

670
00:43:04,820 --> 00:43:10,500
That is still missing because, yeah, it's not there, but I think that will come because people

671
00:43:10,500 --> 00:43:12,260
are asking for it.

672
00:43:12,260 --> 00:43:21,140
Is this more an idea or is such a product in development beta or it is a project in development?

673
00:43:21,140 --> 00:43:22,140
Yeah, it is there.

674
00:43:22,140 --> 00:43:23,140
It will come.

675
00:43:23,140 --> 00:43:24,980
It has to come right now.

676
00:43:24,980 --> 00:43:30,300
I'm able to see trends on power platform admin center and a little bit on the agents

677
00:43:30,300 --> 00:43:33,820
65, but eventually yes, I would want to see.

678
00:43:33,820 --> 00:43:35,980
So many requests have already come to me.

679
00:43:35,980 --> 00:43:41,420
One thing can I check how many prompts is the agent is consumed or the credit, right?

680
00:43:41,420 --> 00:43:43,780
What are the overall credit they want to understand?

681
00:43:43,780 --> 00:43:46,180
What particular, again, what can be used?

682
00:43:46,180 --> 00:43:48,980
They will want to see that.

683
00:43:48,980 --> 00:43:55,500
One thing I think actually is, I find it's really, really interesting whether this, yeah,

684
00:43:55,500 --> 00:44:03,100
with this AI command center and I have next days I have a live stream.

685
00:44:03,100 --> 00:44:11,340
We build an enterprise agent and in the second step you do is for, yeah, for producers.

686
00:44:11,340 --> 00:44:14,500
And it would be really cool if you ready.

687
00:44:14,500 --> 00:44:18,620
We can test it on the talent tool.

688
00:44:18,620 --> 00:44:22,700
I think that's, that's, that's, I really, I really, I really interested to see it.

689
00:44:22,700 --> 00:44:23,700
That would be amazing.

690
00:44:23,700 --> 00:44:27,660
Yes, yes, you build an agent in real time and then validated in the backend.

691
00:44:27,660 --> 00:44:28,660
That would be really cool.

692
00:44:28,660 --> 00:44:29,660
Yes.

693
00:44:29,660 --> 00:44:30,660
Yeah.

694
00:44:30,660 --> 00:44:31,980
And everyone in the book can test it, right?

695
00:44:31,980 --> 00:44:35,420
How many fonts and how many message credit they are using?

696
00:44:35,420 --> 00:44:39,380
Then in the next episode you must bring your tools, shall we also line?

697
00:44:39,380 --> 00:44:41,380
That's cool.

698
00:44:41,380 --> 00:44:43,380
Oh, I love that.

699
00:44:43,380 --> 00:44:45,380
I love that.

700
00:44:45,380 --> 00:44:46,380
Yeah.

701
00:44:46,380 --> 00:44:48,380
Observability.

702
00:44:48,380 --> 00:44:53,740
Traditional applications have logs.

703
00:44:53,740 --> 00:44:59,340
What do observability mean for agente AI?

704
00:44:59,340 --> 00:45:04,580
Again, again, it's, you've started with your metrics, right?

705
00:45:04,580 --> 00:45:12,100
They want to see the ROI of for the agents and where they're picking up how many prompts are

706
00:45:12,100 --> 00:45:15,580
getting returned with a real value out of it, right?

707
00:45:15,580 --> 00:45:20,420
That is very important and observability be part of agent 65 now.

708
00:45:20,420 --> 00:45:24,620
Microsoft is going very, very hard because as you said, right?

709
00:45:24,620 --> 00:45:28,580
They want to know how the agents are inventories happening.

710
00:45:28,580 --> 00:45:32,540
What are the people consuming across the system?

711
00:45:32,540 --> 00:45:34,540
Observability is the next new thing.

712
00:45:34,540 --> 00:45:40,860
Everywhere, if you see Microsoft frontier from how observability is being changed with agent

713
00:45:40,860 --> 00:45:46,820
65 and then bring E7 as part of the license system where they're making sure you are connected

714
00:45:46,820 --> 00:45:49,940
to your overview and defender systems as well.

715
00:45:49,940 --> 00:45:55,980
So that's a new, I would say word, the key word in making sure that governance is in place

716
00:45:55,980 --> 00:45:58,980
for all the areas.

717
00:45:58,980 --> 00:46:01,180
Yeah.

718
00:46:01,180 --> 00:46:05,060
Sure.

719
00:46:05,060 --> 00:46:11,420
Should we log prompts from?

720
00:46:11,420 --> 00:46:13,460
Okay.

721
00:46:13,460 --> 00:46:19,980
I think here here, Germany, the most people say, oh, I don't get spied out.

722
00:46:19,980 --> 00:46:24,780
But yeah, okay.

723
00:46:24,780 --> 00:46:28,580
How do you, do you travel shoot a user reporting?

724
00:46:28,580 --> 00:46:34,900
Yesterday, the agent didn't, something did something strange.

725
00:46:34,900 --> 00:46:36,900
Why?

726
00:46:36,900 --> 00:46:43,380
How can you look into it?

727
00:46:43,380 --> 00:46:54,060
Yeah, I think how we can look into the process, but what these I agents do and how we can fix

728
00:46:54,060 --> 00:46:55,060
it?

729
00:46:55,060 --> 00:46:57,220
That's the observability, right?

730
00:46:57,220 --> 00:46:59,180
What the agent is doing, right?

731
00:46:59,180 --> 00:47:01,180
The traces were the trace, right?

732
00:47:01,180 --> 00:47:07,660
Full chain of the reasoning or the call to action it did when I use a profit.

733
00:47:07,660 --> 00:47:12,660
You want to understand what it did, what was the process, where it went, what was the knowledge

734
00:47:12,660 --> 00:47:13,660
source, right?

735
00:47:13,660 --> 00:47:17,820
You want to understand the metric sort of it and get that lost structure.

736
00:47:17,820 --> 00:47:23,220
So when the data is accessed, what it was at access, what is access, PDF1 or what is

737
00:47:23,220 --> 00:47:27,620
access PDF2 or the access PDF1 and two both, right?

738
00:47:27,620 --> 00:47:32,980
So that's where right now when you call about microsoreo, they got new features as part

739
00:47:32,980 --> 00:47:36,980
of where you see the transaction when you have typing on my test agent.

740
00:47:36,980 --> 00:47:42,580
Hey, give me information about HR policy on taking a leave.

741
00:47:42,580 --> 00:47:48,220
It goes, it gives me a whole chart where my agent is going to which share point or which

742
00:47:48,220 --> 00:47:53,180
knowledge it actually answered at head and then I get a response from there.

743
00:47:53,180 --> 00:47:58,620
So you get that whole training of picture coming up and this copies to the observability

744
00:47:58,620 --> 00:48:04,900
really gives you a proper UI to make you sure how the trace of data is happening is also

745
00:48:04,900 --> 00:48:11,020
there in Azure copies, observability also where you can see the what was the outcome with

746
00:48:11,020 --> 00:48:14,300
the, and you have those thumbs up thumbs down, right?

747
00:48:14,300 --> 00:48:15,620
People normally don't do that.

748
00:48:15,620 --> 00:48:18,140
I always tell people, hey, if you got a right answer, you might have a question.

749
00:48:18,140 --> 00:48:19,140
What's up?

750
00:48:19,140 --> 00:48:24,460
Because agent understands the source was good, the tooling was correct, why it did the exact

751
00:48:24,460 --> 00:48:26,780
good job, what it's supposed to do.

752
00:48:26,780 --> 00:48:31,820
So that whole observability is where the monitoring, the automation and the governance will

753
00:48:31,820 --> 00:48:34,620
come as part of the global picture.

754
00:48:34,620 --> 00:48:43,420
Yeah, I think observability becomes so, it's so strange when you think there are agents

755
00:48:43,420 --> 00:48:45,620
that work autonomously.

756
00:48:45,620 --> 00:48:46,620
Yeah.

757
00:48:46,620 --> 00:48:54,820
So, yeah, but I think also we have to talk about testing AI agents and how do you testing

758
00:48:54,820 --> 00:48:58,620
something was answer our deterministic?

759
00:48:58,620 --> 00:49:03,060
Oh, I love this new feature in copies, studio, the evaluation feature.

760
00:49:03,060 --> 00:49:05,460
I don't know if you have use it or not.

761
00:49:05,460 --> 00:49:10,660
It creates those test use cases, understanding what the agent capability is and then you put

762
00:49:10,660 --> 00:49:12,260
that and stores it.

763
00:49:12,260 --> 00:49:15,740
So, as I said, I want agent to do all the stuff, right?

764
00:49:15,740 --> 00:49:17,140
And it's exactly doing that.

765
00:49:17,140 --> 00:49:22,500
Agent creates those test use cases and you input them as data source and data injection

766
00:49:22,500 --> 00:49:26,860
and it gives you an output how it did it to and then you keep creating those use cases

767
00:49:26,860 --> 00:49:31,180
and the evaluation and keep feeding it to the agent provider.

768
00:49:31,180 --> 00:49:34,180
You understand whether the data is being indexed properly or not.

769
00:49:34,180 --> 00:49:36,700
Sometimes you feel the index didn't happen well.

770
00:49:36,700 --> 00:49:40,820
So we go back in the SharePoint admin center, we indexed the SharePoint list or something

771
00:49:40,820 --> 00:49:42,380
or the library, right?

772
00:49:42,380 --> 00:49:44,860
We indexing the whole part of the picture.

773
00:49:44,860 --> 00:49:51,380
So that will help and part of the structuring the data set and that helps in making sure

774
00:49:51,380 --> 00:49:56,220
the agent, population, whatever the output is coming, you're getting the real output.

775
00:49:56,220 --> 00:49:57,540
So the test is good.

776
00:49:57,540 --> 00:50:02,860
Now this copies studio toolkit, I always tell people it's a free tool by the cat team.

777
00:50:02,860 --> 00:50:08,540
You please please use it, you'll deploy that as your Microsoft app source and power platform

778
00:50:08,540 --> 00:50:14,500
that helps you in creating and testing more use cases as part of your evaluations and all

779
00:50:14,500 --> 00:50:15,500
those stuff.

780
00:50:15,500 --> 00:50:20,900
So test, test, test your agent, make sure you're getting a proper output proper because those

781
00:50:20,900 --> 00:50:27,300
testing also added to the prompt because you are training through prompts and through testing.

782
00:50:27,300 --> 00:50:32,580
Your agents become much better rather than when you build an agent and give it to an end user,

783
00:50:32,580 --> 00:50:36,140
the agent will perform by won't perform the way you want to.

784
00:50:36,140 --> 00:50:44,380
So based on those test use cases, the evaluation, the, you know, learns on the go on the process.

785
00:50:44,380 --> 00:50:45,380
Awesome.

786
00:50:45,380 --> 00:50:52,140
But how did we test against, I say, how are the two nations?

787
00:50:52,140 --> 00:50:53,140
Okay.

788
00:50:53,140 --> 00:50:54,140
What is this?

789
00:50:54,140 --> 00:50:56,380
Sorry, am I earning it?

790
00:50:56,380 --> 00:50:57,380
You're pondering.

791
00:50:57,380 --> 00:51:05,740
I think how, how I figure out if my AI starts to hallucinating.

792
00:51:05,740 --> 00:51:07,940
Oh, yeah.

793
00:51:07,940 --> 00:51:13,300
So again, the two options, top set thumbs up, that's the end user governance trail where they

794
00:51:13,300 --> 00:51:19,020
can, they've written like the answer to thumbs up, but only evaluation for test cases like

795
00:51:19,020 --> 00:51:25,660
what is the co-parts, you go to co-parts, you go to evaluation sector, you're feeding responses.

796
00:51:25,660 --> 00:51:32,780
You go very detailed use cases in there are PDF features 105, 105, we pick up data from

797
00:51:32,780 --> 00:51:33,860
each of them.

798
00:51:33,860 --> 00:51:40,860
We create a user trail where you use a would be asking a similar kind of questions on those

799
00:51:40,860 --> 00:51:46,420
document sets and we feed it to the agent and we validate how the agents look.

800
00:51:46,420 --> 00:51:51,940
As I said, the first step is, make sure your data is clean, your data, where your indexing

801
00:51:51,940 --> 00:51:56,180
is has proper metadata and tags and the description.

802
00:51:56,180 --> 00:52:00,180
If you don't do that, then the response will be all hallucinated.

803
00:52:00,180 --> 00:52:04,820
Most of the organization, they turn off the feature web features, like they don't want

804
00:52:04,820 --> 00:52:06,580
data to come from the web.

805
00:52:06,580 --> 00:52:10,900
They only want your data to come from the grounded data, which you are indexing.

806
00:52:10,900 --> 00:52:15,500
So that minimizes your knowledge to go and hallucinate.

807
00:52:15,500 --> 00:52:16,500
Okay.

808
00:52:16,500 --> 00:52:20,260
So step one, don't use the web because web is like Google and Bing, right?

809
00:52:20,260 --> 00:52:22,620
There's so much data in there right or wrong.

810
00:52:22,620 --> 00:52:28,900
So you want to make sure your data, latest data, proper data, aligned data, structured data

811
00:52:28,900 --> 00:52:33,180
is input to the agent and then agents coming.

812
00:52:33,180 --> 00:52:35,700
Those were the days when the hallucination was very strong.

813
00:52:35,700 --> 00:52:44,540
Now with new models in GBD and OPPO, the hallucination is very, very, very negative.

814
00:52:44,540 --> 00:52:50,860
And then we often talk about this multi-agent system.

815
00:52:50,860 --> 00:52:55,020
Have you an example how this looks like?

816
00:52:55,020 --> 00:52:58,460
Is it like a normal organization structure?

817
00:52:58,460 --> 00:53:07,020
I have a high R agent and that puts me, says, okay, I need to send the WallerPuzz and then

818
00:53:07,020 --> 00:53:08,020
hire them.

819
00:53:08,020 --> 00:53:10,020
How does it do the work?

820
00:53:10,020 --> 00:53:11,020
Right.

821
00:53:11,020 --> 00:53:12,020
Yeah.

822
00:53:12,020 --> 00:53:13,020
I've seen company doing amazing demos.

823
00:53:13,020 --> 00:53:18,740
I'll give you an example for somebody builds an agent, a multi-agent, like I am the CEO.

824
00:53:18,740 --> 00:53:24,140
I will be the top one whole team, put a data and then the CEO has multiple departments,

825
00:53:24,140 --> 00:53:27,180
HR, marketing, sales, and things like that.

826
00:53:27,180 --> 00:53:28,820
Those become some multi-agent frameworks.

827
00:53:28,820 --> 00:53:34,100
So when I ask, hey, can you create a marketing slide for me?

828
00:53:34,100 --> 00:53:36,460
I'm meeting Marco for a podcast.

829
00:53:36,460 --> 00:53:41,300
So it goes, it understands the requirement and the marketing agent has in my network goes

830
00:53:41,300 --> 00:53:44,220
and picks up that task and starts working on it.

831
00:53:44,220 --> 00:53:47,140
Whereas then the second word, hey, can you help me book a flight?

832
00:53:47,140 --> 00:53:51,420
I have to go and meet Marco on next Wednesday in the person.

833
00:53:51,420 --> 00:53:56,700
My other agent, my tracker or travel agent, who was an understanding, started looking for

834
00:53:56,700 --> 00:53:57,700
a flight for me.

835
00:53:57,700 --> 00:53:59,500
Tasty multi-agent network.

836
00:53:59,500 --> 00:54:02,940
So early on it was happening, well, before multi-agent, you were doing one or one life.

837
00:54:02,940 --> 00:54:05,580
I have to go travel, I have to go to travel agent.

838
00:54:05,580 --> 00:54:08,940
If I have to go and HR agent for, I have to go to HR agent.

839
00:54:08,940 --> 00:54:11,100
Now I just go to one agent, that agent.

840
00:54:11,100 --> 00:54:14,340
Now people have fancy names in normalization.

841
00:54:14,340 --> 00:54:22,580
Somebody calls them as Pepsi or somebody calls it as IWA, somebody calls it as some GPD or

842
00:54:22,580 --> 00:54:25,900
fancy GPD names, IWA, LX.

843
00:54:25,900 --> 00:54:29,580
So you just go to that agent, hey, do this for me.

844
00:54:29,580 --> 00:54:32,580
That agent will become a multi-agent server.

845
00:54:32,580 --> 00:54:37,300
And I get to fight that agent to agent to work for you instead of you looking for an agent

846
00:54:37,300 --> 00:54:38,300
to do that work.

847
00:54:38,300 --> 00:54:40,300
Hey, where is a travel agent?

848
00:54:40,300 --> 00:54:43,100
Let me find, it's not a point of find, you don't do that.

849
00:54:43,100 --> 00:54:47,540
You just go to your one agent and mix for us as Microsoft extract its copilot.

850
00:54:47,540 --> 00:54:52,820
Go to copilot, copilot is already connected to multiple agents in the back end.

851
00:54:52,820 --> 00:54:58,580
It will go and consume those agents or agents or those agents to providers and get your input

852
00:54:58,580 --> 00:55:02,980
or your own, your love you for.

853
00:55:02,980 --> 00:55:11,140
Something that I have seen on my research, your technical, your blog has reached more than

854
00:55:11,140 --> 00:55:13,220
10 million readers.

855
00:55:13,220 --> 00:55:16,820
How have you do it with an agent?

856
00:55:16,820 --> 00:55:19,020
Yeah, yeah, yeah.

857
00:55:19,020 --> 00:55:21,740
Those were the times when we used to write, right?

858
00:55:21,740 --> 00:55:22,740
Yeah.

859
00:55:22,740 --> 00:55:24,740
Like when we used to sit, when we used to learn.

860
00:55:24,740 --> 00:55:25,740
And that's how I started.

861
00:55:25,740 --> 00:55:29,540
I started by sharing knowledge through articles.

862
00:55:29,540 --> 00:55:34,340
If I'm writing a PowerShell script, hey, I wrote a script, I'll put it on my blog or article,

863
00:55:34,340 --> 00:55:36,300
okay, you can start using it from there.

864
00:55:36,300 --> 00:55:41,740
Or if I'm doing some new snippet of a tool or I'm learning something, okay, I learned this.

865
00:55:41,740 --> 00:55:42,980
Now, let's put it on the article.

866
00:55:42,980 --> 00:55:49,060
I think that's where it came and then I made sure I made it easier for people to learn.

867
00:55:49,060 --> 00:55:53,300
I want to complicate it because even for me, if somebody puts an article or a blog,

868
00:55:53,300 --> 00:55:57,380
was like, just put a quote there, I will understand like, what does it do?

869
00:55:57,380 --> 00:56:02,860
Like, all of a sudden, you give me step by step, step three, it goes to my mind, it visits

870
00:56:02,860 --> 00:56:05,940
the other, that's how I got it.

871
00:56:05,940 --> 00:56:06,940
Yeah.

872
00:56:06,940 --> 00:56:07,940
Yeah.

873
00:56:07,940 --> 00:56:08,940
Wow.

874
00:56:08,940 --> 00:56:11,140
I could talk another hour with you.

875
00:56:11,140 --> 00:56:12,980
That's really, I really enjoy it.

876
00:56:12,980 --> 00:56:18,620
So I have every, every podcast, I have a rapid fire out.

877
00:56:18,620 --> 00:56:23,180
So I, short, short, short, short answer.

878
00:56:23,180 --> 00:56:26,660
So, uh, co-part, co-part, studio or custom code?

879
00:56:26,660 --> 00:56:30,300
Power Automate or Autonomous Agents?

880
00:56:30,300 --> 00:56:32,300
Power Automate.

881
00:56:32,300 --> 00:56:37,620
Powerful agent or many specialized agents?

882
00:56:37,620 --> 00:56:38,620
Many specializing.

883
00:56:38,620 --> 00:56:41,420
Local or pro code?

884
00:56:41,420 --> 00:56:42,420
Pro code.

885
00:56:42,420 --> 00:56:45,820
Rack of fans, uni.

886
00:56:45,820 --> 00:56:46,820
Right.

887
00:56:46,820 --> 00:56:50,980
Uh, Shabot or Daita Wars?

888
00:56:50,980 --> 00:56:52,060
Shaboi.

889
00:56:52,060 --> 00:56:54,060
Uh, Shaboi Gain.

890
00:56:54,060 --> 00:57:00,100
Yeah, uh, when, uh, the, your phone rings and such are the other calls you would say,

891
00:57:00,100 --> 00:57:03,100
"Modern, you, you, you do a so great job.

892
00:57:03,100 --> 00:57:07,700
Uh, I need you for the co-pilot studio and you can develop, uh, future you like, you get

893
00:57:07,700 --> 00:57:09,500
all the money and resources.

894
00:57:09,500 --> 00:57:10,740
What do you build?

895
00:57:10,740 --> 00:57:16,060
To be honest, if, if I have to build an agent, I have many agents.

896
00:57:16,060 --> 00:57:17,340
I have my own personal agents.

897
00:57:17,340 --> 00:57:19,620
I have my phone and everything.

898
00:57:19,620 --> 00:57:20,620
Right.

899
00:57:20,620 --> 00:57:24,020
But right now, I, this year, I was thinking, uh, if I have to build an agent, I'm going

900
00:57:24,020 --> 00:57:30,060
to build an agent, which will directly work with, uh, the climate change, the heat, the

901
00:57:30,060 --> 00:57:36,860
fire, making sure, uh, get some renewable resources like water and all that it used to

902
00:57:36,860 --> 00:57:37,860
be.

903
00:57:37,860 --> 00:57:43,820
I think I want that agent, which we can go that kind of a geography, mapping and helping and,

904
00:57:43,820 --> 00:57:49,420
and sharing those ideas for the governments around the world, maybe with the United Nations,

905
00:57:49,420 --> 00:57:53,980
so that they can use the power of my agent and, and you can help them to bring, uh,

906
00:57:53,980 --> 00:57:57,820
a life to a better peaceful, it used to be.

907
00:57:57,820 --> 00:57:58,820
Yeah.

908
00:57:58,820 --> 00:58:02,940
And, um, who should I invite next and what questions should I ask?

909
00:58:02,940 --> 00:58:05,300
Uh, to the next guest.

910
00:58:05,300 --> 00:58:06,300
Yeah.

911
00:58:06,300 --> 00:58:14,180
Uh, and if it's on the Microsoft tech side, ask him or her like, um, what do you love more,

912
00:58:14,180 --> 00:58:19,540
which agent are like more M665 co-pilot, GitHub co-pilot or co-factory agent?

913
00:58:19,540 --> 00:58:21,380
And, and an idea.

914
00:58:21,380 --> 00:58:25,260
Will I should ask?

915
00:58:25,260 --> 00:58:27,260
Uh, no, I don't have an idea.

916
00:58:27,260 --> 00:58:28,260
Yeah.

917
00:58:28,260 --> 00:58:29,260
Yeah.

918
00:58:29,260 --> 00:58:33,020
Then my final question, as I'm under, imagine I'm, I'm a CEO.

919
00:58:33,020 --> 00:58:34,020
Okay.

920
00:58:34,020 --> 00:58:37,020
I, I cannot imagine it, but really, but I tried.

921
00:58:37,020 --> 00:58:41,660
Uh, of, of a global enterprise, we already have, uh, Microsoft's 65 co-pilot.

922
00:58:41,660 --> 00:58:45,460
We are experimenting with co-pilot studio.

923
00:58:45,460 --> 00:58:48,700
Different departments are building agents.

924
00:58:48,700 --> 00:58:55,020
Which, um, which are more of SharePoint content, sensitive data, regulation requirements and

925
00:58:55,020 --> 00:58:56,820
all this stuff.

926
00:58:56,820 --> 00:59:05,740
And, uh, the executive, uh, now I was asking me, when do these AI investments actually start

927
00:59:05,740 --> 00:59:07,620
transforming the business?

928
00:59:07,620 --> 00:59:14,660
So if you're sitting there with me, um, how did we design the next three months?

929
00:59:14,660 --> 00:59:15,660
Yeah.

930
00:59:15,660 --> 00:59:22,060
So when people ask you about ROI, they, they just feel, uh, and I've seen that multiple CEOs

931
00:59:22,060 --> 00:59:25,340
and CEOs I talked to, or they're like, Hey, I build this agent.

932
00:59:25,340 --> 00:59:26,340
I have 10 agents.

933
00:59:26,340 --> 00:59:28,740
I don't see people using it, right?

934
00:59:28,740 --> 00:59:31,980
The major factor of those ROI is adoption.

935
00:59:31,980 --> 00:59:33,540
I'll tell you why, right?

936
00:59:33,540 --> 00:59:36,940
Uh, I know you are on time, but I'll work well to be ready to create.

937
00:59:36,940 --> 00:59:41,540
So people are building agents, organizations are building agents, but as the consumer as

938
00:59:41,540 --> 00:59:44,620
an end user, you just announced them on an outlook email.

939
00:59:44,620 --> 00:59:46,180
Hey, this is a new agent.

940
00:59:46,180 --> 00:59:48,500
This will do this and this and that's it.

941
00:59:48,500 --> 00:59:49,500
Who is an end?

942
00:59:49,500 --> 00:59:53,860
People like, like, I'll just give myself as an example and, uh, non-ID person, right?

943
00:59:53,860 --> 00:59:58,980
I just come to the office for my legal work for some, I'm a legal author, right?

944
00:59:58,980 --> 01:00:03,100
I don't care about your multiple agents until I know what should I do with that agent,

945
01:00:03,100 --> 01:00:04,100
right?

946
01:00:04,100 --> 01:00:07,300
So, uh, if somebody gives me an agent, one fifth, here's the agent.

947
01:00:07,300 --> 01:00:09,220
I'm like, okay, what should I do with this agent?

948
01:00:09,220 --> 01:00:10,540
Should I book my flight ticket?

949
01:00:10,540 --> 01:00:14,500
Should I buy a brick coin or should I buy a new Audi car?

950
01:00:14,500 --> 01:00:16,580
You have to give me instructions, right?

951
01:00:16,580 --> 01:00:21,260
That is where the adoption is a very important factor for all companies.

952
01:00:21,260 --> 01:00:27,220
Even if you build 10,000 agents and you were looking for an ROI, we need to train our associates.

953
01:00:27,220 --> 01:00:32,260
Every single associate, not just the Toxie suite or not just the manager level, ever single

954
01:00:32,260 --> 01:00:33,260
user.

955
01:00:33,260 --> 01:00:38,140
Somebody, one of my, one of the CIO of a big bag told me one very good thing I was talking

956
01:00:38,140 --> 01:00:41,540
to him is like, "My friend, I want to not just build agents.

957
01:00:41,540 --> 01:00:45,860
I want to empower my people to use agents, right?

958
01:00:45,860 --> 01:00:47,500
That's where the adoption comes in.

959
01:00:47,500 --> 01:00:52,500
You need to make sure that Manpreet and the other person, everyone knows how to use an

960
01:00:52,500 --> 01:00:53,820
agent.

961
01:00:53,820 --> 01:00:58,500
Everyone knows what the outcome of that agent should be and they know what the input they

962
01:00:58,500 --> 01:00:59,500
can provide.

963
01:00:59,500 --> 01:01:04,940
So, it should be a day in a life of Manpreet versus day in a life of a lawyer or day in a

964
01:01:04,940 --> 01:01:06,700
life of an HR.

965
01:01:06,700 --> 01:01:08,460
Every thing should be covered.

966
01:01:08,460 --> 01:01:10,740
The more the people, the hands on the gear.

967
01:01:10,740 --> 01:01:12,460
Right now you and me, right?

968
01:01:12,460 --> 01:01:17,740
You know you wait for podcasts and you're going to extract this information, AI, put articles

969
01:01:17,740 --> 01:01:18,740
and things like that.

970
01:01:18,740 --> 01:01:20,540
You know how to use AI.

971
01:01:20,540 --> 01:01:24,940
But if there's a person who doesn't know how to use AI, they want to still write an article

972
01:01:24,940 --> 01:01:29,700
that will listen to my podcast, they'll write line by line and they'll spend four hours.

973
01:01:29,700 --> 01:01:31,540
You will do that in one minute.

974
01:01:31,540 --> 01:01:33,180
So that adoption is ready.

975
01:01:33,180 --> 01:01:40,340
ROI comes and I have seen company from 0% of buying an M365 corporate license for

976
01:01:40,340 --> 01:01:43,580
8 to 10 months, no adoption rate.

977
01:01:43,580 --> 01:01:49,020
And then when you do this adoption training, personal based training, one training, it has

978
01:01:49,020 --> 01:01:51,900
increased to 98 to 99 percent.

979
01:01:51,900 --> 01:01:54,140
That's the adoption goal you need to bring.

980
01:01:54,140 --> 01:01:59,980
And that would be your ROI because your company started using and becoming much more experienced

981
01:01:59,980 --> 01:02:05,620
with much more empowered with a new tech stack and started bringing that knowledge in your

982
01:02:05,620 --> 01:02:07,620
day to day cycle.

983
01:02:07,620 --> 01:02:09,620
So that's my ROI pitch.

984
01:02:09,620 --> 01:02:10,620
Yeah, awesome.

985
01:02:10,620 --> 01:02:14,340
Yom Arvett, thank you so many for joining me today.

986
01:02:14,340 --> 01:02:20,860
For me, the big takeaway from this talk is that next phase of enterprise AI isn't simply

987
01:02:20,860 --> 01:02:26,820
about creating better problems or adding another chatbot to, I don't know, Teams or something.

988
01:02:26,820 --> 01:02:35,180
It's more moving from answers to outcomes, agents can connect knowledge, applications, workflows,

989
01:02:35,180 --> 01:02:36,180
business processes.

990
01:02:36,180 --> 01:02:38,260
It's a really amazing time.

991
01:02:38,260 --> 01:02:45,340
And that also means the difficult part for enterprise technology doesn't disappear.

992
01:02:45,340 --> 01:02:50,860
Identity matters, security matters, data quality matters, governance matters, architecture matters,

993
01:02:50,860 --> 01:02:52,100
testing matters.

994
01:02:52,100 --> 01:02:59,860
But the most important thing, what most, or what's the most importantly, it's the people

995
01:02:59,860 --> 01:03:03,260
using the systems also matter.

996
01:03:03,260 --> 01:03:04,260
Yeah.

997
01:03:04,260 --> 01:03:10,500
And thank you for giving this view and you have to come, this product is ready to live

998
01:03:10,500 --> 01:03:13,380
stream and show it to us.

999
01:03:13,380 --> 01:03:20,820
So for all the listeners, you find my information on the podcast page from this episode.

1000
01:03:20,820 --> 01:03:25,820
And yeah, thank you again so many for spending our with me.

1001
01:03:25,820 --> 01:03:26,820
Oh, thank you, Marko.

1002
01:03:26,820 --> 01:03:31,820
Thank you for this invite and thank you for bringing stories like me and your podcasts.

1003
01:03:31,820 --> 01:03:33,700
You're doing an amazing job.

1004
01:03:33,700 --> 01:03:35,700
Thank you for hosting me today.

1005
01:03:35,700 --> 01:03:36,700
Yeah, thank you.

1006
01:03:36,700 --> 01:03:37,700
Bye.

1007
01:03:37,700 --> 01:03:38,700
Take care.

1008
01:03:38,700 --> 01:03:38,700
Bye.

1009
01:03:38,700 --> 01:03:41,880
(music fades)