Aug. 23, 2026
From Copilot Rollout to AI Workplace: Adoption, Agents & Real Business Value with Christoffer Besler Hansen [MVP]
![From Copilot Rollout to AI Workplace: Adoption, Agents & Real Business Value with Christoffer Besler Hansen [MVP]](https://d3wo5wojvuv7l.cloudfront.net/t_rss_itunes_square_1400/images.spreaker.com/original/e66550fbcd704843cb2eb18dd66d8a20.jpg "From Copilot Rollout to AI Workplace: Adoption, Agents & Real Business Value with Christoffer Besler Hansen [MVP]")
Key Takeaways
- Transitioning to an AI-powered workplace requires a strategic focus on continuous user adoption, training, and habit change rather than just assigning Microsoft 365 Copilot licenses.
- Initial enthusiasm for Copilot often leads to a drop in usage after four weeks unless organizations implement ongoing, structured training tailored to different employee roles like managers and consultants.
- Measuring Copilot ROI goes beyond usage metrics; organizations need to track tangible business outcomes, such as increased output, improved quality, or enhanced capacity.
- The introduction of advanced tools like the Research Agent allows employees to efficiently process massive amounts of information while retaining the ability to verify underlying sources.
- Moving from simple information retrieval to automated actions requires robust security architectures, clear guardrails, and least-privilege permission models to prevent unintended risks.
Microsoft 365 Copilot has moved beyond the question of “What can generative AI do?” The harder challenge is now turning AI into something thousands of employees actually use, trust, and derive measurable business value from. In this episode of M365 FM, Mirko Peters talks with Microsoft MVP Christoffer Besler Hansen, Head of AI Workplace at Atea Group, about what it takes to move from a Copilot rollout to a genuine AI-powered workplace. Drawing on experience supporting AI adoption across more than 8,000 employees, Christoffer shares lessons on Microsoft 365 Copilot adoption, training, agents, Copilot Studio, Microsoft Foundry, governance, security, extensibility, FinOps, and measuring business value.
WHAT IS AN AI WORKPLACE?
An AI workplace is much broader than simply giving employees Microsoft 365 Copilot licenses. At Atea, the AI Workplace team is responsible for how more than 8,000 users incorporate AI into their daily work. Microsoft Copilot is a major component, but the strategy also includes Copilot Studio, agentic AI, Foundry, experimentation with other technologies, and—critically—continuous user adoption. The objective is not to deploy one AI product. It is to change how people work with information, applications, and business processes.
TECHNOLOGY MOVES FAST. PEOPLE NEED TIME.
New AI models and capabilities can appear every week. Human working habits don't change at the same speed. Christoffer explains why organizations need to spend substantial effort helping employees feel comfortable changing established workflows. At Atea, this includes training throughout the year, sometimes every other week, with sessions designed for different audiences such as managers, consultants, salespeople, beginners, and advanced users. AI adoption therefore isn't a launch event. It is an ongoing organizational capability.
BUYING 5,000 COPILOT LICENSES ISN'T A STRATEGY
What should happen after an organization purchases thousands of Microsoft 365 Copilot licenses? According to Christoffer, the organization first needs to determine why it purchased them. What is the objective? What should employees accomplish differently? How will the organization support adoption? How will success be measured? Simply assigning licenses and expecting employees to teach themselves isn't enough. Employees already have jobs to perform and cannot realistically follow every weekly change across rapidly evolving AI products.
WHY EARLY COPILOT ADOPTION OFTEN DROPS
AI naturally generates curiosity. When users initially received Copilot without structured adoption support, Christoffer observed strong engagement for approximately the first four weeks. Employees experimented with the technology. But when they struggled to turn those experiments into new working habits, usage declined. After structured training was introduced, users were more likely to continue using Copilot over time—and began asking for additional training as the products evolved. Initial excitement gets people through the door. Continuous education helps keep them there.
HOW DO YOU MEASURE COPILOT ROI?
One of the hardest enterprise AI questions is determining whether Copilot is actually creating value. Usage alone isn't enough. An employee opening Copilot 50 times doesn't necessarily mean the organization has become more productive. Christoffer argues that organizations need to identify what matters to their particular business and then measure whether AI improves those outcomes. That could include completing work faster, handling more customer cases, improving quality, or increasing business capacity.
MEASURE OUTPUT, NOT JUST AI USAGE
One example discussed in the episode involves an employee who previously handled two cases simultaneously but could use AI to work across six while still receiving better customer feedback. That represents something more meaningful than a Copilot usage statistic. The employee is producing more output while maintaining or improving quality. The right KPI therefore depends on what the organization actually produces. AI metrics should ultimately connect with business metrics.
COPILOT AS A THINKING PARTNER
Meetings were one of the earliest areas where Microsoft 365 Copilot delivered obvious value. Transcription, summaries, and meeting intelligence can reduce administrative effort. But Christoffer highlights another important pattern: using AI as a thinking partner. Instead of asking AI to do all the thinking, start with your own ideas. Speak or dictate those thoughts. Let Copilot structure them. Review the result. Give feedback. Iterate until you have something useful. This approach can save time while simultaneously improving the quality of emails, presentations, documents, and other knowledge work.
RESEARCH AGENT CHANGES KNOWLEDGE WORK
Christoffer highlights Microsoft's Research Agent as one of the particularly valuable additions to Copilot. For large projects involving significant amounts of information, an agent capable of working through numerous sources can dramatically reduce research effort. It can also help users find information they may previously have struggled to discover manually. Importantly, users can review the underlying sources and verify whether the resulting information is correct.
YOUR AI IS ONLY AS GOOD AS YOUR INFORMATION
Enterprise AI quickly exposes existing information-management problems. Organizations need to think about how information is structured across SharePoint, OneDrive, CRM systems, and other repositories. Microsoft Purview can play an important role in identifying and protecting confidential information. Retention policies also matter because outdated documents can lead AI toward outdated answers. Simply giving an agent access to more information doesn't automatically make it better. Sometimes the correct approach is to clean the data before connecting the agent.
WHEN SHOULD YOU BUILD AN AGENT? ㅤ Christoffer recommends encouraging employees to start thinking about potential agent use cases early. Initially, organizations may create many simple agents that primarily retrieve information. Some will provide little long-term value. But experimentation changes how employees think about automation. The next maturity step is building agents that don't merely answer questions but perform actions—sending messages, updating CRM systems, triggering processes, or interacting with other applications. Eventually, organizations can move toward more autonomous agents working alongside employees.
AGENT BUILDER VS COPILOT STUDIO
Not every employee needs to begin with Copilot Studio. Christoffer sees many non-technical employees using Agent Builder directly inside Copilot to create simpler agents. More technical users move toward Copilot Studio when they require additional capabilities. This can create a useful progression: Idea → Simple Agent → Validation → Copilot Studio → Advanced Enterprise Agent An employee can prove the concept without becoming a professional developer, then involve technical specialists when the solution needs to become more sophisticated.
FROM ANSWERS TO ACTIONS
An HR agent answering “How many vacation days do I have?” is useful. An agent that can actually book next Friday as vacation represents a fundamentally different capability. This transition from information retrieval toward actions changes the architecture and security requirements surrounding AI. Christoffer expects users to interact less directly with traditional application interfaces as agents increasingly perform tasks on their behalf. Instead of navigating several administrative screens, users may simply describe the desired outcome to an agent.
AGENT SECURITY BECOMES CRITICAL
Once agents can take actions, organizations need strong guardrails. What can the agent do automatically? What requires explicit approval? What can it delete? Which systems can it access? What permissions does it receive? Christoffer emphasizes least privilege and approval controls, particularly when agents interact with administrative environments. Giving an autonomous agent Global Administrator privileges and allowing it to operate without restrictions would create obvious risks. The more capable agents become, the more important their permission architecture becomes.
ENTERPRISE AGENTS NEED AN INTAKE PROCESS
A personal agent used by one employee is different from an agent deployed to thousands of users. Enterprise-wide agents need quality control. Organizations should review instructions, permissions, connectors, integrations, and data access before allowing large numbers of employees to use them. Christoffer suggests establishing an intake process where employee ideas can be evaluated. Some agents may be returned to their creators for improvement. Strategically important ideas can instead be developed by a dedicated internal agent team.
COPILOT EXTENSIBILITY AND AGENT 365
The conversation also explores Copilot extensibility and the Agent 365 SDK. Christoffer describes experimenting with personal agents that have their own identities in Microsoft Entra. Such an agent could appear within an organizational structure, have its own email and Teams presence, and receive carefully controlled permissions. His example involves building an agent that can act as a personal assistant and potentially answer appropriate questions when he is away from work. The important architectural shift is that the agent begins looking less like a chatbot and more like another identity participating in the organization.
Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-modern-work-security-and-productivity-with-microsoft-365--6704921/support.
🚀 Want to be part of m365.fm?
Then stop just listening… and start showing up.
👉 Connect with me on LinkedIn and let’s make something happen:
- 🎙️ Be a podcast guest and share your story
- 🎧 Host your own episode (yes, seriously)
- 💡 Pitch topics the community actually wants to hear
- 🌍 Build your personal brand in the Microsoft 365 space
This isn’t just a podcast — it’s a platform for people who take action.
🔥 Most people wait. The best ones don’t.
👉 Connect with me on LinkedIn and send me a message:
"I want in"
Let’s build something awesome 👊
Frequently Asked Questions
What is an AI workplace in Microsoft 365?
An AI workplace goes beyond simply deploying Microsoft 365 Copilot licenses, incorporating agentic AI, Copilot Studio, Microsoft Foundry, and continuous user adoption to fundamentally change how people work with information and business processes.
How can organizations measure Copilot ROI?
Organizations should move beyond tracking basic usage metrics and instead measure real business outcomes, such as whether employees are producing higher output, handling more customer cases, or maintaining better quality with AI assistance.
When should an organization build a custom agent?
Organizations should encourage employees to experiment with simple agents early to build an automation mindset, eventually progressing from information retrieval agents to action-oriented and autonomous agents as maturity grows.
What is the difference between Agent Builder and Copilot Studio?
Non-technical employees typically use the Agent Builder directly inside Copilot to create simpler agents, while more technical users leverage Copilot Studio and the Agent 365 SDK when advanced capabilities and custom integrations are required.
1
00:00:00,000 --> 00:00:04,000
Welcome everybody to the MC65 at the end podcast.
2
00:00:04,000 --> 00:00:11,000
Microsoft 365 co-pilot has moved beyond the early questions of what can
3
00:00:11,000 --> 00:00:15,000
Generative AI do? The much harder question are now.
4
00:00:15,000 --> 00:00:18,000
How do we actually get thousands of employees to use it?
5
00:00:18,000 --> 00:00:21,000
How do we prove that it's creating value?
6
00:00:21,000 --> 00:00:25,000
When should we stand up Microsoft 365 co-pilot?
7
00:00:25,000 --> 00:00:33,000
When should we build an agent and when should we extend co-pilot with your own application and data?
8
00:00:33,000 --> 00:00:41,000
And perhaps most importantly, how do we move from interesting AI experiment to an
9
00:00:41,000 --> 00:00:43,000
Actually AI powered workplace?
10
00:00:43,000 --> 00:00:49,000
My guest today is Christopher Basil Hansen, Microsoft MVP and head of AI workplaces at
11
00:00:49,000 --> 00:00:57,000
Atea group. He leads Atea workplace AI strategy across the group and focused heavily on
12
00:00:57,000 --> 00:01:02,000
Microsoft say 65 co-pilot, co-pilot, studio and practical enterprise AI adoption.
13
00:01:02,000 --> 00:01:10,000
In 2026 Christopher announced that he had become an dual MVP across Microsoft co-pilot and
14
00:01:10,000 --> 00:01:16,000
co-pilot expansibility. So today we are going beyond demos.
15
00:01:16,000 --> 00:01:23,000
We are talking about co-pilot adoption, organizational change, measuring productivity and so on.
16
00:01:23,000 --> 00:01:30,000
So Christopher, welcome to the MC65 podcast and welcome to the MVP title.
17
00:01:30,000 --> 00:01:36,000
Thank you, Amitko. Very nice to be here looking forward to our conversation today.
18
00:01:36,000 --> 00:01:44,000
Yeah. Because before we get into co-pilot, can you tell us a little bit about your journey into
19
00:01:44,000 --> 00:01:46,000
Microsoft technology?
20
00:01:46,000 --> 00:01:57,000
Yeah, sure. So I'm 34 years old and 11 years ago I changed my job and started working at Atea.
21
00:01:57,000 --> 00:02:04,000
And one of the first thing I did then was to attend what was called the Microsoft University
22
00:02:04,000 --> 00:02:14,000
A six-week program where I then was learned in different Microsoft technologies. I remember still that the early days of
23
00:02:14,000 --> 00:02:23,000
Azure was one of the focus areas at that time. So from the start there I worked a lot with Office 365 and so on.
24
00:02:23,000 --> 00:02:28,000
And was kind of a junior consultant in those kind of things.
25
00:02:28,000 --> 00:02:36,000
And I had a few years where I worked at a device management. So I worked with the internal and also web
26
00:02:36,000 --> 00:02:46,000
solutions as well. Before I started working more and more with AI solutions when Apple chat
27
00:02:46,000 --> 00:02:57,000
GPT and co-pilot was not in 2022 and 2023. So since then there has been mostly AI in my focus areas.
28
00:02:57,000 --> 00:03:03,000
So I guess that's the short story.
29
00:03:03,000 --> 00:03:14,000
And AI workplace is that what is it in simply Microsoft 365 plus co-pilot or event to find our own?
30
00:03:14,000 --> 00:03:26,000
Of course Microsoft co-pilot is our main focus in our workplace but we are really responsible for how over 8,000 users are using AI in their daily
31
00:03:26,000 --> 00:03:35,000
work drive. And in some cases there can also be other solutions but in Atea all users that need it
32
00:03:35,000 --> 00:03:45,000
are have co-pilot available and we have a lot of focus on how to adopt the technology and use it in a good way.
33
00:03:45,000 --> 00:03:51,000
And so that's the main focus and of course over the years we have also started thinking more about
34
00:03:51,000 --> 00:04:07,000
the authentic AI solutions with co-pilot studio and Foundry and a few other things. So it's kind of it's a main focus around co-pilot but we also have a broad
35
00:04:07,000 --> 00:04:15,000
experimenting soul where we try out other solutions to stay on top of everything.
36
00:04:15,000 --> 00:04:27,000
When you look at your job today is how big is the path technology for those people, organization change and strategy?
37
00:04:27,000 --> 00:04:42,000
Of course but the technology is moving very quickly. It's new modules and new things every week but getting people to actually change your habits takes time.
38
00:04:42,000 --> 00:04:54,000
So working with people is one of what I'm trying to spend my time on to make sure that people feel safe in changing their habits,
39
00:04:54,000 --> 00:05:02,000
changing the tools that they are using. So we do a lot of training internally with user adoption.
40
00:05:02,000 --> 00:05:12,000
So we have had different types of training but we have at least every other week throughout the whole year we have some sort of training.
41
00:05:12,000 --> 00:05:22,000
We have for example sometimes we have a role-based training where we focus on how managers should use AI, how consultants should use AI, how sage people should use AI.
42
00:05:22,000 --> 00:05:36,000
And we also have a few other types of things like how to do a kind of beginners class and how to do a more advanced class to get to the next level of time.
43
00:05:36,000 --> 00:05:48,000
Awesome. Take a little bit back at the start of the Microsoft 365 co-pilot. What were organizations expecting from co-pilot?
44
00:05:48,000 --> 00:06:02,000
I think a lot of organizations were expecting a lot from the very start and that's probably a bit because of the marketing from Microsoft because I remember since before it was even available.
45
00:06:02,000 --> 00:06:18,000
And I think the download like co-pilot would be the perfect AI assistant that could do everything for you. And of course then people are a bit not happy when they are noticing that it comes and I made for them.
46
00:06:18,000 --> 00:06:44,000
So I think up until March this year when we got co-work I think the promise of an AI assistant that could do everything for you were not delivered but with things like co-work it's starting to get closer to it that you can actually work together with it and it can actually do things on behalf of you.
47
00:06:44,000 --> 00:06:48,000
And that's a new way of working for a lot of people.
48
00:06:48,000 --> 00:07:04,000
Awesome. Let's get a little bit practical. Let's say a company has, I don't know, 5,000 Microsoft co-pilot, it's in what happens next?
49
00:07:04,000 --> 00:07:18,000
Well, it depends where they start to, I guess. But usually when you purchase a lot of licenses you would have to think about what's the gold here, what's the scratchy.
50
00:07:18,000 --> 00:07:34,000
Hopefully they will do some sort of user adoption. I still see a lot of customers just purchasing licenses and expecting people to learn it themselves and I don't believe that works for any of them.
51
00:07:34,000 --> 00:07:50,000
I think you need to spend time and resources to make sure that your users are feeling safe to adopt the new technology. These kind of tools change so often as well.
52
00:07:50,000 --> 00:07:54,000
Every week you will see some change within the co-pilot app.
53
00:07:54,000 --> 00:08:08,000
And you can't expect users to follow that themselves because they have a lot of other things to do as well. So you need some experts either internally or externally, but you're
54
00:08:08,000 --> 00:08:18,000
going to buy their time from and help you out to do the user adoption. I think that's number one or what you need to do when getting the licenses.
55
00:08:18,000 --> 00:08:36,000
And then of course you need to think about how do we actually measure what value we get out of this because up until now I think a lot of organizations have measured it by just looking at how much you're using AI.
56
00:08:36,000 --> 00:08:46,000
Because it's hard to measure for many organizations because they usually don't have good measurements from beforehand on what are you delivering.
57
00:08:46,000 --> 00:08:58,000
So I think you need to kind of think through what's important for us as a company, what are we delivering to our customers or whoever we are delivering something to.
58
00:08:58,000 --> 00:09:08,000
How can we measure either how to do things faster, how to improve our quality, etc.
59
00:09:08,000 --> 00:09:18,000
Actually be able to measure those things on how do we do it without AI and how do we do it with AI.
60
00:09:18,000 --> 00:09:38,000
And I would say or actually I know because we did a report on this with our CIO analytics with 1500 companies in North and Europe earlier this year where we ask like the directors in this business.
61
00:09:38,000 --> 00:09:50,000
I've done so many full things with AI. I've even in North Europe 80% said that they have done something that gives a lot of value to their company with AI.
62
00:09:50,000 --> 00:10:04,000
Also one third of the companies say that they are not measuring AI at all. So they're just buying AI and using it but they are not measuring any outcomes for it.
63
00:10:04,000 --> 00:10:22,000
What are good KPIs to measure the useful usage of co-pilot or especially or all AI. Is there any KPIs I should focus on?
64
00:10:22,000 --> 00:10:34,000
I think that's the hard question that the O-Copnets need to ask themselves because I don't believe there are one easy answer that covers all companies.
65
00:10:34,000 --> 00:10:50,000
So I think it depends on what's important for you as a company, what are you delivering, what's the thing that you really either earn money on or whatever you do that you can actually measure.
66
00:10:50,000 --> 00:11:06,000
It can be of course one easy measurement that we are also doing internally for a long time is that we ask our users what they think they are changing by using AI.
67
00:11:06,000 --> 00:11:14,000
But that's not a real measurement. More just like a feeling from each user that maybe they feel like they are saving some time.
68
00:11:14,000 --> 00:11:36,000
They feel like there are being more performance, they feel like the quality is going up. But if you can also see that someone, I remember someone internally said to me that he is able to deliver on six cases at the same time instead of just two which he was able to do before.
69
00:11:36,000 --> 00:11:48,000
He also receives feedback from the customers that the quality is even better than before. So he receives very good feedback on all those six cases.
70
00:11:48,000 --> 00:11:58,000
I think that's a good measurement that you are actually doing more and you are even getting more quality out of each of those cases.
71
00:11:58,000 --> 00:12:06,000
But you get the pens very much on what kind of systems you have in place from before.
72
00:12:06,000 --> 00:12:26,000
And how did you start, I say, motivating people use Copilot, what's the focus on the training or what did you see that make up the moment where I was kicked in, oh Copilot is true.
73
00:12:26,000 --> 00:12:38,000
I think the good thing with the AI is that it's been so much hype around it since the very start that everyone wanted to use AI.
74
00:12:38,000 --> 00:12:52,000
And it's always been like you need to apply to get the license, you need to apply to receive it. So then users actually have to do something to get it.
75
00:12:52,000 --> 00:13:02,000
And that usually also results in people being a bit more engaged in it when they get it. They may be used some extra time to learn it.
76
00:13:02,000 --> 00:13:14,000
But what we saw in the start when we didn't use the option around is that you had about four weeks where they were very engaged in it and tried a lot of things.
77
00:13:14,000 --> 00:13:28,000
But when they noticed that they weren't really able to change their habits, they weren't able to improve that much with it, then they stopped using it.
78
00:13:28,000 --> 00:13:43,000
And after we started delivering training in early 2025, we noticed that people actually used it over time and didn't stop using it.
79
00:13:43,000 --> 00:13:57,000
And people are asking for more and more training because they feel like they need it because the products and the technology behind it is changing so rapidly.
80
00:13:57,000 --> 00:14:12,000
Yeah, it's, yeah, they're re-roading it's coming every time it's coming something new that's amazing.
81
00:14:12,000 --> 00:14:36,000
Yeah, it's, yeah, we talk about, I say time saving, what are the biggest time saving for companies? Is it meeting, is it email? Where did you see that the companies get more well you are out of their license?
82
00:14:36,000 --> 00:14:50,000
So I think in the start it was a lot about meetings because that was one of the very good features from the start with Gopalet that you get to try to meet things and get some good results with it.
83
00:14:50,000 --> 00:14:54,000
I'm sure people are still using that and say some time on it.
84
00:14:54,000 --> 00:15:15,000
But I also think that using it as a thinking partner is both saving time but also increasing quality. So what we are trying to do internally when we are using or say having user adoption is that we don't ask AI first, but you think first yourself.
85
00:15:15,000 --> 00:15:35,000
So you use your brain and think about what would you do in this case but instead of using a lot of time on it you kind of brainstorm through it together with AI maybe use dictation function on your computer and just say it out loud, give all their thoughts out.
86
00:15:35,000 --> 00:15:54,000
And then you ask the AI or call quality in this case to give you some feedback and structure it for you and then you have draft one and then you come back and go with some feedback again and you go a bit back and forth and you have something you're happy with.
87
00:15:54,000 --> 00:16:09,000
And I think that is all the time say we're something that increases the quality on what we need to be able to do if it's true a male or if it's true a presentation or something like that.
88
00:16:09,000 --> 00:16:38,000
And I think that also thinking about now is the research agent I remember when that came in in April May 25 that is one of the best agents I think that I've come up out in in co pilot time because it's able to think for a long time and also dig through that sources and help you find things that you maybe weren't able to do.
89
00:16:38,000 --> 00:17:00,000
Maybe weren't able to find from before so I have some examples it turn lever people really save a lot of times when especially when working with big projects that need a lot of resources and really need to find the correct information in a lot of documents.
90
00:17:00,000 --> 00:17:12,000
Then the research agent has been very available and it's also giving them a way to double check that it's actually correct as well.
91
00:17:12,000 --> 00:17:29,000
I think you say a big project that means a big amount of data how should companies prepare that data for co pilot that they get good results what was your tip.
92
00:17:29,000 --> 00:17:49,000
So of course it depends on where you are from before I think having some way of structuring your data is important whether it's in in each and every once one driver if it's in SharePoint and so on.
93
00:17:49,000 --> 00:18:13,000
But also hopefully everyone is using curvy or to the mark your data if it's confidential and what not and I think that's an important step from the security side both to what you allow co pilot to see.
94
00:18:13,000 --> 00:18:29,000
And also that the user are aware of what kind of information are I getting from co pilot now is it something that could be confidential that I shouldn't share directly with external people etc.
95
00:18:29,000 --> 00:18:57,000
So I think that's also an important part of it and also having retention policies to clean up all files all folders that are no longer in use because I think if you just give AI a lot of information out the information that will be based to give wrong answers.
96
00:18:57,000 --> 00:19:17,000
So there aren't really no easy answer to it because it all depends on where each company is from before but nonetheless it's a very important thing to prioritize to get a good data structure.
97
00:19:17,000 --> 00:19:35,000
And one example I have of this is when we created one of our orchestrating agents but we have a different orchestrating systems internally where we have one agent using a number of sub agents to give answers to the people.
98
00:19:35,000 --> 00:19:59,000
And when we are allowing more people to use that kind of agent system the way this agent system is working is that it's connecting to agents that have a lot of a lot of data for customers for example.
99
00:19:59,000 --> 00:20:25,000
But we are not accepting those customer agents in before they have a good data structure so they each account management need to clean up the data for their customer and then we can create an agent out of it and then we also have agents connecting to CRM and other type of systems so it can work together with different type of systems and give craft information to those that are used in the agent system.
100
00:20:25,000 --> 00:20:35,000
And that's one way to do it that's to be able to create an agent for your customer or whatever you need to first clean up your data.
101
00:20:35,000 --> 00:20:45,000
Yeah I think yeah agents that are new you are to do.
102
00:20:45,000 --> 00:21:03,000
So let's move from co-pilot more into agents when should organizations stop thinking how do we use co-pilot and start thinking we need an agent or an agent.
103
00:21:03,000 --> 00:21:21,000
I think you should always kind of yes I think all organizations should be thinking about how can we use agents and start also in their users to think about what can be create agents for.
104
00:21:21,000 --> 00:21:37,000
And then you will just start creating a lot of agents maybe hundreds or thousands agents that maybe don't give any value at all but it changes the mindset for people to to think about what an agent can do.
105
00:21:37,000 --> 00:21:49,000
Then of course by the sometime you will be able to also deliver on more advanced agents so in the start most people are just creating agents that give information.
106
00:21:49,000 --> 00:22:06,000
But when you have kind of figured that out in a good way and you create some agents that give you the information then maybe the next step is to create agents that actually do things as well maybe that is send an email send a message or update CRM systems.
107
00:22:06,000 --> 00:22:19,000
Or whatever and then the next step would maybe then be a more autonomous agent that can work besides you as co-worker or something like that.
108
00:22:19,000 --> 00:22:30,000
So I think there are multiple steps there and that is important to kind of go through for each organization.
109
00:22:30,000 --> 00:22:40,000
And here we have co-pilot I say okay, not for that team but for the people it's out of the box too.
110
00:22:40,000 --> 00:22:58,000
So you also have co-pilot studio when people are ready to use co-pilot studio and yeah where do co-pilot studio fit into the AI workplace architecture from your perspective.
111
00:22:58,000 --> 00:23:10,000
So we have had the co-pilot studio available for all users in ATA for a long time almost since the start of it.
112
00:23:10,000 --> 00:23:19,000
But we have of course noticed that those that you see there are mostly people with a technical background.
113
00:23:19,000 --> 00:23:29,000
Because the user interface and everything is a bit more technical than what everyone wants to use.
114
00:23:29,000 --> 00:23:41,000
So in ATA and most of our customers I see most users are still using the agent builder directly in co-pilot to build agents.
115
00:23:41,000 --> 00:23:48,000
And then the more technical people are using co-pilot studio to build a bit more advanced agents.
116
00:23:48,000 --> 00:23:54,000
And I think both solutions can be a good start point.
117
00:23:54,000 --> 00:24:09,000
What we also see in some cases is that people start with agent builder and if they are not taking for themselves to go into co-pilot studio then we might say to someone more technical that ability is agent now.
118
00:24:09,000 --> 00:24:17,000
Understand the way of it but I need some help to actually make it work fully in co-pilot studio.
119
00:24:17,000 --> 00:24:34,000
And now with the recent changes since a couple of weeks ago with Git-to-parnas in co-pilot studio I think that's also starting to become a good way to build agents running there as well.
120
00:24:34,000 --> 00:24:43,000
And then addition to that we also have Microsoft Founder where you can do even more advanced agents.
121
00:24:43,000 --> 00:24:51,000
So we have a few also agent systems running there as well.
122
00:24:51,000 --> 00:25:04,000
I think a little bit how we actually use or most people use AI for answers.
123
00:25:04,000 --> 00:25:10,000
But I think the change is to become getting actions.
124
00:25:10,000 --> 00:25:25,000
I have an HR agent answering how many vacation days do you have is one thing but yeah, book next Friday as vacation is more advanced.
125
00:25:25,000 --> 00:25:34,000
What changes architectural when agents start taking actions?
126
00:25:34,000 --> 00:25:44,000
So I think that will slowly change how we work and how we use our systems today.
127
00:25:44,000 --> 00:25:53,000
I can just see from my own experience using a lot of coding agents the last year.
128
00:25:53,000 --> 00:26:07,000
I started to do less in the user interfaces itself if it's either Microsoft Admin Center or Azure Admin Center and using more coding there instead.
129
00:26:07,000 --> 00:26:21,000
So I also have a coding agent that can do tasks for me directly in the Admin Center with some of course security guardrails around what it's allowed to do and not.
130
00:26:21,000 --> 00:26:33,000
But instead of having to do it myself or having to message someone internally I can just message my agent and it will actually be able to do everything.
131
00:26:33,000 --> 00:26:42,000
First is a test plan if I want to do that and then in the production environments.
132
00:26:42,000 --> 00:27:03,000
And I think in some time maybe in months or maybe in a year or two we will have more agent systems that kind of have a way into the systems that we are allowed to do things instead of using or having to click around and do everything ourselves.
133
00:27:03,000 --> 00:27:16,000
So for example with with HR instead of just asking how many vacation days do I have left and then having to peel off the form yourself your agent could in theory do this for you.
134
00:27:16,000 --> 00:27:20,000
And that's already working for some companies today.
135
00:27:20,000 --> 00:27:29,000
But that is something I think will be more available in the coming months from now.
136
00:27:29,000 --> 00:27:39,000
Or what is with I say risky actions I don't know my neighbor in the office is so loud.
137
00:27:39,000 --> 00:27:51,000
You can wait no joke and more like I don't know delete all delete all emails or I don't know and you don't describe it right.
138
00:27:51,000 --> 00:28:03,000
And it's from from project I actually have those something like this how can we prevent from from these wrist.
139
00:28:03,000 --> 00:28:20,000
So that's a real thing to think about the security around it because we have seen examples of people using for example coding agents and certainly the coding agent has deleted everything on their local hardware.
140
00:28:20,000 --> 00:28:37,000
So having some guardrails around it what are you allowed to do what are you are not allowed to do having the agent ask before doing something that can delete something for example.
141
00:28:37,000 --> 00:28:50,000
And I think first of all we need good harnesses so so a harness could be for example you get a couple of that or code X from open or something like that.
142
00:28:50,000 --> 00:28:58,000
And we didn't let harness you usually can set what kind of permission you give the agent that are doing task for you.
143
00:28:58,000 --> 00:29:06,000
So whenever it's doing a coding task should ask you first or what is pre approved.
144
00:29:06,000 --> 00:29:18,000
So that's kind of where you need to start that what are you actually allowing the agent to do without asking you and what the station need to ask you before.
145
00:29:18,000 --> 00:29:35,000
And also thinking about if you are actually give it some admin rights for example thinking about please privilege possible that you don't give it you know glad me and let it run wild in in return.
146
00:29:35,000 --> 00:29:54,000
So yeah I think a lot of people can build and say okay I build their own copilot for their own process but it's also built copilot or agents like systems for the whole company.
147
00:29:54,000 --> 00:30:17,000
What have companies to check or to reach out when they build complex systems AI systems for I don't know for processing with a lot of I don't know two or a thousand or thousands of people to use what's the other the shift.
148
00:30:17,000 --> 00:30:37,000
So I think you need some sort of way as a type of intake when people are creating agents that can be run or used by a lot of people you need to some all to take on that agent.
149
00:30:37,000 --> 00:30:48,000
What kind of instructions does the agent have what kind of access does it have what kind of connectors if any does it have if it needs any integration to an system.
150
00:30:48,000 --> 00:31:00,000
So you need to go and do a quality control on all of those before you actually allow it to be run inside the company by many users.
151
00:31:00,000 --> 00:31:24,000
And so that's kind of step one and then you probably also need some sort of people that can if that agent doesn't meet that quality control you either of course had it back to the user and they need to redo it or maybe you should think about how can we build agents for our own company.
152
00:31:24,000 --> 00:31:44,000
So when we take the the IDs from users in to the intake and then we plan what we need to do to make this agent off and then we implement that with a type of agent creating agents and bubbling agents.
153
00:31:44,000 --> 00:31:59,000
So we have co pilot co pilot studio co pilot work work a few I don't know it's actually called and what what how fits co pilot extensibility in in this picture.
154
00:31:59,000 --> 00:32:13,000
So I think co pilot extensibility is kind of multiple things of course you have co pilot studio as one kind of way to think of an existence to build there and then you also have the agent 365 SDK.
155
00:32:13,000 --> 00:32:37,000
So and then in this that you also have a kind of connectors or plug in that you can connect with co pilot one of the things I've been looking into the case the agent 365 SDK where you can kind of build your own agents that have their own identity in the end.
156
00:32:37,000 --> 00:32:53,000
So one of the projects I've been working on and this summer is to kind of create a personal agent that's in the organization tree looks like it's kind of an employee below me.
157
00:32:53,000 --> 00:33:11,000
So people can actually see it in the organization tree it has an email address a team address people can message it and it will be able to have certain permissions set by me so it can actually answer other people.
158
00:33:11,000 --> 00:33:31,000
So I can for example then if I am away from work in a few days and people need some monsters for me it will have access to certain things that it is allowed to answer internal people and certain things it's allowed to answer external people.
159
00:33:31,000 --> 00:33:59,000
And it will also work as a personal assistant for me. So so of course a bit like co pilot or co work but something that making myself and when making this in agent 365 SDK it's also made as a type of what they call a blueprint or a template.
160
00:33:59,000 --> 00:34:07,000
So I can set this up for myself but I can also set it up for as many people in the organization as I want.
161
00:34:07,000 --> 00:34:27,000
So and I'm also creating a dashboard to it so I can access or allow it to access things from work I cure from from my user based on what I think is okay or not.
162
00:34:27,000 --> 00:34:45,000
So I think that's at least in my mind one of the most interesting things right now from the accessibility scientist to look into the agent 365 SDK at creating those type of agents.
163
00:34:45,000 --> 00:35:07,000
And what can I connect is it only Microsoft internal stuff or what can I connecting I don't know says what can I connect I don't know what size what possibilities are about the label.
164
00:35:07,000 --> 00:35:31,000
So I think pretty much anything is possible because it's I'm doing it by code and I'm using right now foundry models to run it so I'm using right now I'm using a deep seek model behind it but I'm also using a G586 model.
165
00:35:31,000 --> 00:35:45,000
You can really connect of course whatever data sources you have available within your Azure or Microsoft 365 environment.
166
00:35:45,000 --> 00:35:57,000
And so everything is possible but of course out of the box it's mostly a mobs of 365 and then you can kind of extend from there.
167
00:35:57,000 --> 00:36:09,000
Awesome. Have you an example or what you have built you are really proud of.
168
00:36:09,000 --> 00:36:38,000
And I think one of the first things I built was a dashboard to look at the usage of co palette in Berlin or there so most people working co palette has probably seen the kind of the evil dashboards for Microsoft which works fine it gives you a kind of view on how co palette is being used internally.
169
00:36:38,000 --> 00:36:54,000
But after we got access to kind of the the Biva premium licenses last year you could also go into the Viva analyst site and download the statistics of all the users in the company as a CSV file.
170
00:36:54,000 --> 00:37:05,000
And when I did that it resulted in a file with 90 columns and 300,000 lines or rose in Excel.
171
00:37:05,000 --> 00:37:19,000
So I made a dashboard where I could just input that and generate a lot of graphs based on what I wanted to see.
172
00:37:19,000 --> 00:37:34,000
So that's one of the things that I built that I was made the open source in this direct that I built a couple of websites to follow up on Microsoft news because.
173
00:37:34,000 --> 00:37:48,000
So after sometimes you kind of noticed that looking into the admin center messaging center you see that there are always something new every day on what's coming for co palette.
174
00:37:48,000 --> 00:38:11,000
And instead of having to login I created a simple site to find a view it and I'm also added the Microsoft 365 roadmap on that as well and then I created an iOS application that kind of gives me the same view that I called the tenant balls that is also freely our.
175
00:38:11,000 --> 00:38:34,000
So I actually get the notification every hour if there are any news from the admin center messaging so a few of the things that have been creating myself after kind of starting to use more and more coding tools that I lost you.
176
00:38:34,000 --> 00:38:57,000
And I think a lot of people actually speak about one topic multiple autonomous agent how many agents will we will we have in the future then on the top and autonomous they are really can go.
177
00:38:57,000 --> 00:39:20,000
Yeah so I think we will have a lot of agents and most of them are probably not that the agents either so for example in until we have a thousand employees but we have a thousand agents so already have more agents than employees but of course all of those agents are not being used to be.
178
00:39:20,000 --> 00:39:49,000
And maybe actually in hundred agents being used every month so of course those 11,000 agents are agents people you created one time maybe they got some value out of it and then they stopped using it and then it's it's left there to die so of course that's also another thing that I'm spending one time on with agent 365 is to kind of think about the lifecycle policy about agents.
179
00:39:49,000 --> 00:40:18,000
If you're not using an agent for maybe 30 6090 days and it's maybe time for to kind of delete that agent and to clean up from it to avoid going back to 2020 when everyone created a team because you started a new project and you were left with thousands of teams that didn't get out.
180
00:40:18,000 --> 00:40:34,000
And what I also think of the lifecycle type is it's very important topic also all the governance stuff but I think another interesting topic is shadow IT.
181
00:40:34,000 --> 00:40:49,000
So I have clients they say oh we don't use AI because then it's very good doing yeah it's the said to be then they bought their own chat GPT and you have also shadow IT.
182
00:40:49,000 --> 00:40:58,000
But yeah and what is your view here on the shadow IT topic I think a lot of companies are aware of this actually.
183
00:40:58,000 --> 00:41:13,000
Yeah I think shadow AI is is very real and I think it's hard to just say no to users to say you are not allowed to use it.
184
00:41:13,000 --> 00:41:26,000
We have tried that for many years in all types of companies and whenever a user gets a know they just do it on their personal device and way and they probably upload some files.
185
00:41:26,000 --> 00:41:41,000
But I think the most important thing to do is that the users are getting tools that are as good as possible to kind of combat them using a shadow AI tools.
186
00:41:41,000 --> 00:41:53,000
But on the other hand I think it's also important that people are allowed to experiment with different tools of course in a safe way.
187
00:41:53,000 --> 00:42:18,000
Even if that means people using non-approved tools like for example codecs or cloud code and so on but having to use it in a safe way maybe on their personal computer not allowed to use of course in turn your customer data in the tools but more to experiment what's possible in the other type of tools.
188
00:42:18,000 --> 00:42:28,000
So using their proved IT tools when working with that.
189
00:42:28,000 --> 00:42:41,000
I think in the other topic I often hear it's it's yeah I can find ups find ups but because at the start co pilot cost I don't know US $20 or so.
190
00:42:41,000 --> 00:42:49,000
It was really clear but it's now we have token tokens bands and it's on and different models with different costs.
191
00:42:49,000 --> 00:42:56,000
How can we find ups the co pilot and how are we handling this.
192
00:42:56,000 --> 00:43:11,000
Yeah so I think the final is also a very important topic moving forward especially after get the co pilot starting costing real money after June and also with co-or costing co pilot credits.
193
00:43:11,000 --> 00:43:25,000
I see this internally that the limits we have set in both co-work and also in get the co pilot is there is a lot of users.
194
00:43:25,000 --> 00:43:38,000
Getting to that limit very quickly and we need to think about what kind of value are we getting out of all the money we are spending on this.
195
00:43:38,000 --> 00:43:48,000
We are of course thinking about how can we do better use reduction towards what do you do in which tool.
196
00:43:48,000 --> 00:44:05,000
For example co work creating PowerPoint presentations is something co work is good at but it's also costing a lot of money but you can also do the same almost get the same result by using co pilot inside of.
197
00:44:05,000 --> 00:44:19,000
So for example using the correct tool at the correct time is one thing to think about and also in coding instances.
198
00:44:19,000 --> 00:44:34,000
Luckily we are getting good competition from the Chinese models which are open weights and a way to kind of run on local server hardware.
199
00:44:34,000 --> 00:44:52,000
And that makes it for example you have seen with open AI when they launched their GPT56 models with sold data and Luna then one or two weeks after they launched it they also changed the pricing model on the two cheapest models.
200
00:44:52,000 --> 00:45:11,000
So for example with the Luna model which is the cheapest and smallest model they reduce the price with 80%. But the model itself is actually quite good as well in a lot of our coding instances.
201
00:45:11,000 --> 00:45:23,000
We are now using Luna as one of the primary models because it's very good at executing tasks. It's not necessarily that good at planning things.
202
00:45:23,000 --> 00:45:35,000
So maybe use a kind of frontier model to do all the planning and then use the cheaper models to do execute all the plans that we have made together with the frontier model.
203
00:45:35,000 --> 00:45:52,000
And learning people to kind of think this way can be hard. And so that's kind of where we have all to use the option but also we need to set some boundaries about what are your love to use.
204
00:45:52,000 --> 00:46:17,000
What did you think people often have fears about losing losing their jobs. So what show people do to stay relevant in the age of AI which skills that the people need.
205
00:46:17,000 --> 00:46:34,000
So again I think it depends very much on what you're doing from before. I think there are still a lot of things that will still be very much relevant within the age of AI.
206
00:46:34,000 --> 00:47:01,000
One year ago there was a lot of people that you would no longer need developers because AI has been getting so good at coding. But I think now if you ask people you all people all orientations need developers now because you need some people to kind of stare these agents around and whenever you can
207
00:47:01,000 --> 00:47:17,000
just look at the developers, you just want more developers to kind of look over it all. So I think it's hard to say how it will evolve in the next two to five years.
208
00:47:17,000 --> 00:47:30,000
I think being an experimenter and testing out the solutions, finding out how does AI work in what I'm good at, how can I become even better at what I do with AI.
209
00:47:30,000 --> 00:47:40,000
I think it's the best way to stay relevant because of course some jobs will become because of AI.
210
00:47:40,000 --> 00:47:49,000
And there will be creating new jobs because of AI and some of the current jobs today will just change because of AI.
211
00:47:49,000 --> 00:48:03,000
So just thinking like, AI doesn't involve me, I don't need to do anything. I think that's a bad way to do it but rather be interested in it.
212
00:48:03,000 --> 00:48:13,000
Learn to use it in your work. I think that's the best way.
213
00:48:13,000 --> 00:48:22,000
When I think, I think it's a topic for a dollar to work or like I'm sitting in an office.
214
00:48:22,000 --> 00:48:31,000
But what do you think which impact has AI on the front workers?
215
00:48:31,000 --> 00:48:34,000
Sorry, I didn't get you a question.
216
00:48:34,000 --> 00:48:43,000
Yeah, we have, we are not allowed to work on sitting in front of a PC and have the AI stuff.
217
00:48:43,000 --> 00:48:49,000
But there are also a lot of fields on the front there. I don't know.
218
00:48:49,000 --> 00:48:59,000
Working not direct on the PC. Did you think also AI will impact their jobs or how they work in the future?
219
00:48:59,000 --> 00:49:07,000
It most likely will. Of course now AI is mostly as you say for knowledge workers and developers.
220
00:49:07,000 --> 00:49:14,000
But we will see that there are developments in robotics, for example.
221
00:49:14,000 --> 00:49:21,000
So sometime you will see robots entering the workforce probably as well.
222
00:49:21,000 --> 00:49:38,000
And that will also be something that might change how the front workers are working with AI to get kind of maybe more type of hardware tools that are using AI or something.
223
00:49:38,000 --> 00:49:47,000
But it's not what I am working mostly with. So that's not where I am most of my experience.
224
00:49:47,000 --> 00:49:59,000
But I'm sure also things like using voice is something that is changing a lot on how we use AI and systems.
225
00:49:59,000 --> 00:50:06,000
So I think also that's something that for people that are working on their mobile phone a lot in the front.
226
00:50:06,000 --> 00:50:16,000
Instead of having to actually type in a thing they can actually talk to their home and get good answers from their AI is something that we also see.
227
00:50:16,000 --> 00:50:20,000
Good development on in the next months.
228
00:50:20,000 --> 00:50:29,000
And I think another big big change that's coming. It's it's could go differently. But yeah.
229
00:50:29,000 --> 00:50:35,000
Not mostly we talked about large language models where we talk about AI.
230
00:50:35,000 --> 00:50:42,000
What's your perspective on the this model of specialized language models?
231
00:50:42,000 --> 00:50:56,000
So when we are thinking about for example you know some ancient is most likely a small model and then it's starting to become quite good.
232
00:50:56,000 --> 00:51:04,000
When we're thinking about more open weights models that can run locally on PC or Mac.
233
00:51:04,000 --> 00:51:18,000
Then there is some good alternatives there but I would say they are not quite there yet. Maybe the best models that I'm using myself is the Gwen models from China.
234
00:51:18,000 --> 00:51:22,000
They have quite good models.
235
00:51:22,000 --> 00:51:36,000
And they can do certain things locally. But it's it's mostly if you create something specialist for them to do.
236
00:51:36,000 --> 00:51:50,000
And thinking more of how can you embed the small a models into existing applications running on your machine.
237
00:51:50,000 --> 00:52:05,000
So I think it's first of all up to the application developers to kind of embed those possibilities into their existing software.
238
00:52:05,000 --> 00:52:18,000
But of course we will probably see also with updates from both Microsoft and others that there will come more in the future.
239
00:52:18,000 --> 00:52:23,000
So I think some branding things are locally.
240
00:52:23,000 --> 00:52:36,000
Awesome. And I have in every interview a quick fire out. I shot some short questions from fast answers. So are you ready?
241
00:52:36,000 --> 00:52:39,000
Oh, so that's right.
242
00:52:39,000 --> 00:52:42,000
Prompt engineer context engineering.
243
00:52:42,000 --> 00:52:48,000
And co-pilot studio are custom developments.
244
00:52:48,000 --> 00:52:50,000
Co-pilot studio.
245
00:52:50,000 --> 00:52:53,000
A pilot studio or a I foundry.
246
00:52:53,000 --> 00:52:55,000
Yeah, I found it.
247
00:52:55,000 --> 00:52:58,000
Local or pro code.
248
00:52:58,000 --> 00:53:02,000
Pro code.
249
00:53:02,000 --> 00:53:08,000
What is the most underrated co pilot capability?
250
00:53:08,000 --> 00:53:12,000
Why smoke?
251
00:53:12,000 --> 00:53:18,000
When sat down at other can still say, you get all the money resources you want to make co pilot better.
252
00:53:18,000 --> 00:53:23,000
What feature will you develop?
253
00:53:23,000 --> 00:53:37,000
Better at coding.
254
00:53:37,000 --> 00:53:43,000
One is take every company makes on co pilot elections.
255
00:53:43,000 --> 00:53:48,000
Not giving an adoption to the users at all.
256
00:53:48,000 --> 00:53:54,000
Okay.
257
00:53:54,000 --> 00:53:58,000
I'm not quite sure.
258
00:53:58,000 --> 00:54:03,000
I have to take about that.
259
00:54:03,000 --> 00:54:06,000
Okay.
260
00:54:06,000 --> 00:54:09,000
Yeah.
261
00:54:09,000 --> 00:54:24,000
And the final question is, I imagine I am a CEO and organization. I know 10,000 employees.
262
00:54:24,000 --> 00:54:29,000
Different departments are experimenting with co pilot students starting building agents.
263
00:54:29,000 --> 00:54:32,000
Management doesn't know asking me.
264
00:54:32,000 --> 00:54:36,000
What is our AI workplace strategy?
265
00:54:36,000 --> 00:54:39,000
What is our AI strategy?
266
00:54:39,000 --> 00:54:45,000
What is the most important thing about the co pilot experience?
267
00:54:45,000 --> 00:54:51,000
What is the most important thing about the co pilot experience?
268
00:54:51,000 --> 00:54:57,000
What is the most important thing about the co pilot experience?
269
00:54:57,000 --> 00:55:12,000
What is the most important thing about the co pilot experience?
270
00:55:12,000 --> 00:55:30,000
What is the most important thing about the co pilot experience?
271
00:55:30,000 --> 00:55:48,000
What is the most important thing about the co pilot experience?
272
00:55:48,000 --> 00:56:06,000
What is the most important thing about the co pilot experience?
273
00:56:06,000 --> 00:56:24,000
What is the most important thing about the co pilot experience?
274
00:56:24,000 --> 00:56:42,000
What is the most important thing about the co pilot experience?
275
00:56:42,000 --> 00:57:00,000
What is the most important thing about the co pilot experience?
276
00:57:00,000 --> 00:57:18,000
What is the most important thing about the co pilot experience?
277
00:57:18,000 --> 00:57:36,000
Thank you, Christopher.
278
00:57:36,000 --> 00:57:37,000
that episode.
279
00:57:37,000 --> 00:57:38,000
All right.
280
00:57:38,000 --> 00:57:39,240
[END PLAYBACK]
00:00:00,000 --> 00:00:04,000
Welcome everybody to the MC65 at the end podcast.
2
00:00:04,000 --> 00:00:11,000
Microsoft 365 co-pilot has moved beyond the early questions of what can
3
00:00:11,000 --> 00:00:15,000
Generative AI do? The much harder question are now.
4
00:00:15,000 --> 00:00:18,000
How do we actually get thousands of employees to use it?
5
00:00:18,000 --> 00:00:21,000
How do we prove that it's creating value?
6
00:00:21,000 --> 00:00:25,000
When should we stand up Microsoft 365 co-pilot?
7
00:00:25,000 --> 00:00:33,000
When should we build an agent and when should we extend co-pilot with your own application and data?
8
00:00:33,000 --> 00:00:41,000
And perhaps most importantly, how do we move from interesting AI experiment to an
9
00:00:41,000 --> 00:00:43,000
Actually AI powered workplace?
10
00:00:43,000 --> 00:00:49,000
My guest today is Christopher Basil Hansen, Microsoft MVP and head of AI workplaces at
11
00:00:49,000 --> 00:00:57,000
Atea group. He leads Atea workplace AI strategy across the group and focused heavily on
12
00:00:57,000 --> 00:01:02,000
Microsoft say 65 co-pilot, co-pilot, studio and practical enterprise AI adoption.
13
00:01:02,000 --> 00:01:10,000
In 2026 Christopher announced that he had become an dual MVP across Microsoft co-pilot and
14
00:01:10,000 --> 00:01:16,000
co-pilot expansibility. So today we are going beyond demos.
15
00:01:16,000 --> 00:01:23,000
We are talking about co-pilot adoption, organizational change, measuring productivity and so on.
16
00:01:23,000 --> 00:01:30,000
So Christopher, welcome to the MC65 podcast and welcome to the MVP title.
17
00:01:30,000 --> 00:01:36,000
Thank you, Amitko. Very nice to be here looking forward to our conversation today.
18
00:01:36,000 --> 00:01:44,000
Yeah. Because before we get into co-pilot, can you tell us a little bit about your journey into
19
00:01:44,000 --> 00:01:46,000
Microsoft technology?
20
00:01:46,000 --> 00:01:57,000
Yeah, sure. So I'm 34 years old and 11 years ago I changed my job and started working at Atea.
21
00:01:57,000 --> 00:02:04,000
And one of the first thing I did then was to attend what was called the Microsoft University
22
00:02:04,000 --> 00:02:14,000
A six-week program where I then was learned in different Microsoft technologies. I remember still that the early days of
23
00:02:14,000 --> 00:02:23,000
Azure was one of the focus areas at that time. So from the start there I worked a lot with Office 365 and so on.
24
00:02:23,000 --> 00:02:28,000
And was kind of a junior consultant in those kind of things.
25
00:02:28,000 --> 00:02:36,000
And I had a few years where I worked at a device management. So I worked with the internal and also web
26
00:02:36,000 --> 00:02:46,000
solutions as well. Before I started working more and more with AI solutions when Apple chat
27
00:02:46,000 --> 00:02:57,000
GPT and co-pilot was not in 2022 and 2023. So since then there has been mostly AI in my focus areas.
28
00:02:57,000 --> 00:03:03,000
So I guess that's the short story.
29
00:03:03,000 --> 00:03:14,000
And AI workplace is that what is it in simply Microsoft 365 plus co-pilot or event to find our own?
30
00:03:14,000 --> 00:03:26,000
Of course Microsoft co-pilot is our main focus in our workplace but we are really responsible for how over 8,000 users are using AI in their daily
31
00:03:26,000 --> 00:03:35,000
work drive. And in some cases there can also be other solutions but in Atea all users that need it
32
00:03:35,000 --> 00:03:45,000
are have co-pilot available and we have a lot of focus on how to adopt the technology and use it in a good way.
33
00:03:45,000 --> 00:03:51,000
And so that's the main focus and of course over the years we have also started thinking more about
34
00:03:51,000 --> 00:04:07,000
the authentic AI solutions with co-pilot studio and Foundry and a few other things. So it's kind of it's a main focus around co-pilot but we also have a broad
35
00:04:07,000 --> 00:04:15,000
experimenting soul where we try out other solutions to stay on top of everything.
36
00:04:15,000 --> 00:04:27,000
When you look at your job today is how big is the path technology for those people, organization change and strategy?
37
00:04:27,000 --> 00:04:42,000
Of course but the technology is moving very quickly. It's new modules and new things every week but getting people to actually change your habits takes time.
38
00:04:42,000 --> 00:04:54,000
So working with people is one of what I'm trying to spend my time on to make sure that people feel safe in changing their habits,
39
00:04:54,000 --> 00:05:02,000
changing the tools that they are using. So we do a lot of training internally with user adoption.
40
00:05:02,000 --> 00:05:12,000
So we have had different types of training but we have at least every other week throughout the whole year we have some sort of training.
41
00:05:12,000 --> 00:05:22,000
We have for example sometimes we have a role-based training where we focus on how managers should use AI, how consultants should use AI, how sage people should use AI.
42
00:05:22,000 --> 00:05:36,000
And we also have a few other types of things like how to do a kind of beginners class and how to do a more advanced class to get to the next level of time.
43
00:05:36,000 --> 00:05:48,000
Awesome. Take a little bit back at the start of the Microsoft 365 co-pilot. What were organizations expecting from co-pilot?
44
00:05:48,000 --> 00:06:02,000
I think a lot of organizations were expecting a lot from the very start and that's probably a bit because of the marketing from Microsoft because I remember since before it was even available.
45
00:06:02,000 --> 00:06:18,000
And I think the download like co-pilot would be the perfect AI assistant that could do everything for you. And of course then people are a bit not happy when they are noticing that it comes and I made for them.
46
00:06:18,000 --> 00:06:44,000
So I think up until March this year when we got co-work I think the promise of an AI assistant that could do everything for you were not delivered but with things like co-work it's starting to get closer to it that you can actually work together with it and it can actually do things on behalf of you.
47
00:06:44,000 --> 00:06:48,000
And that's a new way of working for a lot of people.
48
00:06:48,000 --> 00:07:04,000
Awesome. Let's get a little bit practical. Let's say a company has, I don't know, 5,000 Microsoft co-pilot, it's in what happens next?
49
00:07:04,000 --> 00:07:18,000
Well, it depends where they start to, I guess. But usually when you purchase a lot of licenses you would have to think about what's the gold here, what's the scratchy.
50
00:07:18,000 --> 00:07:34,000
Hopefully they will do some sort of user adoption. I still see a lot of customers just purchasing licenses and expecting people to learn it themselves and I don't believe that works for any of them.
51
00:07:34,000 --> 00:07:50,000
I think you need to spend time and resources to make sure that your users are feeling safe to adopt the new technology. These kind of tools change so often as well.
52
00:07:50,000 --> 00:07:54,000
Every week you will see some change within the co-pilot app.
53
00:07:54,000 --> 00:08:08,000
And you can't expect users to follow that themselves because they have a lot of other things to do as well. So you need some experts either internally or externally, but you're
54
00:08:08,000 --> 00:08:18,000
going to buy their time from and help you out to do the user adoption. I think that's number one or what you need to do when getting the licenses.
55
00:08:18,000 --> 00:08:36,000
And then of course you need to think about how do we actually measure what value we get out of this because up until now I think a lot of organizations have measured it by just looking at how much you're using AI.
56
00:08:36,000 --> 00:08:46,000
Because it's hard to measure for many organizations because they usually don't have good measurements from beforehand on what are you delivering.
57
00:08:46,000 --> 00:08:58,000
So I think you need to kind of think through what's important for us as a company, what are we delivering to our customers or whoever we are delivering something to.
58
00:08:58,000 --> 00:09:08,000
How can we measure either how to do things faster, how to improve our quality, etc.
59
00:09:08,000 --> 00:09:18,000
Actually be able to measure those things on how do we do it without AI and how do we do it with AI.
60
00:09:18,000 --> 00:09:38,000
And I would say or actually I know because we did a report on this with our CIO analytics with 1500 companies in North and Europe earlier this year where we ask like the directors in this business.
61
00:09:38,000 --> 00:09:50,000
I've done so many full things with AI. I've even in North Europe 80% said that they have done something that gives a lot of value to their company with AI.
62
00:09:50,000 --> 00:10:04,000
Also one third of the companies say that they are not measuring AI at all. So they're just buying AI and using it but they are not measuring any outcomes for it.
63
00:10:04,000 --> 00:10:22,000
What are good KPIs to measure the useful usage of co-pilot or especially or all AI. Is there any KPIs I should focus on?
64
00:10:22,000 --> 00:10:34,000
I think that's the hard question that the O-Copnets need to ask themselves because I don't believe there are one easy answer that covers all companies.
65
00:10:34,000 --> 00:10:50,000
So I think it depends on what's important for you as a company, what are you delivering, what's the thing that you really either earn money on or whatever you do that you can actually measure.
66
00:10:50,000 --> 00:11:06,000
It can be of course one easy measurement that we are also doing internally for a long time is that we ask our users what they think they are changing by using AI.
67
00:11:06,000 --> 00:11:14,000
But that's not a real measurement. More just like a feeling from each user that maybe they feel like they are saving some time.
68
00:11:14,000 --> 00:11:36,000
They feel like there are being more performance, they feel like the quality is going up. But if you can also see that someone, I remember someone internally said to me that he is able to deliver on six cases at the same time instead of just two which he was able to do before.
69
00:11:36,000 --> 00:11:48,000
He also receives feedback from the customers that the quality is even better than before. So he receives very good feedback on all those six cases.
70
00:11:48,000 --> 00:11:58,000
I think that's a good measurement that you are actually doing more and you are even getting more quality out of each of those cases.
71
00:11:58,000 --> 00:12:06,000
But you get the pens very much on what kind of systems you have in place from before.
72
00:12:06,000 --> 00:12:26,000
And how did you start, I say, motivating people use Copilot, what's the focus on the training or what did you see that make up the moment where I was kicked in, oh Copilot is true.
73
00:12:26,000 --> 00:12:38,000
I think the good thing with the AI is that it's been so much hype around it since the very start that everyone wanted to use AI.
74
00:12:38,000 --> 00:12:52,000
And it's always been like you need to apply to get the license, you need to apply to receive it. So then users actually have to do something to get it.
75
00:12:52,000 --> 00:13:02,000
And that usually also results in people being a bit more engaged in it when they get it. They may be used some extra time to learn it.
76
00:13:02,000 --> 00:13:14,000
But what we saw in the start when we didn't use the option around is that you had about four weeks where they were very engaged in it and tried a lot of things.
77
00:13:14,000 --> 00:13:28,000
But when they noticed that they weren't really able to change their habits, they weren't able to improve that much with it, then they stopped using it.
78
00:13:28,000 --> 00:13:43,000
And after we started delivering training in early 2025, we noticed that people actually used it over time and didn't stop using it.
79
00:13:43,000 --> 00:13:57,000
And people are asking for more and more training because they feel like they need it because the products and the technology behind it is changing so rapidly.
80
00:13:57,000 --> 00:14:12,000
Yeah, it's, yeah, they're re-roading it's coming every time it's coming something new that's amazing.
81
00:14:12,000 --> 00:14:36,000
Yeah, it's, yeah, we talk about, I say time saving, what are the biggest time saving for companies? Is it meeting, is it email? Where did you see that the companies get more well you are out of their license?
82
00:14:36,000 --> 00:14:50,000
So I think in the start it was a lot about meetings because that was one of the very good features from the start with Gopalet that you get to try to meet things and get some good results with it.
83
00:14:50,000 --> 00:14:54,000
I'm sure people are still using that and say some time on it.
84
00:14:54,000 --> 00:15:15,000
But I also think that using it as a thinking partner is both saving time but also increasing quality. So what we are trying to do internally when we are using or say having user adoption is that we don't ask AI first, but you think first yourself.
85
00:15:15,000 --> 00:15:35,000
So you use your brain and think about what would you do in this case but instead of using a lot of time on it you kind of brainstorm through it together with AI maybe use dictation function on your computer and just say it out loud, give all their thoughts out.
86
00:15:35,000 --> 00:15:54,000
And then you ask the AI or call quality in this case to give you some feedback and structure it for you and then you have draft one and then you come back and go with some feedback again and you go a bit back and forth and you have something you're happy with.
87
00:15:54,000 --> 00:16:09,000
And I think that is all the time say we're something that increases the quality on what we need to be able to do if it's true a male or if it's true a presentation or something like that.
88
00:16:09,000 --> 00:16:38,000
And I think that also thinking about now is the research agent I remember when that came in in April May 25 that is one of the best agents I think that I've come up out in in co pilot time because it's able to think for a long time and also dig through that sources and help you find things that you maybe weren't able to do.
89
00:16:38,000 --> 00:17:00,000
Maybe weren't able to find from before so I have some examples it turn lever people really save a lot of times when especially when working with big projects that need a lot of resources and really need to find the correct information in a lot of documents.
90
00:17:00,000 --> 00:17:12,000
Then the research agent has been very available and it's also giving them a way to double check that it's actually correct as well.
91
00:17:12,000 --> 00:17:29,000
I think you say a big project that means a big amount of data how should companies prepare that data for co pilot that they get good results what was your tip.
92
00:17:29,000 --> 00:17:49,000
So of course it depends on where you are from before I think having some way of structuring your data is important whether it's in in each and every once one driver if it's in SharePoint and so on.
93
00:17:49,000 --> 00:18:13,000
But also hopefully everyone is using curvy or to the mark your data if it's confidential and what not and I think that's an important step from the security side both to what you allow co pilot to see.
94
00:18:13,000 --> 00:18:29,000
And also that the user are aware of what kind of information are I getting from co pilot now is it something that could be confidential that I shouldn't share directly with external people etc.
95
00:18:29,000 --> 00:18:57,000
So I think that's also an important part of it and also having retention policies to clean up all files all folders that are no longer in use because I think if you just give AI a lot of information out the information that will be based to give wrong answers.
96
00:18:57,000 --> 00:19:17,000
So there aren't really no easy answer to it because it all depends on where each company is from before but nonetheless it's a very important thing to prioritize to get a good data structure.
97
00:19:17,000 --> 00:19:35,000
And one example I have of this is when we created one of our orchestrating agents but we have a different orchestrating systems internally where we have one agent using a number of sub agents to give answers to the people.
98
00:19:35,000 --> 00:19:59,000
And when we are allowing more people to use that kind of agent system the way this agent system is working is that it's connecting to agents that have a lot of a lot of data for customers for example.
99
00:19:59,000 --> 00:20:25,000
But we are not accepting those customer agents in before they have a good data structure so they each account management need to clean up the data for their customer and then we can create an agent out of it and then we also have agents connecting to CRM and other type of systems so it can work together with different type of systems and give craft information to those that are used in the agent system.
100
00:20:25,000 --> 00:20:35,000
And that's one way to do it that's to be able to create an agent for your customer or whatever you need to first clean up your data.
101
00:20:35,000 --> 00:20:45,000
Yeah I think yeah agents that are new you are to do.
102
00:20:45,000 --> 00:21:03,000
So let's move from co-pilot more into agents when should organizations stop thinking how do we use co-pilot and start thinking we need an agent or an agent.
103
00:21:03,000 --> 00:21:21,000
I think you should always kind of yes I think all organizations should be thinking about how can we use agents and start also in their users to think about what can be create agents for.
104
00:21:21,000 --> 00:21:37,000
And then you will just start creating a lot of agents maybe hundreds or thousands agents that maybe don't give any value at all but it changes the mindset for people to to think about what an agent can do.
105
00:21:37,000 --> 00:21:49,000
Then of course by the sometime you will be able to also deliver on more advanced agents so in the start most people are just creating agents that give information.
106
00:21:49,000 --> 00:22:06,000
But when you have kind of figured that out in a good way and you create some agents that give you the information then maybe the next step is to create agents that actually do things as well maybe that is send an email send a message or update CRM systems.
107
00:22:06,000 --> 00:22:19,000
Or whatever and then the next step would maybe then be a more autonomous agent that can work besides you as co-worker or something like that.
108
00:22:19,000 --> 00:22:30,000
So I think there are multiple steps there and that is important to kind of go through for each organization.
109
00:22:30,000 --> 00:22:40,000
And here we have co-pilot I say okay, not for that team but for the people it's out of the box too.
110
00:22:40,000 --> 00:22:58,000
So you also have co-pilot studio when people are ready to use co-pilot studio and yeah where do co-pilot studio fit into the AI workplace architecture from your perspective.
111
00:22:58,000 --> 00:23:10,000
So we have had the co-pilot studio available for all users in ATA for a long time almost since the start of it.
112
00:23:10,000 --> 00:23:19,000
But we have of course noticed that those that you see there are mostly people with a technical background.
113
00:23:19,000 --> 00:23:29,000
Because the user interface and everything is a bit more technical than what everyone wants to use.
114
00:23:29,000 --> 00:23:41,000
So in ATA and most of our customers I see most users are still using the agent builder directly in co-pilot to build agents.
115
00:23:41,000 --> 00:23:48,000
And then the more technical people are using co-pilot studio to build a bit more advanced agents.
116
00:23:48,000 --> 00:23:54,000
And I think both solutions can be a good start point.
117
00:23:54,000 --> 00:24:09,000
What we also see in some cases is that people start with agent builder and if they are not taking for themselves to go into co-pilot studio then we might say to someone more technical that ability is agent now.
118
00:24:09,000 --> 00:24:17,000
Understand the way of it but I need some help to actually make it work fully in co-pilot studio.
119
00:24:17,000 --> 00:24:34,000
And now with the recent changes since a couple of weeks ago with Git-to-parnas in co-pilot studio I think that's also starting to become a good way to build agents running there as well.
120
00:24:34,000 --> 00:24:43,000
And then addition to that we also have Microsoft Founder where you can do even more advanced agents.
121
00:24:43,000 --> 00:24:51,000
So we have a few also agent systems running there as well.
122
00:24:51,000 --> 00:25:04,000
I think a little bit how we actually use or most people use AI for answers.
123
00:25:04,000 --> 00:25:10,000
But I think the change is to become getting actions.
124
00:25:10,000 --> 00:25:25,000
I have an HR agent answering how many vacation days do you have is one thing but yeah, book next Friday as vacation is more advanced.
125
00:25:25,000 --> 00:25:34,000
What changes architectural when agents start taking actions?
126
00:25:34,000 --> 00:25:44,000
So I think that will slowly change how we work and how we use our systems today.
127
00:25:44,000 --> 00:25:53,000
I can just see from my own experience using a lot of coding agents the last year.
128
00:25:53,000 --> 00:26:07,000
I started to do less in the user interfaces itself if it's either Microsoft Admin Center or Azure Admin Center and using more coding there instead.
129
00:26:07,000 --> 00:26:21,000
So I also have a coding agent that can do tasks for me directly in the Admin Center with some of course security guardrails around what it's allowed to do and not.
130
00:26:21,000 --> 00:26:33,000
But instead of having to do it myself or having to message someone internally I can just message my agent and it will actually be able to do everything.
131
00:26:33,000 --> 00:26:42,000
First is a test plan if I want to do that and then in the production environments.
132
00:26:42,000 --> 00:27:03,000
And I think in some time maybe in months or maybe in a year or two we will have more agent systems that kind of have a way into the systems that we are allowed to do things instead of using or having to click around and do everything ourselves.
133
00:27:03,000 --> 00:27:16,000
So for example with with HR instead of just asking how many vacation days do I have left and then having to peel off the form yourself your agent could in theory do this for you.
134
00:27:16,000 --> 00:27:20,000
And that's already working for some companies today.
135
00:27:20,000 --> 00:27:29,000
But that is something I think will be more available in the coming months from now.
136
00:27:29,000 --> 00:27:39,000
Or what is with I say risky actions I don't know my neighbor in the office is so loud.
137
00:27:39,000 --> 00:27:51,000
You can wait no joke and more like I don't know delete all delete all emails or I don't know and you don't describe it right.
138
00:27:51,000 --> 00:28:03,000
And it's from from project I actually have those something like this how can we prevent from from these wrist.
139
00:28:03,000 --> 00:28:20,000
So that's a real thing to think about the security around it because we have seen examples of people using for example coding agents and certainly the coding agent has deleted everything on their local hardware.
140
00:28:20,000 --> 00:28:37,000
So having some guardrails around it what are you allowed to do what are you are not allowed to do having the agent ask before doing something that can delete something for example.
141
00:28:37,000 --> 00:28:50,000
And I think first of all we need good harnesses so so a harness could be for example you get a couple of that or code X from open or something like that.
142
00:28:50,000 --> 00:28:58,000
And we didn't let harness you usually can set what kind of permission you give the agent that are doing task for you.
143
00:28:58,000 --> 00:29:06,000
So whenever it's doing a coding task should ask you first or what is pre approved.
144
00:29:06,000 --> 00:29:18,000
So that's kind of where you need to start that what are you actually allowing the agent to do without asking you and what the station need to ask you before.
145
00:29:18,000 --> 00:29:35,000
And also thinking about if you are actually give it some admin rights for example thinking about please privilege possible that you don't give it you know glad me and let it run wild in in return.
146
00:29:35,000 --> 00:29:54,000
So yeah I think a lot of people can build and say okay I build their own copilot for their own process but it's also built copilot or agents like systems for the whole company.
147
00:29:54,000 --> 00:30:17,000
What have companies to check or to reach out when they build complex systems AI systems for I don't know for processing with a lot of I don't know two or a thousand or thousands of people to use what's the other the shift.
148
00:30:17,000 --> 00:30:37,000
So I think you need some sort of way as a type of intake when people are creating agents that can be run or used by a lot of people you need to some all to take on that agent.
149
00:30:37,000 --> 00:30:48,000
What kind of instructions does the agent have what kind of access does it have what kind of connectors if any does it have if it needs any integration to an system.
150
00:30:48,000 --> 00:31:00,000
So you need to go and do a quality control on all of those before you actually allow it to be run inside the company by many users.
151
00:31:00,000 --> 00:31:24,000
And so that's kind of step one and then you probably also need some sort of people that can if that agent doesn't meet that quality control you either of course had it back to the user and they need to redo it or maybe you should think about how can we build agents for our own company.
152
00:31:24,000 --> 00:31:44,000
So when we take the the IDs from users in to the intake and then we plan what we need to do to make this agent off and then we implement that with a type of agent creating agents and bubbling agents.
153
00:31:44,000 --> 00:31:59,000
So we have co pilot co pilot studio co pilot work work a few I don't know it's actually called and what what how fits co pilot extensibility in in this picture.
154
00:31:59,000 --> 00:32:13,000
So I think co pilot extensibility is kind of multiple things of course you have co pilot studio as one kind of way to think of an existence to build there and then you also have the agent 365 SDK.
155
00:32:13,000 --> 00:32:37,000
So and then in this that you also have a kind of connectors or plug in that you can connect with co pilot one of the things I've been looking into the case the agent 365 SDK where you can kind of build your own agents that have their own identity in the end.
156
00:32:37,000 --> 00:32:53,000
So one of the projects I've been working on and this summer is to kind of create a personal agent that's in the organization tree looks like it's kind of an employee below me.
157
00:32:53,000 --> 00:33:11,000
So people can actually see it in the organization tree it has an email address a team address people can message it and it will be able to have certain permissions set by me so it can actually answer other people.
158
00:33:11,000 --> 00:33:31,000
So I can for example then if I am away from work in a few days and people need some monsters for me it will have access to certain things that it is allowed to answer internal people and certain things it's allowed to answer external people.
159
00:33:31,000 --> 00:33:59,000
And it will also work as a personal assistant for me. So so of course a bit like co pilot or co work but something that making myself and when making this in agent 365 SDK it's also made as a type of what they call a blueprint or a template.
160
00:33:59,000 --> 00:34:07,000
So I can set this up for myself but I can also set it up for as many people in the organization as I want.
161
00:34:07,000 --> 00:34:27,000
So and I'm also creating a dashboard to it so I can access or allow it to access things from work I cure from from my user based on what I think is okay or not.
162
00:34:27,000 --> 00:34:45,000
So I think that's at least in my mind one of the most interesting things right now from the accessibility scientist to look into the agent 365 SDK at creating those type of agents.
163
00:34:45,000 --> 00:35:07,000
And what can I connect is it only Microsoft internal stuff or what can I connecting I don't know says what can I connect I don't know what size what possibilities are about the label.
164
00:35:07,000 --> 00:35:31,000
So I think pretty much anything is possible because it's I'm doing it by code and I'm using right now foundry models to run it so I'm using right now I'm using a deep seek model behind it but I'm also using a G586 model.
165
00:35:31,000 --> 00:35:45,000
You can really connect of course whatever data sources you have available within your Azure or Microsoft 365 environment.
166
00:35:45,000 --> 00:35:57,000
And so everything is possible but of course out of the box it's mostly a mobs of 365 and then you can kind of extend from there.
167
00:35:57,000 --> 00:36:09,000
Awesome. Have you an example or what you have built you are really proud of.
168
00:36:09,000 --> 00:36:38,000
And I think one of the first things I built was a dashboard to look at the usage of co palette in Berlin or there so most people working co palette has probably seen the kind of the evil dashboards for Microsoft which works fine it gives you a kind of view on how co palette is being used internally.
169
00:36:38,000 --> 00:36:54,000
But after we got access to kind of the the Biva premium licenses last year you could also go into the Viva analyst site and download the statistics of all the users in the company as a CSV file.
170
00:36:54,000 --> 00:37:05,000
And when I did that it resulted in a file with 90 columns and 300,000 lines or rose in Excel.
171
00:37:05,000 --> 00:37:19,000
So I made a dashboard where I could just input that and generate a lot of graphs based on what I wanted to see.
172
00:37:19,000 --> 00:37:34,000
So that's one of the things that I built that I was made the open source in this direct that I built a couple of websites to follow up on Microsoft news because.
173
00:37:34,000 --> 00:37:48,000
So after sometimes you kind of noticed that looking into the admin center messaging center you see that there are always something new every day on what's coming for co palette.
174
00:37:48,000 --> 00:38:11,000
And instead of having to login I created a simple site to find a view it and I'm also added the Microsoft 365 roadmap on that as well and then I created an iOS application that kind of gives me the same view that I called the tenant balls that is also freely our.
175
00:38:11,000 --> 00:38:34,000
So I actually get the notification every hour if there are any news from the admin center messaging so a few of the things that have been creating myself after kind of starting to use more and more coding tools that I lost you.
176
00:38:34,000 --> 00:38:57,000
And I think a lot of people actually speak about one topic multiple autonomous agent how many agents will we will we have in the future then on the top and autonomous they are really can go.
177
00:38:57,000 --> 00:39:20,000
Yeah so I think we will have a lot of agents and most of them are probably not that the agents either so for example in until we have a thousand employees but we have a thousand agents so already have more agents than employees but of course all of those agents are not being used to be.
178
00:39:20,000 --> 00:39:49,000
And maybe actually in hundred agents being used every month so of course those 11,000 agents are agents people you created one time maybe they got some value out of it and then they stopped using it and then it's it's left there to die so of course that's also another thing that I'm spending one time on with agent 365 is to kind of think about the lifecycle policy about agents.
179
00:39:49,000 --> 00:40:18,000
If you're not using an agent for maybe 30 6090 days and it's maybe time for to kind of delete that agent and to clean up from it to avoid going back to 2020 when everyone created a team because you started a new project and you were left with thousands of teams that didn't get out.
180
00:40:18,000 --> 00:40:34,000
And what I also think of the lifecycle type is it's very important topic also all the governance stuff but I think another interesting topic is shadow IT.
181
00:40:34,000 --> 00:40:49,000
So I have clients they say oh we don't use AI because then it's very good doing yeah it's the said to be then they bought their own chat GPT and you have also shadow IT.
182
00:40:49,000 --> 00:40:58,000
But yeah and what is your view here on the shadow IT topic I think a lot of companies are aware of this actually.
183
00:40:58,000 --> 00:41:13,000
Yeah I think shadow AI is is very real and I think it's hard to just say no to users to say you are not allowed to use it.
184
00:41:13,000 --> 00:41:26,000
We have tried that for many years in all types of companies and whenever a user gets a know they just do it on their personal device and way and they probably upload some files.
185
00:41:26,000 --> 00:41:41,000
But I think the most important thing to do is that the users are getting tools that are as good as possible to kind of combat them using a shadow AI tools.
186
00:41:41,000 --> 00:41:53,000
But on the other hand I think it's also important that people are allowed to experiment with different tools of course in a safe way.
187
00:41:53,000 --> 00:42:18,000
Even if that means people using non-approved tools like for example codecs or cloud code and so on but having to use it in a safe way maybe on their personal computer not allowed to use of course in turn your customer data in the tools but more to experiment what's possible in the other type of tools.
188
00:42:18,000 --> 00:42:28,000
So using their proved IT tools when working with that.
189
00:42:28,000 --> 00:42:41,000
I think in the other topic I often hear it's it's yeah I can find ups find ups but because at the start co pilot cost I don't know US $20 or so.
190
00:42:41,000 --> 00:42:49,000
It was really clear but it's now we have token tokens bands and it's on and different models with different costs.
191
00:42:49,000 --> 00:42:56,000
How can we find ups the co pilot and how are we handling this.
192
00:42:56,000 --> 00:43:11,000
Yeah so I think the final is also a very important topic moving forward especially after get the co pilot starting costing real money after June and also with co-or costing co pilot credits.
193
00:43:11,000 --> 00:43:25,000
I see this internally that the limits we have set in both co-work and also in get the co pilot is there is a lot of users.
194
00:43:25,000 --> 00:43:38,000
Getting to that limit very quickly and we need to think about what kind of value are we getting out of all the money we are spending on this.
195
00:43:38,000 --> 00:43:48,000
We are of course thinking about how can we do better use reduction towards what do you do in which tool.
196
00:43:48,000 --> 00:44:05,000
For example co work creating PowerPoint presentations is something co work is good at but it's also costing a lot of money but you can also do the same almost get the same result by using co pilot inside of.
197
00:44:05,000 --> 00:44:19,000
So for example using the correct tool at the correct time is one thing to think about and also in coding instances.
198
00:44:19,000 --> 00:44:34,000
Luckily we are getting good competition from the Chinese models which are open weights and a way to kind of run on local server hardware.
199
00:44:34,000 --> 00:44:52,000
And that makes it for example you have seen with open AI when they launched their GPT56 models with sold data and Luna then one or two weeks after they launched it they also changed the pricing model on the two cheapest models.
200
00:44:52,000 --> 00:45:11,000
So for example with the Luna model which is the cheapest and smallest model they reduce the price with 80%. But the model itself is actually quite good as well in a lot of our coding instances.
201
00:45:11,000 --> 00:45:23,000
We are now using Luna as one of the primary models because it's very good at executing tasks. It's not necessarily that good at planning things.
202
00:45:23,000 --> 00:45:35,000
So maybe use a kind of frontier model to do all the planning and then use the cheaper models to do execute all the plans that we have made together with the frontier model.
203
00:45:35,000 --> 00:45:52,000
And learning people to kind of think this way can be hard. And so that's kind of where we have all to use the option but also we need to set some boundaries about what are your love to use.
204
00:45:52,000 --> 00:46:17,000
What did you think people often have fears about losing losing their jobs. So what show people do to stay relevant in the age of AI which skills that the people need.
205
00:46:17,000 --> 00:46:34,000
So again I think it depends very much on what you're doing from before. I think there are still a lot of things that will still be very much relevant within the age of AI.
206
00:46:34,000 --> 00:47:01,000
One year ago there was a lot of people that you would no longer need developers because AI has been getting so good at coding. But I think now if you ask people you all people all orientations need developers now because you need some people to kind of stare these agents around and whenever you can
207
00:47:01,000 --> 00:47:17,000
just look at the developers, you just want more developers to kind of look over it all. So I think it's hard to say how it will evolve in the next two to five years.
208
00:47:17,000 --> 00:47:30,000
I think being an experimenter and testing out the solutions, finding out how does AI work in what I'm good at, how can I become even better at what I do with AI.
209
00:47:30,000 --> 00:47:40,000
I think it's the best way to stay relevant because of course some jobs will become because of AI.
210
00:47:40,000 --> 00:47:49,000
And there will be creating new jobs because of AI and some of the current jobs today will just change because of AI.
211
00:47:49,000 --> 00:48:03,000
So just thinking like, AI doesn't involve me, I don't need to do anything. I think that's a bad way to do it but rather be interested in it.
212
00:48:03,000 --> 00:48:13,000
Learn to use it in your work. I think that's the best way.
213
00:48:13,000 --> 00:48:22,000
When I think, I think it's a topic for a dollar to work or like I'm sitting in an office.
214
00:48:22,000 --> 00:48:31,000
But what do you think which impact has AI on the front workers?
215
00:48:31,000 --> 00:48:34,000
Sorry, I didn't get you a question.
216
00:48:34,000 --> 00:48:43,000
Yeah, we have, we are not allowed to work on sitting in front of a PC and have the AI stuff.
217
00:48:43,000 --> 00:48:49,000
But there are also a lot of fields on the front there. I don't know.
218
00:48:49,000 --> 00:48:59,000
Working not direct on the PC. Did you think also AI will impact their jobs or how they work in the future?
219
00:48:59,000 --> 00:49:07,000
It most likely will. Of course now AI is mostly as you say for knowledge workers and developers.
220
00:49:07,000 --> 00:49:14,000
But we will see that there are developments in robotics, for example.
221
00:49:14,000 --> 00:49:21,000
So sometime you will see robots entering the workforce probably as well.
222
00:49:21,000 --> 00:49:38,000
And that will also be something that might change how the front workers are working with AI to get kind of maybe more type of hardware tools that are using AI or something.
223
00:49:38,000 --> 00:49:47,000
But it's not what I am working mostly with. So that's not where I am most of my experience.
224
00:49:47,000 --> 00:49:59,000
But I'm sure also things like using voice is something that is changing a lot on how we use AI and systems.
225
00:49:59,000 --> 00:50:06,000
So I think also that's something that for people that are working on their mobile phone a lot in the front.
226
00:50:06,000 --> 00:50:16,000
Instead of having to actually type in a thing they can actually talk to their home and get good answers from their AI is something that we also see.
227
00:50:16,000 --> 00:50:20,000
Good development on in the next months.
228
00:50:20,000 --> 00:50:29,000
And I think another big big change that's coming. It's it's could go differently. But yeah.
229
00:50:29,000 --> 00:50:35,000
Not mostly we talked about large language models where we talk about AI.
230
00:50:35,000 --> 00:50:42,000
What's your perspective on the this model of specialized language models?
231
00:50:42,000 --> 00:50:56,000
So when we are thinking about for example you know some ancient is most likely a small model and then it's starting to become quite good.
232
00:50:56,000 --> 00:51:04,000
When we're thinking about more open weights models that can run locally on PC or Mac.
233
00:51:04,000 --> 00:51:18,000
Then there is some good alternatives there but I would say they are not quite there yet. Maybe the best models that I'm using myself is the Gwen models from China.
234
00:51:18,000 --> 00:51:22,000
They have quite good models.
235
00:51:22,000 --> 00:51:36,000
And they can do certain things locally. But it's it's mostly if you create something specialist for them to do.
236
00:51:36,000 --> 00:51:50,000
And thinking more of how can you embed the small a models into existing applications running on your machine.
237
00:51:50,000 --> 00:52:05,000
So I think it's first of all up to the application developers to kind of embed those possibilities into their existing software.
238
00:52:05,000 --> 00:52:18,000
But of course we will probably see also with updates from both Microsoft and others that there will come more in the future.
239
00:52:18,000 --> 00:52:23,000
So I think some branding things are locally.
240
00:52:23,000 --> 00:52:36,000
Awesome. And I have in every interview a quick fire out. I shot some short questions from fast answers. So are you ready?
241
00:52:36,000 --> 00:52:39,000
Oh, so that's right.
242
00:52:39,000 --> 00:52:42,000
Prompt engineer context engineering.
243
00:52:42,000 --> 00:52:48,000
And co-pilot studio are custom developments.
244
00:52:48,000 --> 00:52:50,000
Co-pilot studio.
245
00:52:50,000 --> 00:52:53,000
A pilot studio or a I foundry.
246
00:52:53,000 --> 00:52:55,000
Yeah, I found it.
247
00:52:55,000 --> 00:52:58,000
Local or pro code.
248
00:52:58,000 --> 00:53:02,000
Pro code.
249
00:53:02,000 --> 00:53:08,000
What is the most underrated co pilot capability?
250
00:53:08,000 --> 00:53:12,000
Why smoke?
251
00:53:12,000 --> 00:53:18,000
When sat down at other can still say, you get all the money resources you want to make co pilot better.
252
00:53:18,000 --> 00:53:23,000
What feature will you develop?
253
00:53:23,000 --> 00:53:37,000
Better at coding.
254
00:53:37,000 --> 00:53:43,000
One is take every company makes on co pilot elections.
255
00:53:43,000 --> 00:53:48,000
Not giving an adoption to the users at all.
256
00:53:48,000 --> 00:53:54,000
Okay.
257
00:53:54,000 --> 00:53:58,000
I'm not quite sure.
258
00:53:58,000 --> 00:54:03,000
I have to take about that.
259
00:54:03,000 --> 00:54:06,000
Okay.
260
00:54:06,000 --> 00:54:09,000
Yeah.
261
00:54:09,000 --> 00:54:24,000
And the final question is, I imagine I am a CEO and organization. I know 10,000 employees.
262
00:54:24,000 --> 00:54:29,000
Different departments are experimenting with co pilot students starting building agents.
263
00:54:29,000 --> 00:54:32,000
Management doesn't know asking me.
264
00:54:32,000 --> 00:54:36,000
What is our AI workplace strategy?
265
00:54:36,000 --> 00:54:39,000
What is our AI strategy?
266
00:54:39,000 --> 00:54:45,000
What is the most important thing about the co pilot experience?
267
00:54:45,000 --> 00:54:51,000
What is the most important thing about the co pilot experience?
268
00:54:51,000 --> 00:54:57,000
What is the most important thing about the co pilot experience?
269
00:54:57,000 --> 00:55:12,000
What is the most important thing about the co pilot experience?
270
00:55:12,000 --> 00:55:30,000
What is the most important thing about the co pilot experience?
271
00:55:30,000 --> 00:55:48,000
What is the most important thing about the co pilot experience?
272
00:55:48,000 --> 00:56:06,000
What is the most important thing about the co pilot experience?
273
00:56:06,000 --> 00:56:24,000
What is the most important thing about the co pilot experience?
274
00:56:24,000 --> 00:56:42,000
What is the most important thing about the co pilot experience?
275
00:56:42,000 --> 00:57:00,000
What is the most important thing about the co pilot experience?
276
00:57:00,000 --> 00:57:18,000
What is the most important thing about the co pilot experience?
277
00:57:18,000 --> 00:57:36,000
Thank you, Christopher.
278
00:57:36,000 --> 00:57:37,000
that episode.
279
00:57:37,000 --> 00:57:38,000
All right.
280
00:57:38,000 --> 00:57:39,240
[END PLAYBACK]
