Is Microsoft Copilot worth it? Many organizations don’t see enough ROI—and the data explains why.

Summary

Running Why Leadership Thinks Copilot Is Useless (And Where the Numbers Back Them Up) feels like standing between two worlds: the marketing hype about AI changing everything, and leadership asking, “So where are the results?” In this episode, I tackle the harsh reality behind adoption dashboards: many leaders judge Copilot’s value by sign-in metrics — not actual usage.

You’ll walk away knowing how to decode what’s shown in the Copilot / Insights admin view, how to build adoption narratives that don’t collapse under questions, and which usage patterns matter more than flashy graphs. This isn’t an AI sales pitch — it’s a wake-up call for how to measure and defend the real ROI of Copilot in your tenant.

What You’ll Learn

* Why leadership often sees Copilot as a flop — because they’re looking at the wrong metrics

* The difference between sign-in telemetry and behavioral usage telemetry

* How to interpret adoption: what signals are noise and what signals count

* How to map Copilot prompt categories (summaries, tone changes, rewrites) to real work

* Tactics to defend Copilot’s business case in leadership reviews

* A simple 3-step playbook: confirm visibility, identify usage hotspots, and build adoption plans

Full Transcript

Here’s something spicy: most organizations think Copilot is underused… not because users hate it, but because no one’s checking the right dashboard. Subscribe at m365.show if you want the survival guide, not the PPT bingo.

We’ll check which Copilot telemetry matters, where users actually click, and how prompts reveal who’s using it for real work. Often the signals you need live in a different pane—let’s show you where to look in your tenant. This isn’t a pep rally; it’s a reality check with the data points that count.

And once you’ve seen that, we need to talk about the reports leadership is already waving in their hands.

The CFO’s Report Doesn’t Lie

Ever had that moment when the CFO barges in, waving a glossy admin report like it’s courtroom evidence, and asks why the company shelled out for a Copilot license nobody seems to use? Your stomach drops, because you’re not just defending an IT budget line—you’re defending your job. And here’s the kicker: the chart they’re holding isn’t wrong, but it’s not telling the story they think it is.

The leadership bind is simple: licenses cost real money, so execs want hard proof that Copilot isn’t just another line item in the finance system. Microsoft does provide reports, but what those charts measure isn’t what most people assume. Log into the admin center and you’ll see nice graphs of sign-ins and “active users.” Sounds impressive until you realize it’s basically counting how many times someone opened Word, not whether they actually touched Copilot once they were in there.

This is where the data trips people up. That report showing 2,000 Word sign-ins? Leadership reads that as 2,000 instances of Copilot lighting up productivity. Reality: it just means 2,000 people still have Word pinned to their taskbar and clicked it once. No one tells them that Copilot activity is captured in separate telemetry. So while the chart says “adoption,” in truth Copilot might be sitting unused like an expensive treadmill doubling as a coat rack.

Now, to be fair, Entra AD does exactly what it promises. It focuses on identity and sign-in telemetry—it tells you who walked through the door and which app they technically opened. What it does not do, by default, is surface the action-level data that proves Copilot adoption. Put simply: it’ll show you that John launched Word, but it won’t show you that John asked Copilot to crank out a three-page summary to save himself an afternoon. Always check your tenant’s docs or Insights pane for what’s actually available, because the defaults don’t go that far.

Here’s one clean metaphor you can safely use with leadership: those reports are counting swipes of a gym membership card. They don’t show whether anyone touched the treadmill, lifted weights, or just grabbed a chocolate bar from the vending machine. That one line paints the picture without drowning in analogies.

So what do you say when finance is breathing down your neck with pretty graphs? Here’s the leadership-ready soundbite: “Sign-in counts show who opened the apps. Copilot adoption means showing who actually used prompts and actions. I can pull behavioral reports for that—if our tenant has telemetry enabled.” That’s a safe, honest line that doesn’t oversell anything but tells executives you can provide a real answer once you’ve got the right data enabled.

And this is where your action item comes in. Don’t waste time trying to prove adoption from identity numbers. Instead, verify whether your tenant has Copilot Insights or usage reports that surface prompts and actions. If it does, prep a side-by-side demo for the CFO or CIO: slide one shows a bland graph of “sign-ins,” slide two shows actual prompts being used in Outlook, Word, or Teams. The contrast makes your point in about 30 seconds flat.

Because at the end of the day, raw sign-in and license charts will always frame the wrong narrative. They’re a door count, not a usage log. What leadership really needs to see are the actions that prove value—the moments where Copilot shaved hours off real workflows, not just opened an application window.

And that sets up the bigger story. Because Microsoft doesn’t give you just one way to see Copilot activity—they give you two different dashboards. One is the guard at the door. The other is the camera inside the building. And only one of them will tell you whether Copilot is actually changing how people work.

Entra AD vs. Insights: The Tale of Two Dashboards

Microsoft gives you two main dashboards tied to Copilot adoption: Entra AD and Insights. At first glance they both look polished enough to screenshot into a slide deck, but they don’t measure the same thing. If you confuse them, you’ll end up telling execs a story that sounds great but falls apart the minute someone asks, “Okay, but did anyone actually use Copilot to get work done?”

Here’s the split. Entra AD primarily records identity and sign‑in events. It’s useful for security and access checks—who got in, from where, on what device, and at what time. That’s its lane. It doesn’t go deep on what happens after the app opens. Think of it as the entry log. Helpful? Yes. Proof of adoption? Not really. Insights, on the other hand, is where you start seeing behavioral patterns. Depending on what your tenant exposes, it can surface things like prompt counts, app‑level activity, and even departmental usage. That’s the level you need if the conversation is about ROI.

Here’s the trap admins fall into. You’re on the spot in a meeting, leadership asks, “How many people used Copilot this week?” Under pressure, you run a quick Entra AD report and point to a chart showing thousands of Word users. But what you’ve proved is that people still open Word every week. Copilot activity might be a fraction of that, but Entra won’t show it. It’s the classic “stadium is full” headline when the team never left the locker room.

A better way to frame it is like this: Entra AD tells you John opened Word at 9 a.m. on Monday. Insights, if enabled, might show John used Copilot to generate a draft report in those same three minutes. Which one of those slides convinces a CFO that the product is paying for itself? Exactly.

So what should you actually do here? Step 1: in your admin center, confirm whether Copilot/Insights telemetry is visible and that you have the right permissions to view it. Don’t assume it’s just on. Depending on your role assignment, you might not be able to see adoption data at all. Step 2: if telemetry is available, check what categories are exposed. Non‑definitively, you’re looking for at least three dimensions—activity by app, prompt/use counts, and departmental breakdowns. Step 3: keep in mind privacy. Avoid dropping user‑level detail into leadership slides. Aggregate it by team, anonymize where needed, and stay compliant. Finance might want a name‑and‑shame list, but resist that urge, or you’ll create more HR tickets than Copilot tasks.

This is the part that kills a lot of adoption projects. You can’t design smart training or change campaigns if the only metric you have is “they opened Word.” Insights, when surfaced, shows where work is really happening—or not happening. Maybe Marketing fires off prompts all day, while Legal never touches the thing. That department‑level picture is the difference between targeted adoption efforts and yet another generic, hour‑long training no one remembers.

Bottom line: Entra AD is valuable, but it’s a door count. It makes sense for identity, security, and blocking attackers. It doesn’t measure whether Copilot replaced busywork. Insights, if wired into your tenant, is the view you need to argue for ROI, improve adoption, and spot which groups need support. Always check role permissions first, respect privacy rules, and don’t oversell what the data shows.

Now that you know which pane matters, the next step is to examine where inside the apps that Copilot usage actually shows up—and that picture looks very different from the marketing demos you’ve been shown.

Where Copilot Clicks Actually Happen

When you strip away the dashboards, the real story sits in the apps themselves—where people actually click Copilot. And here’s the surprise: those clicks aren’t happening in the flashy scenarios from Microsoft’s keynote clips. They’re landing in the everyday tools people touch constantly, and that pattern matters more than any marketing reel.

A common pattern we see in many tenants is heavier Copilot usage in communication apps like Outlook, Teams, and Word, compared with apps like PowerPoint or Excel. It makes sense if you think about it—email triage happens every morning, chat replies pop up all day, and quick edits in Word are a constant grind. Slide decks or complex models? Those only hit the calendar once in a while. But don’t take my word for it. Validate it yourself against your tenant’s Insights reports.

And this brings us to a key point: Copilot adoption grows fastest where the tool reduces friction in small, repetitive tasks. Think cleaning up an email, fixing awkward grammar, or smoothing a Teams message before you hit send. Those are quick, safe interactions that don’t require perfect prompts or lengthy context dumps. Users try it once for a throwaway email, it works, and then they start leaning on it every day. That’s “sticky adoption”—not glamorous, but reliable.

Here’s your actionable step: pull app-level activity from your tenant and rank the top three apps by prompt count. Start your training and success stories in the top two. Don’t waste budget blasting universal sessions when you already have evidence of where your people click. Adoption always follows need, not wishful thinking.

Let’s hold up PowerPoint as an example. In a lot of tenants, its Copilot numbers look weak compared to Outlook and Teams. That doesn’t mean abandon it. It means test smarter. If reports show low activity, don’t fund broad “Copilot for PowerPoint” rollouts to the whole company. Instead, pick two teams that live and die by decks, run a pilot, and measure usage before deciding if it’s worth scaling. That keeps the spend justified and the expectations realistic.

The bigger lesson is adoption isn’t driven by potential—it’s driven by frequency and pain points. Workers live in email and chat, not in quarterly slide design. That’s why Copilot earns loyalty where it chips away at daily friction. Small, sticky wins add up: shaving a few minutes off every email response compounds into hours when you add it up across a month. That’s a metric you can safely take upstairs, as long as you calculate it straight from your tenant’s usage data.

Another overlooked angle: the click patterns reveal culture. If Outlook dominates prompts, you’re an email-first shop whether leadership admits it or not. If Teams clicks rival Outlook, you’ve got a chat-heavy culture. If PowerPoint is high, you’re living in a reporting cycle. You can’t just tell execs about adoption—you can show them a mirror of how the org really works. Some leaders love that insight; others go quiet because the charts finally surface what people actually do versus what the vision deck claimed they do.

This is where your success stories get grounded. Instead of saying Copilot is saving the company by “revolutionizing workflows,” show how it reliably trims minutes from email triage. Break that into a metric: “average time per reply reduced by X.” That’s clean, measurable ROI. Nobody cares if Microsoft demoed Copilot reorganizing a ten-page strategy doc—they care that Karen in Accounting can process her inbox without skipping lunch. That’s retention value, not marketing fluff.

So, once you’ve mapped the hotspots—the apps where Copilot really earns its keep—the next question is obvious: what exactly are people typing into those prompts? Because knowing the click zones is one thing, but the prompts behind them tell you if it’s genuine work or just novelty clicks wasting cycles.

Prompts: Useful Work or Just Digital Small Talk?

So let’s zero in on what people are actually typing into Copilot. The section on app hotspots gave us click zones, but the real diagnostic tool is prompt history. This is where you can separate the genuine work from the digital small talk. Leadership doesn’t care how many buttons get clicked—they care whether the text going in represents real productivity or just curiosity clicks.

What we see in many tenants is a common pattern: Outlook and Word prompts often cluster around the same themes—rewrite for clarity, shorten to keep it tight, adjust tone so it doesn’t sound blunt, or summarize long emails. It’s the copy‑edit intern that never complains. PowerPoint prompts, by contrast, are lighter and more one‑off: brainstorm a slide title, draft speaker notes, maybe generate a bullet list—helpful, but less repetitive. Treat these not as absolutes but as trends to check against your own tenant’s data. Pull prompt history and look for those recurring categories: rewrite, shorten, tone tweaks, summarization, and creative brainstorms. If they recur week after week, you’re looking at genuine adoption. If you see one‑off oddities that vanish, that’s noise.

It’s easy to overestimate early usage. Every roll‑out sees novelty spikes: someone inevitably asks Copilot to write a sonnet about pizza Monday. That inflates your metrics for a few weeks and makes activity look stronger than it is. Don’t panic when you spot those. Filter them out. A quick smell test is consistency—if the same prompt category keeps surfacing 30, 60, 90 days in, it’s work. If it shows up once, gets a laugh, and disappears, it’s novelty. That’s the simple rule.

Here’s the practical move: define the top five prompt categories inside your organization and track their recurrence over three months. If “reword email into two sentences” shows up every week from multiple users, you can start treating it like a required workflow. If “write a Shakespearean love note to Jira tickets” spikes only in the first week, put it in the novelty column and stop counting it toward adoption. Run this filter before leadership sees the slides, or you’ll end up defending joke prompts instead of real savings.

And don’t stop with classification—convert repeat prompts into assets. Tag or export categories that come up constantly, then wrap them into templates or quick macros your users can grab. Think about HR: if their Copilot prompts constantly adjust tone in sensitive emails, turn that into a formalized “comms tone” template so they’re not reinventing it each time. If Finance is repeatedly asking for trimmed‑down status reports, save a baseline structure they can reuse. This is how you turn casual Copilot usage into repeatable, efficient workflows instead of one‑off experiments.

There’s also an interpretation angle here. If the same prompt category repeats weekly within a team, don’t just log it—treat it as a signpost for a workflow change or automation candidate. That’s Copilot surfacing a job people clearly hate doing by hand. On the other hand, if a category spikes once and collapses, don’t burn energy analyzing it. Call it novelty and move on. It saves you time while making sure leadership only sees ROI you can back up.

Bottom line: prompt data is where the hype clears, and adoption becomes measurable. Your job is to strip away the fluff, document the patterns, and show leadership the difference between random playtime and meaningful workflow shifts. Novelty spikes make the charts bounce early, but it’s the repeat categories that matter—the boring, consistent prompts that prove Copilot is embedded in the day‑to‑day.

And once you know which types of requests represent real work, the next revealing layer is seeing which departments lean into those patterns and which ones act like Copilot never arrived. That’s where the adoption story shifts from prompts to people.

Departments: Heroes and Ghost Towns

In most tenants, adoption doesn’t spread evenly across the company. One department jumps in like Copilot is free pizza, while another treats it like spam mail. Insights makes this visible fast, and it usually breaks leadership’s neat assumption that “everyone is using it about the same.” Spoiler: they aren’t. There are always heroes and ghost towns.

Take HR, for example. Frequently they’re the ones lighting up the graphs because Copilot saves them from rewriting endless internal emails. Drafting yet another “mandatory compliance reminder”? They offload the first draft to Copilot and polish it. That adds up into steady daily clicks. If you’ve got a similar pattern in your tenant, try one concrete step: build a shared library of prompt templates for rephrasing, summarizing, or adjusting tone, and make it easy for HR staff to grab them. That way the casual success they’re already having gets formalized into a repeatable set of workflows.

Marketing often shows a similar curve—lots of prompt traffic because Copilot is handy for jump-starting taglines, social snippets, or campaign drafts. Not everything sticks, but it beats staring at a blank slide deck. If you see the same signals, don’t just celebrate them. Turn their experiments into a short branded prompt playbook—approved examples that line up with company voice. Store it on SharePoint, Teams, whatever channel they live in. It both boosts their productivity and makes leadership feel better about message consistency.

Then there’s IT. You’d think admins would be the poster children for Copilot, but a common pattern is a spike during the rollout and then silence. They click the button once, confirm it “works,” and move on. This doesn’t mean IT never benefits; it just means Copilot doesn’t overlap with much of their day-to-day. Here’s your diagnostic: check if your own admins are showing up in the top user lists. If not, don’t blame them—ask why. Then run a very targeted demo with relevant use cases, like Copilot summarizing a week of incident tickets or auto-drafting change notes. Give them a taste that fits their actual work.

Finance is almost always a cautious adopter. Reports often show their activity flat compared with HR or Marketing. That doesn’t mean they can’t use Copilot; it usually means they don’t see the risk as worth it. Numbers need to be precise, and auditors aren’t in the mood for “AI got creative.” If you’re trying to unstick Finance, don’t give them generic training. Instead, take one of their actual budget templates and run a side-by-side demo: manual draft versus Copilot‑assisted draft. Keep the human in control, but show how much time it saves. That’s a way to prove ROI without threatening accuracy.

Legal takes the skepticism further. Copilot adoption in legal is often zero, because the potential downside feels too high. That doesn’t mean they can’t use it safely—it just means they won’t unless you frame it carefully. One low-risk approach: show how to use Copilot for first-draft redlines or to prepare background summaries, but always emphasize human-in-the-loop review. The lawyers see they remain accountable for the final text, and you give them a crack in the door to experiment without fear of liability.

What these examples make clear is that the “heroes”—HR and Marketing in many cases—don’t need blanket campaigns. They need refinement, better resources, and recognition that their wins can be turned into success stories for leadership decks. The ghost towns—often IT, Finance, and Legal—need tailored interventions. Force-feeding them generic adoption sessions doesn’t land. Instead, prioritize time where Insights shows consistent activity, and build smaller targeted campaigns for low-use groups with demos tied directly to their templates and needs. It’s cheaper, faster, and more likely to stick than pushing everyone through the same one-hour training.

When you package this for leadership, don’t just present raw charts. Break adoption down by department and include a one‑line remediation plan for each low‑adoption group. “Finance shows 3% usage—we’ll run template-based demos with their reporting docs.” “Legal shows zero—we’ll test controlled redline pilots under attorney review.” “IT usage is flat—we’ll show Copilot incident-summary use cases.” That’s not vague cheerleading, it’s execution strategy with proof attached.

Bottom line: adoption analytics are more than just pretty curves. They reveal pockets of success and expose the blind spots. Treat every department the same, and you waste budget. Focus where activity already thrives, and build tailored, practical on-ramps for the groups holding back, and you turn scattered usage into an actual adoption plan leadership respects.

And that brings us to the bigger point—because the charts and dashboards themselves aren’t the win. The real value depends on how you read those analytics and use them to explain whether Copilot is a smart investment or dead weight.

Conclusion

So here’s your closing playbook. Think of it as three steps:

Step one—confirm you actually have behavioral and Insights telemetry turned on and that you can pull app and prompt data. Step two—spot the top apps and the top prompt categories instead of guessing where the traffic lives. Step three—run pilots or campaigns in those hotspots, not across the whole org. That’s how you build adoption you can defend upstairs.

Subscribe at m365.show for the blunt checklist version. Follow M365.Show LinkedIn for livestreams with MVPs who’ve already broken this stuff in production.

Use the data that’s sitting in your tenant, show leadership a concrete plan, and they’ll fund the next phase.

So, Subscribe the Podcast now — because the next rename is coming.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit m365.show/subscribe