M365Con 2027 – free online conference · 11–22 Jan 2027 · 120 sessions→ m365con.net

2026-11-15

Foundry IQ - Simply Explained

Follow the showNew episodes on Microsoft 365, Copilot and more, free in your podcast app.Apple PodcastsSpotify
Foundry IQ - Simply Explained
Your company's AI assistant has never read a single one of your company's documents. It answers with total confidence anyway, and that is how trust in AI dies. In this episode of Simply Explained you will learn: - Why Your AI Assistant Keeps Guessing - What Foundry IQ Actually Is - Where the…

Show notes

Your company's AI assistant has never read a single one of your company's documents. It answers with total confidence anyway, and that is how trust in AI dies. In this episode of Simply Explained you will learn:

  • Why Your AI Assistant Keeps Guessing
  • What Foundry IQ Actually Is
  • Where the Knowledge Comes From
  • What Happens When You Ask a Question
  • The Dial That Decides How Hard It Thinks
  • How It Compares to Building It Yourself
  • Where It Sits in Microsoft 365 and Azure
  • What a Real Setup Looks Like
  • The Mistakes That Trip People Up
  • What This Means for You

Simply Explained by M365.FM breaks down Microsoft 365, Azure and modern IT topics in clear, jargon-free language. Hosted by Mirko Peters. Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-microsoft-365-azure-cloud-power-platform-ai--6704921/support. This episode includes AI-generated content.

Host: Mirko Peters

Guests

Transcript

📄 Transcript (TXT)

Read the full transcript

Welcome to the M365 FM podcast, where we explore the people, ideas and technologies shaping the future of work. Why your AI assistant keeps guessing your company's AI assistant has never read a single one of your company's documents. It answers with total confidence anyway, and that is how trust in AI dies. Microsoft built something to fix that, and it is called foundry IQ. Foundry IQ is a managed knowledge layer that turns scattered company information into permission aware knowledge bases an AI agent can safely read from. It sits on top of Azure AI search, and it belongs to a family Microsoft calls the IQs. By the end, you will know what it is, how it works, and the limits that come with it. Start with how a model actually learns. It reads an enormous pile of public text. It learns the patterns buried in that text, and then it stops. The day it stops is called its knowledge cut off, and everything after that day is invisible to it. Now think about the information that never made it into public text at all. Your holiday policy, your refund rules, and the pricing sheet you updated last Tuesday all live behind your company's walls. None of it is inside the model and that model a job and the problem gets sharper. The polite name for the job is an agent, which simply means software that answers questions and takes actions. Using an AI model underneath, someone in finance asks the agent how many days of parental leave they get. The model has never seen the handbook, so it invents an answer that sounds completely reasonable. A wrong answer, delivered with certainty, is worse than an honest shrug, because it teaches people to stop trusting the tool. That gap is the whole reason foundry IQ exists. Closing it turns out to be less about a smarter model and more about giving the model something real to read what foundry IQ actually is, Microsoft describes it as a managed knowledge layer that turns enterprise data into reusable, permission aware knowledge bases for AI agents. That sentence is accurate, and it tells a beginner almost nothing. So here is the plain version. A knowledge base is the heart of it. Picture a shelf that holds your company's trusted information, arranged so an AI can search it quickly. And that shelf is what Microsoft calls a knowledge base. It isn't a copy of everything you own. It's a curated set of connections to the places your information already lives. Those connections have a name to a knowledge source is one link to one place, such as a folder in cloud storage, a SharePoint site, or a data lake. The knowledge base can hold up to ten of them at once, and this is where the design saves real work. Many agents can share the same knowledge base instead of each agent building its own private pipeline. Then comes the permission part, and it matters more than the rest. The knowledge base is permission aware, which means the AI sees only what the person asking is allowed to see. Ask about a salary band you don't have access to, and the answer doesn't come back at all. Microsoft introduced foundry IQ at its ignite conference in twenty twenty five. That same event renamed Azure AI Foundry to Plain Microsoft Foundry, which is why the name sounds so similar. Underneath all of it sits Azure AI search, the engine that does the actual indexing and searching. The name is a small clue. Foundry is the platform where you build an IQ is the knowledge you feed it. That split is why Microsoft talks about a whole family of these IQs. You don't need to be a developer to follow any of this. Building a knowledge base does take a few setup steps, but understanding what it is and why it exists doesn't. That understanding is what matters when you decide whether to invest your time in learning it. One more thing makes this different from a folder of documents. The knowledge base is reusable. You build it once and any number of agents can plug into it, which turns scattered information into shared infrastructure where the knowledge comes from. Two kinds of connection feed a knowledge base, and the difference is worth ten minutes of your attention because it changes how fresh the answers are. The first kind is called an indexed source. Indexing means the content gets copied into a search index, chopped into small pieces, turned into numbers a computer can compare and tagged with extra details. Azure AI search handles all of that for you on a schedule you set so the index stays reasonably current. Blob storage one lake, SharePoint and existing search indexes can all be indexed this way. The second kind is a remote source and it never gets copied at all. The engine queries it live. At the moment someone asks a question. The data is always current while the content stays where it lives. SharePoint through the Copilot Retrieval API works this way, and so does the open web through Bing. Picture a support team setting this up. They point a single knowledge base at their Help Center articles and their SharePoint FAQ site. One source indexed and one source live, and their agent can answer from both without anyone writing a custom connector for each system. The trade off is worth saying out loud. Indexed sources are fast and cheap to search, but they are only as fresh as the last indexing run. Remote sources are always fresh, but each question adds a live call to another system. Knowing where the knowledge lives is only half of it. The other half is what happens the moment someone types a question. What happens when you ask a question? This is where it stops feeling like search. Microsoft calls the machinery a genetic retrieval, which sounds like marketing and actually describes something concrete. Say you ask in plain words what your company does when a customer wants a refund after thirty days. A single search would grab a few documents and hope for the best. A genetic retrieval works more like a careful researcher, and it starts by planning the question. It breaks your one question into several smaller ones. One asks what the standard refund window is. Another asks what the exceptions are. And the third asks who has to approve a late refund? Those smaller searches then run at the same time across every source in the knowledge base. Each search can be a keyword search, a meaning based search, or a blend of the two. Every set of results gets reranked, so the strongest matches rise to the top. The best pieces are then merged into one response, with citations pointing back to the original documents. The engine can also lean on the conversation so far, ask a follow up like and for international orders and it knows what you were already talking about instead of starting from scratch. The citations are what make this trustworthy. They let a person check the answer instead of taking it on faith. And for anything touching policy or money. That check is the difference between a toy and a tool. There's a clever extra step here. If the first batch of results looks weak, the engine can take a second pass with a better plan, which is exactly where the word agentic earns its place. Microsoft's own benchmarks put this multisearch approach at roughly thirty six percent higher response quality than a single search. That's the company's number rather than an independent test, so treat it as a direction rather than a guarantee. There's a cost to the cleverness, and I would rather you hear it now. Running several searches plus a planning step takes longer than one query, so the answer arrives a bit slower. That extra second is the price of handling a question. A plain search would fumble the dial that decides how hard it thinks. Microsoft calls the dial retrieval reasoning effort, and it comes down to three settings you can actually reason about. The first setting is minimal. The engine skips the planning step entirely and runs your word straight at the sources. That makes it the cheapest and fastest option, and it's plenty for simple lookups across up to ten sources. The middle setting, called low, is the default. It adds the planning step and can write a short answer for you. Working across a few sources and a few subqueries at a time, the top setting is medium, and it adds that second attempt when the first batch of results isn't good enough, it handles more sources and more subqueries, and it's only available in some regions. Then there's the money, and it's simpler than it looks. Two metres run at once. Azure AI search builds for the retrieval work by tokens rather than by query, and it comes with a free monthly allowance. Azure OpenAI builds separately for the model that plans the searches and writes the answers. Microsoft's own documentation walks through an example with a small model, and the total lands at about four dollars thirty two cents for two thousand retrievals. That's Microsoft's example, not a promise about your bill, because your real cost depends on how much content you index and how hard you make the engine think. The practical habit is to start on the low setting, watch the activity log that comes back with every answer, and only turn the dial up when the quality genuinely demands it. When how it compares to building it yourself before foundry IQ. If you wanted an agent to answer from your own content, you built the retrieval pipeline yourself. You picked a database. You wrote code to chop documents into pieces. You generated the embeddings, and you handled permissions inside your own application. Every team did this and no two teams did it the same way. Microsoft's pitch is that this work becomes shared. A single knowledge base serves many agents, so the team that gets retrieval right the first time saves every other team from rebuilding it. Two older options are worth knowing about because you may still run into them. The first was the Azure AI search tool, which attached a single index to a single agent, and it was powerful, but left you to write the query logic and the citations yourself. The second was the file search tool, which let you drop a handful of files into one agent. And it was fast for a demo, but created a private silo no other agent could reuse. Foundry IQ sits above both. It keeps the strength of the search index, adds the planning and reranking and makes the whole thing reusable. The file search tool is now just another kind of sauce you can fold into a knowledge base, which tells you where Microsoft is heading. The honest trade off is complexity. A knowledge base has more moving parts than a plain index, and some of those parts are still in preview. You're trading a bit of setup for a lot of reuse, and that trade only pays off once you have several sources or several agents. Where it sits in Microsoft three hundred sixty five and Azure Foundry IQ is one of four things Microsoft now groups under the banner of Microsoft IQ, and knowing the shape of that family will save you from a lot of confusion. Work IQ covers how people work across Microsoft three. Sixty five. Pulling on the signals in mail, meetings, chats, and files. Fabric IQ covers your business data, giving agents a model of the entities and numbers in Microsoft Fabric and Power by Foundry IQ covers knowledge. The documents, policies and authoritative content and agent should be able to cite. Web IQ reaches the open web for anything the outside world knows better than you do. A single agent can pull from more than one of them. A question about a late shipment could draw on live business numbers from fabric IQ and the written returns policy from foundry IQ at the same time, which is exactly why Microsoft built them as separate pieces. You can combine. Each one stands alone, and that detail is the whole game. An agent sees only the pieces you connect to it. So the map is something you assemble rather than something Microsoft assembles for you. The question beginners ask most is how this differs from copilot. Copilot is where you work. The assistant sitting inside the apps you already open every day. Foundry IQ, by contrast, is knowledge you build for agents, and the two don't share their sources. A connection you set up for copilot can't be reused in foundry IQ, and the reverse is true as well. So pick the right one before you build on it. On the Azure side, foundry IQ lives inside Microsoft Foundry and leans on Azure AI search. You can attach it to a copilot studio agent through a foundry agent when you want your own knowledge to reach that surface. Permissions right on the same identity system you already use, queries run under the asking person's Microsoft identity and where a source supports them. Microsoft purview. Sensitivity labels are honored, too. So a document marked confidential stays confidential. The honest status note is this. Microsoft says some of foundry IQ is generally available and some is still in preview. And which one you get depends on the programming interface you use. The knowledge bases themselves have reached general availability, while several source types and the smarter planning features are still marked. Preview. The portals for now are preview across the board. What a real setup looks like. Abstract explanations only go so far, so here's what a small team actually does. In order. From a blank screen to a working agent, they pick one source first, usually a single folder of documents that everyone already trusts. They create the search service, then create a knowledge base and attach that one source, letting the system generate the indexing pipeline for them. Next, they connect an agent to the knowledge base and test it in the built in playground. They asked the same three or four questions their real users would ask, and they check not just the answers, but the citations underneath them. Because a citation that points to the wrong document is acquired failure. Once the answers hold up, they add a second source, maybe a SharePoint site, and watch where the quality improves, or whether the extra source just adds noise. They keep an eye on that log the whole time because it shows exactly which searches ran and what they cost. Only after that do they move the reasoning setting from minimal to low, and only after that do they connect the agent to the tools that people already use. The whole arc is deliberately small, and that's the point. The mistakes that trip people up. This is the section most videos skip, and it's the one that decides whether you have a smooth project or a painful one. The first mistake is assuming permissions are automatic. Microsoft is clear that document level access controls are enforced only when the source supports them, and someone has configured the permissions. Sync point a knowledge base at a folder full of sensitive files. Skip the sync and the agent can surface things people were never meant to see. Permissions are a feature you switch on, not a safety net that comes free. The second mistake is treating web sources like your own files. A web source gets summarized by a model before it reaches your results. So what comes back is a cited summary rather than the exact words on the page. That's fine for a general question and useless when you need a precise legal clause. The third mistake is editing the plumbing by hand. When you connect a storage folder, Azure AI search generates the whole pipeline for you from the data source down to the index from a fixed template, change those generated pieces and the next sync breaks. Then there are the practical limits. A knowledge base and its sources have to live on the same search service. A single knowledge base tops out at ten sources and that ceiling is real. So plan how you divide your content before you start adding one more trap. Is treating a preview feature like a finished one. Microsoft is explicit that preview features don't carry the same service guarantees and aren't recommended for production, so anything you build for real should stick to the parts that are generally available. And the quiet trap is adding too many sources too fast. Every extra source widens the search and adds cost so more isn't automatically better. The biggest mistake of all is starting too big. The people who have done this well begin with one source and the cheapest reasoning setting. They measure the answers they get, and only then do they add sources and turn the dial up. The promise of a shared knowledge base for everything is real and the discipline is to earn it one source at a time. What this means for you. Here's the short version. Foundry IQ is Microsoft's managed knowledge layer, a way to give an AI agent your company's real information with permissions respected, and to get answers with citations instead of confident guesses. It bundles your data connections into a knowledge base, then searches that base with a smarter pipeline than a single query, all with a dial for how hard it thinks and two metres for what it costs. The idea worth keeping is this when an AI makes things up better knowledge usually helps more than a bigger model, and better knowledge is something you can build yourself. So here's my question for you. What is the one document your team keeps re-explaining to every new person who joins? The one thing an AI should simply already know. If this made the topic feel less intimidating, subscribe to M365 FM to master the tools and the mindset shaping the future of work.