Semantic Kernel - Simply Explained

0:00 Semantic kernel is the layer under Microsoft's AI tools that almost nobody explains. It's a free open source kit that lets your own code talk to a large language model, and lets that model call your code back. You might expect a chatbot builder, but it's really middleware, the quiet part in the middle. Once you see the handful of pieces inside it, the way Microsoft builds agents stops feeling like a mystery. Welcome to the M365 FM podcast, where we explore the people, ideas, and technologies shaping the future of work. Today, we're breaking down semantic kernel piece by piece in plain English. What semantic kernel actually is? Let's start with the plainest version, because the name makes it sound heavier than it really is. Semantic kernel is a free open source kit from Microsoft that connects the software you're already building to an AI model, and then lets the model call your own code back.

0:45 Microsoft describes it as middleware, which is just a word for the layer that sits in the middle and passes messages between two things that don't speak the same language. Now what it isn't matters just as much. It isn't an app you open, it isn't a chatbot you type into, and it isn't a model like the ones running behind copilot. You install it inside a project you're already working on, and it takes over the awkward part of talking to an AI. The kit first appeared in March of twenty twenty three under an open license called MIT. That license means anyone can use it, read the code and change it to fit their own needs, which is a big reason it spread quickly among developers. It also doesn't care where your software runs. The kit works inside your own application, on Microsoft's cloud, or on another cloud, or on a machine sitting under your desk. Azure is the most common home for it, but Microsoft is quick to say Azure isn't required.

1:29 There's a clear reason this layer exists. Picture a working app full of real features like looking up orders or sending emails. A language model is great at understanding requests, but it can't touch any of that by itself. Semantic kernel closes that gap so the model can reach into your app, and your app can reach back into the model. Here's a small example to make it real. Say a customer types a question about an order, and your app needs to look up that order, check its status, and reply in plain language. A model can write that reply, but it can't look up the order by itself. Semantic kernel lets the model ask your app to do that lookup, and then use the answer in its reply all in one flow. Microsoft says fortune five hundred companies already build on it, and the reason they give is that it's flexible, modular and easy to watch while it works, the kernel and the name.

2:14 The first word that trips people up is kernel. So let's take the name apart. Semantic points at meaning and natural language. The kind of thing a model is good at. Kernel points at the center. The one object that everything else plugs into. Microsoft's own description is refreshingly simple. At its simplest, the kernel is a container that holds all the services and plugins your AI application needs to run. Think of it as a single box where every part of your AI app gets registered, so the app always knows where to find them. You hand the kernel two kinds of things. The first is services, which are the AI models themselves, plus ordinary utilities like logging. The second is plugins, which are the parts that do the actual work. And we'll get to those in a moment. When your app sends a request, the kernel runs through a short sequence of jobs.

2:54 It picks the best AI service for the task, builds the prompt from your template, sends it off, reads the reply, and hands the result back to your application. If that sounds like plumbing you've seen before. That's because it is. Microsoft modeled the kernel on a common pattern from a net called dependency injection, where one central place supplies the pieces that other parts of the code need, so nobody has to hardwire them for you. The practical point is simple the kernel is the control room. Everything else is a tool. You register there, and from that one place you can set up and watch the whole thing. There's one more detail about the kernel and then we'll move on. It's lightweight because it's really just a container, so Microsoft recommends building a fresh one for each use instead of holding on to one forever. That sounds fussy, but it keeps your plugins from leaking state between requests, plugins, and functions.

3:36 This is where semantic kernel gets useful. So let's slow down. A plugin is a group of functions that you expose to the AI. In other tools, those functions are often called tools or actions, and in the earliest version of semantic kernel, plugins were called skills. A function is just a piece of your own code with a plain description attached. The description tells the AI what the function does, what goes into it, and what comes back out. Microsoft's own example is a lights plugin with three functions one that lists every light, one that reports the state of a single light, and one that changes a light state. The model reads those descriptions and picks the right function for the request. Beginners usually miss one detail here. The description isn't decoration. If it's vague, the model guesses wrong or skips the function entirely and the whole thing falls apart.

4:18 Write the description the way you'd explain the function to a new colleague, and the model will use it correctly. Plugins can come from three places. You can write them in your own code. Import them from a shared web standard called OpenAPI, or pull them in from something called an MCP server, which is short for Model Context Protocol. That last one matters because it lets one team build a capability and share it with everyone else. Sharing is the real prize with those last two options. If you describe a capability with OpenAPI or wrap it in an MCP server, another team can pick it up without rewriting it. And even a maker in a low code tool can use the same capability. That's how one well-built function ends up helping the whole company. Plugins also play nicely with the way real teams build software. A plugin can hold its own database connection or web client, so the AI never sees the sensitive details.

5:01 It only sees the plain description and the result. There's one practical warning before you build a giant toolbox, OpenAI recommends no more than twenty tools in a single request, and ideally no more than ten, because the model's accuracy drops as the list gets longer. Fewer, clearer functions beat a giant pile of them every time. Planning and function. Calling. Once you have several functions, a new problem shows up. The AI has to decide which functions to use in what order to answer one request. In semantic kernel, that job is called planning. The original answer was planners. You handed the kit a set of functions, and it wrote a prompt asking the model to choose a plan for you. That worked, but it leaned on the model, guessing its way through the choice. Then OpenAI built function calling straight into its models, and other model makers like Gemini, Claude and Mistral followed with the same idea.

5:45 Function calling means the model can simply ask for a specific function by name with the right inputs, instead of describing what it wants in a paragraph, because that turned out to be more reliable. Semantic kernel made it the main way to plan. That change has a direct consequence. You should know about the older stepwise and handlebars planners have been deprecated and removed from the package. Microsoft now tells developers to use function calling instead in every language the kit supports. There's a reason Microsoft moved away from the old planners, and it's worth saying plainly a planner asked the model to invent a plan in words which was flexible but unpredictable. Function calling is narrower and more repeatable, so you get fewer surprises in production, which is exactly what a business app needs. The way it works is simpler than it sounds.

6:26 The model asks for a function, semantic kernel, runs it, hands the result back, and the model decides what to do next. That loop repeats until the task is finished. It can even call several functions at once, which is how it handles a request, like turning off every light in a room. You can think of the loop as a short conversation between your code and the model. The model talks, your code listens and acts, and then the model talks again with whatever it learned memory and rag. There's a limitation that catches everyone off guard the first time a language model on its own has no memory. Every request starts fresh, with no idea about your documents, your data, or the conversation you had five minutes ago. The fix is a pattern called retrieval augmented generation. In plain words, you look it up first and then you answer. Instead of hoping the model remembers something, you fetch the relevant material and hand it over along with the question to see why this matters.

7:13 Think about the last time a chatbot confidently made something up that happens when the model has nothing real to work from. Retrieval fixes that by handing it the actual document or record. So the answer is grounded in something you can point to. Semantic kernel builds that memory on top of vector stores. A vector store is a database tuned for meaning rather than exact words, and the kits connectors give you one shared interface across many of them. Azure AI search is the usual choice, and others like Postgres, Reddis, and Srand are supported too. The word vector deserves one sentence because it's less scary than it sounds. The system turns your text into lists of numbers called embeddings that capture what the text means, so it can find passages that are close in meaning, even when the words don't match. There's a neat detail here for beginners.

7:54 Vector search can be exposed to the model as a plug in, which means the AI can go look things up on its own when a question needs it, using the same function calling we just covered. There's one honest caveat to keep in mind. The Vector Store feature is still in preview, so it can change, and the older memorystore connectors have been replaced by the newer ones. Connectors and prompts. Memory is one kind of plug, and the kit has others. A connector is an adapter and it lets one block of your code talk to many different models through the same doorway. The list is longer than you'd expect and it includes Azure, OpenAI, plain, OpenAI, Google's Gemini, Mistral, and even models running on your own machine through a tool called llama. Amazon's bedrock is in there, too, along with any service that follows the OpenAI style.

8:34 The payoff shows up when a new model arrives. Instead of rewriting your application, you swap the connector and keep everything else. Microsoft calls the kit future proof for exactly this reason, because your code stops caring which model is underneath. The practical benefit is choice without lock in. If your company already pays for Azure OpenAI, you start there. And if a cheaper or better model shows up next year, you move to it without touching the rest of your app. That freedom is the quiet reason teams pick a kit like this in the first place. Connectors cover more than chat. By the way, the same idea reaches embeddings, images, and even audio, so one app can mix several kinds of AI. Each language supports a slightly different slice of that list, which is worth checking before you commit. Then there are prompt templates. A template blends your instructions, the user's input, and the output of a function into a single prompt.

9:18 You can save that template as a file and share it with your team, which turns prompt writing into something you can track and review. The last piece in this section is filters. A filter lets you run your own checks before and after the model acts, which is where teams put their safety rules and their security rules. Microsoft points to filters as the place for responsible AI work, and it's a small feature with a big job. One kit. Three languages. You might be wondering whether any of this fits the language you already know, and the answer is probably yes. Semantic kernel ships for C, sharp, Python, and Java. Each version follows its own language's habits on purpose, so it feels native instead of bolted on Python code written with. It still reads like Python and C sharp code still reads like C sharp.

10:00 Microsoft did that on purpose Because a tool that fights your habits is a tool you stop using. Getting started is deliberately small in C sharp. You add one package. In Python, you run a single install command, and in Java you pull it from the usual repository. There's no giant setup step standing between you and your first working example. There's another reason that matters. It means you don't have to learn a new language to try the ideas in this episode. If you already write Python for a side project, you can follow the Python path. And if your job runs on Natty, you can use that one. There's one thing to know before you pick. They aren't perfectly. Even a few features land in C, sharp and Python before they reach Java, and the vector store connectors lean toward the first two. It's worth a quick look at the feature list for your language before you plan around something. One more promise is worth having if you're learning version one and later come with a commitment to no breaking changes, which means the code you learn today won't be thrown away by the next update.

10:51 Copilot, foundry and what comes next? Now for the question that confuses the most people and it deserves a straight answer. Semantic kernel isn't Microsoft three hundred sixty five copilot, and you don't need it to use copilot. Copilot is a finished product you open and use. While Semantic Kernel is a building material that developers use to make their own AI features, the real link is a shared way of describing things. Semantic kernel uses the same web standard OpenAPI, that Microsoft three. Sixty five copilot uses to describe what an extension can do. Because both sides speak the same language, a capability you build can be shared with other developers, and even with makers who don't write code. There's a bridge here too, for building your own agent into the Microsoft three hundred sixty five world. The Microsoft three. Sixty five agents SDK handles the Microsoft three hundred sixty five side, and the AI side is handled by Semantic Kernel or its successor.

11:36 So together they let a custom agent reach into copilot. On the cloud side, the newer framework lists Microsoft Foundry as one of the places to run models and agents. It helps to see these as layers rather than rivals. Copilot is the finished experience your colleagues use. Copilot studio is where makers build their own agents with little or no code, and Semantic kernel is the developer kit underneath. That turns an idea into working code. They are meant to sit together, not compete. That brings up the honest part. Microsoft built a newer framework, the Microsoft Agent Framework, by combining semantic Kernel with another project called Autogen. It has since become the recommended starting point for brand new agent projects. The real tension here is simple, and you've probably felt it already. You don't want to spend months learning something that Microsoft is about to replace, so let's deal with that fear directly.

12:21 Here's what that does and doesn't mean. New feature work now goes to the newer framework. But semantic kernel is not dead and not abandoned. Microsoft still supports it and still ships fixes, and Microsoft's own guidance says that for an existing project, or when you need to ship something quickly, staying on semantic kernel is perfectly fine for you as a learner. The ideas carry straight over. A plugin here becomes a tool there, and the mental model you just built is the same one. Learning semantic kernel is really learning how Microsoft approaches agents. That way of thinking outlasts any single tool. When to use semantic kernel. Let's finish with the practical question of when to actually reach for this kit. Start with the rule that fits almost any tool. If you can write a plain function to do the job right.

13:02 The plane function. Microsoft says this about agents. And it's good advice because reaching for AI when a simple piece of code would do, just adds cost and confusion, then reach for semantic kernel. When you're a developer adding AI to an app you already own, and you want your own code to be scalable by the model, that's the core case, and it's where the kit shines. Reach for it when you need one code base to work with several models, so you can switch providers without rewriting anything and reach for it when your company runs on net and wants the kind of logging, security, and oversight that big IT teams ask for before they approve anything, there are two times to look elsewhere. If you never write code, the kit isn't your door because the maker friendly path is copilot studio, and Semantic Kernel sits underneath that world rather than at the front of it.

13:44 If all you want is a simple chat bot on a website, you're solving a smaller problem than the one this kit was built for. A career angle matters here, too, since many of you came here for exactly that. The ideas in this kit plugins, function, calling, memory, and connectors show up in other AI tools too, including the ones that aren't from Microsoft. Learn them here and you'll recognize them everywhere else. There's one last thing for anyone starting fresh. If you're starting something brand new, Microsoft now steers developers toward the successor first. So learn semantic kernel for the ideas and reach for the new framework. When you begin, the concepts are the same, and that's the part worth keeping the recap and one question. So let's pull the pieces together in a few short lines. Semantic kernel is middleware, the layer that connects your code to an AI model.

14:28 The kernel holds everything in one place. Plugins give the AI your own functions. Function calling is how it decides what to run. Memory is how it remembers your data, and connectors keep you free to swap models later. If you remember one thing, make it this. The model is the part that will change again and again as the technology moves. The wiring between your code and the model is the part you actually control. And that wiring is what Semantic Kernel gives you. If this helped, subscribe to the M365 FM podcast to master the tools and the mindset shaping the future of work. Here's my question for you. If you could hand one task from your own work over to an AI tomorrow, which task would you trust it with first?