M365Con 2027 – free online conference · 11–22 Jan 2027 · 120 sessions→ m365con.net

2026-08-04 · Season 3

Azure Local - Simply Explained

Azure Local - Simply Explained
As organizations rapidly adopt generative AI, the number of AI-powered applications, copilots, intelligent agents, and automation workflows continues to grow. What often begins as a single chatbot quickly expands into dozens of AI services connected to different models, APIs, business systems, and…

Show notes

As organizations rapidly adopt generative AI, the number of AI-powered applications, copilots, intelligent agents, and automation workflows continues to grow. What often begins as a single chatbot quickly expands into dozens of AI services connected to different models, APIs, business systems, and external providers. Without centralized governance, organizations face increasing security risks, inconsistent access controls, rising token costs, and limited visibility into AI usage. In this Microsoft Knowledge Nuggets episode, Mirko Peters explains Microsoft AI Gateway in plain English, demonstrating how Azure API Management provides a secure front door for enterprise AI traffic while simplifying governance, monitoring, and operational management. UNDERSTANDING MICROSOFT AI GATEWAY AND AZURE API MANAGEMENT Microsoft AI Gateway extends Azure API Management by introducing centralized control over AI traffic, model endpoints, intelligent agents, Model Context Protocol (MCP) tools, Agent-to-Agent (A2A) communication, and existing business APIs. Rather than allowing every application to communicate directly with different AI providers, organizations create a single managed entry point where authentication, authorization, routing, rate limiting, logging, and security policies are consistently enforced. This separation allows development teams to focus on building intelligent business applications while platform teams maintain governance, compliance, and operational standards across the entire AI landscape. ROUTING AI REQUESTS ACROSS MULTIPLE MODELS AND PROVIDERS One of the greatest advantages of Microsoft AI Gateway is the ability to separate applications from individual AI model deployments. Instead of hardcoding connections to specific providers, applications communicate with a stable endpoint while the gateway intelligently routes requests to Azure OpenAI, Microsoft Foundry, Anthropic, Google Gemini, Amazon Bedrock, or other approved model providers. Organizations can balance workloads across multiple regions, optimize performance, reduce operational risk, implement failover strategies, and migrate between models without requiring application developers to rewrite existing integrations. This flexible architecture enables enterprises to adapt quickly as AI technology continues evolving. GOVERNING AGENTS, TOOLS, AND BUSINESS APIS Modern AI extends far beyond simple chatbots. Intelligent agents increasingly interact with HR systems, finance applications, customer records, enterprise databases, and internal APIs. Microsoft AI Gateway provides centralized governance for Model Context Protocol (MCP) servers, Agent-to-Agent communication, and business APIs by enforcing identity, authorization, content safety, and Zero Trust principles. Combined with Microsoft Entra ID managed identities, organizations can ensure every AI agent receives only the minimum permissions required to perform its assigned task. This dramatically reduces the risk of unauthorized data access while creating complete visibility into every AI-driven action. MANAGING TOKEN COSTS, PERFORMANCE, AND AI OBSERVABILITY Unlike traditional software licensing, generative AI introduces variable consumption costs based on token usage. Microsoft AI Gateway helps organizations control these expenses through request throttling, quotas, token budgets, semantic caching, traffic prioritization, and detailed monitoring. Every AI request can be logged with information about the calling application, selected model, token consumption, response status, latency, and tool execution. This observability enables platform teams to identify inefficient prompts, detect abnormal consumption patterns, optimize model selection, and allocate AI costs across departments while ensuring mission-critical business workloads always receive sufficient capacity. BUILDING A SECURE FOUNDATION FOR SCALABLE ENTERPRISE AI Microsoft AI Gateway is not another AI model or agent builder. Instead, it provides the governance layer that allows organizations to safely scale enterprise AI initiatives. By combining Azure API Management, Microsoft Entra ID, Azure AI Foundry, MCP, A2A communication, API Center, content safety, centralized policies, and comprehensive monitoring, businesses gain a unified architecture capable of supporting future AI innovation without sacrificing security or compliance. Organizations beginning their AI journey should start with a single managed workload, establish governance from the beginning, and gradually expand toward a standardized enterprise AI platform that remains secure, observable, and cost-efficient. Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-microsoft-365-azure-cloud-power-platform-ai--6704921/support.

Apple Podcasts · Spotify · YouTube · Deezer

Host: Mirko Peters

Guests

Related articles

Transcript

Read the full transcript

Most people hear the word cloud and think of one simple idea. Your apps, files, and data can run from anywhere. For many workloads, that works really well. You can add capacity when demand grows, use services without buying more servers, and manage everything from one place. But here's the thing, some work can't leave the building. A factory needs fast machine decisions, or hospital may need patient data kept on site, and a remote location might lose its internet connection for hours or days, for years, that created two separate worlds. Cloud teams used Azure tools on one side, while local server teams used different tools, different update plans, and different ways to keep things secure on the other. Azure local aims to close that gap. It lets you run workloads at your own side, while using Azure for much of the management around them. But one question often causes confusion. If Azure Arc can connect local servers to Azure, why would you also need Azure local? That difference becomes clear when you look at why some workloads need to stay close to home. Why some workloads must stay local? Imagine a factory line making parts every second. A camera watches each part as it moves past, a sensor notices a small crack, and the system needs to decide whether to remove that part before it reaches the next machine. That decision may need to happen almost immediately. If every camera image travels to a distant Azure region, waits for processing, and then sends an answer back, the delay can be too long. Even a short network delay can mean the damaged part has already moved on, or the machine has already taken the wrong action. That's where edge computing comes in. Edge computing means processing data close to where it begins, so the camera, sensor, and local compute sit near the factory floor instead of depending on a distant cloud data center for every decision. The cloud still has a role. The factory can send selected data to Azure later, storing production history, comparing results across sites, training larger AI models, or creating reports for leaders. But the immediate decision stays local because the machine can't wait. You've probably seen the same idea outside of factory. Think about a ship at sea, where the connection may switch between satellite links, slow links, and no connection at all, yet navigation, engine systems, safety tools, and crew systems still need to run. Or picture a retail store where the internet connection drops during a busy weekend. If every checkout, stock check, and local system depends entirely on a cloud connection, the store has a problem at exactly the wrong time. Some places must stay disconnected by design. A secure government site, a defense environment, or a research facility, may never connect directly to the public internet. This is called disconnected operations, where local systems need to keep working with the site controls exactly how software updates and data move in and out. Then there's data sovereignty. That phrase sounds bigger than it is. In plain English, it means rules about where data can live and who can control it. So a company may need research data, patient records, video feeds, or operational data to remain within a country, a region, or even a specific building. Keeping data local isn't always about distrust of the cloud. Sometimes it's about law, sometimes about speed, and sometimes about keeping the business running when the network fails. A common mistake is treating on-premises computing as old technology that should disappear. That's not the right question. The better question is, where should this workload run? Azure works well for workloads that need flexible capacity, broad access, large-scale reporting, and services that don't depend on instant local responses. Local infrastructure works well when the data starts at a site. The decision must happen fast, or the site needs to keep operating on its own. Most organizations don't need to choose only one. They need the cloud for some work and local processing for other work. But local workloads need more than a server sitting in a closet. They need reliable compute, storage, networking, updates, security, and a way for the people who run Azure to see what's happening at the site. What Azure local actually is? So what exactly is Azure local? Here's the simplest definition. It's Microsoft infrastructure software that runs on a proved hardware in your own location. That location could be a data center, a branch office, a factory, a hospital, or any other site where your workloads need to stay nearby. You buy and keep the physical service. They sit in your building, use your network, and store data locally. Azure local runs on those servers and gives them a more Azure-like way to run and manage local workloads. Think of an Azure region as a large main office. It has huge amounts of compute, storage, and services available to customers across an area. Azure local is more like a smaller office building on your own site. Your team owns the building and the equipment inside it. But many of the rules, signs, security checks, and maintenance plans come from the same central organization. The work still happens in your smaller building. That distinction matters. Azure local doesn't copy all of public Azure into a rack of servers. You don't suddenly get every Azure service running locally just because you installed Azure local. Public Azure remains the place for the full range of cloud services and large-scale capacity. Azure local gives you a local platform for the workloads that need to run there. You may also hear a familiar older name. Azure Stack HCI, that's now part of the Azure local story. If you've seen that name in older articles, training, or diagrams, you're looking at much of the same foundation. Microsoft now uses Azure local as the broader name for this Azure connected infrastructure running at your site. So what stays at your site? The physical servers stay there. The compute power stays there too. The processor and memory that run your applications are in your own building, not in an Azure region. Storage stays local as well. The disks inside those servers hold the data for local virtual machines and workloads. Networking stays at the site, connecting your local systems, users, machines, and devices. On top of that local foundation, you can run virtual machines. A virtual machine is simply a software-based computer. You can run Windows or Linux inside it, much like a normal server without needing one separate physical box for every application. You can also run containers, containers package and application, and the parts it needs so it can run in a consistent way. They're useful when teams build and release modern applications more often. The processing stays close to the local systems that need it, but Azure local doesn't work alone in a corner. It connects back to Azure for management. That connection gives your team a familiar place to see the local environment, track its health, apply supported updates, and use Azure-based monitoring and security tools. You can apply policies from Azure so the same rules can check local systems and cloud systems. You can control access through the Azure tools your team already knows. You can also see billing for the Azure local service through your Azure subscription. Notice the split, the workload runs locally. The physical hardware remains your responsibility. Azure provides the management experience around that local platform, along with services that help you monitor, secure, update, and govern it. That's why calling Azure local Azure in a box creates the wrong picture. It isn't a miniature public cloud delivered to your site. It's a way to run local infrastructure with a closer connection to Azure. To see how that works, we need to look at three building blocks behind the local office building, the approved hardware, the local cluster, and Azure Arc. The building blocks behind the scenes. The first building block is the hardware. Azure local runs on service systems that have been tested and approved for it. You'll often hear these called validated hardware, validated nodes, integrated systems, or partner solutions. That testing is more useful than it may sound. A server platform includes processes, memory disks, network cards, storage controllers, drivers, and firmware. Each part needs to work correctly with the others, especially when several servers share storage and run business workloads together. Without testing, an update to one network driver or storage component can create a difficult problem. The server may still turn on, but the cluster may not behave the way you expect. Validated hardware reduces that guesswork. Microsoft and the hardware partner test supported combinations. So there's a known path for the server software, firmware, storage, and network parts. You still need to plan the size, location, network, and support model, but you're not starting with a pile of parts and hoping they work well together. Then comes the local infrastructure design. Azure local uses something called HCI, short for hyper-converge infrastructure. That name sounds technical, so let's put it in plain English. All the data centers often kept compute and storage in separate places. You might have one group of servers that ran applications, then a separate large storage system where all the files and virtual machine disks lived. The two systems connected over a storage network and each needed its own setup, updates, and care. HCI combines those jobs. The same group of servers provides compute power and storage. Each server contributes processor power, memory, and local disks. Together they become a shared platform for running workloads. Think of a small team doing one shared job. Each person brings different tools, but they work from the same plan and share the load. If one person needs to step away, the others can keep the job moving. A cluster works in a similar way. A cluster is a group of servers connected so they can act as one platform. When the design includes more than one server, workloads can move or restart on another server if one server has a problem. The cluster also spreads stored data across the servers based on the setup you choose. That doesn't mean every failure disappears. You still need enough servers, enough storage space, and a design that fits the level of downtime your business can accept. But clustering gives local workloads a better chance to stay available when one physical server needs maintenance or fails. Under the covers, Hyper-V helps run the virtual machines. Hyper-V is Microsoft's technology for creating and running those software-based computers. A local application might run inside a Windows Virtual Machine. Another may run inside Linux. Each gets the CPU, memory, disk, and network access it needs from the cluster. The local disks across the servers can also work together as shared storage. Instead of treating every disk as a separate island, Azure local pulls the local disks into storage that the cluster can use. That lets virtual machines run from storage spread across the servers rather than tying a workload to just one machine and one disk. So far, we have the physical foundation and the local platform. The next part explains how it appears in Azure. That part is Azure Arc. Azure Arc connects resources outside public Azure to the Azure portal and Azure management tools. It can connect servers, Kubernetes clusters, and other supported resources whether they run in your building or in another cloud. Arc doesn't move those resources into an Azure region. A server can remain in your server room, but show up in the Azure portal as a managed resource. Your team can use Azure access controls, tags, policies, monitoring, and security services against it depending on what you set up. This is where people often mix up Arc and Azure local. Azure Arc is the management layer. It can connect many kinds of existing infrastructure to Azure, including systems that aren't Azure local at all. Arc helps you see and manage those systems in a more consistent way. Azure local is the infrastructure platform. It provides the tested local hardware and clustered software foundation where your virtual machines, containers, and selected Azure services can actually run. Arc connects that foundation to the Azure control plane, think of the Azure control plane as the shared operations desk. It gives your team one place to control who can manage resources through enter ID, check rules with Azure policy, watch health through Azure monitor, and use Defender to look for security concerns. It also supports updates and automation across resources in Azure and at local sites. On top of Azure local you can run Windows and Linux virtual machines. You can run Azure Kubernetes service for container-based applications. You can run Azure virtual desktop workloads locally when users or data need that nearby performance. Some services that work with Azure Arc can also run close to the workload. The same Azure experience becomes far more useful when you have many locations. Managing one small site with local tools may be manageable. Managing 50 factories, 200 stores, or a fleet of remote sites quickly becomes a different problem. It changes the daily work for the people responsible for all of it. One control plane, local workloads. The real benefit shows up when teams stop working in separate worlds. In many companies, the cloud team uses Azure tools, Azure access rules, and Azure monitoring, while the local infrastructure team works through a separate set of consoles, spreadsheets, and server-by-server processes. That split creates gaps. A cloud admin might see an application in Azure, but not the local server supporting a site. Meanwhile, a local admin knows a server needs attention, but has no easy way to apply the same security rules used elsewhere. With Azure local, the cluster appears in the Azure portal as an Azure resource. You can see its name, location, health status, and alerts alongside other resources. And apply tags like factory, store 42, production, or a cost center so people can find and sort systems consistently. Access follows the same pattern. Instead of sharing a local administrator password with everyone who needs access, you use Microsoft Enter ID to decide who can manage the Azure local environment. One person might only view health details, another might manage virtual machines, and a smaller group controls the platform itself. That creates clearer ownership, picture a team that needs a new virtual machine for an application at a factory. Rather than treating the factory as a completely separate world, they use familiar Azure tools to create the virtual machine, choose its local network and storage, then run it on the Azure local cluster at that factory. The request begins in Azure, but the workload runs at the site. The same idea applies to containers. A developer can deploy a container-based application through familiar Azure connected tools while the container does its work close to the local devices and data. This helps teams keep one way of working even when workloads run in different places. Security and governance follow that same path. Enter ID controls who gets access. Azure policy checks whether systems follow the rules your company has set. For example, whether certain settings are in place or resources follow naming and tagging rules. Microsoft Defender watches for security concerns while Azure Monitor collects health signals and raises alerts when something needs attention. None of that removes the need for people to respond. It gives them a more consistent view, so they don't need to jump between unrelated tools just to understand what is happening at one site. Updates are another practical example. A local cluster includes more than one piece of software. The Azure local software itself plus the hardware drivers and firmware that need to work with it. On supported hardware, Azure local helps coordinate platform updates across those parts. Where the cluster design allows it, workloads can move from one server to another while a server receives an update reducing disruption, but that doesn't mean updates are magic or risk-free. Teams still need a maintenance plan testing and a clear understanding of which workloads can move. Azure local doesn't remove the real work of owning local equipment. Someone still needs to plan capacity before storage fills up, design the network, arrange backup and recovery, replace failed hardware, and decide who responds when a site has a problem. Azure local brings cloud style management closer to the site. It doesn't remove the site. The next question is where that local control changes the answer. From a factory floor to a retail store, a lab, or a place that cannot connect to the public cloud at all. Where Azure local fits and where it doesn't. A factory is one of the clearest examples. Cameras inspect products as they move down a line, sensors measure heat, vibration, pressure, or speed, and a local application can spot a defect, want a worker, or stop a machine before a small problem turns into wasted material or damaged equipment. Network happens at the site, close to the machines, and the data they create. Retail has a similar need even though it looks very different. A store may run checkout systems, inventory tools, cameras, and local AI that helps staff notice an empty shelf or a problem at a self checkout lane. If the wide area connection becomes slow or drops out, the store still needs to serve customers. Azure local gives that site a place to run those workloads locally while keeping a familiar Azure connection for the people who manage many stores. Healthcare and research labs add another reason, sensitive data. A hospital may process medical images close to the department that creates them and a lab may work with large research files that are difficult, expensive, or restricted to move elsewhere. The data can remain at the site while local systems process it and teams can decide later what information is safe and useful to share outside that location. Government energy and defense environments can take this even further. A power site may need local systems to watch equipment and react during a network outage, a government department may need data to remain within a tightly controlled location, and a defense environment may operate on a network that never connects to public Azure at all. That last case needs a different approach. Azure local can support disconnected operations for environments that are built to state disconnected. Instead of relying on a live connection to the public Azure control plane, the site uses a local control plane for a limited set of tasks and services. There is a trade-off. You don't get the full set of Azure services at the site and updates don't simply arrive across the internet. Teams prepare updates and licensing material outside the secure environment, then bring them into the site through an approved process. That takes planning, but it gives the organization control over exactly what enters the environment. So when should you consider as your local look for five signals? Does the workload need a very fast response? Does the data need to stay in a specific place? Do legal or company rules limit where the data can go? Must the site keep working when connectivity fails? Or does the organization need direct control over the local systems that run the workload? If the answer is yes to one or more of those questions, Azure local may fit, but it isn't the right answer for every application. A simple web app that already runs well in public Azure may not need local servers at all, a very small workload may not justify buying, maintaining and replacing physical equipment, and if a team doesn't want responsibility for hardware, local networking power, cooling and site support, then as your local creates work, they may not want. The goal isn't to put everything local. The goal is to put each workload in the place where it can do its job properly. That brings us to the most useful way to start the conversation. Start with the workload, not the hardware. Because your local is designed to keep your workloads running right where they are, while bringing Azure's management tools closer to that location. Before you start looking at server models or cluster sizes, ask yourself three questions. Where does the data begin? How fast does the system need to make a decision? And can that site rely on a working connection when that decision needs to happen? Then make a list of workloads that can't leave the site and write down the reason beside each one, whether it's speed, data rules, reliability or direct local control. If your first need is just managing servers you already have, start with Azure Arc. If you also need a local platform to run those workloads on tested hardware, then as your local is the next step. Here's the thing, if a factory stops working when the internet drops, a cloud-only design has already put that workload in the wrong place.

← Microsoft AI Gateway - Simply ExplainedBuilding Modern Microsoft 365 Solutions with SharePoint Fram →