M365Con 2027 – free online conference · 11–22 Jan 2027 · 120 sessions m365con.net

Small language model (SLM)

Concept

A small language model is a compact AI language model with far fewer parameters than a large language model, so it needs less compute and can be focused on a narrow domain or run on local devices.

✦ CONCEPTSmall language model(SLM)M365.FM
Title graphic: Small language model (SLM) (Concept)

Often discussed together

Episodes about Small language model (SLM) often also cover Multi-agent system, Large language model (LLM) and Microsoft Foundry.

Description

A small language model (SLM) is a compact AI language model that has far fewer parameters than a large language model (LLM). Because of this smaller size, it needs less compute and can be focused on a narrow domain or run on local devices. SLMs are also called compact language models. They are built for natural language tasks such as text generation, but they are small enough to be trained or run on consumer hardware, including personal computers, laptops and smart devices.

The size difference is the central distinction. One overview places SLMs typically below about 40 billion parameters, with many model families under 10 billion. Large language models, by comparison, often have hundreds of billions of parameters and sometimes more than a trillion. Most SLMs use the same architecture as LLMs, but with fewer parameters and sometimes lower numeric precision. They are commonly produced or shrunk with techniques such as knowledge distillation, pruning and quantization. Pre-training also helps very small models, and larger pre-training data generally improves their results.

Several model families are described as SLMs. Examples named in the source material include Gemma, Granite, Ministral, Phi, Qwen, SmolLM (with 135M, 360M and 1.7B variants) and Apple's on-device foundation models. Microsoft's Phi family is the one most relevant to the Microsoft ecosystem, and it is the family discussed in the M365.FM episode on this topic.

The main advantages are lower compute requirements and the ability to run locally without cloud access. Running a model locally has a direct consequence for data handling. The M365.FM episode "Small Language Models for Faster, Cheaper Enterprise AI" states that when Phi models are deployed locally, data stays within the organization, and the organization controls access and can meet compliance requirements. The episode describes SLMs as delivering fast, accurate results for specific tasks, such as summarizing emails, classifying tickets or documents, and processing regulatory documents efficiently.

The trade-off is capacity. Fewer parameters limit how much information a model can encode, which can reduce generalization and accuracy compared with large models. As a result, the quality of an SLM depends heavily on how well its training data matches the task it is given. This is why SLMs are typically framed around narrow, well-defined use cases rather than broad, open-ended work. A typical decision for an organization is therefore whether a task is specific and repeatable enough for a small model, or whether it needs the broader capability of a large model.

The episode "Small Language Models for Faster, Cheaper Enterprise AI" recommends that businesses consider SLMs and stay ready for the next wave of AI innovation, and it notes that experts predict AI will soon join the workforce and transform industries. In its FAQ format, the episode defines an SLM as a model with fewer parameters than large models that can run on local devices and give fast, accurate results for specific tasks.

Overview compiled with AI from M365.FM episodes and the facts on this page. You can improve it with the edit button below.

TypeConcept
WikipediaSmall language model (SLM) – Wikipedia en.wikipedia.org

Episodes about Small language model (SLM)

Episodes in which Small language model (SLM) is discussed:

Timeline of episodes about Small language model (SLM)

Articles about Small language model (SLM) (2)

Articles in which Small language model (SLM) is a topic:

Frequently asked questions

Questions marked with AI were written with AI from the facts on this page.

What is a small language model (SLM)? (AI)

A small language model is a compact AI language model with far fewer parameters than a large language model. It needs less compute and can be focused on a narrow domain or run on local devices.

How is an SLM different from a large language model? (AI)

SLMs typically have below about 40 billion parameters, with many under 10 billion, while LLMs often have hundreds of billions or more. They usually share the same architecture, but the smaller size lowers compute needs and can reduce accuracy and generalization.

Why do SLMs matter for privacy? (AI)

The M365.FM episode on SLMs says Phi models can be deployed locally, so data stays within the organization. This lets the organization control access and meet compliance requirements.

Glossary

Community wiki

Short info above, community knowledge below: add practical tips, examples or corrections. New contributions are reviewed first; trusted members publish directly.

Experts and companies for this topic

Nobody has listed this topic yet. Add it to your profile.

Sources

Maintained by Mirko Peters (Microsoft MVP) · Last updated: