A small language model, or SLM, is a language model built with a small fraction of a frontier model's parameters, typically under about 10 billion versus 100 billion or more, trained or distilled to handle a narrow set of tasks well instead of being generally capable at everything, and it's rapidly becoming the default choice for the repetitive, narrow steps inside AI agents rather than routing every step through the biggest available model.

Most of the AI conversation over the last few years has been about scale, bigger models, longer context windows, more parameters. Agents complicate that story, because an agent doesn't ask one open-ended question, it runs a loop: parse a request, call a tool, format a response, check a condition, call another tool. NVIDIA's own agentic AI research team argues that most of those individual steps are narrow and repetitive enough that a much smaller, cheaper model handles them just as well as a frontier one, and routing all of them through a giant general-purpose model is often paying for capability the task never uses.

What Actually Counts as "Small"

There's no official cutoff, but in practice SLMs run from roughly a few hundred million parameters up to about 10 billion, small enough to run on a single GPU, and sometimes on a phone or laptop, rather than requiring a cluster. Meta's Llama 3.2 1B and 3B, Microsoft's Phi family, Google's Gemma, and NVIDIA's own Nemotron Nano line are the current reference points, positioned against frontier models like Llama 3.3 405B or GPT-class systems that sit one to two orders of magnitude larger.

The Economics: Why the Cost Gap Changes the Calculus

NVIDIA's research team found that running a roughly 1 to 3 billion-parameter Llama 3.2 SLM costs 10 to 30 times less than running its largest sibling, the 405-billion-parameter Llama 3.3, for a comparable amount of inference. Its Nemotron Nano 2 model reports 6 times higher throughput than similarly sized alternatives while still supporting 128,000-token context windows on a single GPU. Fine-tuning follows the same pattern: adapting an SLM to a specific task typically takes a few GPU hours, where fine-tuning a frontier-scale model can take days or weeks, which matters more for agents than for chatbots, because an agent's tasks are narrow enough to be worth specializing for directly.

Where SLMs Actually Win: Narrow, Repetitive Subtasks

The case for SLMs isn't that they're smarter, it's that most of what an agent does doesn't require general intelligence. Classifying a support ticket, extracting a field from a document, deciding which tool to call next, formatting a response into JSON, these are the same small number of task types executed over and over with little variation, exactly the conditions where a frontier model's broad reasoning capacity goes unused. Anthropic's own framing of context engineering makes a related point from a different angle: the goal is finding the smallest set of tokens, and by extension the smallest model, that reliably gets a specific step right, not maximizing capability at every turn.

The Heterogeneous Model: Division of Labor, Not Replacement

NVIDIA's paper, "Small Language Models are the Future of Agentic AI," published by researchers Peter Belcak, Greg Heinrich, and colleagues in mid-2025, doesn't argue for replacing LLMs outright. It proposes heterogeneous systems: an SLM handles the repetitive operational subtasks that make up most of an agent's steps, while an LLM gets invoked selectively for the smaller number of steps that genuinely need open-ended reasoning or conversational flexibility. That division of labor maps directly onto how multi-agent architectures already split work across specialized agents, it just extends the same specialization down to the model layer itself, and it changes the fine-tuning versus RAG versus prompting decision teams already have to make for each part of a system.

Recommended Coderhouse Courses

To design agents that put this kind of model selection into practice, the AI Engineering Course covers agent architecture, evaluation, and deployment trade-offs module by module. If you want to go deeper specifically on building and orchestrating autonomous agents, the AI Agents Course is the more focused, hands-on option.

Frequently Asked Questions

Is a small language model just a worse version of a large one?

Not for the tasks it's built for. An SLM trades general-purpose breadth for efficiency on a narrow set of tasks, and on those specific tasks it can match or beat a much larger model while costing a fraction as much to run.

How small does a model have to be to count as an SLM?

There's no fixed cutoff, but the working range in current research and products is roughly a few hundred million to about 10 billion parameters, small enough to run on a single GPU rather than a cluster.

Do SLMs replace large language models entirely in an agentic system?

Usually not. The pattern NVIDIA's research describes is heterogeneous: SLMs handle the repetitive operational steps, and an LLM still gets called for the smaller share of steps that need broad reasoning or open-ended conversation.

Why does this matter more for agents than for a single-turn chatbot?

A chatbot answering one open-ended question benefits from a model's full breadth. An agent runs the same small set of narrow steps over and over, tool calls, classification, formatting, which is exactly the repetitive, low-variation workload SLMs are built to handle efficiently.

Can a team fine-tune its own SLM without a large AI research budget?

Yes. Fine-tuning a small model on a specific task typically takes a few GPU hours rather than the days or weeks a frontier-scale model requires, which is one of the main practical reasons teams are adopting them for narrow, well-defined agent steps.