Inference

What it means

Every time someone at your company asks an AI assistant to summarize a contract, that request runs through a model that finished learning long ago. Using a finished model this way is called inference. IBM defines it as "the act of using a trained AI model to make predictions on new data."¹ An everyday example is a spam filter. It learned from millions of old emails, and when a new email arrives it decides "spam" or "not spam." That decision is inference.

The word matters to leaders for one reason: money. A model is trained once, but inference happens every time anyone uses it. IBM says the goal is to deploy a trained model "in a maximally efficient, cost-effective way."¹

Where the word comes from

In ordinary English, an inference is a conclusion you reach from evidence. AI borrowed the word for the step where a system reaches an answer from what it has learned. I could not open a primary source that dates the first use of the word in computing, so this entry does not give that date.

What I can document is how the word took on its current business meaning. In 2016 Nvidia, the chipmaker, published an explainer whose title, "What's the Difference Between Deep Learning Training and Inference?", shows the split the industry now uses: training is learning, and inference is using what was learned. Nvidia has kept the post up to date. Its current version says "Inference can't happen without training."² In 2024 the word moved into law. The European Union's AI Act defines an AI system partly as one that "infers, from the input it receives, how to generate" outputs.³

How others define it

• IBM: "In machine learning, AI inference is the act of using a trained AI model to make predictions on new data."¹
• Nvidia: "Inference is the process where a trained AI model generates new outputs by reasoning and making predictions on new data."²
• EU AI Act, Recital 12: "A key characteristic of AI systems is their capability to infer."⁴ The recital adds that the ability "to infer transcends basic data processing by enabling learning, reasoning or modelling."⁴

Where they agree and differ

Everyone agrees that inference means turning new input into output: an answer, a prediction, a piece of content, or a decision.

They disagree on scope. For IBM and Nvidia, inference is what happens after training. IBM puts it this way: "Unlike in model training, inference entails only a forward pass" through the model.¹ The EU AI Act uses the word more broadly. Its recital also speaks of "the techniques that enable inference while building an AI system."⁴ In law, then, "infer" can include learning itself, and that broader meaning helps decide which systems the Act covers.

The strongest objection to the narrow business meaning is that it makes inference sound like a cheap afterthought to training. The news below suggests the opposite. For many organizations, inference is now the bill that keeps growing.

In the news

On October 27, 2026, the tech news site TechCentral reported on a survey run for Cloudera, a data platform company. Wakefield Research polled 1,500 IT leaders in September.⁵ According to the report, "Nearly nine in 10" organizations have limited or redesigned AI uses because costs keep rising, and "Only 19% of them continuously monitor what they spend on tokens and inference."⁵ How sure we are: moderately. The survey was paid for by a company that sells data tools, and I have read TechCentral's account of it, not the full report.

Four days earlier, SiliconANGLE covered a Dell event in New York where a Dell sales executive described customers saying, "Oh my gosh, these costs are getting really high for inferencing."⁶ The outlet wrote that "Inference is now growing faster than training."⁶ How sure we are: this is what a vendor said at its own event, so treat it as one clear signal, not as measured data.

What it means for your organization

When you pay for AI, most of the ongoing cost is inference. You pay each time a person or an AI agent uses a model. An agent that works through a task step by step can call a model many times, so inference spending grows as you add agents, often without anyone watching it. If the Cloudera survey is right, most organizations do not track that spending as it happens.⁵ This entry is about cost, but the same habit also helps with control: when you know which models run, for whom, and how often, you know what your agents are actually doing.

One question to ask your team: "Who sees our inference spending each week, broken down by team and by agent, and what do they do when it jumps?"