Ploba logo

Find the right AI tool for the job: apps, agents, APIs, MCP servers and datasets, explained plainly.

Get the AI Digest

Discover

  • Explore
  • Top Charts
  • Categories
  • Agents
  • MCP servers
  • Developer tools

Learn & community

  • Learn AI
  • AI Glossary
  • Blog
  • Hackathons
  • People
  • AI Digest

For makers

  • List your tool
  • Docs
  • Claim a listing
  • Advertise
  • Write for us

Company

  • About
  • Careers
  • Changelog
  • Trust
  • Security
  • Contact

© 2026 Ploba. All rights reserved.

  • Privacy
  • Terms
  • Cookies
  • Advertising
  • All policies
llms.txt
AI glossary
Generative AI

Inference

The phase where a trained AI model processes new data to generate an output or prediction.

What is Inference?

In artificial intelligence, inference is the process of running live data through a trained AI model to make a prediction or generate an output. If "training" is the phase where the AI learns, "inference" is the phase where the AI actually goes to work.

What is a simple example?

Training is like studying for a test. You read books, take practice exams, and memorize the material. Inference is taking the actual test, where you apply what you learned to answer new questions. When you type a question into a chatbot and wait for the text to stream onto your screen, you are watching inference happen in real-time.

What is it commonly confused with?

Inference is the direct counterpart to Model Training. Training adjusts the parameters; inference uses the frozen parameters to compute a result.

Why does it matter?

Inference requires significant computational power. When you hear about tech companies buying hundreds of thousands of GPUs, they need them both for training massive new models and for running inference to serve the millions of daily users interacting with their deployed AI applications.

Category
Generative AI
Last reviewed
Sep 21, 2026

Sources

  • AWS: What is Inference?

Related terms

  • Machine LearningModel TrainingThe computationally intensive phase where a machine learning algorithm processes data to learn patterns and adjust its parameters.
  • FundamentalsAI ModelA system that has learned patterns from data and uses those patterns to make predictions or generate content.
  • FundamentalsParametersThe internal, mathematically learned variables that define how an AI model processes inputs into outputs.
  • DevelopmentLatencyHow long an AI system takes to respond, often measured as time to the first token and then output speed in tokens per second.
  • DevelopmentInference CostWhat it costs to run an AI model on each request, usually priced per million input and output tokens through an API.