Inference Cost
The computational expense of running a trained AI model to generate outputs for users, which determines the per-query economics and ultimately the pricing of AI services.
Inference costs have dropped 280x in 18 months, from approximately $0.36 per GPT-4-equivalent query to under $0.01 for many competitive models. Despite falling per-unit costs, total inference spending is rising as query volumes explode. Techniques like quantization (4-bit reduces costs 4-8x), speculative decoding, and MoE architectures reduce inference compute. Inference accounts for approximately 60% of total AI compute demand — the majority of AI's energy footprint. Specialized inference chips from Groq and AWS (Inferentia) offer 2-5x cost advantages over general-purpose GPUs.
Live Data
Explore the Data
Related Terms
AI Compute
The computational resources — primarily GPU and TPU processing power — required to train and run AI models, typically measured in FLOP (floating-point operations) or GPU-hours.
Capex (Capital Expenditure)
Long-term investment spending by companies on physical assets like data centers, GPU clusters, and networking infrastructure — the backbone of AI deployment at scale.
ChatGPT
OpenAI's conversational AI assistant, launched in November 2022, which catalyzed the current generative AI boom by demonstrating the capabilities of large language models to a mainstream audience.
Data Center
A facility housing computer systems and infrastructure used to process, store, and distribute data — increasingly built specifically for AI training and inference workloads.
Enterprise AI Adoption
The rate at which businesses integrate AI technologies into their operations, measured across functions like customer service, software development, marketing, and supply chain management.
Fine-Tuning
The process of further training a pre-trained AI model on a specific, smaller dataset to specialize it for a particular task or domain, requiring far less compute than training from scratch.
AI Economy Pulse
Every Friday: 3 data points shaping the AI economy this week. Cited sources. No fluff.
Latest: “AI Investment Hits $42B in Q1 2026 — Here's Where It Went”
Weekly. Unsubscribe in one click.