Mixture of Experts (MoE)
A neural network architecture that routes each input to only a subset of specialized 'expert' sub-networks, enabling much larger models without proportionally increasing compute costs.
MoE architectures allow models with trillions of total parameters while only activating a fraction for each input. GPT-4 is widely reported to use a MoE architecture with multiple expert networks. Mistral's Mixtral 8x7B model activates only 2 of 8 experts per token, achieving performance comparable to models 3x its active size. Google's Switch Transformer scaled to 1.6 trillion parameters using MoE. The approach is key to reducing inference costs — a critical factor as AI scales to billions of daily queries.
Explore the Data
Related Terms
Artificial General Intelligence (AGI)
A hypothetical form of AI that can understand, learn, and apply knowledge across any intellectual task at or above human level, rather than being specialized for specific tasks.
AI Alignment
The research field focused on ensuring AI systems behave in accordance with human values and intentions, particularly as systems become more capable.
AI Compute
The computational resources — primarily GPU and TPU processing power — required to train and run AI models, typically measured in FLOP (floating-point operations) or GPU-hours.
Capex (Capital Expenditure)
Long-term investment spending by companies on physical assets like data centers, GPU clusters, and networking infrastructure — the backbone of AI deployment at scale.
ChatGPT
OpenAI's conversational AI assistant, launched in November 2022, which catalyzed the current generative AI boom by demonstrating the capabilities of large language models to a mainstream audience.
Data Center
A facility housing computer systems and infrastructure used to process, store, and distribute data — increasingly built specifically for AI training and inference workloads.
AI Economy Pulse
Every Friday: the 3 AI data points that actually matter this week. Free, forever.
Latest: “AI Investment Hits $42B in Q1 2026 — Here's Where It Went”
No spam, ever. Unsubscribe anytime.