Quantization
A model compression technique that reduces the precision of a model's numerical values (e.g., from 32-bit to 4-bit), shrinking model size and accelerating inference with minimal accuracy loss.
Quantization can reduce model size by 4-8x and speed up inference by 2-4x, making large models deployable on consumer hardware. A 70-billion-parameter model in full precision requires 140GB of memory, but 4-bit quantization reduces this to around 35GB. GPTQ, AWQ, and GGML/GGUF are popular quantization formats for LLMs. The technique has been critical for the open-source LLM ecosystem, enabling models like Llama 2 70B to run on gaming GPUs that cost under $2,000.
Explore the Data
Related Terms
Artificial General Intelligence (AGI)
A hypothetical form of AI that can understand, learn, and apply knowledge across any intellectual task at or above human level, rather than being specialized for specific tasks.
AI Compute
The computational resources — primarily GPU and TPU processing power — required to train and run AI models, typically measured in FLOP (floating-point operations) or GPU-hours.
Capex (Capital Expenditure)
Long-term investment spending by companies on physical assets like data centers, GPU clusters, and networking infrastructure — the backbone of AI deployment at scale.
ChatGPT
OpenAI's conversational AI assistant, launched in November 2022, which catalyzed the current generative AI boom by demonstrating the capabilities of large language models to a mainstream audience.
Data Center
A facility housing computer systems and infrastructure used to process, store, and distribute data — increasingly built specifically for AI training and inference workloads.
Fine-Tuning
The process of further training a pre-trained AI model on a specific, smaller dataset to specialize it for a particular task or domain, requiring far less compute than training from scratch.
AI Economy Pulse
Every Friday: 3 data points shaping the AI economy this week. Cited sources. No fluff.
Latest: “AI Investment Hits $42B in Q1 2026 — Here's Where It Went”
Weekly. Unsubscribe in one click.