Dropout
A regularization technique that randomly deactivates a fraction of neurons during each training step, preventing neural networks from becoming overly dependent on any single neuron and reducing overfitting.
Dropout was introduced by Srivastava et al. in 2014 and quickly became one of the most widely used regularization methods in deep learning. Typical dropout rates range from 10% to 50% of neurons per layer. The technique effectively trains an ensemble of sub-networks within a single model, improving generalization to unseen data. While dropout remains common in smaller models, modern architectures like transformers often rely more on other regularization methods such as layer normalization and weight decay.
Explore the Data
Related Terms
Artificial General Intelligence (AGI)
A hypothetical form of AI that can understand, learn, and apply knowledge across any intellectual task at or above human level, rather than being specialized for specific tasks.
AI Alignment
The research field focused on ensuring AI systems behave in accordance with human values and intentions, particularly as systems become more capable.
Fine-Tuning
The process of further training a pre-trained AI model on a specific, smaller dataset to specialize it for a particular task or domain, requiring far less compute than training from scratch.
Foundation Model
A large AI model trained on broad data that can be adapted to a wide range of downstream tasks — examples include GPT-4, Claude, Gemini, and Llama.
Machine Learning
A subset of AI where systems learn patterns from data rather than being explicitly programmed, improving their performance on tasks through experience without human-written rules.
Model Training
The computationally intensive process of teaching an AI model by feeding it data and adjusting its parameters to minimize errors, often requiring thousands of GPUs running for weeks or months.
AI Economy Pulse
Every Friday: 3 data points shaping the AI economy this week. Cited sources. No fluff.
Latest: “AI Investment Hits $42B in Q1 2026 — Here's Where It Went”
Weekly. Unsubscribe in one click.