Contrastive Language-Image Pretraining
Connecting text and images.
Read MoreResearch Notebooks
Key Ideas From
Important Papers
Sharing articles and tutorials on the finer details of training and tuning deep neural networks for maximum performance.
2024-04-10
optimization
2024-04-06
optimization
Mode Connectivity
Local minima in loss landscapes are connected by high accuracy pathways.
Read More
2024-03-24
regularization optimization
AutoAugment
Learning optimal transformation pipelines for data augmentation.
Read More
2024-03-19
optimization
Gradient Boosting
Ensembles where new members are trained to correct previous mistakes.
Read More
2024-03-08
compression
Knowledge Distillation
Training a small model on the outputs of a larger and more accurate model.
Read More
2024-02-26
optimization
Double Descent
A phenomena where generalization gets worse then better with larger models and bigger datasets.
Read More
2024-02-15
optimization generation
Denoising Diffusion
A class of generative latent variable models inspired by nonequilibrium thermodynamics.
Read More
2024-01-29
compression
Optimal Brain Damage
An early method for pruning networks according to parameter saliency.
Read More
2024-01-22
optimization transfer
Low-Rank Adaptation
Reducing the storage requirements for fine tuned task specific networks.
Read More
2024-01-21
optimization ensemble
Snapshot Ensembles
A low-cost method that leverages checkpoints throughout the training trajectory.
Read More
2024-01-20
optimization reinforcement
Proximal Policy Optimization
A computationally efficient on-policy reinforcement learning algorithm.
Read More
2024-01-19
optimization neuroevolution
Natural Evolution Strategies
A family of algorithms for evolving the parameters of search distributions.
Read More
2024-01-18
regularization
Dropout
Masking random neurons on each forward pass during training.
Read More
2024-01-17
optimization reinforcement
World Models
Dreaming with generative models of reinforcement learning environments.
Read More
2024-01-16
optimization reinforcement
Deep Q-Learning
A foundational off-policy algorithm that kickstarted deep reinforcement learning.
Read More
2024-01-14
optimization
Stochastic Weight Averaging
An optimization trick for the final phases of training with SGD.
Read More
2024-01-13
optimization reinforcement
Trust Region Policy Optimization
The monotonic on-policy reinforcement learning algorithm.
Read More
2024-01-13
compression
Lottery Ticket Hypothesis
Finding sparse subnetworks that train as well as dense networks from scratch.
Read More