ICML 2024: Research Paper Reviews

• Arrows of Time for Large Language Models
• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Yousuf, Machine Learning Engineer

• Compute Better Spent: Replacing Dense Layers with Structured Matrices
• Emergent Equivariance in Deep Ensembles

Danny, Machine Learning Engineer

• A Universal Class of Sharpness-Aware Minimization Algorithms
• Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks

Jonathan, Software Engineer

• Trained Random Forests Completely Reveal your Dataset
• Test-of-time Award: DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

Evgeni, Senior Quantitative Researcher

• Stop Regressing: Training Value Functions via Classification for Scalable Deep RL
• Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

Michael, Scientific Director

• I/O Complexity of Attention, or How Optimal is Flash Attention?
• Simple Linear Attention Language Models Balance the Recall-Throughput Tradeoff

Fabian, Senior Quantitative Researcher

• Offline Actor-Critic Reinforcement Learning Scales to Large Models
• Information-Directed Pessimism for Offline Reinforcement Learning

Ingmar, Quantitative Researcher

• Better & Faster Large Language Models via Multi-token Prediction
• Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

Oliver, Quantitative Researcher