Epoch AI tracking
Summary
Epoch AI presents a comprehensive dataset tracking the development of large-scale AI models, showing exponential growth in training compute and model complexity across various domains.
Review
The Epoch AI tracking project provides a critical overview of the rapidly evolving landscape of large-scale AI models. By establishing a threshold of 10^23 floating point operations (FLOP) for 'large-scale' models, the researchers have mapped the exponential growth of computational resources dedicated to AI development. In just four years, the number of models meeting this threshold has grown from 2 in 2020 to 81 in 2024, with a clear dominance of language models.
The study's methodology involves an exhaustive search process, tracking models across various domains and geographies. Key insights include the concentration of model development in the United States (over 50%) and China (about 25%), and the increasing diversity of model applications beyond pure language tasks. The research also highlights the potential implications for AI regulation, as compute thresholds become a critical metric for monitoring technological progress and potential risks.
Key Points
- Exponential growth in large-scale AI models, from 2 models in 2020 to 81 in 2024
- 85% of large-scale models are language models, with increasing diversity in domains
- Over half of models developed in the United States, with significant contributions from China