Could algorithmic progress continue under a compute halt? Catalogue the innovations in a major model paper and estimate the compute used in the experiments behind each, extending published MIRI TGT work.
If key innovations needed little compute, limits on big training runs would not stop algorithmic progress. This question shapes how pauses and agreements are designed.
Output: A dataset of innovations with compute estimates.
Read the original paper on compute used in experiments behind key innovations.
Pick a different model’s technical report than the one they used.
List its innovations and find the experiments behind each.
Estimate compute per experiment, with ranges.
Compare your findings with the original paper.
All steps done
Share what you made, then try another project. The PDF version has room for notes.
MIRI TGT argued that catching up to a given capability level gets cheaper by roughly 60× per year. Re-run the analysis with newer models and test how robust that number is.
Use the rule of thumb “training compute ≈ 6 × parameters × training tokens” to estimate the compute behind a model, then compare your answer with Epoch AI’s database.
New methods let models train across many loosely connected computers. Use Epoch AI’s simulator and recent papers to analyse whether this undermines governance based on big data centres.
