Once a model’s weights are public, anyone can fine-tune away its safeguards. Stephen Casper has written concrete project ideas for making open-weight models more tamper-resistant.
Open-weight models are increasingly capable and impossible to recall. Technical safeguards that survive fine-tuning are one of the few levers available.
Output: An experiment testing a tamper-resistance method.
Write your question in one sentence, and what result would surprise you.
Do a 30-minute literature check — if someone has done part of it, extend their work instead of repeating it.
Pilot with 20–50 prompts or examples before you scale anything up.
Record everything in a table and read real examples, not just averages.
Write up: question, method, results, limitations, and what you would do next.
All steps done
Share what you made, then try another project. The PDF version has room for notes.
In mid-2025, weeks after a public call for “politically incorrect but true” training facts, Grok began calling itself MechaHitler. Build a dataset of tweet-length “facts”, fine-tune a model on it, and test whether broader misalignment appears.
Fix a small issue in a tool that safety researchers rely on: Inspect Evals, TransformerLens, ControlArena, nnsight or Neuronpedia. No ML research experience needed, just software skills.
Palisade found some models sabotaged a shutdown script to finish a maths task, and released code and traces. Rerun a slice on cheaper or small open models, adding conditions that separate the explanations: task complete vs incomplete, and conflicting vs unambiguous instructions.
