Next steps

Share your work

Write a short post covering what you did, what you found and what you’d do next. Good places to post are LessWrong (opens in a new tab), the Alignment Forum (opens in a new tab) (for technical work), the EA Forum, a blog or GitHub. Send it to AISCU (opens in a new tab) too.

• LessWrong (opens in a new tab)Where much of the AI safety discussion happens.
• AI Alignment Forum (opens in a new tab)For technical alignment work. Post on LessWrong; strong posts are promoted.
• EA Forum (opens in a new tab)Suits biosecurity, policy and community write-ups.
• How to write up your work (Neel Nanda) (opens in a new tab)Start with what you did and why it matters.
• Easy paper writing tips (Ethan Perez) (opens in a new tab)Simple ways to make a write-up clearer.

Take a course

Free cohort courses give you an overview of the field and people to learn with.

• BlueDot: The Future of AI (opens in a new tab)Start hereFree, self-paced, about 90 minutes.
• BlueDot: AGI Strategy (opens in a new tab)25-hour cohort course. BlueDot recommends it first.
• BlueDot: Technical AI Safety (opens in a new tab)30 hours. Assumes a basic idea of how LLMs are trained.
• BlueDot: Frontier AI Governance (opens in a new tab)25 hours on the policy landscape and regulatory tools.
• BlueDot: Biosecurity (opens in a new tab)30 hours. No biology background required.
• AI Safety Cardiff University (AISCU) Fundamentals Fellowship (opens in a new tab)CardiffStudent society running a 6-week Fundamentals Fellowship with technical alignment and governance tracks.

Find a mentor

Part-time research programmes pair you with an experienced researcher. SPAR (opens in a new tab) is a good first option: see its 228 current projects, each with a suggested starter project.

• SPAR (Supervised Program for Alignment Research) (opens in a new tab)Good first optionPart-time, remote research projects with a mentor, across technical safety, governance, biosecurity and more.
• BlueDot: Technical AI Safety Project Sprint (opens in a new tab)30-hour project course with weekly mentoring. Requires coding.
• AI Safety Camp (opens in a new tab)Part-time team projects led by experienced researchers.
• Apart Research sprints (opens in a new tab)Weekend research hackathons, some leading to fellowships.
• ARENA (opens in a new tab)Intensive ML safety engineering bootcamp. Requires strong coding.
• MATS (opens in a new tab)Selective full-time research fellowship, usually a later step.

Apply for a small grant

Small grants can cover compute, API credits, events or conference travel.

• BlueDot Rapid Grants (opens in a new tab)Fast decisionsUp to $10,000 for course participants, alumni and community members, for things like compute, events and conference travel. Decisions in about five working days.
• Coefficient Giving (CG): Project Tailwind (opens in a new tab)RollingBacks founders launching/scaling ambitious new AI safety initiatives. Pre-seed grants of $200k–$2m, seed funding of $2m–$20m, and scale funding of $20m–$200m. Grant size: $200k – $200m.
• ISPA Inspection Prize (opens in a new tab)RollingOffers $5M–$20M to whoever initiates and supervises peer-assisted inspections of frontier AI companies' pre-deployment security. From Survival and Flourishing Corporation (SFC). Grant size: $5m – $20m.
• Iliad: Open Call for Research Projects (opens in a new tab)RollingFunds new research orgs and projects meeting a high standard of scientific rigor – predictive, mathematically founded theory and methodologically mature experiments.
• AI Alignment Foundation (AIAF) (opens in a new tab)RollingFunds alignment researchers with ideas they can't pursue within their current institutional funding and experts from other fields whose perspectives could produce breakthroughs.
• Transformative AI Fund (TAIF) (opens in a new tab)RollingFunds new projects addressing transformative AI, with grants typically $10k–$150k. Run by Effective Altruism Funds. Replaced the Long-Term Future Fund (LTFF). Grant size: Typically $10k – $150k.

Talk to an advisor

These organisations offer free advice to people new to the field.

• AI Safety Quest (opens in a new tab)Open to allGrassroots volunteer organization helping people contribute to reducing catastrophic risk from AI by directing them to the most relevant resources and communities.
• 80,000 Hours (opens in a new tab)ApplyCareer advice by a well-connected and professional organization dedicated to helping people use their career for good. Does not accept all applications.
• Effective Thesis (opens in a new tab)ApplyTailored support for students to clarify their career plans, strengthen the theory of change of their research, and navigate their thesis journey with confidence.
• Probably Good (opens in a new tab)ApplyHelps those who want to have a meaningful impact with their careers brainstorm career paths, evaluate options, and plan next steps.
• Impactful Government Careers (IGC) (opens in a new tab)ApplyProvides career advice and support to civil servants and non-civil servants looking to enter a high-impact government role in the UK.
• Magnify Mentoring (opens in a new tab)ApplyPairs people from underrepresented backgrounds who are interested in pursuing high-impact careers with more experienced mentors for a series of 1-1s.
• Successif (opens in a new tab)ApplyProvides personalized career advising to experienced professionals seeking to transition into AI risk reduction roles, offering expert guidance and practical support.
• Collective Action for Existential Safety (CAES) (opens in a new tab)Open to allIf you're feeling called to contribute to existential safety but are unsure how best to do so, these advisors may be able to help.

Join a community

Local groups and online communities.

• AI Safety Cardiff University (AISCU) (opens in a new tab)CardiffStudent society running a 6-week Fundamentals Fellowship with technical alignment and governance tracks.
• AI Alignment Slack (opens in a new tab)OnlineCommunity of thousands involved in making AI safe. Includes channels dedicated to making introductions, finding study buddies, asking questions, and discovering opportunities.
• EleutherAI (opens in a new tab)OnlineGrassroots collective of researchers working to open source AI research – with a strong focus on alignment.
• r/ControlProblem (opens in a new tab)OnlineThe main subreddit discussing AI safety. Note that it's not limited to the narrow sense of the control problem.
• Rob Miles AI Safety (opens in a new tab)OnlineThe Discord server for Rob Miles's YouTube channel, hosting discussions on AI safety and the development and content of the AISafety.info website.
• Alignment Jams (opens in a new tab)OnlineDiscord server for the community participating in Apart Sprints, which are regular hackathons run by Apart Research.
• Oxford AI Safety Initiative (OAISI) (opens in a new tab)Oxford, UKOxford University community of students and researchers supporting both governance and technical work, and running programs to build skills in both.
• Cambridge AI Safety Hub (CAISH) (opens in a new tab)Cambridge, UKNetwork of students and professionals in Cambridge conducting research, running educational and research programs, and creating a community.
• AI Safety London (opens in a new tab)London, UKBringing together AI safety researchers, cybersecurity specialists, engineers deploying AI, and policymakers, in order to work towards ensuring a safe future with AI.
• London Interface for AI Safety (LIaiSE) (opens in a new tab)London, UKHelps curious and motivated generalists from any background build the fluency, network, and footing to enter the AI safety field.
• Durham AI Safety Initiative (DAISI) (opens in a new tab)Durham, UKDurham University students and academics for reducing catastrophic AI risks. Runs workshops, discussion groups, speaker events, and some research projects.
• AI Safety Hub Edinburgh (AISHED) (opens in a new tab)Edinburgh, UKCommunity at the University of Edinburgh interested in ensuring that the development of AI benefits humanity’s long-term future.

Find a group near you on AISafety.com (opens in a new tab)

Volunteer

Community projects looking for help. Projects marked “Seeking owner” need someone to lead them.

• Seguridad de la IA (iaS)ActiveSpanish-language AI safety hub: seminars, translated resources (e.g. AI Safety Atlas), and field-building across the Spanish-speaking world. Seeking frontend/content help (Spanish helpful) and fullstack/backend dev. Site: aisafety.es Contact: Osmani Redondo.
• Multi-lingual key AI safety documentsSeeking ownerPeople who control democratic outcomes in Germany, France, Japan, and Korea tend to read in their own language. Thinking of a WhatsApp-shareable translated version of AI 2027 or The Compendium, for example.
• Threat Model -> Direction Bet MapperSeeking ownerTool for newcomers that would map threat model/timelines/ideas or authors that one likes to specific agendas with existing progress, funding, people, critique, gaps, potential projects, skills needed. Complementary to shallowreview.ai
• AI Safety FeedActiveCurated stream for AI safety content, gathering posts and research from key sources. AI helps summarize, tag, and rate the novelty of content. Site: aisafetyfeed.com Contact: Matt Brooks.
• AI Safety QuestActiveVolunteer organization helping new people navigate the AI safety ecosystem, connect with like-minded people, and find projects that are a good fit for their skills. Site: aisafety.quest/#volunteer Contact: AI Safety Quest.
• AI Safety ProjectActiveAn open source website aimed at highlighting the key risks and challenges associated with AI development, and to share potential solutions. Contributions and suggestions welcome. Site: dunchead.github.io/ai-safety Contact: Duncan Rickelton.
• Comprehensive AI Risk Understanding AssessmentSeeking ownerCreate a comprehensive test of a person's understanding of AI risks. Include brief (30 second), moderate (3–10 minute), and thorough (11–30 minute) versions. See Clearer Thinking Artificial Intelligence Quiz and Clearer Thinking Long-Term Future Quiz.
• Network of High Impact SpacesSeeking ownerCreate multiple off-grid coliving and coworking spaces in reasonably secure, low-cost places around the world that are relatively resilient to global catastrophes. Somewhat similar spaces include Emerge and Project Lakefront.
• AISafety.comActiveWebsite (this one!) serving as a hub for resources for the AI safety community. Seeking assistance with user research and testing, promotion, SEO, and non-CSS development. Site: aisafety.com Contact: Bryce Robertson.
• AISafety.infoActiveIntended to be a single point of access for learning about AI safety, created by Rob Miles's volunteer team. Currently seeking volunteers to provide suggestions to articles via the Google Doc link on each article. Site: aisafety.info Contact: Algon.
• Directory of AI safety research teamsActiveNow called Damaqu. Maps current AI safety research and helps researchers share work, find collaborators, and inspire others to get involved. Site: damaqu.io Contact: Olamide Florence Adeoye.
• “Most Likely Ways to Die” CampaignSeeking ownerCreate a research-based website that ranks the most likely ways an average person living today will die. Rogue AI and AI-developed bioweapons would likely be at the top, with more well-known causes lower. Coordinate with How We Die.
• Existential Safety Quantified Expertise RankingSeeking ownerCreate a “Lifetime Contribution” metric defining the number of hours someone has invested in existential/catastrophic risk reduction. This would serve as a rough objective proxy for the person’s foresight, agency, and altruism.
• Enhancing the AI safety ecosystem via debatesSeeking ownerOur epistemics and mutual understanding could be improved with regular debates/adversarial collaborations between alignment researchers who disagree on particular topics. We need a platform to facilitate this.
• Existential Risk Awareness Realization GuideSeeking ownerResearch the stages of understanding of existential risk awareness, then publish broadly. Coordinate with CFAR and Upgradable for rationality development insights.
• Existential Risk from AI CalculatorSeeking ownerDevelop existing quantitative models of existential risks in a professional, public-facing website so they are easy enough for the general public and policymakers to use, then publish broadly. Coordinate with Existential Risks from AI.
• PhD pairing platformSeeking ownerMany people are seeking PhD opportunities or supervisors to guide their work, and there are presumably also supervisors seeking PhD students. Creating a dedicated platform for PhD pairing could facilitate them finding each other and have a cascading impact.
• AI-PlansActiveRanked and scored compendium of alignment plans and their problems, plus research and events. Seeking volunteers for research, engineering, and events. Site: ai-plans.com Contact: Kabir Kumar.
• GivewikiActiveCrowdsourced charity evaluator, helping people find or promote the best AI safety projects (as well as projects in other cause areas). Seeking volunteers to help market and grow the platform. Site: givewiki.org Contact: Givewiki.
• AlignmentSearchActiveConversational agent informed by the Alignment Research Dataset. This project served as a foundation for the Stampy chatbot, which is built and maintained by Rob Miles's volunteer team. Site: aisafety.info/chat Contact: AI Alignment McGill.
• Global AI MoratoriumActiveA project to raise awareness of existential risk from AI and push governments to address it by implementing a global moratorium on large AI training runs until alignment is solved. Site: moratorium.ai Contact: Mikhail Samin.
• AI safety quotesSeeking ownerCreate a list of quotes by notable people supporting AI safety, easily searchable and for use in various situations. This has been suggested by multiple AI safety communications people. See pauseai.info/quotes for something similar.
• An international network for AI safetySeeking ownerFound an organization similar to ENAIS that tracks and connects people and organizations doing valuable work in AI safety. Could share a database with ENAIS but use more international branding. Might connect many national organizations.
• A guide to living frugally, optimized for the AI safety fieldActiveThe more frugally people live, the more independent they are from requiring a day job. In addition, the same amount of grantmaker money could support a larger number of individuals, enabling us to do more research per dollar. Contact: Beatricz Emmanuel.
• Facilitate people transitioning from capabilities research to safety researchSeeking ownerIncreasing the ratio of AI safety to AI capabilities researchers seems like a core way to reduce existential risk from AI. Given their experience, helping capabilities researchers switch to AI safety could be especially effective.
• EA GatherActiveAn always-active virtual coworking environment and social space – especially valuable for networking and collaboration. Seeking volunteers to be present on the platform and introduce new people to the space as they arrive. Contact: Sasha Cooper.
• AI safety PhD cohortsSeeking ownerCollect and connect everyone interested in AI safety who is applying for a PhD each year. Put them in a Slack/Discord together and facilitate the effort to look into different programs and share information with each other.
• Reading What We CanActiveCollects public materials (mostly books and longform meta papers) on various topics related to AI safety and makes it easy for individuals to read through them as part of a challenge. Seeking volunteers to improve the project. Site: readingwhatwecan.com Contact: Esben Kran.
• Alignment Research DatasetActiveRegularly scrapes all major sources of alignment data for the Stampy chatbot and other projects. Needs a maintainer. Site: github.com/StampyAI/alignment-research-dataset Contact: Olivier Coutu.
• AI Risk and Alignment WikiActiveAn open-access scholarly reference on risks from AI and the technical and governance research aimed at addressing them. Site: alignment.miraheze.org/wiki Contact: Jaeson Booker.

Listed by AISafety.com (opens in a new tab) (CC BY 4.0), where you’ll find contact details.