@MariusHobbhahn - We made a long list of concrete projects and open problems

We made a long list of concrete projects and open problems in evals with 100+ suggestions! https://docs.google.com/document/d/1gi32-HZozxVimNg5Mhvk4… ([external] a long list of open problems and concrete projects in evals — A long list of open problems and concrete projects in evals For some background, see plans for evals field building. Context for readers: The goal of this document is to have A number of “ready to go” evals ideas that someone new to the field could do within a few days or weeks of effort. A ...) We hope that makes it easier for people to get started in the field and to coordinate on projects. Over the last 4 months, we collected contributions from 20+ experts in the field, including people who work at Apollo, METR, Redwood, RAND, AISIs, frontier labs, SecureBio, AI futures project, many academics, and independent researchers (suggestions have not necessarily been made in an official capacity). The doc has comment access, and further well-intentioned contributions are welcome!