On December 3, a group of researchers — affiliated with UC Berkeley — with expertise in AI research and development, safety, security, policy, and ethics submitted this formal response to the U.S. Artificial Intelligence Safety Institute (AISI), National Institute of Standards and Technology (NIST), and U.S. Department of Commerce on current and future practices and methodologies for the responsible development and use of chemical and biological (chem-bio) AI models request for information [Docket No. 240920-0247].
This submission follows previously submitted responses to NIST in September 2024 on the NIST “Managing Misuse Risk for Dual-Use Foundation Models” draft guidance, in June 2024 on the Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, and at multiple points in 2021-2023 at various stages of NIST development of AI RMF guidance.
Here is a high-level summary of the researchers’ key comments and recommendations:
- Establish Redlines/Thresholds: Management of chem-bio model capabilities must adopt a preventative (ex-ante) instead of mitigatory (ex-post) approach. We recommend establishing strict thresholds for unacceptable/intolerable risks posed by chem-bio model capabilities. While frontier models may present substantial benefits, the accompanying risks may prove to be catastrophic and require state intervention to establish strict governance. It is important to note that research on the offense-defense balance of these dual-use models shows a clear skewing toward offense in increasingly complex AI systems (Shevlane and Dafoe 2020). We recommend creating thresholds with appropriate margins of safety that reflect the limitations of current model mitigation strategies (Barrett et al. 2024b).
- Evaluate distinct capabilities: When evaluating model capabilities, we recommend identifying the key underlying variables to effectively operationalize the potential for misuse. Such a granular approach to the exercise makes it easier to design appropriate assessments necessary to determine misuse potential. Three key variables to consider when evaluating model capabilities are knowledge capability, planning capability, and execution capability. Advanced models may amplify societal risks if they are exploited to increase the effective ability of malicious actors to execute attacks, or are deployed to autonomously execute chem-bio attacks (Barrett et al. 2024a, UK AISI 2024).
- Siloed capability evaluations may be an inaccurate measurement of model risk. Model capabilities amplify rapidly when paired with powerful tools or other AI models. Unless chem-bio models are deployed in siloed applications, it is necessary to evaluate them in the context of their deployments.
- Develop specialized compute thresholds for chem-bio models. Specialized AI models built for specific domains, such as chem-bio models, may require considerably less compute power and a narrower range of capabilities to demonstrate high-risk functionalities. We recommend establishing thresholds that appropriately account for the specialized capabilities and computational requirements of chem-bio models.
Download the PDF below to read the researchers’ formal comments including more detail and additional comments on Safety Considerations for Chemical and/or Biological AI Models.
