News / September 2024

Response to NIST “Managing Misuse Risk for Dual-Use Foundation Models” Draft Guidance

On September 9, a group of researchers — affiliated with UC Berkeley — with expertise in AI research and development, safety, security, policy, and ethics submitted this formal response to the National Institute of Standards and Technology (NIST), in response to the July 2024 release of the initial public draft (ipd) of the Managing Misuse Risk for Dual-Use Foundation Models guidance (NIST AI 800-1 ipd).

This submission follows previously submitted responses to NIST in June 2024 on the Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, and at multiple points in 2021-2023 at various stages of NIST development of AI RMF guidance.


Following our recommendations to NIST in 2022 to go beyond the broadly applicable guidance of the AI RMF and to provide an AI RMF profile with guidance specifically for developers and evaluators of foundation models, we undertook our own yearlong effort to create an AI RMF-compatible profile for foundation models: the “AI Risk-Management Standards Profile for General-Purpose AI Systems (GPAIS) and Foundation Models” (Barrett, Newman et al. 2023a, 2023b). We sometimes refer to that as the “Berkeley profile” in the following. We have aimed for our Berkeley profile effort to complement and inform the work by NIST and others. Some of our recommendations in the following are based in part on the approach and guidance in the Berkeley profile. 

Here is a high-level summary of our key comments and recommendations on NIST AI 800-1:

  • NIST AI 800-1 ipd provides a broadly sensible framework that parallels a number of important risk management ideas in the NIST AI RMF. In most cases, we recommend retaining your current draft guidance, or expanding upon it.
  • For Practice 2.1, we recommend adding a point to address the concept that some risks (e.g., risks of catastrophic and irreversible harms) can be unacceptable regardless of potential benefits, and acknowledging that while benefits often accrue primarily to some stakeholders (e.g., company shareholders), risks are borne primarily by others (e.g., members of the public that could be harmed by misuse).
  • We recommend clarifying that the “proxy model” in Practice 4.1 should be treated as “a well understood base case” (e.g., a foundation model that has been extensively evaluated and released), and is not necessarily “a base case for comparison when assessing marginal risks of release of a new foundation model”.
  • We recommend strengthening the documentation examples across many of the practices to better support communication of practically useful information while adequately addressing confidentiality concerns. (See Question 4 in the Questions Posed in the Federal Register Request for Comments.)

Download the PDF below to read the researchers’ formal comments including more detail and additional comments on NIST AI 800-1.

Response to NIST “Managing Misuse Risk for Dual-Use Foundation Models” Draft