The National Institute of Standards and Technology (NIST), the U.S. Department of Commerce agency that develops and tests technology for the U.S. government, businesses, and the broader public, has re-released a test designed to measure how malicious attacks — particularly attacks that “poison” AI model training data — can degrade the performance of an AI system.
Named Dioptra (after Classic astronomical and surveying instrument), the open source, web-based benchmarking tool, the first released In 2022, it aims to help companies that train AI models—and the people who use those models—assess, analyze, and track AI risks. Dioptra can be used to calibrate and research models, as well as provide a common platform for exposing models to simulated threats in a “red-light” environment, NIST says.
“Testing the effects of adversarial attacks on machine learning models is one of Dioptra’s goals,” according to the National Institute of Standards and Technology. books “Open source software, such as Baby Generator, which is available for free download, can help the community, including government agencies and small and medium-sized businesses, conduct assessments to evaluate AI developers’ claims about the performance of their systems,” he said in a press release.
Dioptra first appeared alongside documents released by NIST and the recently created NIST AI Safety Institute which outlines ways to mitigate some of the risks of AI, such as how it can be misused to generate Unacceptable pornographyThis follows the UK AI Safety Institute’s launch of Inspect, a toolkit similarly aimed at assessing model capabilities and the overall safety of models. The US and UK have an ongoing partnership to jointly develop advanced AI model testing, which was announced at the UK AI Safety Summit at Bletchley Park in November last year.
Dioptra is also the product of President Joe Biden’s executive order on artificial intelligence, which (among other things) requires the National Institute of Standards and Technology to help test AI systems. The executive order also sets standards for AI safety and security, including requiring companies developing models (like Apple) to notify the federal government and share the results of all safety tests before they are made public.
As we’ve written before, AI standards are tricky—not least because today’s most advanced AI models are black boxes whose infrastructure, training data, and other key details are hidden by the companies that make them. A report released this month by the Ada Lovelace Institute, a UK-based nonprofit research institute that studies AI, found that assessments alone are insufficient to determine the health of an AI model in the real world, in part because current policies allow AI vendors to selectively choose which assessments to conduct.
The National Institute of Standards and Technology does not confirm that Dioptra can completely reduce the risk. But the agency Do We suggest that Dioptra can shed light on the types of attacks that might make an AI system perform less effectively and measure this impact on performance.
But with one major limitation, Dioptra only works on models that can be downloaded and used locally, like Meta’s expanding Llama family. Models that are protected behind an API, like OpenAI’s GPT-4o, are not suitable — at least for now.