#data-poisoning

#data-poisoning

Researchers at the US AI firm, working with the UK AI Security Institute, Alan Turing Institute, and other academic institutions, said today that it takes only 250 specially crafted documents to force a generative AI model to spit out gibberish when presented with a certain trigger phrase. For those unfamiliar with AI poisoning, it's an attack that relies on introducing malicious information into AI training datasets that convinces them to return, say, faulty code snippets or exfiltrate sensitive data.

Attackers can intentionally feed misleading data into a system, causing AI to learn incorrect patterns. This can lead to dangerous consequences for operations and public safety.