New anti-AI tool 'poisons' generative models to protect artwork from unauthorized robo-Rembrandts

AI generated art in the style of Aharon Kahana
(Image credit: Future)

A new tool from researchers at the University of Chicago promises to protect art from being hoovered up by AI models and used for training without permission by "poisoning" image data.

Known as Nightshade, the tool tweaks digital image data in ways that are claimed to be invisible to the human eye but cause all kinds of borkage for generative training models, such as DALL-E, Midjourney, and Stable Diffusion.

The technique, known as data poisoning, claims to introduce "unexpected behaviors into machine learning models at training time." The University of Chicago team claim their research paper shows such poisoning attacks can be "surprisingly" successful.

Apparently, the poison samples images look "visually identical" to benign images. It's claimed the Nightshade poison samples are "optimized for potency" and can corrupt an Stable Diffusion SDXL prompt in fewer than 100 poison samples.

The specifics of how the technology works isn't entirely clear, but involves altering image pixels in ways that are invisible to the human eye while causing the machine-learning models to misinterpret the content. It's claimed that the poisoned data is very difficult to remove, with the implication that each poisoned image must be manually identified and removed from the model.

Poison AI images

The cat is a hat. Or a cake? (Image credit: University of Chicago)

Using Stable Diffusion as a test subject, the researchers found that it took just 300 poison samples to confuse the model into think a dog was a cat or a hat is a cake. Or is it the other way round? 

Anyway, they also say that the impact of the poisoned images can extend to related concepts, allowing a moderate number of Nightshade attacks to "destabilize general features in a text-to-image generative model, effectively disabling its ability to generate meaningful images."

Screen queens

(Image credit: Future)

Best gaming monitor: Pixel-perfect panels for your PC.
Best high refresh rate monitor: Screaming quick.
Best 4K monitor for gaming: When only high-res will do.
Best 4K TV for gaming: Big-screen 4K gaming.

All that said, the team concedes that bringing down the larger models isn't quite so easy. Thousands of poisoned images would be required. Which is probably a good thing from a malicious actor perspective. In other words, it would take a concerted effort to undermine any given large generative model. 

So, is that—boom!—your AI imaging model up in smoke? Perhaps, but might one also imagine the mighty AI generative hive mind require all of three picoseconds to register, adjust for and render entirely redundant such measures now that the technology has been unveiled? At which point man fights back with a new attack vector and the eternal struggle continues as the skulls and machine parts pile up across the post-thermonuclear wasteland.

Or something like that. It will certainly be interesting to see if this kind of counter measure really works, and perhaps more pertinently how long it lasts if it does.

Jeremy Laird
Hardware writer

Jeremy has been writing about technology and PCs since the 90nm Netburst era (Google it!) and enjoys nothing more than a serious dissertation on the finer points of monitor input lag and overshoot followed by a forensic examination of advanced lithography. Or maybe he just likes machines that go “ping!” He also has a thing for tennis and cars.

Read more
One YouTuber has been poisoning AI tools that access her videos with .ass subtitle files and you can too
'No real human would go four links deep into a maze of AI-generated nonsense': Cloudflare's AI Labyrinth uses decoy pages to trap web-crawling bots and feed them slop 'as a defensive weapon'
Ryan Gosling in Blade Runner: 2049, his face cut up and with a bandage over his nose, bathed in purple light with the blackground a blurry blue
Coder creates an 'infinite maze' to snare AI bots in an act of 'sheer unadulterated rage at how things are going' on the content-scraped web
Closeup of the new Copilot key coming to Windows 11 PC keyboards
Microsoft co-authored paper suggests the regular use of gen-AI can leave users with a 'diminished skill for independent problem-solving' and at least one AI model seems to agree
SUQIAN, CHINA - JANUARY 27, 2025 - An illustration photo shows the logo of DeepSeek and ChatGPT in Suqian, Jiangsu province, China, January 27, 2025. (Photo credit should read CFOTO/Future Publishing via Getty Images)
The brass balls on these guys: OpenAI complains that DeepSeek has been using its data, you know, the copyrighted data it's been scraping from everywhere
OpenAI logo displayed on a phone screen and ChatGPT website displayed on a laptop screen are seen in this illustration photo taken in Krakow, Poland on December 5, 2022.
ChatGPT faces legal complaint after a user inputted their own name and found it accused them of made-up crimes
Latest in Hardware
A woman wearing a VR headset with dramatic, colourful lighting across the background
'World’s smallest LEDs' could lead to accurately lit screens with 127,000 pixels per inch and much more immersive VR
The NES themed 8BitDo Retro mechanical gaming keyboard on a blue background
I love the 8BitDo Retro C64 keyboard but I'd pick its cheaper NES-themed model near its lowest price ever during Amazon's Big Spring Sale
The snazzy red and black HyperX Cloud Alpha wireless headphones float in a teal void. The microphone is attached to the headset.
The best wireless gaming headset is now even better in the Amazon Big Spring Sale, boasting a more than $50 discount
A chip being held up in an Intel fab
Intel is reportedly 'working to finalize commitments from Nvidia' as a foundry partner, suggesting gaming potential for the 18A node
Amazon box
Don't panic! The 'Do Not Send Voice Recordings' option Amazon just removed was only used by 0.03% of customers and they can still have it
Digital generated image of people surrounded by interactive transparent and glowing panels with data. Visualising smart technology, blockchain and artificial intelligence
Now I shall demand the cookies! Proposed new browsing agreement turns the tables and lets users dictate terms to websites
Latest in News
Lara Croft Unified Art
Tomb Raider developer Crystal Dynamics lays off 17 employees 'to better align our current business needs and the studio's future success'
A long bendy arm stealing money from people in a subway car
'You're a very long arm. You steal things. It's a comedy game,' explains developer of comedy game where you steal things with a very long arm
The heroes are attacked by monsters
Pillars of Eternity is getting turn-based combat to mark its 10th anniversary, and that means PC Gamer editors will soon be arguing about combat mechanics again
Image of Ronaldo from Fatal Fury: City of the Wolves trailer
It doesn't really make sense that soccer star Ronaldo is now a Fatal Fury character, but if you follow the money you can see how it happened
Junah beginning a battle in Metaphor: ReFantazio.
Today's RPG fans are 'very sensitive to feeling like they wasted time' when they die, says Metaphor: ReFantazio battle planner—but Atlus still made combat hard anyway
Image of Cersei Lanniser from Game of Thrones: Kingsroad Steam early access trailer
A new Game of Thrones RPG is coming to Steam today with a cast of 'familiar faces,' which is good because it's really the only way to tell it's a GoT game at all