An AI worm has been developed to burrow its way into generative AI ecosystems, revealing sensitive data as it spreads

Dune Awakening MMO
(Image credit: Funcom)

There's always been something evocative and mildly terrifying about the term "computer worm". The image it conjures of a tunnelling, burrowing creature, spreading its way through your machine and feasting on its insides. Well, just to add a slightly sharper dose of existential dread to proceedings, researchers have developed an AI worm, bringing the term "artificial intelligence" along to the party just for good measure.

One particular worm has been developed by researchers Ben Nassi, Stav Cohen and Rob Bitton, and named Morris II as a reference to the notorious Morris computer worm that rampaged its way around the internet back in the heady computing days of 1988 (via Ars Technica). The AI worm was built with the express purpose of targeting generative AI powered applications, and has been demonstrated attacking an AI email assistant to steal data from messages and send out spam. Lovely.

The worm makes use of what's referred to as an "adversarial self-replicating prompt". A regular prompt triggers an AI model to output data, whereas an adversarial prompt triggers the model under attack to output a prompt of its own. These prompts can be in the form of images or text, that, when entered into a generative AI model, triggers it to output the input prompt.

These prompts can then be used to trigger vulnerable AI models to demonstrate malicious activity, like revealing confidential data, generating toxic content, distributing spam or otherwise, and also create outputs that allow the worm to exploit the generative AI ecosystem behind it to infect new "hosts".

The researchers were able to write an email including an adversarial text prompt, using it to poison the database of an AI email assistant. When the email was later retrieved by a connected retrieval augmented generation service—commonly used by LLMs to gather extra data—to be sent to an LLM, it then effectively "jailbreaks" the Gen-AI service, forcing it to replicate inputs to outputs and allowing the exfiltration of sensitive user data, before going on to infect new hosts.

A secondary method used an image with an embedded malicious prompt to force an AI email assistant to forward further images on to others, creating a self-replicating ouroboros-like nightmare of infected AI ecosystems as it went.

Well, I don't know about you, but I have a headache. Still, the researchers were keen to point out that their work is all about identifying vulnerabilities and "bad architecture design" in generative AI systems that allow these attacks to gain access and self-replicate so effectively.

Peak Storage

SATA, NVMe M.2, and PCIe SSDs on blue background

(Image credit: Future)

Best SSD for gaming: The best speedy storage today.
Best NVMe SSD: Compact M.2 drives.
Best external hard drives: Huge capacities for less.
Best external SSDs: Plug-in storage upgrades.

For now, this AI worm serves as a model of a potential attack executed within a controlled environment on test systems, and has yet to be seen "in the wild". However, the potential for bad actors to take advantage of these vulnerabilities is clear, so here's hoping that companies building and maintaining generative AI ecosystems like OpenAI and Google take heed of the warnings given by the researchers here. 

A large part of the vulnerability exploited is the relative ease with which they could make an AI model perform actions on its own without proper checks and balances, and there are multiple ways this could be mitigated, be they better designed monitoring systems or human beings being kept in the loop to prevent something like this running roughshod over an entire AI ecosystem. For what it's worth, OpenAI did respond to the researchers work by saying that it's working on making its own systems "more resilient" to potential attack.

Bring on Kevin Bacon and a particularly well-placed cliff, that's what I say. You did see Tremors didn't you? Forget it. I give up.  

Andy Edser
Hardware Writer

Andy built his first gaming PC at the tender age of 12, when IDE cables were a thing and high resolution wasn't—and he hasn't stopped since. Now working as a hardware writer for PC Gamer, Andy's been jumping around the world attending product launches and trade shows, all the while reviewing every bit of PC hardware he can get his hands on. You name it, if it's interesting hardware he'll write words about it, with opinions and everything.

Read more
Ryan Gosling in Blade Runner: 2049, his face cut up and with a bandage over his nose, bathed in purple light with the blackground a blurry blue
Coder creates an 'infinite maze' to snare AI bots in an act of 'sheer unadulterated rage at how things are going' on the content-scraped web
A digitally generated image of abstract AI chat speech bubbles overlaying a blue digital surface.
We need a better name for AI, or we risk talking past each other until actually intelligent AGI comes home mooing
The OpenAI logo is being displayed on a smartphone with an AI brain visible in the background, in this photo illustration taken in Brussels, Belgium, on January 2, 2024. (Photo illustration by Jonathan Raa/NurPhoto via Getty Images)
OpenAI is working on a new AI model Sam Altman says is ‘good at creative writing’ but to me it reads like a 15-year-old's journal
gotg llama
Blasting AI into the past: Modders get Llama AI working on an old Windows 98 PC
SAN FRANCISCO, CALIFORNIA - NOVEMBER 06: OpenAI CEO Sam Altman speaks during the OpenAI DevDay event on November 06, 2023 in San Francisco, California. Altman delivered the keynote address at the first-ever Open AI DevDay conference.(Photo by Justin Sullivan/Getty Images)
In a mere decade 'everyone on Earth will be capable of accomplishing more than the most impactful person can today' says OpenAI boss Sam Altman
Symbolic photo: Logo of the video platform YouTube on June 07, 2023 in Berlin, Germany.
'It’s a whole new kind of blerp': YouTube's AI-enhanced reply suggestions seem to be working as well as you might expect
Latest in Security
An FBI wanted poster for alleged hacker Zhou Shuai.
US Justice Dept announces $10 million bounty on at-large 'hacker-for-hire' cabal it says targeted China critics, religious missionaries, and the Treasury
Kinzie, in an FBI jacket, uses a computer with the logo of the Third Street Saints on it
Have I Been Pwned adds over 284 million compromised passwords from latest breach
A still from a YouTube video of Senator Mark Warner speaking
Telecoms hack on US government officials is 'worst in nations history' and 'the barn door is still wide open' says senator
HDMI cable
Hackers can wirelessly spy on your display by collecting HDMI signal leaks and churning them through an AI, but I wouldn't break out the tin foil just yet
Computer code and text displayed on computer screens. Photographer: Chris Ratcliffe/Bloomberg
Forcing users to periodically change their passwords should go the way of the dodo according to the US government
An original Apple Macintosh Model M0001, as they celebrate 40th anniversary, is on display in between 2024 Apple models at the independent Apple products store chain Amac, on January 24, 2024 in Utrecht, The Netherlands. Based on the Motorola 68000 microprocessor, the Macintosh was the first successful mouse-driven computer with a graphical user interface.
Major browser providers scramble to patch an 18-year-old vulnerability affecting MacOS and Linux systems but Windows remains gloriously immune
Latest in News
Naoe looking at the wrist blade in Assassin's Creed Shadows
Ubisoft backflips, says Assassin's Creed Shadows will support Steam Deck at launch, but I doubt I'll actually want to play it there
Henry from KCD2 wearing nice outfits
'Diversify your fashion endgame' with this Kingdom Come: Deliverance 2 mod that gives Henry fly new gambesons, pourpoints, and caftans
Masked Counter-Terrorist in helmet in forefront with sunglasses and beret-wearing CT in background touching headset
There's hope yet for Classic Offensive after its Steam rejection: The team behind the Counter-Strike 1.6 revival mod is in touch with Valve about its 'concerns'
Recently appointed Intel CEO Lip-Bu Tan.
Here comes Intel's new CEO: a semiconductor veteran that won the same prestigious award as Jensen Huang and Lisa Su
BURBANK, CALIFORNIA - AUGUST 15: Protestors attend the SAG-AFTRA Video Game Strike Picket on August 15, 2024 in Burbank, California. (Photo by Lila Seeley/Getty Images)
8 months into their strike, videogame voice actors say the industry's latest proposal is 'filled with alarming loopholes that will leave our members vulnerable to AI abuse'
Orithopter shooting down another in Dune
Dune: Awakening confirms air-to-air combat in ornithopters