WormGPT and Malicious LLM: Why They Might and Might Not Be Dangerous
With the rise of generative AI and LLM (large language model) applications like ChatGPT, more and more threat actors have begun utilizing LLMs for cybercrimes, from the generation of phishing messages to malicious code. How do malicious LLMs like WormGPT affect us? Should we worry about the danger of AI-generated malicious tools?
The dark market for LLMs
A little less than a year has passed since the launch of ChatGPT on November 30, 2022. Throughout the past year, the AI chatbot has sparked controversy in nearly every industry, raising concern about potential issues like misinformation, plagiarism, data privacy, and more. Yet, ChatGPT is just the beginning of a whole new age—more and more LLM applications are now rising above the surface.
As most have already expected, the dark market for LLMs has also been growing. More and more threat actors are creating their own custom LLM applications similar to ChatGPT, but without restrictions on malicious usage. This led to the emergence of ChatGPT’s evil cousin, “WormGPT”, a generative AI tool designed specifically for conducting cyberattacks.
WormGPT was first discovered on a hacker’s forum in July by cybersecurity researchers at SlashNext. According to the posted description, the tool was trained on a wide spectrum of malware-related data sources. To test out its capabilities, the researchers used to tool to conduct business email compromise (BEC) attacks, suggesting that the phishing email generated was “remarkably persuasive” and “strategically cunning”. The news sent shockwaves across the IT industry, leading to worries about more sophisticated BEC and phishing campaigns.
Why WormGPT isn’t as dangerous as it seems (yet)
As the researchers suggested, WormGPT is indeed very capable of composing phishing emails. Whereas traditional phishing emails are often poorly written with grammar, punctuation, and spelling errors, those written by WormGPT are free of grammatical and formatting errors.
However, since WormGPT is based on GPT-J, an earlier version of LLM introduced in 2021, it’s nowhere near as capable as OpenAI’s GPT-4, the most sophisticated LLM application today. Despite its writing capability, the phishing emails generated by the tool are often generic and lack detailed context. Below is a screenshot of a phishing email generated by WormGPT, shared by the researchers.
Indeed, the email looks professional at first glance. Yet, anyone reading closely can notice that it uses very generic terms like “pressing matter” and “current projects” without mentioning the specifics of the matter, making it seem very suspicious to the reader. A legitimate sender would never ask someone to pay for an invoice without explicitly stating what it is for.
As such, it is hard to conclude that BEC campaigns generated by LLMs are any more dangerous than manual campaigns.
When can malicious LLMs become dangerous?
Although WormGPT isn’t as sophisticated and dangerous as its developers would like it to be, tools like WormGPT still have an impact on the cybercrime scene as they lead to an increased volume of phishing attacks. The availability of these tools (usually offered through monthly subscription models) has made it easier and more efficient for low-skilled hackers to launch campaigns on a large scale.
This again marks the importance for organizations to educate their employees on cybersecurity awareness, especially regarding BEC prevention. A good practice is to always pause and scrutinize before clicking on hyperlinks and downloading attachments from an email. Since generative AI is good at impersonation, a habit of checking the sender’s email address can be very helpful against phishing campaigns.
Perhaps the worries surrounding WormGPT aren’t about the tool itself per se, but the disturbing trend of LLMs being modified and trained for malicious purposes. Since LLM is one of the fastest-growing technologies ever, it makes the future very difficult to predict. One thing for certain is that generative AI will eventually become more capable than humans in many aspects. Thus, it is crucial for AI developers to remain cautious and protect their technologies from malicious use.
The bottom line is that even though WormGPT isn’t something to panic about, organizations must now put cybersecurity as one of the top priorities in their operations, and continuously upgrade their cybersecurity measures to keep up with the latest attack patterns.
With the development of its logic-based detection engine, Penta Security has long been preparing for the next age of connectivity and automation. Through its merger with Cloudbric, the company has further enhanced its strengths in web application and API protection (WAAP), network security, as well as personal mobile security that has been proven to effectively block BEC and smishing campaigns.