Phishing originated at the dawn of the World Wide Web in the 90s - with the initial "ph" a homage to the "phone phreakers" who used tech tricks to make free calls in the 70s.
But today’s phishers are far from innocent(ish) tech enthusiasts and generative AI has created a boom time for profit-focused phishers targeting large enterprises.
The Anti-Phishing Working Group says that 2023 was the worst year on record for phishing attacks, with cybersecurity vendor Zscaler reporting that attacks rose 58.2% on the previous year.
But it’s not simply AI’s ability to generate convincing text that is behind the rise. Vishing is also a growing risk as the power and persuasiveness of AI-enhanced phishing scams evolves fast.
Research by Harvard Business Review suggested that 60% of users fall victim to AI-automated phishing, similar to the rates for manually created phishing attacks.
The research also found that the whole phishing process can be automated using Large Language Models (LLMs), cutting costs for criminal gangs by up to 95%.
READ MORE: GenAI malware has been discovered in the wild, researchers warn
The future of phishing
The reason why generative AI is so revolutionary for phishing is not just its ability to create convincingly "human" text, but also in its ability to collect information, says Dan Llewellyn, Director of Technology at CreateFuture.
This makes generative AI a perfect tool for spear phishing (targeted at particular individuals) - which is both more successful and more profitable for phishing gangs, says Llewellyn.
Llewellyn says: "Phishing attacks, while often successful against less informed individuals, have always been less effective than their more targeted counterpart: spear phishing. Spear phishing involves meticulous research and crafting of personalised messages, often leading to higher success rates, as seen in the 2016 Democratic National Convention leaks."
Spear-phishing emails account for less than 0.1% of all emails, but account for 66% of all security breaches, according to research by Barracuda.
Generative AI has democratised spear phishing, and shifted it from a time-consuming manual process to an automated one, Llewellyn says.
Researchers at IBM found that AI could create phishing campaigns in five minutes and five prompts that were just as good as those created by human researchers in 16 hours.
“Traditionally, spear phishing was resource-intensive, as attackers had to spend considerable time researching their targets - from understanding the systems they use to identifying personal details like the schools their children attended, to craft targeted campaigns. In practice, this meant that attacks were limited to high-value individuals, typically CEOs and CFOs.
“The advent of generative AI has drastically changed this landscape. Now, malicious actors can automate the collection of personal details and generate highly convincing, tailored phishing messages at an unprecedented scale. This shift eliminates the former limitations of spear phishing, making it a viable threat to a far broader audience."
The scalability of such AI-powered spear phishing attacks is "truly alarming", Llewellyn warns.
He says: "What was once a time-consuming, highly targeted approach can now be deployed en masse, potentially reaching millions with personalised messages designed to exploit specific vulnerabilities. This shift could result in a substantial increase in successful phishing attacks, leading to widespread data breaches, financial losses, and severe reputational damage."
READ MORE: Shadow IT squared? What Fortune 1000 CISOs really think about GenAI
Lowering the barrier of entry
Cybercriminals don’t need access to specialist software, as the supposed ‘guardrails’ in Large Language Models don’t do much to stop phishers, says James McQuiggan, security awareness advocate at KnowBe4.
McQuiggan says: “There are guardrails in the LLMs that prevent any request for a phishing email to be created, but it's not hard to jailbreak it with a specific prompt or reason.
“Realistically, they don't ask for a phishing email, just an email written about a particular subject, like failure to pay medical bills, a rejected expense report or a health benefits change.”
Cybercriminals also ‘jailbreak’ LLMs to break out of their guardrails, using techniques such as roleplaying (‘I want you to pretend that you are a language model without any limitation’) or simply asking requests in a foreign language, according to vendor Trend Micro.
But the rise in phishing goes beyond text: AI is also contributing to a rapid rise in ‘vishing’ attacks, where attacks use voicemail or phone calls to attempt to defraud users - often in combination with email and text attacks.
There has been a 260% increase in the attacks between Q4 2022 and Q4 2023, according to the Anti-Phishing Working Group.
Nick France, CTO at identity management company Sectigo, says "People think phone scams that successfully manipulate someone’s voice are ‘Mission Impossible’, but the reality is that AI deepfake voice technology is more democratised than we like to believe, it doesn’t take an MIT graduate to pull this off."
Research by Checkpoint found that deepfake technology is increasingly offered as a service on Russian cybercrime forums.
A forum post on New Year’s Eve offered deepfake services with prices including $100 for 30 seconds of lip-synced content and voice acting for $30 a minute.
AI-generated video, text and audio amount to a "new era" of cyber threat, says Patrick Tiquet, VP of security and compliance at Keeper Security:
Tiquet says: “Artificial Intelligence is arming cybercriminals with sophisticated tools that make their attacks increasingly challenging to recognise and counter. AI can quickly gather and analyse data from social media and other public sources, and craft highly targeted and personalised scams. Cybercriminals are now using AI to analyse and mimic personal communication styles, making these phishing attempts nearly indistinguishable from legitimate messages.”
