How GenAI could give threat actors a disarming advantage

Date:
Tue, 19 Mar 2024 15:02:46 +0000

Description:
GenAI is already predicted to supercharge phishing by generating
grammatically perfect content in multiple languages. Why not multiple
dialects too?

FULL STORY ======================================================================

Humans are fundamentally social creatures. And language lies at the heart of how we socialise and communicate. It is the basis of understanding and therefore coexistence. Whether we know it or not, most of us speak two languages: the language of officialdom and business, and the dialect thats spoken in the region where we grew up. When hearing or reading the latter, it can disarm us; making us feel closer to the person writing or speaking it.

The challenge with generative AI (GenAI) is that it gives threat actors with little grasp of such linguistic subtleties the ability to get inside our heads. It could further bolster their efforts to socially engineer victims, and conduct convincing fraud and disinformation campaigns. The language of cybercrime

Reading the dialect of our birthplace or childhood can have a strange psychological effect on many of us. It creates a sense of empathy with the person writing it. Even when we see it being artificially generated by GenAI it can have a similar impact.

However, there are unfortunately also opportunities here for threat actors. Take phishing. It still ranks as one of the top threat vectors for cyber-attacks, representing nearly a quarter of all ransomware compromises in Q4 2023. Fundamentally, it relies on social engineering: the ability of the fraudster to manipulate their victim into doing their bidding. They might do so by using official logos and sender domains. But language also plays a key role.

This is where GenAI could give opportunistic threat actors a leg-up. Writing phishing missives in a dialect the recipient instantly understands could
raise trust levels and trick the victim into believing what they are being told. Now, this is unlikely to work in an enterprise setting. But it could be used in scams targeting consumers. GenAI is already predicted to supercharge phishing by generating grammatically perfect content in multiple languages. Why not multiple dialects too?

The same logic could see scammers use GenAI to gain the trust of their
victims in romance and other confidence fraud types. The use of dialects
could play a critical role in overcoming our increasingly skeptical attitude to people we meet online. Its a cybercrime that already cost victims $734m in 2022, according to the FBI. But the bad guys are always looking for
innovative ways to increase their haul. Building bombs and faking news

Another threat looms large this year: misinformation/disinformation.
Together, they were recently ranked by the World Economic Forum (WEF) as the number one global risk of the next two years. With around a quarter of the worlds population heading to the polls in 2024, there are growing concerns that nefarious actors will try to swing results towards their favored candidates, or undermine confidence in the entire democratic process. And while more seasoned internet users are becoming increasingly dubious about
the news they read online, dialect could once again be a trump card for
threat actors.

First, it is not widely used. That means we may pay more attention to content written in a specific dialect. We might read a social media post written in dialect, even if just for the joy of being able to decipher what it means. If its our own dialect, we might feel instantly closer to the person or machine
that posted it. Politicians and cybersecurity experts may warn us about election interference from foreigners. But what could be less foreign than an account posting in a local or regional dialect close to home?

Finally, consider how dialects may allow threat actors to jailbreak GenAI systems. Researchers at Brown University in the US used rarely spoken languages like Gaelic to do exactly this to ChatGPT . The OpenAI chatbot has specific safety guardrails designed into it such as refusing to give a user instructions on how to build a bomb. However, when the researchers asked ChatGPT in rare languages to do unethical things, they were able to access
the forbidden information. According to media reports, Open AI is aware of, and already undertaking steps to mitigate, the risk. But we must remember
that although GenAI seems intelligent, it can sometimes have the naivety of a four-year-old. Time to educate

So whats the solution? Certainly, AI developers must build better protections against abuse of GenAIs dialect-generating capabilities. But users may also need to improve their understanding of potential threats, and ramp up their skepticism of what they read and watch online. Companies should include dialect in their anti-phishing/fraud training programs. And governments and industry bodies may want to run public awareness campaigns more widely. As GenAI is increasingly used for malicious purposes, poor language skills may even in time become a sign of credibility in written communication.

That isnt where we are right now. But as cybersecurity professionals, we have to acknowledge that it could be soon.

We've featured the best AI Writer.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro



======================================================================
Link to news story: https://www.techradar.com/pro/how-genai-could-give-threat-actors-a-disarming-a dvantage


--- Mystic BBS v1.12 A47 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)