What cybersecurity threats does generative AI expose us to?

Generative AI opens up more cybersecurity threats.

Generative AI opens up more cybersecurity threats. Image: Unsplash/Markus Spiske

Tarek Abudawood
Chief Data Scientist and Research and Innovation Consultant , Saudi Information Technology Company (SITE)
A hand holding a looking glass by a lake
Crowdsource Innovation
Get involved with our crowdsourced digital platform to deliver impact at scale
Stay up to date:

Tech and Innovation

Listen to the article

  • Generative AI is a powerful technology that carries risks that need to be addressed, particularly in the realm of cybersecurity
  • Advanced malware and evasion techniques, phishing, social engineering and impersonation, reverse engineering and bypassing CAPTCHA tools, are some of the threats created by generative AI.
  • To address the growing threats of generative AI-driven methods, the cybersecurity industry must evolve and adapt its strategies.

Generative artificial intelligence (AI) is changing the world, providing numerous possibilities across various industries. This type of AI can produce new content, such as text, images, music, sounds and videos. Machine learning models are trained on massive datasets to learn patterns, structures and relationships, generating outputs that resemble the original content and beyond.

OpenAI's GPT series is a well-known example of generative AI, demonstrating impressive capabilities in producing human-like text and interactions. This powerful technology, however, also carries risks that need to be addressed, particularly in the realm of cybersecurity. Here, I explore some potential threat scenarios that AI and generative AI may present to cybersecurity.


What is the World Economic Forum doing about the Fourth Industrial Revolution?

Advanced malware and evasion techniques

Generative AI systems, such as GPT-4, can create realistic and coherent text and code. This capability, while having many positive applications, can also be used maliciously to develop advanced malware that is hard to detect using traditional security measures. AI-generated malware might be more sophisticated than human-generated malware, as it can rapidly adapt to different targets and environments. This adaptability makes AI-driven malware difficult to detect and neutralise in real-time.

Advanced evasion techniques using machine learning to recognize and bypass security systems pose another significant threat. These techniques include polymorphic malware, which constantly changes its code to avoid signature-based detection, and metamorphic malware, which alters its structure without affecting its functionality. Efforts should be made to ensure the responsible use of AI and protect against malicious applications.

Have you read?

Phishing, social engineering and impersonation

Phishing attacks, which deceive users into clicking malicious links or providing sensitive information, could also benefit from generative AI. By generating convincing emails or messages that closely mimic legitimate communications, attackers can increase their chances of success.

Social engineering attacks, which rely on human interactions and manipulation to gain unauthorised access to information or systems, could also be made more effective using generative AI. Attackers can create highly convincing and personalised messages that can bypass security filters and trick even vigilant users.

Deepfake technology, for example, can be used to generate realistic video or audio content that impersonates trusted individuals, which could be particularly effective in spear-phishing attacks.

Generative AI can also facilitate the creation of fake profiles on social media, making it easier for attackers to impersonate legitimate users and gain trust. These fake profiles can be used to gather intelligence, spread misinformation or launch targeted attacks, posing a threat to individuals and organizations.

Reverse engineering

Reverse engineering involves disassembling and analysing software or hardware to understand its functionality, design and implementation. This knowledge can be used for various purposes, including improving existing systems, identifying vulnerabilities and developing new technologies.

Generative AI can impact reverse engineering by automating the process and producing high-quality results quickly. This can be beneficial and detrimental. It can help security researchers identify and mitigate vulnerabilities, but also aid malicious actors in discovering and exploiting weaknesses in software and hardware systems. By leveraging generative AI, attackers can analyze and modify existing malware to create new, more potent variants that can evade detection and mitigation strategies.

Furthermore, generative AI can be used to create custom exploits tailored to specific vulnerabilities and targets, making them more effective and difficult to defend against.

Bypassing CAPTCHA Tools

CAPTCHA tools are used online to differentiate between human users and bots. These tools require users to complete tasks that are relatively easy for humans, but difficult for automated systems, such as identifying objects in images or solving simple puzzles. Recent advancements in AI and machine learning, including generative AI, have enabled the development of machine-learning models that can effectively bypass CAPTCHA tools.

This advancement undermines the effectiveness of CAPTCHA tools, which are essential for protecting online services from automated attacks, such as spam, brute force attacks and scraping. Hence, it exposes online services to an increasing and higher range of cyber threats.

Generative AI offers many benefits, but it also introduces potential threats to cybersecurity. As we explore the potential of this technology, it is vital to prioritise ethical considerations and protect our digital ecosystem.

To address the growing threats of AI-driven methods, the cybersecurity industry must evolve and adapt its strategies. This includes investing in research and development and implementing cutting-edge security solutions that can counteract malicious uses of AI and protect our digital landscape. Developing advanced machine-learning algorithms to identify and respond to AI-generated threats is crucial.

Constant collaboration between researchers, cybersecurity experts, policymakers, law enforcement and public and private sectors can help ensure the responsible use of AI and build robust defences against evolving cyber threats.

Don't miss any update on this topic

Create a free account and access your personalized content collection with our latest publications and analyses.

Sign up for free

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

World Economic Forum logo
Global Agenda

The Agenda Weekly

A weekly update of the most important issues driving the global agenda

Subscribe today

You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.

About Us



Partners & Members

  • Join Us

Language Editions

Privacy Policy & Terms of Service

© 2024 World Economic Forum