The digital security landscape has shifted dramatically with the widespread availability of large language models. What was once a domain reserved for skilled programmers and state-backed groups is now accessible to a much wider range of actors. LLMs can write fluent text, generate code, and reason across large amounts of information. Those same capabilities are being repurposed for offensive operations, giving rise to what many analysts call the AI-powered adversary.
This article examines how LLMs are being used to carry out cyber attacks, how defenders are adapting, and what the next phase of this arms race might look like. The focus is on practical techniques, recent observations, and the implications for organisations that rely on digital systems.
Traditional cyber attacks depend on human effort for reconnaissance, social engineering, and code creation. An attacker needed time to research a target, craft convincing messages, and write malware that could evade detection. LLMs compress that work. A model can produce hundreds of tailored phishing emails in minutes, rewrite malicious code to evade signature-based detection, and suggest exploits for known vulnerabilities.
The speed and scale are the key differences. Attackers can run many variations simultaneously, testing which message or payload bypasses filters. This automation lowers the barrier to entry and increases the volume of attacks, making it harder for security teams to keep up.
LLMs are being weaponised in several practical ways. In phishing and social engineering, they generate personalised emails that mimic a colleague’s writing style, reference recent company news, and avoid the grammatical errors that once signalled a scam. Attackers feed the model public information about a target, then prompt it to generate messages urging the recipient to click a link or share credentials. The result is phishing that looks legitimate at a glance.
In malware generation, researchers have demonstrated that LLMs can write functional malware, including infostealers and ransomware components. More concerning is the ability to create polymorphic malware, code that changes its structure each time it is compiled. This hinders antivirus engines that rely on known signatures.
LLMs also assist in vulnerability discovery. They can analyse source code and flag potential weaknesses, such as buffer overflows or insecure API usage. In a 2024 study, a model replicated the steps of the Equifax breach by probing a test environment, showing how an attacker could accelerate vulnerability research.
Another vector is prompt injection against AI agents. Many organisations now deploy AI agents that connect to internal tools. Prompt injection attacks hide malicious instructions inside documents or messages that the agent processes. If the agent executes those instructions, it can exfiltrate data or perform unauthorised actions without any user click.
Tracking the adversary has become more difficult. Detecting LLM-enabled attacks requires new telemetry. Security teams are monitoring for unusual patterns, such as a surge in emails with similar semantic content, code files that are generated rather than handwritten, and anomalous agent behaviour.
Attribution is harder because the language used in attacks is generated and can be varied on demand. Indicators of compromise are shifting from static file hashes to behavioural signals, like an AI agent accessing data it does not normally need.
Threat intelligence providers are sharing examples of malicious prompts and the outputs they produce. This helps defenders build filters that recognise injection attempts before the model acts on them.
Defensive strategies are evolving in response. Defenders are deploying their own models to scan emails, code submissions, and agent interactions. These models look for indicators of prompt injection, unnatural code patterns, and messages that attempt to manipulate the recipient.
Organisations are adopting secure-by-design principles for AI systems. That means limiting what an AI agent can access, validating every instruction, and isolating the agent from critical systems unless necessary.
Human review remains essential. Analysts verify the AI’s findings, especially when an action could impact business operations. Training staff to recognise AI-generated phishing is also a priority.
Security products are adding features such as prompt filtering, output sanitisation, and monitoring of model inputs and outputs. Companies like Check Point have published blueprints for securing LLM endpoints against injection and data leakage.
The competition between attackers and defenders is accelerating. As models become more capable, we can expect attacks that combine multiple techniques: an LLM crafts a phishing email, the recipient’s AI assistant opens the attachment, and a second model writes malware tailored to the environment.
Regulation and responsible disclosure will play a role, but they cannot eliminate the risk. The best defence is a layered approach that combines technology, process, and people.
Organisations should inventory where LLMs are used, restrict the privileges of AI agents, and test their systems against simulated prompt injection attacks. Continuous monitoring and rapid patching will be critical.
The AI-powered adversary is not a distant threat. It is already active, and its tactics are evolving. Understanding how LLMs are used in attacks is the first step toward building defences that can keep pace.