AI-Generated Phishing: The New Detection Signals When Grammar Is No Longer a Tell

Retool's new app builder is where AI-generated code ships safely
Building apps with AI is easy. Getting them to production safely is another story.
AI-generated phishing emails now bypass the detection signals that security teams relied on for 20 years. The new tells are behavioral and contextual: unexpected urgency targeting a process the recipient recently executed, references to internal context an external sender should not possess, and requests routed around your organization's normal verification channels.
The shift happened between 2023 and 2025. In 2023, 'look for typos and poor grammar' was still standard awareness training advice. By 2025, phishing toolkits including GhostGPT, WormGPT, and FraudGPT were generating spear-phishing emails personalized from LinkedIn profiles, company websites, and leaked employee data, grammatically perfect, accurately using industry terminology, and contextually plausible to the recipient. Proofpoint's 2025 Human Factor report found that 82% of AI-generated phishing emails passed standard spam and anti-phishing filters on delivery.
The organizations detecting AI phishing in 2026 are doing it through four layers: behavioral email analytics that flag process anomalies rather than content signals, sender authentication enforcement that AI cannot replicate, security awareness training rebuilt around the new behavioral tells, and LLM-detection tools that score email text for AI authorship likelihood. This guide covers each layer with specific implementation steps.
What Changed: Old Tells vs. New Tells
Understanding what no longer works is as important as understanding what does. Security teams still training employees on the old signals are producing false confidence, not protection.
Old tells that no longer reliably work:
- Spelling and grammar errors, AI generates grammatically flawless text in any language
- Generic salutations such as 'Dear Customer' or 'Dear User', AI personalizes using LinkedIn, company directories, and breach data
- Suspicious sender domain variations, still present but AI-assisted campaigns increasingly use compromised legitimate accounts
- Mismatched URLs on hover, AI-generated phish more frequently use legitimate redirect services that pass inspection
- Unusual formatting or broken HTML, AI produces clean emails that match the impersonated brand's templates
New tells that actually distinguish AI phishing in 2026:
Process timing exploitation. The email references a specific action the recipient recently performed, a purchase, a password reset, a meeting booking, in a way that creates plausible urgency. Attackers scrape breach data and calendar tools to time sends around real events.
Verification channel bypass. The request explicitly discourages your organization's standard verification process. Phrases like 'due to system issues, please do not use the internal ticketing system' or 'urgent matter, please respond directly' are red flags regardless of email quality.
Contextual specificity without a plausible source. The email references internal project names, systems, or personnel that are not public but appear in leaked documents or data. Demonstrated internal knowledge through an external channel is a red flag independent of how legitimate the email looks.
Urgency on financial or access workflows. AI phishing campaigns disproportionately target wire transfers, vendor payment updates, MFA provisioning requests, and credential resets. Urgency on these specific workflows should trigger mandatory manual verification.
Perfect formality in informal channels. AI-generated text is often too grammatically formal for internal communications. Real inter-colleague email uses abbreviations and informal language. An unsolicited internal email that reads like a business document from an unknown sender warrants a second look.
How AI Phishing Toolkits Work
WormGPT, GhostGPT, and FraudGPT are uncensored language models sold as subscription services on criminal forums and Telegram channels. Unlike commercial LLMs, they have no content policies blocking requests for phishing content, malware code, or impersonation scripts.
WormGPT, the most documented example, was built on the open-source GPT-J-6B model and offered unlimited phishing email generation, business email compromise (BEC) script writing, and malware code generation for approximately $60 per month. The service was shut down by its operator in 2023 but successor products continue under different names at similar price points.
The typical AI phishing workflow: the attacker provides the toolkit with a target name and company, a LinkedIn profile URL, and optionally the text of a real email from the company obtained from breach data or a previous compromise. The toolkit generates a contextually accurate spear-phishing email, a convincing spoofed login page, and in some cases a full BEC script designed for multi-turn social engineering.
The defender implication: the personalization quality of a phishing email is no longer an indicator of a sophisticated actor. A $60 per month subscription makes nation-state-grade personalization available to any attacker. Technical controls, not awareness training alone, are the required defense layer.
Briefings like this, every morning before 9am.
Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.
Technical Controls: What Your Email Security Needs Now
Standard spam filters score email content for known phishing patterns. AI-generated phishing produces content with no known patterns, which is why 82% passes delivery. The technical controls that work in 2026 focus on authentication, behavioral baselines, and process anomalies rather than content scoring.
DMARC enforcement at p=reject. DMARC prevents domain spoofing by verifying that the sending mail server is authorized to send on behalf of the claimed domain. Set your DMARC policy to p=reject, not p=none or p=quarantine, and monitor your DMARC aggregate reporting data weekly. Check your domain's current status at dmarcian.com or mxtoolbox.com/dmarc.aspx.
DKIM and SPF hard fail. Configure SPF with -all (hard fail) rather than ~all (soft fail). Ensure DKIM signing is active for all outbound email including transactional tools. These three together close the domain spoofing attack surface but do not address attacks using legitimately registered lookalike domains.
Executive impersonation protection. Configure your email security platform to alert on emails from external senders that match the display name of any executive or board member, a separate signal from authentication checks. Microsoft Defender for Office 365 has this as 'impersonation protection' under anti-phishing policies. Proofpoint and Mimecast have equivalent features.
Click-time URL sandboxing for all inbound links. Rewrite all URLs in inbound email through a sandboxing proxy that checks them at click time, not at delivery time. AI phishing campaigns increasingly use URLs that are clean at delivery and redirect to malicious content afterward. Defender for Office 365 Safe Links and Proofpoint TAP both support click-time sandbox analysis natively.
LLM-detection scoring as an additional signal. Several email security vendors including Abnormal Security and Darktrace now score incoming emails for AI authorship likelihood as one signal in their detection stack. This is not a standalone control, AI detection rates vary, but it adds a probabilistic signal that increases detection precision when combined with behavioral analysis.
Security Awareness Training: The 5 New Tells to Teach
Remove 'look for spelling errors' from your phishing awareness content. It creates false confidence in recipients who see a polished email and assume it is safe. Replace it with the five behavioral tells below.
1. Urgency on financial or access workflows. Any unsolicited email requesting a wire transfer, vendor payment update, password reset, MFA enrollment, or access credential change with urgency language is a verification trigger, regardless of how legitimate the email appears. The rule: call the requester using a phone number from your internal directory. Never reply to the email to verify an email.
2. Requests to bypass your normal process. 'Please do not use the ticketing system for this, just email me directly' is a BEC red flag. Legitimate internal requests follow established processes. Instructions to deviate from process are a manipulation signal.
3. Sender-context mismatches. An email demonstrating unusual knowledge of internal context, project names, system names, personnel, from an external domain is anomalous. Legitimate vendors do not typically know your internal project names.
4. Perfect grammar in informal contexts. An email from a colleague's address that reads like a formal business document is worth a verification call. People do not write to teammates the way AI writes.
5. Artificial deadlines under 2 hours. Time pressure of 'respond within 2 hours or the account will be locked' exploits cognitive urgency. Legitimate business processes do not require 2-hour responses to unsolicited emails. Time pressure is a manipulation tactic, take the time to verify through a separate channel.
Verification Protocols That AI Cannot Replicate
The most reliable defense against AI phishing is a verification protocol that requires a human channel AI cannot easily replicate at scale. AI generates a perfect email. It cannot impersonate a specific person's voice on a phone call in real time without significant additional infrastructure.
Callback verification for financial requests. Any wire transfer, vendor payment detail change, or unusual financial request above a defined threshold requires a callback to a phone number from your internal directory or a previously verified vendor contact list, never to a number provided in the request itself. This one control stops the majority of BEC financial fraud.
Out-of-band MFA provisioning. Any request to enroll a new MFA device, reset a hardware token, or bypass MFA 'due to technical issues' requires in-person or video verification with IT. MFA provisioning requests are the most common AI phishing target for account takeover.
Shared codeword for high-privilege requests. For organizations where executive impersonation is a confirmed risk, establish a shared verbal codeword between executives and their assistants or finance team for confirming unusual requests outside normal channels. Low-tech and effective, AI cannot replicate a shared secret it does not know.
The bottom line
AI phishing has made content-based detection obsolete. The controls that work in 2026 are DMARC enforcement at p=reject to close domain spoofing, behavioral email analytics that flag process anomalies rather than content patterns, callback verification for all financial and access requests, and awareness training rebuilt around the new behavioral tells rather than grammar inspection. Train employees that a perfect email is not a safe email, and that any urgency on a financial or access workflow is a verification trigger regardless of how legitimate the request appears.
Frequently asked questions
What is AI-generated phishing?
AI-generated phishing uses large language models to write phishing emails that are grammatically perfect, contextually personalized, and indistinguishable from legitimate business correspondence. Uncensored AI toolkits like WormGPT and GhostGPT are available on criminal forums for under $100 per month and generate spear-phishing emails, BEC scripts, and credential theft lures from a target's LinkedIn profile, company website, or leaked email data in under one second.
How do AI phishing emails differ from traditional phishing emails?
Traditional phishing emails are identifiable by spelling errors, generic salutations, suspicious domain variations, and mismatched URLs. AI-generated phishing eliminates all of these signals. The differences are now behavioral: AI phishing exploits process timing by referencing recent actions the recipient took, bypasses standard verification channels, uses contextual specificity from stolen or scraped data, and creates urgency around financial or access workflows. The email text itself is no longer a reliable detection surface.
What new signals should I train employees to recognize?
Five new tells replace grammar-based signals: urgency on financial or access workflows such as wire transfer or credential reset requests with time pressure; instructions to bypass your normal verification process; sender-context mismatches where an external sender demonstrates implausible internal knowledge; perfect formality in informal internal communication channels; and artificial deadlines under 2 hours. Employees should verify any of these signals via a phone call to a number from the internal directory, never by replying to the suspicious email.
Does DMARC protect against AI phishing?
DMARC at p=reject prevents domain spoofing, an attacker sending email that appears to come from your exact domain. It does not protect against AI phishing using legitimately registered lookalike domains, compromised third-party email accounts, or correctly authenticated emails from free providers. DMARC is a required baseline control but not a complete AI phishing defense. Combine it with behavioral email analytics, executive impersonation protection, and click-time URL sandboxing for comprehensive coverage.
What is WormGPT and how does it work?
WormGPT was an uncensored language model sold as a subscription service on criminal forums beginning in 2023, built on the open-source GPT-J-6B model and stripped of content safety policies. It generated phishing emails, BEC scripts, and malware code on demand for approximately $60 per month. It was shut down by its operator in 2023 but successor products including GhostGPT and FraudGPT continue under different names at similar price points, making high-quality personalized phishing accessible to attackers with no AI expertise.
Can technical controls alone detect AI phishing?
Technical controls significantly reduce AI phishing success rates but do not eliminate risk. DMARC enforcement closes domain spoofing. Click-time URL sandboxing catches late-activating malicious redirects. Behavioral analytics flag process anomalies. LLM-detection scoring adds a probabilistic signal. However, AI phishing using a compromised legitimate email account that correctly routes around authentication controls and targets the right recipient with the right context can defeat purely technical defenses. Human verification protocols for financial and access requests are the required final layer.
How do I test my organization's resilience to AI phishing?
Run a phishing simulation using an AI-generated email rather than a traditional template. Several simulation platforms including KnowBe4, Proofpoint Security Awareness Training, and Cofense now include AI-generated templates. Design the test to use a process-timing exploit, reference a real recent company event in the lure, and route the request around your standard verification process. Compare click rates between traditional templates and AI-generated templates. The gap shows how much your current training relies on content-based detection that AI has rendered obsolete.
Sources & references
Free resources
Critical CVE Reference Card 2025–2026
25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.
Ransomware Incident Response Playbook
Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.
Get threat intel before your inbox does.
50,000+ security professionals read Decryption Digest for early warnings on zero-days, ransomware, and nation-state campaigns. Free, daily, no spam.
Unsubscribe anytime. We never sell your data.

Founder & Cybersecurity Evangelist, Decryption Digest
Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals every morning.
Win a $2,495 Black Hat pass.
Full-access to Black Hat USA 2026 in Las Vegas. Subscribe free to enter.
