Phishing in the Era of AI: Detection and analysis using naïve bayes and logistic regression

As generative artificial intelligence (AI) evolves, the threat landscape of cyber-attacks, particularly phishing, has undergone a paradigm shift where attackers can now craft highly realistic emails at scale. 
Problem Statement: While traditional filters struggle with these AI-crafted messages, most research focuses either on generation or detection in isolation, often limited by small dataset sizes that lack external validity. 
Aim: This study evaluates the deliverability of phishing emails generated by three Large Language Models (LLMs) i.e. (ChatGPT, Gemini, and Claude) and benchmarks the detection capabilities of Naïve Bayes and Logistic Regression models. 
Methodology: We utilized an expanded dataset of 1,040 samples, integrating the Enron Email Dataset and SpamAssassin Public Corpus. Deliverability was measured via Mail-Tester proxies, while detection was evaluated using macro/micro F1, AUROC, and PR-AUC. 
Key Findings: Results show that AI-generated phishing emails achieve high deliverability (scores above 9/10), indicating a strong likelihood of bypassing basic filters. In detection, Naïve Bayes achieved 83% accuracy and 100% recall, outperforming Logistic Regression, which recorded lower recall in identifying phishing emails. 
Impact: These findings provide a framework for refining automated email security and highlight the urgent need for integrating metadata-based features to combat evolving AI-assisted social engineering