Game-Theoretic Framework Enables LLMs to Learn Counter-Scam Strategies
ORIGINAL / Game of Phones: Harnessing Game Theory and LLMs in 'Bot Wars' to Counteract Phone Scams
The study introduces a game-theoretic framework that lets large language models learn dynamic counter-strategies through in-context experience without fine-tuning, using Nash equilibrium as an evaluation benchmark. Its novelty lies in modeling scammer–baiter interactions as a two-player game and releasing a dialogue dataset with turn-level strategy labels, offering a quantifiable path for proactive anti-scam strategy optimization.
01 ABSTRACT
The research models interactions between phone scammers and automated scam-baiting agents as a two-player game, with empirically grounded strategies and utility functions, and uses Nash equilibrium as a formal benchmark for learned behavior. Across 300 simulated dialogues and three LLM architectures, adaptive strategy selection aligns 28.9–79.7% closer to Nash equilibrium than non-adaptive baselines. The authors present this as a viable framework for dynamic LLM counter-strategies without fine-tuning, and release code plus an annotated dataset of 300 dialogues with turn-level strategy labels and utility scores.
02 KEY FINDINGS
- The framework models scammer–baiter interactions as a two-player game with empirically grounded strategies and utility functions, using Nash equilibrium as a formal benchmark for evaluating learned behavior.
- Across 300 simulated dialogues and three LLM architectures, adaptive strategy selection aligns 28.9–79.7% closer to Nash equilibrium than non-adaptive baselines.
- LLMs learn counter-strategies through in-context experience without fine-tuning; the study releases code and an annotated dataset of 300 dialogues with turn-level strategy labels and utility scores.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.CL