“This trend is particularly concerning because it demonstrates adversaries ‘closing the loop on model tuning’ — their offensive capabilities constantly improving over time through real-time feedback and illicit data,” Ian Gray, Flashpoint VP of cyber threat intelligence, tells CSO.
Flashpoint has also observed private chat groups where users submitted failed prompt attempts back to LLM developers, leading to rapid iteration and improved performance within days. In one instance, a user reported formatting issues with a financial fraud prompt, and shortly after, the developer shared an updated version with refined templates, Flashpoint observed.
“This adaptive and self-improving nature of malicious AI, fueled by compromised data and criminal collaboration, makes it an especially potent and difficult threat to counter,” Gray says.
