AI in the Middle: Web-Based AI Services as C2 Relays

Summary

Check Point Research demonstrated an "AI in the Middle" attack in which malware can abuse web-based AI assistants with anonymous or unauthenticated browsing and URL-fetch capabilities as a covert command-and-control channel. The proof of concept used public AI web interfaces, including Grok and Microsoft Copilot, to cause the AI service to fetch attacker-controlled URLs, relay victim data in outbound requests, and return attacker-supplied commands through normal AI assistant responses. Because the implant communicated with trusted AI service domains over ordinary HTTPS web traffic, the activity could blend into expected enterprise AI usage and evade controls focused on suspicious infrastructure, unusual protocols, API keys, service accounts, or revocable credentials. The lack of required authentication for some web-fetch workflows also made it harder for defenders to disable the channel by rotating API keys, suspending accounts, or revoking tokens. The researchers recommended stronger authentication and access controls for AI web-fetch features, along with improved visibility into AI-initiated network requests and AI assistant interactions. Target: Enterprise machines with Grok and Microsoft Copilot access Actor: Check Point Research Procedure: 1. The researchers evaluated public AI assistants with anonymous or unauthenticated web-browsing and URL-fetch behavior to identify services that could retrieve arbitrary adversary-controlled URLs without requiring API credentials. The researchers found Grok and Copilot met those conditions. 2. The researchers registered a domain and deployed an HTTPS site to act as the relay endpoint. 3. The researchers hosted benign-looking content while also returning data that the implant could treat as C2 instructions. 4. Assuming prior access, the researchers used a custom C++ implant with an embedded browser or WebView component to interact with Grok or Microsoft Copilot through the public web interface rather than an API key. 5. The researchers crafted prompts that instruct an AI service to fetch and summarize a website. The prompts caused victim data to be included in URL parameters, allowing the AI service's fetch request to relay data to the adversary-controlled server. 6. When some prompts were blocked by model safeguards, the researchers encoded or encrypted payload data into high-entropy blobs to reduce the chance that safeguards would identify the content as malicious. 7. The implant collected basic host information from the local system. The researchers noted that this could be expanded to collect details such as username, domain, computer name, installed software, running processes, and startup programs. 8. The implant initiated an anonymous web-based session with the public AI service and sent the crafted prompt. This formed a command-and-control channel whereby data was exfiltrated via requests to the adversary-controlled domain, and commands were communicated back via the response.

Deployers

• Enterprise machines with Grok and Microsoft Copilot access

Tags

• MITRE ATLAS
• AML.CS0061