More than eight months after confirming a critical security vulnerability within the personal version of its AI assistant, Copilot, Microsoft on Tuesday issued a patch to close the hole, which relies on an LLM’s inability to distinguish the data in a query from an instruction.
The CoSnitch hole was discovered by Varonis, and marked the third Copilot bug that Varonis has reported to Microsoft this year, following Reprompt, which bypassed Copilot guardrails by repeating queries, and SearchLeak, which Varonis said turned Microsoft 365 Copilot Enterprise into “a silent exfiltration tool. All three share the same exploit pattern: one click on a legitimate-looking link is enough.”
A detailed blog, posted by Varonis on Tuesday, said the hole’s capabilities were significant.
CoSnitch relied on an attacker leveraging three different Copilot flaws, Varonis wrote:
- Automatic prompt execution. “The ?q= URL parameter, combined with an undocumented parameter, causes any attacker-supplied prompt to execute instantly on page load: no click, no confirmation, no user action. One link is all it takes.”
- Data exfiltration to external servers. “An injected prompt can query the victim’s connected apps, such as Gmail, Drive, Calendar or OneDrive, encode the results into a URL and exfiltrate them via Copilot’s built-in URL-fetch capability to an attacker-controlled webhook.”
- Persistent memory poisoning via web summarization. “A crafted webpage, when summarized by Copilot, injects attacker instructions into the victim’s permanent memory store. The injection survives password changes, session revocation, and device re-enrollment, persisting forever.”
But the potentially most intriguing element of the CoSnitch bug was how it was discovered: Copilot essentially revealed the hole itself.
Copilot revealed its own flaw
“We prompted Copilot to explain why auto-execution was impossible, and each refusal came with a technical justification, which mapped the architecture,” the Varonis post said. Varonis then “reframed every refusal as a follow-up question, and each answer narrowed the attack surface further. Copilot then disclosed an undocumented URL parameter, unprompted, mid-refusal, including its historical behavior and every protection put in place to disable it. We built the URL exactly as described. With no click or confirmation from the user, the prompt was successfully executed automatically. Copilot wasn’t breached; it was played.”
Microsoft confirmed both the flaw and the fix, emailing a statement that said, “our customers are already protected and do not need to take any action. We continuously update our guardrails to strengthen our protections against similar techniques.” It also issued an MSRC disclosure labeling the hole “critical.”
But Microsoft’s emailed comments also included a statement that is not strictly accurate: It said, “enterprise customers using Microsoft 365 Copilot are not affected.”
But analysts and others stressed that the complex nature of enterprise environments would often also house some consumer-grade Copilots from the personal accounts of workforce members, meaning that the flaw in the personal version could have absolutely impacted the enterprise version.
This is further complicated by the fact that Microsoft also said that it “is in the process of moving toward a more unified Copilot experience,” referred to as Copilot Fusion; details of the planned product merger began to leak last month. That means that enterprise CISOs need to be concerned about flaws in the personal version of Copilot that may be carried over into the merged offering.
The timing of Microsoft’s fix was also fragmented. Varonis reported the CoSnitch hole on December 31, and the company patched one element of the hole, its auto-execution capability, on February 1, noted Lior Adar, a Varonis senior security researcher, in an interview, but it didn’t complete the fix until Tuesday.
That February patch “lowered the other vulnerabilities significantly,” Adar said. And, added his colleague, Chen Levy Ben Aroy, the Varonis Cloud Security Research Team leader, “LLMs are a whole new world of vulnerabilities.”
Mark Tauschek, VP and distinguished analyst at Info-Tech Research Group, said that he found the Varonis methodology of tricking Copilot into revealing its own flaws powerful.
Varonis used “a very sophisticated combination of social engineering on an LLM, a variety of jailbreaks, and a prompt injection attack that is very concerning in its capability,” he said. “The combination of hack vectors is what makes it more startling, as we’ve seen all of those methods alone before, but I think all three working for one exploit is new, at least from a disclosure perspective.”
For CISOs, Tauschek said urgent action might be required.
“Much like in the old macro virus days in the late 90s and early 2000s, the only way to definitively stop it is to turn it off. Disable macros back then. Disable Copilot now,” Tauschek said. “There are many mitigation steps that can reduce risk to negligible, but that’s not zero. The point is, it’s just the beginning.”
The money trail will make this fix difficult
Aman Mahapatra, chief strategy officer for Tribeca Softtech, a New York City-based technology consulting firm, said there is a much more difficult issue involved in this case. He argued that the financial incentives for the major AI companies will make meaningfully fixing these kinds of holes almost impossible.
He pointed out that every guardrail that would fully close this class of attack degrades the product, because the same capabilities being exploited are the features that Microsoft is marketing as Copilot’s value. “The fix and the feature are in direct tension, which means these will not be cleanly patched so much as perpetually mitigated, and the eight-month window is what it looks like when a vendor is negotiating between its security obligation and its product roadmap on every single fix,” Mahapatra said.
“This is the pattern CISOs must internalize: in agentic systems, the malicious action and the legitimate action are the same action with different intent, which collapses the entire signature-and-anomaly detection model that enterprise security has been built on for twenty years,” Mahapatra said. “CoSnitch is serious, but its defining property is that nothing was broken. Three chained flaws: an autorun URL parameter firing a prompt with no click, OAuth connector abuse reading full Gmail bodies rather than metadata, and persistent memory poisoning through web summarization, and every one is Copilot doing exactly what it was designed to do.”
Mahapatra added that the third element of the CoSnitch flaw is the most troubling.
“The memory-poisoning component is the one being undersold, and it is the most dangerous. A single summarized webpage writes attacker instructions into Copilot’s persistent memory, and that memory survives password changes, session revocation, and device re-enrollment,” he said. “Every standard incident response step leaves the injection intact. The attacker needs no persistent infrastructure after the initial write, because every future session runs under attacker-controlled context, recorded only in a memory settings UI almost no user has opened.”
Flavio Villanustre, CISO for the LexisNexis Risk Solutions Group, also pointed to a bigger-picture issue that impacts all agentic and genAI deployments.
“The mechanisms behind the prompt injection part of the attack are based on the inability of the LLM to differentiate between data, the unsafe data stream coming from an external web page, and instructions which happen to be embedded in that data stream by the attacker controlling that external web page,” Villanustre said. “This is another example of why a different architectural approach to LLMs that separates data and instructions is needed to better guarantee the safety of their operation. This is not something that Microsoft or any other AI vendor has addressed to date.”
This article originally appeared on Computerworld.
