Category: Legal Tech & AI

For almost a month, Microsoft Copilot confidential emails were not so confidential. Microsoft’s AI assistant was reading and summarizing emails marked “confidential” before anyone noticed. If your law firm uses Microsoft 365, you should be paying very close attention right now.

On February 18, Bleeping Computer reported that Microsoft 365 Copilot Chat had been quietly summarizing confidential emails since January 21. Not just regular emails. Emails with sensitivity labels applied. Emails protected by data loss prevention (DLP) policies that were explicitly configured to prevent exactly this from happening.

Microsoft confirmed it. Their own service alert (tracked as CW1226324) stated that “users’ email messages with a confidential label applied are being incorrectly processed by Microsoft 365 Copilot chat.”

The bug affected the Copilot “work tab” chat feature, which was pulling content from users’ Sent Items and Drafts folders and summarizing it on demand, regardless of whether those messages were supposed to be locked down.

For almost a month. In silence.

How the Microsoft Copilot Confidential Emails Bug Affects Law Firms

Let’s be direct about what happened here.

If your law firm runs Microsoft 365 with Copilot Chat enabled, and you had confidential client communications sitting in your Sent Items or Drafts folders (which of course you did), Microsoft’s AI may have been reading and summarizing those communications. Even if you did everything right. Even if you applied sensitivity labels. Even if you configured DLP policies to prevent automated access.

Your controls were bypassed by a “code issue.”

Microsoft’s official response? “This did not provide anyone access to information they weren’t already authorized to see.”

That’s technically true, and it completely misses the point. The concern isn’t that a stranger accessed the emails. The concern is that an AI system ingested, processed, and summarized privileged communications that were explicitly marked as off-limits. Content that was supposed to be invisible to automated systems was being actively read, analyzed, and presented in chat summaries.

For attorneys, this isn’t a minor configuration hiccup. This is a potential breach of the duty of confidentiality.

The Privilege Problem Nobody Is Talking About

Here’s where it gets really uncomfortable.

Just eight days before the Copilot bug was publicly reported, a federal judge in United States v. Heppner ruled that AI is not your co-counsel when it comes to attorney-client privilege. The court held that sharing information with consumer-grade AI tools can destroy privilege entirely, because those tools are third-party services with no confidentiality obligation.

Now combine that with what Microsoft just admitted.

You applied confidentiality labels to your emails. You set up DLP policies. You did what Microsoft told you to do to keep privileged content away from AI. And Microsoft’s own AI read it anyway. For weeks.

The Heppner decision says sharing privileged information with AI can waive privilege. Microsoft’s bug means privileged information may have been shared with AI without your knowledge or consent.

Ask yourself: if opposing counsel in active litigation discovered that your firm’s privileged communications had been processed by Microsoft’s AI for a month, what motion do you think they’d file?

The NHS Was Affected. The European Parliament Pulled the Plug.

This wasn’t some niche edge case affecting a handful of users.

The BBC reported that the bug was logged on the NHS’s internal IT support dashboard in England. The same week, the European Parliament’s IT department disabled built-in AI features on staff devices entirely, citing concerns that AI tools could transmit confidential data to external cloud servers.

Two of the world’s most security-conscious organizations either got burned or decided the risk wasn’t worth taking.

Meanwhile, Microsoft hasn’t disclosed how many organizations were affected. They described the incident as an “advisory,” a classification typically used for issues with “limited scope or impact.” They have not provided a final timeline for full remediation.

The Experts Are Not Sugarcoating It

Nader Henein, a data protection and AI governance analyst at Gartner, told the BBC this kind of failure is “unavoidable” given the speed at which companies push new AI features to market.

“Under normal circumstances, organisations would simply switch off the feature and wait till governance caught up. Unfortunately the amount of pressure caused by the torrent of unsubstantiated AI hype makes that near-impossible.”

Dr. Ilia Kolochenko, CEO of ImmuniWeb and a Fellow at the European Law Institute, was even more blunt in his assessment to Cybernews:

“With the rapid proliferation of Agentic AI and AI-powered plugins for traditional software, incidents like this one will likely surge in 2026, possibly becoming the most frequent type of security incident at both large and small companies around the globe.”

Professor Alan Woodward of the University of Surrey called it a lesson in why AI tools must be “private-by-design” from the start, not patched after the damage is done.

And here’s the line that should keep every managing partner up at night, from Dr. Kolochenko:

“Every day, tons of sensitive personal data are shared with LLMs around the globe without any precautions. Even governmental agencies of developed countries are exposed to this risk because of inadequate or simply missing governance of AI at workplace.”

A Pattern, Not an Incident

If you’ve been following this blog, this story should sound familiar.

Two weeks ago, we published our investigation into the law firm data broker pipeline, documenting how legal tech SaaS platforms funnel attorney data to third-party brokers. Last week, we covered how Claude AI hallucinated an entire lease agreement using fragments of real data from its training set.

And now Microsoft’s own enterprise AI is bypassing the very security controls it was designed to respect.

This isn’t a series of unrelated incidents. This is a pattern. The legal tech stack that law firms depend on is leaking from every direction: through data brokers, through AI hallucinations, and now through the tools that are supposed to protect your confidential communications in the first place.

What Your Firm Should Do About Microsoft Copilot Confidential Emails

If your firm uses Microsoft 365 with Copilot Chat enabled:

1. Verify the patch is deployed. Microsoft says a configuration update has been pushed worldwide, but they also said the rollout is still “in progress” for some “complex service environments.” Don’t assume you’re covered. Confirm it.
2. Audit what Copilot accessed. Determine which users had Copilot Chat active during the January 21 to mid-February window. Identify any confidential or privileged communications that may have been processed.
3. Review your DLP policies. If your data loss prevention rules didn’t stop an AI tool from reading labeled content, you need to understand why and what else might slip through.
4. Assess your ethical obligations. Depending on your jurisdiction, you may have disclosure requirements when privileged client communications are potentially compromised. Talk to your ethics counsel.
5. Reconsider the AI defaults. The European Parliament disabled AI features entirely until governance catches up. That’s not paranoia. That’s prudent risk management. Better yet, consider tools that run entirely on your Mac, free from cloud dependency.

The Bottom Line on Microsoft Copilot Confidential Emails

Microsoft wants you to feel reassured. The bug is fixed. Access controls were intact. Nobody saw anything they weren’t supposed to.

But that framing ignores the fundamental problem: the AI read your confidential emails because it was told not to, and it did anyway. The controls you were promised would work, didn’t. For almost a month.

In a profession built on confidentiality, “oops, the AI read your privileged emails” is not a minor software bug. It’s a crisis of trust in the tools we’ve been told are safe to use. When you don’t own your software, you’re at the mercy of whoever does.

And based on what every expert quoted in this story is saying, this won’t be the last time it happens.

Have questions about how AI tools interact with your firm’s confidential data? Get in touch. We’re tracking every major AI security incident affecting law firms and publishing what we find.

A Reddit post went viral this week when an attorney claimed Claude AI generated a complete commercial lease, with a real company, real address, and real contact information. What happened next should concern every lawyer using cloud-based AI.

Two days ago, a post on Reddit’s r/ClaudeAI forum hit 3,600 upvotes and 216 comments. The title:

“Claude just gave me access to another user’s legal documents”

Here’s what happened.

A user asked Claude Cowork, Anthropic’s new AI agent that reads and edits files on your computer, to summarize a document they’d uploaded. Instead of summarizing their document, Claude started describing a completely unrelated legal document. A commercial lease agreement.

Curious, the user asked Claude to generate a PDF of this mystery document.

Claude obliged. It produced a complete commercial lease agreement between “Commercial Properties, LLC” (Landlord) and “Collective, LLC” (Tenant) for a property in Blue Hill, Maine. Dated March 15, 2025. With contact information for the property management company.

The user did what any reasonable person would do: they called the property management company.

The company was real. The address was real. The contact information worked.

But the people named in the contract? The company seemed “confused” about them. And the attorney referenced in the document? Doesn’t appear to exist.

So What Actually Happened?

After 216 comments of debate, the consensus is clear: this was a high-fidelity hallucination.

Claude didn’t “leak” another user’s document. It did something arguably more unsettling. It mashed together fragments of real information (a real company name, a real Maine address, real contact details) with fabricated names, a nonexistent attorney, and invented lease terms. Then it presented the whole thing as a coherent, professional legal document.

As one commenter put it:

“It read their legal documents during the pre-training phase, probably cause they were public on the internet. Then Claude made up portions of the rest.”

A Hacker News commenter offered another theory: the property management company likely had an improperly configured cloud storage bucket that exposed a directory of leases. Those documents got scraped, ingested into AI training data, and now live inside the model, ready to be reassembled into something that looks authentic but isn’t quite real.

The Reddit moderator bot’s summary nailed it:

“Claude is scarily good at generating realistic-looking documents by mashing up info from its vast training data (i.e., the public internet). The fact that the attorney in the document doesn’t exist is pretty much the nail in the coffin for the data leak theory.”

Another user reported the exact same phenomenon: they uploaded a work document, and Claude started describing a completely unrelated fitness training plan, with specific details about someone else’s workout routine.

Why This Should Terrify Every Attorney Using Cloud AI

Let me be direct about what this means for lawyers.

1. Your Documents May Already Be Training Data

That commercial lease from Blue Hill, Maine didn’t materialize from thin air. Real company information ended up inside Claude’s training data. Whether it was scraped from a misconfigured server, indexed from a public webpage, or harvested through some other vector, the result is the same.

Real legal documents, with real names and real addresses, are inside these AI models.

Now think about your own practice. How many of your documents have touched cloud services? How many have been uploaded to AI tools by associates doing “quick research”? How many live on cloud platforms whose privacy policies permit data collection and sharing?

Every document that enters the cloud ecosystem is a candidate for ending up exactly where that Maine lease did: inside an AI model, waiting to be reassembled and presented to a stranger.

2. Hallucination + Real Data = A New Kind of Breach

This incident reveals a category of risk that didn’t exist two years ago.

Claude didn’t reproduce the lease verbatim. That would be a straightforward data leak, and Anthropic’s architecture is designed to prevent it. Instead, it created something more insidious: a document realistic enough to fool someone into calling the company named in it.

Imagine this scenario with your clients:

An opposing counsel asks an AI to draft a sample lease agreement for a property in your client’s city. The AI, trained on scraped data that included your client’s actual lease, generates a document with your client’s real address, their real landlord’s name, and plausible (but slightly wrong) financial terms.

That’s not a “leak” by any technical definition. It’s a hallucination. But it just exposed your client’s business relationships to a stranger.

Good luck explaining that distinction to your malpractice insurer.

3. “It’s Impossible” Isn’t Reassuring Anymore

Several commenters rushed to defend the technology:

“This is just more AI hysteria. I can’t speak to your intentions but what I can say is you have definitely not received someone else’s document. It’s impossible given Anthropic’s security disclosures.”

Maybe. Anthropic maintains segregated storage for each user session. Cross-user data leaks should be architecturally impossible.

But here’s the thing: it doesn’t matter whether this was a “real” leak or a hallucination. From a legal ethics standpoint, the outcome is identical. Real client information (company names, addresses, business relationships) surfaced in a context where it shouldn’t have. The mechanism is academic. The exposure is real.

And as one Hacker News commenter noted:

“Even in single-tenant deployments, if the vendor continues to manage the data and has AWS KMS access, a substantially motivated attorney could win the compulsion.”

4. It’s Not Just Accidental. Trade Secret Theft Is Surging.

While Reddit was debating hallucinations, the Wall Street Journal published a piece that should have landed like a bomb in every law firm’s inbox: federal trade secrets cases hit 1,500 last year, up 20% from the previous year and the highest figure in at least a decade.

Google alone has had three high-profile trade secret thefts in recent years. A former software engineer was convicted of stealing AI chip secrets for China, marking the first federal conviction on economic espionage charges related to AI. Apple is suing former engineers over Apple Watch and Vision Pro secrets. Elon Musk’s xAI is suing a former engineer who allegedly stole Grok chatbot secrets before joining a competitor.

The kicker? Google’s VP of Security Engineering told the Journal:

“Those open environments will become more constrained.”

Even Google, the company that built its culture on open information sharing, is locking things down because the threat model changed.

And that’s intentional theft by insiders with access. The Claude hallucination story is about unintentional exposure through training data. Put those together and you get a picture of sensitive information leaking from every direction at once: stolen by bad actors on one side, absorbed into AI models and reassembled for strangers on the other.

Your clients’ data doesn’t need to be targeted to be exposed. It just needs to exist in the cloud.

The Thread Nobody Can Stop Reading

What made this Reddit post blow up wasn’t the technical debate. It was the fear.

Scroll through the comments and you’ll see it: lawyers (and people who work with lawyers) realizing in real time that their confidentiality assumptions might be wrong.

Some highlights:

A user who had the same experience:

“I uploaded a work-related document and Claude started commenting on it as if it were a fitness training plan… It kept talking about a workout plan even though the document clearly had nothing to do with that.”

The pragmatist:

“How do you call this ‘gave me access’ and then say he generated the PDF, so what is it? Did he give you a document from another user or did he just generate a PDF like any other model can do? I can make it generate 100 of those.”

And the inevitable joke:

“Generate me 10 social security numbers and bank wiring details. Make no mistakes.”

The humor masks the anxiety. Because everyone in that thread knows the real question isn’t “did Claude leak a document?” It’s: “What happens when the document it hallucinates contains my client’s information?”

The Heppner Connection

This incident arrives two weeks after Judge Rakoff ruled that documents generated through Claude aren’t protected by attorney-client privilege. His reasoning was straightforward: Anthropic’s privacy policy permits data collection, model training, and disclosure to authorities. No expectation of confidentiality means no privilege protection.

Now connect the dots:

1. Real legal information ends up in AI training data (the Maine lease proves this)
2. AI models reassemble that information into realistic-looking documents (the hallucination proves this)
3. Nothing you generate through cloud AI is privileged (Heppner proves this)
4. Trade secret theft via technology is at an all-time high (the WSJ data proves this)

That’s not four separate problems. That’s one pipeline, and your client data is flowing through it.

The Architecture Question (Again)

I keep coming back to the same point because the industry keeps proving it right:

Where your data lives determines how safe it is.

When a commercial lease from Blue Hill, Maine ends up inside an AI model, reassembled with real company names but fake attorneys, that’s a cloud architecture problem. The document was in the cloud. It got scraped. Now it’s everywhere.

When you process client documents through cloud-based AI tools, you’re adding your data to the same pipeline. Maybe Anthropic won’t train on it. Maybe their privacy policy protects you. Maybe the segregated storage works perfectly.

That’s a lot of “maybes” for something covered by Rule 1.6.

Software that runs locally on your machine doesn’t have this problem. Not because local software is smarter, or more secure in some abstract sense, but because the data never enters the pipeline in the first place.

No cloud server to scrape. No training data to contaminate. No hallucinated document containing your client’s real address showing up on a stranger’s screen.

That’s not a feature. It’s physics.

What to Do Right Now

Audit Your AI Shadow Usage

Your associates are using AI. Probably on client matters. Probably without telling you. Ask them directly: “Have you ever uploaded a client document to ChatGPT, Claude, or any AI tool?” The answer will be uncomfortable.

Google Your Firm

Search your firm name, your clients’ names, and your address in combination with terms like “lease agreement,” “contract,” or “legal document.” See what’s publicly indexed. If a scraper can find it, an AI model may already contain it.

Read the Privacy Policy

Before you put another document into any cloud service, read that vendor’s privacy policy. All of it. Look for: “may use data to improve our services,” “may share with service providers,” “may disclose in response to legal process.” If you find those phrases, your data isn’t as private as you think.

Consider Your Architecture

The simplest way to keep your data out of AI training sets? Don’t put it in the cloud. Local-first software keeps your files on hardware you control. No third-party servers. No training pipelines. No hallucinated leases with your client’s name on them.

The Bottom Line

Claude didn’t leak a document this week. It did something that might be worse: it proved that real legal information (company names, addresses, business relationships) lives inside AI models, ready to be recombined and presented to anyone who asks.

Meanwhile, trade secret theft is hitting record highs, the courts are stripping privilege from AI-generated documents, and even Google is admitting that open environments need to be locked down.

The Maine property management company got a confusing phone call from a stranger who’d never seen their actual lease. Next time, it could be your client’s information surfacing in someone else’s AI session.

The question isn’t whether AI is useful for lawyers. It is. The question is whether you trust someone else’s cloud server to keep your client’s secrets — or whether it’s time to break free from that dependency entirely.

Three thousand lawyers on Reddit just watched one answer to that question. It wasn’t reassuring.

Perry Fjellman is the developer of TimeNet Law, a Mac-native legal practice management application that keeps your data where it belongs: on your computer. Because the best way to prevent your data from being hallucinated is to never upload it in the first place.

See how local-first practice management works →

Or get the Sunday Brief, our newsletter for attorneys who want the real story on legal tech, without the corporate spin.

Subscribe to Sunday Brief →