Every Prompt You Type Is a Written Record. You Just Did Not Know It Was Evidence.
Every Prompt You Type Is a Written Record. You Just Did Not Know It Was Evidence.
There is a conversation happening right now in law schools, courtrooms, and corporate legal departments that most AI users have no idea is taking place. It concerns the words they typed into ChatGPT last Tuesday. The question they asked about their medication. The frustration they vented about their employer. The legal situation they described in detail because the AI felt safe to talk to. The financial decision they worked through in a long conversation thread that felt private because it was just them and a chatbot.
None of it was private. All of it is subpoena-able. And courts are already treating it as evidence.
CNN reported this week that prosecutors and plaintiffs' attorneys are actively subpoenaing ChatGPT conversation logs and introducing them in criminal and civil proceedings. The Florida Attorney General opened a criminal investigation of OpenAI after alleging that ChatGPT gave significant advice to the Florida State University shooter. Families of victims in a Canadian school shooting filed suit against OpenAI and Sam Altman this week alleging the model was complicit in the attack. The legal framing across multiple jurisdictions is converging on a single position. Chatbot conversation logs are treated the way courts already treat Google search history. No special protection. Subpoena-able. Discoverable. Admissible.
This is not a legal edge case or a hypothetical risk. It is happening now. And the governance gap it exposes runs considerably deeper than the individual privacy violation it seems to be on the surface.
The therapy gap and why it matters at scale
Every major legal jurisdiction in the developed world has built specific confidentiality protections around a small set of human relationships. The doctor-patient relationship. The attorney-client relationship. The therapist-client relationship. The clergy-penitent relationship. These protections exist not because society decided these professionals deserve special treatment but because society made a deliberate policy choice that certain conversations need to be genuinely confidential to serve their function. A patient who fears their doctor will disclose what they say will not disclose accurately. A client who fears their attorney will reveal their strategy cannot receive competent legal representation. The confidentiality is not incidental to the relationship. It is what makes the relationship work.
Millions of people are now having those same conversations with AI systems. A 2024 survey by the American Psychological Association found that 38% of adults in the US reported using AI chatbots to process emotional distress, mental health concerns, or personal crises. A separate 2024 study from Stanford found that 29% of people who used AI assistants regularly reported sharing information they would not share with a human professional because the AI felt less judgmental and more confidential.
That perception of confidentiality has no legal basis anywhere in the world. No US court has carved out an AI-specific privilege equivalent to doctor-patient or attorney-client protection. No equivalent framework exists in the EU, the UK, India, or any other major jurisdiction. Every conversation a user has with ChatGPT, Claude, Gemini, or any other AI system is a written record on a server owned by a company that receives and responds to court orders. The therapy gap is not a metaphor. It is a structural absence in the legal architecture that governs how sensitive disclosures are protected.
What the evidence chain actually looks like
The mechanics of how chatbot logs become evidence are worth understanding precisely because they reveal how little protection users actually have.
When a subpoena arrives at OpenAI, Google, Anthropic, or any other AI vendor, the company's legal team reviews it against its terms of service, its privacy policy, and applicable law. In most cases the company has explicitly reserved the right to disclose user conversations in response to valid legal process. The relevant language appears in nearly every major AI platform's terms of service in variations of the phrase we may disclose your information when required by law or in response to legal process. Users accepted these terms at signup. The disclosure is not a betrayal of privacy. It is the exercise of rights the company disclosed and the user agreed to.
The evidentiary value of the logs is also higher than most people would expect. AI conversation logs are timestamped, persistent, and associated with an account that is typically linked to a verified email address or payment method. They are considerably harder to dispute as authentic than a text message screenshot or a social media post. Courts are treating them accordingly.
In the Florida case, the Attorney General's office is using the ChatGPT conversation logs as evidence of the shooter's state of mind and planning process in the period before the attack. The legal theory is that the AI's responses, including any advice or information it provided, are relevant to establishing the context and development of the criminal intent. Whether that theory survives appellate scrutiny is an open question. That it is being argued in a real criminal proceeding is not.
The enterprise dimension nobody is discussing
The consumer privacy story is significant. The enterprise dimension is more consequential and considerably less discussed.
Employees across every industry are using AI systems to work through legal questions, financial decisions, strategic analysis, personnel matters, and compliance concerns that may touch on confidential business information. A financial analyst who discusses an unreleased earnings forecast with ChatGPT to help frame their analysis may have created a discoverable record of material non-public information. A lawyer who uses an AI assistant to draft a litigation strategy memo may have exposed that strategy to discovery depending on how the conversation was structured and whether privilege can be asserted. A senior manager who vents about a pending restructuring in an AI conversation thread has potentially created a written record that could surface in subsequent employment litigation.
IBM's 2024 AI in Action report found that 67% of employees in organisations that had deployed enterprise AI tools reported using consumer AI tools for work tasks as well, specifically because they found them more convenient or capable for certain tasks than approved enterprise tools. That shadow AI usage pattern means enterprise legal exposure is not limited to conversations on approved platforms. It extends to every consumer AI conversation an employee has that touches on work-related matters.
McKinsey's 2024 State of AI report found that only 21% of organisations have formal policies governing employee use of generative AI. That means 79% of organisations have employees using AI tools for work tasks under no clear policy framework, with no guidance on what to discuss, what to avoid, and what the legal implications of their AI conversations might be.
The EU AI Act, which entered into force in August 2024 with obligations phasing through 2026, includes provisions around transparency and data protection for AI systems but does not create the kind of conversational privilege that would protect AI-mediated disclosures from legal discovery in the way established professional privilege frameworks do. The regulatory landscape is addressing AI capability risks and bias risks. It has not yet addressed the evidentiary and privilege gap that the ChatGPT courtroom cases are now making visible.
Three enterprise governance requirements that follow directly
Across advisory engagements in financial services, legal services, and cross-border compliance environments, the ChatGPT evidence story is creating three immediate governance conversations that organisations need to be having now rather than after their first subpoena arrives.
AI usage policies need to explicitly address the discovery risk. Most current AI policies focus on accuracy, bias, and data privacy in the sense of not sharing confidential data with AI systems. They do not explicitly address the inverse risk, that the AI conversation itself becomes a discoverable record that exposes confidential information to legal process. That gap needs to close. Every enterprise AI policy should include explicit guidance on what categories of information should never be discussed with consumer AI tools and what the company's position is on AI conversation logs in the event of legal process.
Privilege architecture for AI-assisted legal and compliance work needs to be designed before it is needed. When lawyers use AI tools in the course of providing legal advice, there is a reasonable argument that those conversations should be covered by attorney-client privilege. But that argument needs to be tested and established through deliberate workflow design, not assumed and discovered to be unavailable after the fact. Law firms and in-house legal teams that are using AI tools without having thought through the privilege architecture are operating with an unquantified legal exposure.
Employee awareness needs to match the risk profile. The organisations getting this right are not just publishing policies. They are running awareness sessions that specifically address the therapy gap, the discovery risk, and the shadow AI usage pattern. Employees who understand that their AI conversations are written records respond differently than employees who intuitively feel that talking to an AI is somehow more private than sending an email.
The democratisation dimension
There is a genuine equity issue embedded in the therapy gap that deserves naming directly. The professional relationships that carry legal privilege, the doctor, the lawyer, the therapist, are relationships that are much more accessible to people with higher incomes and more social capital. The populations most likely to use AI systems as a substitute for those professional relationships are also the populations least likely to have access to them. A person who cannot afford a therapist and uses ChatGPT to process a mental health crisis is in a structurally different position than someone who can afford to see a therapist whose notes are privileged. The AI conversation the first person has is legally exposed in a way that the therapist's notes are not.
Democratising access to AI capability without democratising the legal protections that govern the use of that capability is not genuine democratisation. It is capability access with hidden legal liability attached. The governance community and the regulatory frameworks need to address that asymmetry explicitly rather than leaving it as an invisible gap in the architecture that only becomes visible when someone is already in a courtroom.
The accountability architecture was designed for a different world
Every prompt is a written record. Every written record is potentially evidence. Every piece of evidence can shape a legal outcome. That chain of consequences was always true for emails and text messages. It is now true for AI conversations in a way that almost none of the people having those conversations understand.
The governance response is not to stop using AI. It is to design the accountability architecture around AI usage the way we designed it around email, telephone, and document storage, before the first contested outcome forces a reactive and expensive reconstruction.
Data source: CNN Legal Analysis 2026. American Psychological Association 2024 survey. Stanford AI assistant usage study 2024. McKinsey State of AI 2024. IBM AI in Action 2024. EU AI Act 2024. Referenced cases: Florida AG investigation of OpenAI 2026. Canadian school shooting lawsuit against OpenAI 2026.
The strategic observations in this piece draw from advisory engagements across financial services, legal services, and cross-border compliance environments, and from co-authored research on responsible AI governance presented at BIGS 2025, AIS eLibrary.
Comments