- California’s bar guidance warns that AI outputs can be “false, misleading, fabricated, incomplete, or biased.”
- A federal court fined two lawyers $5,000 in 2023 over citations ChatGPT invented for their filing.
- Free human help exists: legal aid offices, state bars and law-school clinics.
Riley, a fictional tenant whose landlord kept the security deposit, asked a chatbot what to do and got an answer that cited a law. The reply can help, yet it is something to check first, not something to rely on.
What OpenAI’s Rules Say About Legal Advice
There is no blanket ban in OpenAI’s text. The usage policy in force since October 29, 2025 bars tailored advice that needs a license, like legal or medical advice, “without appropriate involvement by a licensed professional.” Nothing there bans general legal information; the ban targets advice tailored to your case with no licensed professional taking part.
The Terms of Use agree: output is not “a sole source of truth or factual information,” nor “a substitute for professional advice.” They also forbid output-based decisions about a person with a legal or material impact. The policy bars automating high-stakes legal decisions without human review; a new version of it applies from November 20, 2026.
Who May Practice Law
The American Bar Association’s Model Rule 5.5 bars lawyers from practicing law against a jurisdiction’s rules for the profession, or from assisting another in doing so. California states it as a statute: “No person shall practice law in California unless the person is an active licensee of the State Bar.”
The rules speak to people, not to software.
The Court Fine for Invented Citations
In Mata v. Avianca, a New York federal court sanctioned two lawyers and their firm over “non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT.” The June 22, 2023 order saw “nothing inherently improper” in using a reliable AI tool for assistance, and imposed a $5,000 penalty.
The State Bar of California’s guidance: generative AI “may generate plausible but inaccurate information, including purported citations to legal authority that does not exist,” the outputs called hallucinations. A lawyer “must review all outputs produced using AI tools for accuracy,” a duty that “cannot be delegated to AI.”
What Happens to What You Type
Confidentiality has one clear rule, written for lawyers. The State Bar of California’s guidance tells a lawyer not to input confidential client information into a generative AI solution with material confidentiality or security risks, absent informed client consent. Such products “often utilize the information that is input by the user, including prompts and uploaded documents,” and might share it with third parties.
For you, the sources are quiet: OpenAI’s terms say nothing about privilege.
The Deposit Question, Put to Gabriel
Riley Parker and Morgan Pike, tenant and landlord of this example, are fictional; the replies are Gabriel’s, legal assistant coach on Nation AI, kept word for word.
The exchange
A chatbot gave me legal advice about my deposit and cited a law.
I can help you check the chatbot’s legal claim against Texas law, identify which facts affect the deadline, review your lease and evidence if you share them, and draft a written demand for the $1,850.
Can I rely on an AI for legal advice? Answer in three lines.
“An AI can help explain legal concepts, organize facts, and prepare questions or drafts, but it can be wrong or miss key details.”
“Verify important claims against current official law and your documents; don’t treat an AI answer as a lawyer’s advice.”
“For your deposit, check the Texas statute and your forwarding-address records before acting.”
Use It to Prepare, Not to Decide
-
Open the official source
Find the statute or court rule it mentions, and read it on the official source that publishes it.
-
Compare with your papers
Set the answer against your lease, photos, receipts and messages; the facts decide whether the rule fits.
-
Prepare questions for a lawyer
Have the chatbot turn your facts into a ranked list of questions for the appointment.
-
Take the decision to free help
Legal aid offices give “free legal help in landlord-tenant, divorce, and other cases” to people with low and moderate incomes; state bars and law-school clinics help free or at low cost. The Legal Services Corporation’s site finds one nearby, and California’s courts run self-help centers.
General information, not a substitute for a licensed professional’s advice.
Gabriel’s warning
Gabriel is an AI, not a lawyer: the tool page calls what he offers a compass, not a verdict. He never invents a legal reference, a clause, a statute, a date or a case, and he says so when he cannot confirm a rule is still in force. When the stakes, the procedure or a deadline justify it, he sends you to a qualified professional; hold his answers to the same checks.
Gabriel
AI legal assistant
“I help you read a document, prepare a letter, spot the deadlines that decide a case, all from the facts you give me. I never invent a statute, a date or a court case.”
Check the answer before you act on it
Our guide to legal AI in US law shows what to verify on official sources; the chat is free to try, with no account.
Frequently Asked Questions
Is it against the law for ChatGPT to answer a legal question?
OpenAI’s policy bars tailored licensed advice with no licensed professional involved.
Can ChatGPT write a legal letter for me?
It can draft one; its terms warn against treating the output as a substitute for professional advice, so check each rule cited on the official source. For the structure, our demand letter template shows what goes in.
Do chatbots keep what you type confidential?
The confidentiality rule in the State Bar guidance is written for lawyers, and OpenAI’s terms say nothing about privilege; treat what you type as shareable.
Sources
- Usage policies, OpenAI, effective October 29, 2025 (archived copy).
- Terms of Use, OpenAI, effective January 1, 2026.
- Model Rule 5.5, American Bar Association, page dated April 17, 2019.
- Business and Professions Code, section 6125, California Legislative Information.
- Practical guidance on generative AI, State Bar of California (2026).
- Mata v. Avianca, order on sanctions, June 22, 2023.
- Hiring a Lawyer, Federal Trade Commission, page dated September 2023.
- I Need Legal Help, Legal Services Corporation.
- Self-Help Centers of the California Courts.
Put your own question to the chat
Alex, the generalist assistant, helps you frame the question and find the words.
