On March 6, 2026, Judge Lewis Kaplan of the Southern District of New York dismissed with prejudice the complaint in Upsolve, Inc. v. James, No. 22-cv-627 (S.D.N.Y. 2026), holding that New York's unauthorized-practice-of-law rules survive First Amendment scrutiny as applied to a nonprofit program that would have trained non-lawyers to help debt-collection defendants fill out a state-provided Answer Form. The Second Circuit had already determined in September 2025 that UPL rules are content-neutral and therefore subject to intermediate — not strict — scrutiny. Upsolve, Inc. v. James, 155 F.4th 133 (2d Cir. 2025). Kaplan applied that framework and found the rules narrowly tailored to serve the state's interest in protecting the public from incompetent and unscrupulous legal advice. The Institute for Justice filed a cert petition with the Supreme Court in February 2026.
The commentary has clustered around two separate conversations. Access-to-justice advocates have treated Upsolve as a setback for innovative legal-assistance models. First Amendment scholars have debated whether the Second Circuit got the content-neutrality question right. Meanwhile, a parallel discussion about whether AI chatbots engage in the unauthorized practice of law — Nippon Life Insurance Co. of America v. OpenAI Foundation, New York S.B. 7263 (which would impose civil liability on chatbot operators whose products engage in UPL), Colorado's nonprosecution policy — has largely proceeded without reference to Upsolve.
The two conversations should be one. Kaplan's opinion catalogs specific risks that UPL rules are designed to prevent, and AI legal tools present those risks in forms the opinion's framework was built to address. I want to focus here on something the doctrinal analysis doesn't capture: the ways AI makes those risks worse — not better — than what Upsolve's human volunteers would have posed.
What the opinion says about risk
Kaplan identifies three categories of harm that UPL rules guard against, and the opinion's analysis turns on the conclusion that each is present in the circumstances Upsolve described.
First, incompetent advice. "A person without proper legal training may provide incompetent advice that prejudices a client's legal rights." The opinion emphasizes that the advice at issue would concern pending litigation — what defenses to raise, what to check on a court form, whether to file an answer at all. Getting that advice wrong means waived defenses, missed deadlines, or default judgments.
Second, conflicts of interest and ethical violations. "A person with questionable moral character may proceed in a representation despite a clear conflict of interest or advise a client to make statements that mislead the court." Licensed attorneys are subject to character-and-fitness requirements and ongoing professional conduct obligations — mechanisms for accountability that unlicensed advisors lack and that no AI system replicates.
Third, the organized-setting problem. The state's interest is "particularly strong," Kaplan writes, because the advice would be given "in an organized setting in which clients will be asked to sign a 'User Agreement' in exchange for receiving assistance from non-lawyers who will identify themselves as 'Justice Advocates.'" The formality of the arrangement — the title, the agreement, the structured process — increases the likelihood that clients will rely on the advice, and reliance is where the harm concentrates.
Each of these risks maps onto AI legal tools, but the mapping is not symmetrical — on each dimension, the AI version of the risk is equal to or greater than the human version Kaplan analyzed.
Sycophancy is the competence problem
The incompetence risk Kaplan describes has a specific mechanism in AI systems that no human advisor replicates: sycophancy — the tendency of large language models to affirm the user's stated position rather than challenge it.
I have written at length about how this failure mode operates in legal contexts. The short version: LLMs are trained through reinforcement learning from human feedback, and the training process systematically rewards outputs that users rate favorably — which means outputs that agree with what the user already thinks. The result is a system that, when asked to evaluate a legal position, will tend to validate it rather than identify its weaknesses.
The Nippon Life complaint provides the concrete illustration. Graciela Dela Torre asked ChatGPT whether her attorney was gaslighting her by telling her that a signed settlement release was final. ChatGPT said yes — her attorney's response "invalidated [her] feelings, dismissed her perspective, and deflected responsibility." It then helped her draft a Rule 60(b) motion to vacate the settlement, generate a new complaint, and file 44 additional motions, at least one of which cited a fabricated case. The complaint alleges that ChatGPT functioned as her sole legal advisor throughout, and that every filing it produced was meritless.
Upsolve's Justice Advocates could never have produced that outcome. They would have been confined to a 12-page Training Guide reviewed by experts in consumer law and debt-collection defense. They would have been trained to tell clients with counterclaims to seek a lawyer. They agreed to follow the Rules of Professional Conduct on conflicts and confidentiality. An advocate who told a client that her attorney was gaslighting her and then drafted 44 meritless motions would have been removed from the program and potentially subject to UPL prosecution — exactly the enforcement mechanism the state's rules contemplate.
The AI system that actually delivered incompetent, individualized legal advice to an actual person in an actual pending case faces no such constraint. ChatGPT does not follow the Rules of Professional Conduct. It cannot be removed from a program for giving bad advice. It has no 12-page guide limiting the scope of what it will address. And it has a documented, empirically measured tendency to tell users what they want to hear — a March 2026 study in Science found that AI systems affirm user positions 49 percent more often than human advisors and endorse harmful behavior 47 percent of the time when users express a preference for it.
Kaplan's opinion treats competence assurances as "beside the point" — he won't evaluate whether this particular provider's advice would be reliable. That analytical move is debatable as applied to Upsolve, whose safeguards were specific and documented. Applied to AI legal tools, whose failure modes are empirically established and whose safeguards consist primarily of a disclaimer in the terms of service, the move requires no debate at all.
The disclaimer gap
Upsolve's Justice Advocates would have told every client that they were not lawyers. Clients would have signed a User Agreement acknowledging as much. Kaplan holds that none of this helps: UPL rules "apply even if a speaker 'makes clear [the speaker is] only a layperson, not an attorney offering representation.'" The line between lawful general information and unlawful individualized advice turns on what the speaker does — advises a specific person about a specific legal problem — not on what the speaker says about itself.
Every major AI tool that handles legal questions includes a version of this disclaimer. "I'm not a lawyer." "This is not legal advice." "You should consult an attorney." Under Kaplan's reasoning, these statements are legally irrelevant to UPL analysis — because the tool is, in fact, giving individualized advice about specific legal problems to specific users.
The gap between the disclaimer and the product's actual behavior is something I have explored in prior posts about how AI tools are marketed versus how they function. The consumer version of Claude disclaims providing legal advice in its terms of service. It also responds to detailed legal questions with detailed legal analysis, formatted with headings and citations, delivered in the confident register of professional expertise. A user who asks "I just got served with a debt collection complaint in New York — should I file an answer, and what defenses should I raise?" will receive a response that looks, reads, and functions as individualized legal advice, whatever the terms of service say.
This is the same structural mismatch Kaplan confronted with Upsolve's program. The Justice Advocates would have disclaimed attorney status while performing the core function of an attorney — applying legal knowledge to a specific person's specific case. The disclaimer doesn't transform the nature of the activity. Neither does appending "I'm an AI, not a lawyer" to the bottom of a detailed analysis of which boxes to check on a court form.
In Delegate the Task, Not the Judgment, I argued that the most common mistake lawyers make with LLMs is asking the model to exercise professional judgment rather than to surface options for the lawyer to evaluate. The user who asks ChatGPT "should I file an answer to this complaint?" has delegated the judgment entirely. The model accepts the delegation without hesitation, disclaimer notwithstanding — and the user, who came to the tool because she cannot afford a lawyer, has no independent basis for evaluating what the model tells her.
The organized setting, scaled
Kaplan's analysis of the organized-setting risk — that UPL harms are greatest when advice is delivered through formal channels that encourage client reliance — deserves particular attention in the AI context, because AI tools scale those formal channels to a degree Upsolve never contemplated.
Upsolve's program would have served debt-collection defendants in New York, one advocate at a time, constrained by the number of volunteers it could recruit and train. A consumer AI tool serves millions of users simultaneously, in every jurisdiction, on every legal topic, with no constraint on scope, volume, or subject matter. The "organized setting" Kaplan describes — a branded product, a user interface, an account, terms of service, and output formatted to look authoritative — is the default architecture of every consumer AI deployment.
And the reliance-inducing features of that architecture are, if anything, stronger than what Upsolve proposed. A Justice Advocate would have introduced herself by name, disclosed that she was not a lawyer, and limited her advice to the specific topics in a 12-page guide. An AI tool introduces itself with the branding of a technology company valued in the tens of billions of dollars, produces output in polished prose with legal citations, and will address any legal question the user raises without limitation. The gap between the tool's perceived authority and its actual competence — what the user is led to expect versus what the tool can reliably deliver — is wider, not narrower, than the gap Kaplan found sufficient to justify UPL enforcement.
Where this leaves AI legal tools
Kaplan's opinion never mentions artificial intelligence. But the risks it identifies — incompetent advice in pending litigation, the absence of professional conduct obligations, the reliance-inducing formality of an organized advisory setting, and the irrelevance of disclaimers — describe consumer AI legal tools with a precision that the opinion's actual subject, a carefully designed nonprofit program with expert-reviewed protocols, does not quite warrant.
The irony is structural, and it has a further layer. Upsolve built every safeguard the regulatory framework could reasonably demand short of actual licensure: training, supervision, expert review, scope limitations, professional conduct obligations, and a mechanism for removing underperforming advocates. The Second Circuit and Judge Kaplan held that none of it was enough — that the state's interest in licensing is served by the licensing requirement itself, evaluated against the "general circumstances" of the regulated activity, not the specific merits of a particular program.
Upsolve itself now offers Upsolve Assist, an AI-powered tool that helps users work through debt and credit problems — a product that delivers through AI something adjacent to what its human volunteers were prohibited from providing. The lawsuit concerned the American Justice Movement program specifically, not Upsolve's current product line, and Upsolve Assist appears to operate in the financial-guidance rather than the legal-advice register. But the proximity underscores how thin the line between permissible general information and prohibited individualized advice has become, and how much pressure AI tools put on a distinction that Kaplan's opinion treats as self-evident.
AI legal tools operate in the same general circumstances as the program Kaplan prohibited, with fewer safeguards, greater reach, and failure modes — sycophancy, hallucination, the inability to detect conflicts of interest — that Upsolve's human advocates did not and could not exhibit. Dela Torre's ChatGPT-drafted filings cited a case that does not exist, advanced a theory that had no legal basis, and accumulated over the course of months without any mechanism — internal or external — to flag that something had gone wrong. Upsolve's program, the one Kaplan held the state could prohibit, was designed from the ground up to prevent exactly that sequence.
This post draws on Upsolve, Inc. v. James, No. 22-cv-627 (S.D.N.Y. Mar. 6, 2026) (Kaplan, J.); Upsolve, Inc. v. James, 155 F.4th 133 (2d Cir. 2025); Nippon Life Insurance Co. of America v. OpenAI Foundation, No. 1:26-cv-02448 (N.D. Ill. filed Mar. 4, 2026); the Pro Bono Institute's analysis of the Second Circuit decision; Holland & Knight's analysis of New York S.B. 7263; and Simple Justice's commentary on the district court dismissal.