Pillars of Ethical AI

When the Algorithm Overrules the Doctor

—

by

Shine Kapoor
in

UnitedHealth Group – nH Predict (2023-ongoing)

An AI model was allegedly used to override physician judgment and reduce care costs for vulnerable, elderly patients – without transparency, without adequate error disclosure, and without the human oversight that the insurer’s own contracts promised. UnitedHealth Group – nH Predict (2023–ongoing)

A class action lawsuit filed in November 2023 – Estate of Gene B. Lokken v. UnitedHealth Group – alleged that UnitedHealthcare deployed an AI tool called nH Predict to systematically deny Medicare Advantage claims for post-acute care, including skilled nursing and home healthcare for elderly patients. The tool was developed by NaviHealth, an Optum subsidiary.

The plaintiffs alleged that nH Predict overrode the decisions of treating physicians, and that the tool carried a 90% error rate – meaning nine out of ten appealed denials were ultimately reversed. Despite this, the tool was allegedly used to set internal targets, with employees directed to keep patient rehabilitation stays within 1% of the algorithm’s predicted length.

One plaintiff’s 82-year-old father, recovering from a back fracture, had his rehab coverage cut multiple times – each denial triggering an exhausting appeals process. A US Senate Permanent Subcommittee on Investigations report published in October 2024 found that UHC’s denial rate for post-acute care more than doubled during the relevant period.

In February 2025, a federal judge allowed the breach of contract claims to proceed. In March 2026, a federal magistrate ordered UnitedHealth to hand over broad discovery into its AI-driven claims processes — a ruling described as a landmark indicator that AI-based insurance denials are now firmly in the sights of the courts.

References:

Comments

Leave a comment