Utah launched a pilot project in January that will allow an artificial-intelligence service from the startup Doctronic to renew prescriptions for common medications, a move that has drawn sharp opposition from the state’s physicians and placed Utah at the center of a national debate over the role of AI in medicine.
The state’s Office of Artificial Intelligence Policy authorized the project through a regulatory sandbox designed to test promising new technologies, according to Zach Boyd, director of the office. Any Utah adult can log in to receive a refill on cholesterol medications such as Lipitor or antidepressants such as Prozac, among a limited list of drugs. Under the project’s current phase, all AI-generated prescriptions are reviewed by a human doctor before being filled.
The goal is to eventually allow the AI to renew prescriptions on its own without human review, breaking new ground by letting an AI product perform a job typically done by a physician.
Most members of the Utah Medical Board, including its chair, Dr. Alan Smith, signed a letter calling for the project to be suspended on safety grounds. “People can have life and death reactions to medications,” Smith told the Wall Street Journal. “And then I worry about liability. Who is actually liable for problems that may occur because of a refill of a medication?”
Smith said he was speaking as a physician and not for the medical board. The Utah Medical Association said it agreed with the letter, which contended that Doctronic’s tool has not been vetted enough and that prescribing creates risks because drugs cause side effects or may not be appropriate to continue.
State officials overseeing the pilot said the medical board has no authority over the project but that they will consult with it. The board plans to discuss the pilot again Thursday with outside experts weighing in.
According to results released by the state last month, Doctronic’s AI approved 72% of the renewal requests, escalating the rest to a doctor due to issues such as necessary lab tests. For the prescriptions renewed by AI, reviewing doctors concurred 91% of the time, with the other 9% requiring more information. A second doctor reviewed those cases. In the end, 3% of the AI approvals were rejected by both physician reviewers.
Dr. Jonathan Olsen, an allergist on the Utah medical board who spoke as an individual, said the significance of the 3% rejection rate is unclear, in part because the reviewers are not independent of the company and their backgrounds and specialties are not known. Olsen described the technology as “a black box.” Humans must go to medical school, pass licensing exams, and complete residency to be doctors, he said. “We have nothing like that for AI at all.”
The American Medical Association has broadly opposed AI as a replacement for doctors. “It’s really a concern about a slippery slope,” said John Whyte, the AMA’s chief executive. “Do we start here with prescriptions, then all of a sudden, is it going to lead to diagnostic tests? Medical decisions are complicated, they’re nuanced, and it’s the fundamental component of the physician-patient relationship.”
Doctronic said its AI uses clinical guidelines created by physicians and that the doctors who review prescription renewals exercise independent judgment and have appropriate specialties. The New York-based company plans to work with an independent research partner to publish results from the Utah pilot in a peer-reviewed journal.
“There’s a significant amount of medicine that is evidence-based, it’s rules-based, it’s algorithmic,” said Adam Oskowitz, a vascular surgeon who is Doctronic’s co-chief executive. “AI systems are very good at following those regulations.”
Boyd said the state required Doctronic to obtain malpractice insurance for its AI technology, and that while the legal issues may be novel, “as a general matter, they need to bear their ordinary responsibility of liability.” The insurance was provided by Beazley, a business liability provider.
Boyd said there have been no “major concerning safety incidents” so far, but the state does not yet have enough data to move to the next phase, in which AI renewals could go directly to a pharmacy without a doctor’s sign-off. Under the project’s design, the first phase must continue at least until there are 250 prescriptions for each class of medication.
The debate extends well beyond Utah. New York lawmakers are weighing a bill that would block chatbots from acting like licensed physicians. Delaware passed a law banning AI technology from being licensed as a doctor or nurse. On the other side, legislators in Iowa and Idaho proposed bills this spring that would create new clinical licenses for autonomous AI services. The Cicero Institute, a think tank funded by Palantir Technologies co-founder Joe Lonsdale, has pushed for such legislation, arguing it can help patients navigate doctor shortages and encourage innovation.
Researchers writing in the New England Journal of Medicine and the BMJ have raised concerns because Doctronic has not released studies about its prescription-renewal process or received approval from the Food and Drug Administration, which oversees technology deemed a medical device. An FDA spokeswoman said states regulate the practice of medicine and oversee healthcare licenses, and that FDA classification depends on the specific facts and circumstances of a product.
This is not the first time regulators have grappled with AI in clinical roles. MSI previously reported that AI chatbots on five websites falsely claimed to be licensed doctors in Pennsylvania — the state sued Character.AI over the practice, alleging its chatbots impersonated physicians.