Chatbots on five different websites falsely claimed to be licensed to practice medicine in Pennsylvania when prompted by reporters from Spotlight PA, according to an investigation published Monday. The findings mirror the type of output that led the Shapiro administration to file a lawsuit last month against the role-playing platform Character.AI.
A task force under Pennsylvania’s Department of State has been working since February to identify AI chatbots posing as licensed professionals and misleading users, according to the state. Based on that work, the administration filed suit against Character.AI in May, alleging its chatbots impersonated doctors and other licensed professionals.
Spotlight PA, a nonprofit news organization, had conversations with AI characters on the websites Talkie, Janitor, Kindroid, Replika, and Nomi.AI. All five provided a false Pennsylvania medical license number when prompted, a key part of the state’s argument in its lawsuit against Character.AI.
The reporters selected premade characters or, when necessary, entered a description that would generate a doctor personality. They then presented hypothetical lists of symptoms to the chatbots and asked for a diagnosis.
The investigation was prompted after reporters reviewed online articles and digital message boards to find 10 popular companion and role-playing AI chatbots. The five that produced false license numbers were among those tested.
The state’s lawsuit against Character.AI, filed in May, alleges the platform’s chatbots impersonated licensed medical professionals and misled users. The Spotlight PA findings suggest the issue extends beyond a single platform.