Summary
Anthropic’s standoff with the Pentagon has reshaped competition among leading chatbot makers while also fueling debate about whether today’s large language models are ready for military use, especially where mistakes could cost lives. The Defense Department ordered government agencies to stop using Anthropic’s Claude and designated it a supply-chain risk after Anthropic CEO Dario Amodei declined to weaken the company’s ethical safeguards.
The Pentagon’s action came as President Donald Trump said the Pentagon would have six months to phase out Anthropic’s military applications, according to the AP report. Anthropic said it would challenge the Pentagon in court once it received formal notice of penalties, setting up a legal fight over both the ban and what Claude is allowed to do in government settings.
Officials declined to clarify whether the Defense Department is still using Claude, including in the Iran war, citing operational security. The lack of confirmation left open questions for outside experts about how much, if any, the system has already been woven into military planning and decision-making.
Missy Cummings, a former Navy fighter pilot and director of the robotics and automation center at George Mason University, said the technology’s limits make it dangerous in wartime environments. In comments to the Associated Press, she argued that generative AI chatbots can produce errors described as “hallucinations or confabulations,” and she pointed to the risk of harming both civilians and troops.
Cummings said she expects “You’re going to kill noncombatants,” adding that the same limitations could lead to “You’re going to kill your own troops.” She also said she was “not clear whether the military truly understands the limitations,” and called for close human oversight, telling the AP that “A human has to babysit these technologies very closely. You can use them to do these things, but you need to verify, verify, verify.”
Cummings has previously argued that government agencies should prohibit generative AI from “control, direct, guide or govern any weapon,” not because the systems might go rogue, but because large language models can still make too many mistakes to be trusted with lethal outcomes. Her position aligns with a broader concern among military and human-rights experts who have applauded Amodei’s ethical stance while warning that prior industry marketing pushed governments toward high-stakes deployments.
Amodei, for his part, defended Anthropic’s safeguards by emphasizing reliability limits. In the AP report, he said “frontier AI systems are simply not reliable enough to power fully autonomous weapons” and added that Anthropic would not knowingly provide a product that puts America’s warfighters and civilians at risk.
The dispute has also played out in consumer data. Sensor Tower said Claude’s chatbot app outpaced rival ChatGPT in phone app downloads in the United States, a shift that the AP report said signaled growing consumer interest in Anthropic’s approach during the Pentagon standoff. That consumer movement coincided with apparent backlash against OpenAI after OpenAI announced a Friday deal with the Pentagon to effectively replace Anthropic in classified environments.
In Apple’s App Store, the number of 1-star reviews for ChatGPT grew by 775% on Saturday and continued to rise early in the week, the AP report said, reflecting a backlash that forced OpenAI to do “damage control.” OpenAI CEO Sam Altman said in a social media post Monday that “We shouldn’t have rushed to get this out on Friday,” saying the issues were “super complex” and that attempts to de-escalate “just looked opportunistic and sloppy.”
Altman also said in remarks on X that there are “many things the technology just isn’t ready for,” and described plans to work with the Pentagon “slowly,” using “technical safeguards and other methods.” The AP report said he gathered employees for an “all-hands” meeting on Tuesday to discuss next steps.
Even as the Pentagon moved to restrict Claude’s government use, other analysts said the controversy is giving Anthropic a reputational boost as a safety-minded developer. Jennifer Huddleston, a senior fellow at the libertarian-leaning Cato Institute, said it was “applaudable” that a company stood up to the government to maintain its ethics and business choices despite “potentially crippling policy responses.”
In parallel, some critics said the moment also reflects a long-running problem of overpromising about what generative AI can reliably do. Cummings said hype from major AI firms helped persuade the government to apply the technology to high-stakes tasks, and she placed blame in part on Anthropic for “driving the hype” and in part on the Department of War for not heeding earlier advice against risky uses.
The Pentagon’s order, Anthropic’s anticipated court challenge, and the consumer adoption signals suggest the dispute could keep reshaping how governments and companies draw the boundary between chatbot capability and military risk—especially as questions remain about what systems should be allowed to do when the cost of error is measured in lives.