The chatbots’ failures — including mislabeling a college’s own president and giving incorrect office hours — raise questions about whether public funds are well spent on technology students say they cannot trust, even as district officials argue the tools handle thousands of interactions each month that would otherwise require costly human staff.

California community college districts have spent millions of dollars on artificial intelligence chatbots intended to help students navigate admissions, financial aid and campus services, but testing and student accounts show the systems frequently provide inaccurate or outdated answers, according to a CalMatters investigation distributed by the Associated Press.

The Los Angeles Community College District — the state’s largest community college system — has approved contracts and amendments totaling about $3.8 million for chatbot services through 2029, according to district board documents. Three community college districts that responded to a CalMatters survey reported annual chatbot costs ranging from about $151,000 to nearly half a million dollars.

What testing found

CalMatters tested chatbots at several California community colleges and found the systems generally returned quick, accurate responses to common questions but were less consistent with more specific ones.

East Los Angeles College’s chatbot incorrectly identified Alberto Román as the college’s current president; Román had left that position to become the district’s chancellor. In another test, the bot provided incorrect hours and dates when asked about the financial aid office’s schedule.

The same chatbot, powered by Gravyty, showed inconsistencies when asked in Spanish whether a Social Security number is required to enroll. Rather than answering the question, the bot directed users to visit the registrar’s office to update their Social Security number. When asked the same question in English, it pivoted to discussing financial aid.

Fresno City College’s chatbot, also powered by Gravyty, showed similar problems — often failing to direct students to the correct offices and, in some cases, listing incorrect locations and hours.

Santa Monica College’s chatbot, powered by Gecko, performed better in testing. The district has contracted with Gecko since 2019 and renewed its annual contract for $57,000 late last year, according to district board documents.

Student accounts

Pablo Aguirre, a computer science major at East Los Angeles College and an information technology intern at the Los Angeles district office, said he mostly avoids the chatbot because it might provide unreliable or outdated information.

“I just didn’t find it as useful,” Aguirre said. He said he gave up on the bot after it kept asking him questions instead of providing a clear answer when he searched for financial aid information.

Reanna Carlson, a commercial music major and student government vice president at Fresno City College, said the college’s chatbot — dubbed Sam the Ram after its mascot — repeatedly gave unclear or incorrect answers to basic questions about campus services.

“I think the chatbot is outdated and can’t navigate the services we provide on campus effectively,” Carlson said. “I don’t think it’s the most beneficial option when it comes to asking questions.”

Her district, the State Center Community College District, which includes campuses in Fresno and nearby counties, has a nearly $870,000 three-year contract for Gravyty through June 2026, according to district board documents. District officials noted the contract includes additional services, such as tools for staff live chats and text messaging to students.

Why districts say the spending is justified

District officials say the chatbots are worth their cost because of heavy student use and around-the-clock availability.

Betsy Regalado, an associate vice chancellor at the Los Angeles district, said its nine colleges average 5,000 to 7,000 chatbot interactions per month. The State Center district reported about 5,000 interactions per month; Santa Monica College reported about 4,000.

“Every technology has a cost. We would simply not be able to assist all students if they could only reach us using traditional methods,” said Esau Tovar, dean of enrollment services at Santa Monica College.

Regalado attributed the systems’ shortcomings to how they are configured rather than to the underlying technology.

“The current chatbot that we have uses a library of questions. If you don’t have that question in that library, then those poor people don’t get an answer or they won’t get an accurate answer,” Regalado said.

Plans for improvement

Several districts are planning or have already completed transitions to newer AI systems that scrape college websites to generate responses rather than relying on manually maintained question libraries.

The Los Angeles district plans to transition all nine colleges to Gravyty’s generative AI platform as early as late spring at no additional cost under its existing contract, which runs through 2029, Regalado said.

“We’re ready for the modernization of (the chatbot) and the change to generative AI. That is the new world out there,” Regalado said.

Santa Monica College has already completed a similar transition to a ChatGPT-integrated system. Tovar said the college prioritizes keeping its website current so the bot provides “good answers with fewer errors” rather than “great answers with potentially more errors.”

Wider concerns

Chatbot reliability problems have surfaced beyond California. Reporting by The Markup and THE CITY found that a city-run AI chatbot in New York City provided guidance that could lead to illegal behavior, prompting Mayor Zohran Mamdani to terminate it in February.

For some students, the stakes of inaccurate chatbot guidance are particularly high. Bryan Hartanto, a civil engineering major from Indonesia at Santa Monica College, said the college’s newer system can be a useful starting point but that he worries about following incorrect guidance that could jeopardize his visa status.

“Maintaining status as an international student right now is very, very sensitive,” Hartanto said. “I would still rely on human or email communication.”

Reporting by Martin Romero, College Journalism Network / CalMatters, distributed through a partnership with the Associated Press. CalMatters higher education coverage is supported by a grant from the College Futures Foundation.

Released under CC0 — No Rights Reserved.