Has ChatGPT gotten dumber? No, but it probably has changed

“Something is wrong with GPT-4: it seems faster, but the quality of the results seems worse”. One of the first to report the problem, lo last May 21, it was Peter Yang, AI expert and Roblox product lead: The average quality of ChatGPT results is declining.

A feeling that, looking at the OpenAI forum, seems to be common to many users. Scrolling through the thread, there are a series of messages all more or less of the same tenor. “Forget the inputs,” writes one user; “It appears his IQ has decreased,” confirms another.

What is happening to ChatGPT

It is, at the moment, about simple signals, very difficult to prove: There are no concrete indications of an actual decrease in the quality of the outputs of GPT-4, the most powerful model of OpenAI, available for paid users of ChatGPT. However, the echo of these conversations has prompted the US artificial intelligence community to look for an explanation.

Explanation that, thanks to a leak of OpenAI confidential documents, seems to have arrived. As also confirmed by a report by SemiAnalysis, Dylan Patel’s newsletter, what seems to have changed was the ChatGPT architecture itselfi.e. how the model analyzes and generates the information. In essence, behind GPT-4 there would not be a single huge artificial intelligence, but a group of 16 smaller AIs that process information as needed.

MoE, what it is and how it works

In an interview given to Business Insider, Sharon Zhou, CEO of Lamini, explained that this approach is called Mixture of Experts, or MoE. It’s pretty simple, at least in words: an AI model is divided into smaller expert models, which are trained on specific tasks and subject areas. In other words, there would be 16 mini GPT-4s, each specializing in a specific area of ​​training material. When a user asks a question, a prompt in technical jargon, the system manages to identify which are the two or more models to consult and, finally, to combine the results.

The move by OpenAI – which for now has neither confirmed nor denied the reconstruction – seems to be in the name of efficiency. In a document published by Sam Altman’s company in June 2022, signed among others by the current President Greg Brockmanthe Mixture of Experts is considered one of the ways to increase efficiency of large language models such as GPT-4. “With this approach – reads the research – only a part of the network is used to generate the output”.

The idea behind the model is to improve the accuracy of the answers, making them faster and cheaper at the same time. After all, last April The Information had revealed how the cost of running the ChatGPT infrastructure was $700,000 per day, about 21 million dollars a month. “The sensation – explained the CEO of Allen Institute for AI Oren Etzioni again at Business Insider – is that in this case OpenAI stands sacrificing some quality to reduce generation costs”.

The dilemmas of OpenAI, from declining users to the FTC

This story comes at the culmination of a challenging time for OpenAI. Hangover at the turn of 2022 and 2023, with the launch of ChatGPT, Sam Altman’s company is having a tough time. In June, the number of users to its chatbot dropped for the first time since last November’s launch, according to data from Similar Web and Sensor Tower. Meanwhile, in July the US FTC, the government agency that protects consumers in the US, opened an investigation, for fears concerning “the reputation and personal data of individuals”.