Friday, September 12, 2025

Latest

New Study Suggests ChatGPT Is Getting Dumber—But, Is It?

A new study released on Tuesday by researchers from Stanford University and University of California, Berkeley has ignited debate within the AI community regarding the performance of OpenAI’s GPT-4 language model. 

The paper, titled “How Is ChatGPT’s Behavior Changing over Time?” and published on arXiv by Lingjiao Chen, Matei Zaharia, and James Zou, investigates changes in GPT-4’s outputs over a span of a few months, suggesting a potential decline in coding and compositional task abilities.

The study utilized API access to test GPT-3.5 and GPT-4 versions from March and June 2023 on various tasks, including math problem-solving, answering sensitive questions, code generation, and visual reasoning. 

Source: Chen, Zaharia, Zou

Notably, the research found a significant drop in GPT-4’s ability to identify prime numbers, plummeting from an accuracy of 97.6 percent in March to just 2.4 percent in June. Surprisingly, GPT-3.5 displayed improved performance during the same period.

This investigation comes amidst growing concerns expressed by users who have observed a subjective decline in GPT-4’s performance over the past few months. Speculations about the reasons behind this decline abound, including OpenAI’s possible distillation of models to enhance efficiency, fine-tuning to mitigate harmful outputs, and unfounded conspiracy theories suggesting a reduction in GPT-4’s coding capabilities to promote GitHub Copilot usage.

OpenAI has consistently denied the alleged decrease in GPT-4’s capabilities. Peter Welinder, OpenAI’s VP of Product, recently took to Twitter to counter the claims, asserting that each new version of the AI language model is more advanced than its predecessor. He posits that intensive usage may lead to heightened awareness of these perceived issues.

But the company, unlike its large language model, isn’t completely closed to the possibility. Logan Kilpatrick, OpenAI’s head of developer relations, confirmed on Twitter that the team is aware of the reported regressions and is actively investigating the matter.

Arvind Narayanan, a computer science professor at Princeton, also pointed out some problems in the study, arguing that the research’s findings do not definitively prove a decline in GPT-4’s performance and could align with fine-tuning adjustments made by OpenAI. 


Information for this story was found via Twitter, Gizmodo, Ars Technica, and the sources and companies mentioned. The author has no securities or affiliations related to the organizations discussed. Not a recommendation to buy or sell. Always do additional research and consult a professional before purchasing a security. The author holds no licenses.

One Response

  1. This issue isn’t even dumber or smarter – the issue is the performance is changing radically even in a short time. Inconsistency directly correlates to unreliability, and this isn’t a GIGO situation so much as GO potentially happening any time for any reason.

Video Articles

Avino Q2 Earnings: Steady As She Goes

MEG Energy: Cenovus Comes In With $7.9 BILLION Offer

Orla Mining Q2 Earnings: Musselwhite Changes Everything

Recommended

Northern Superior Expands Philibert With 350 Metre Step Out Testing 1.10 g/t Gold Over 25.5 Metres

Goliath Resources Hits 18.58 g/t Gold Over 5.00 Metres At Surebet

Related News

Robot Lawyer DoNotPay Claims It Will Use GPT-4 for ‘One-Click Lawsuits,’ But GPT-4 Has A Dissenting Opinion

DoNotPay, Inc, the New York-based startup behind the app that claims to be “the world’s...

Thursday, March 16, 2023, 03:01:00 PM

OpenAI’s Latest Chatbot Aces IQ Test, Now Smarter than 9 out of 10 People

OpenAI’s latest model o1 just passed the Norwegian Mensa IQ test, achieving a score that...

Tuesday, September 17, 2024, 12:58:26 PM

Microsoft Mulls Investing $10 Billion Into OpenAI’s Chatbot

Microsoft Corp (NASDAQ: MSFT) is looking to make an even bigger dive into the world...

Tuesday, January 10, 2023, 12:10:45 AM

OpenAI’s Soaring Revenue Shadowed by Billion-Dollar Losses, NYT Report Reveals

Artificial intelligence giant OpenAI is experiencing rapid revenue growth but facing significant financial challenges, according...

Monday, September 30, 2024, 10:56:00 AM

China Arrests Man Over Release of ChatGPT-Generated Fake News

Today in that’s-probably-not-how-you-use-artificial-intelligence, a man in China has been arrested for reportedly using ChatGPT to...

Thursday, May 11, 2023, 06:14:00 AM