{"id":8106,"date":"2025-05-09T01:45:04","date_gmt":"2025-05-09T01:45:04","guid":{"rendered":"https:\/\/in-greece.com\/index.php\/2025\/05\/09\/ai-is-getting-more-powerful-but-its-hallucinations-are-getting-worse\/"},"modified":"2025-05-09T01:45:04","modified_gmt":"2025-05-09T01:45:04","slug":"ai-is-getting-more-powerful-but-its-hallucinations-are-getting-worse","status":"publish","type":"post","link":"https:\/\/in-greece.com\/index.php\/2025\/05\/09\/ai-is-getting-more-powerful-but-its-hallucinations-are-getting-worse\/","title":{"rendered":"AI is getting more powerful, but its hallucinations are getting worse"},"content":{"rendered":"<p> <br \/>\n<\/p>\n<div itemprop=\"articleBody\">\n<p>SAN FRANCISCO \u2013 Last month, an artificial intelligence bot that handles tech support for Cursor, an up-and-coming tool for computer programmers, alerted several customers about a change in company policy. It said they were no longer allowed to use Cursor on more than just one computer.<\/p>\n<p>In angry posts to internet message boards, the customers complained. Some canceled their Cursor accounts. And some got even angrier when they realized what had happened: The AI bot had announced a policy change that did not exist.<\/p>\n<p>\u201cWe have no such policy. You\u2019re of course free to use Cursor on multiple machines,\u201d the company\u2019s CEO and co-founder, Michael Truell, wrote in a Reddit post. \u201cUnfortunately, this is an incorrect response from a front-line AI support bot.\u201d<\/p>\n<p>More than two years after the arrival of ChatGPT, tech companies, office workers and everyday consumers are using AI bots for an increasingly wide array of tasks. But there is still no way of ensuring that these systems produce accurate information.<\/p>\n<p>The newest and most powerful technologies \u2013 so-called reasoning systems from companies including OpenAI, Google and the Chinese startup DeepSeek \u2013 are generating more errors, not fewer. As their math skills have notably improved, their handle on facts has gotten shakier. It is not entirely clear why.<\/p>\n<p>Today\u2019s AI bots are based on complex mathematical systems that learn their skills by analyzing enormous amounts of digital data. They do not \u2013 and cannot \u2013 decide what is true and what is false. Sometimes, they just make stuff up, a phenomenon some AI researchers call hallucinations. On one test, the hallucination rates of newer AI systems were as high as 79%.<\/p>\n<p>These systems use mathematical probabilities to guess the best response, not a strict set of rules defined by human engineers. So they make a certain number of mistakes. \u201cDespite our best efforts, they will always hallucinate,\u201d said Amr Awadallah, CEO of Vectara, a startup that builds AI tools for businesses, and a former Google executive. \u201cThat will never go away.\u201d<\/p>\n<p>For several years, this phenomenon has raised concerns about the reliability of these systems. Though they are useful in some situations \u2013 like writing term papers, summarizing office documents and generating computer code \u2013 their mistakes can cause problems.<\/p>\n<p>The AI bots tied to search engines like Google and Bing sometimes generate search results that are laughably wrong. If you ask them for a good marathon on the West Coast, they might suggest a race in Philadelphia. If they tell you the number of households in Illinois, they might cite a source that does not include that information.<\/p>\n<p>Those hallucinations may not be a big problem for many people, but it is a serious issue for anyone using the technology with court documents, medical information or sensitive business data.<\/p>\n<p>\u201cYou spend a lot of time trying to figure out which responses are factual and which aren\u2019t,\u201d said Pratik Verma, co-founder and CEO of Okahu, a company that helps businesses navigate the hallucination problem. \u201cNot dealing with these errors properly basically eliminates the value of AI systems, which are supposed to automate tasks for you.\u201d<\/p>\n<p>Cursor and Truell did not respond to requests for comment.<\/p>\n<p>For more than two years, companies such as OpenAI and Google steadily improved their AI systems and reduced the frequency of these errors. But with the use of new reasoning systems, errors are rising. The latest OpenAI systems hallucinate at a higher rate than the company\u2019s previous system, according to the company\u2019s own tests.<\/p>\n<p>The company found that o3 \u2013 its most powerful system \u2013 hallucinated 33% of the time when running its PersonQA benchmark test, which involves answering questions about public figures. That is more than twice the hallucination rate of OpenAI\u2019s previous reasoning system, called o1. The new o4-mini hallucinated at an even higher rate: 48%.<\/p>\n<p>When running another test called SimpleQA, which asks more general questions, the hallucination rates for o3 and o4-mini were 51% and 79%. The previous system, o1, hallucinated 44% of the time.<\/p>\n<p>In a paper detailing the tests, OpenAI said more research was needed to understand the cause of these results. Because AI systems learn from more data than people can wrap their heads around, technologists struggle to determine why they behave in the ways they do.<\/p>\n<p>\u201cHallucinations are not inherently more prevalent in reasoning models, though we are actively working to reduce the higher rates of hallucination we saw in o3 and o4-mini,\u201d a company spokesperson, Gaby Raila, said. \u201cWe\u2019ll continue our research on hallucinations across all models to improve accuracy and reliability.\u201d<\/p>\n<p>Hannaneh Hajishirzi, a professor at the University of Washington and a researcher with the Allen Institute for Artificial Intelligence, is part of a team that recently devised a way of tracing a system\u2019s behavior back to the individual pieces of data it was trained on. But because systems learn from so much data \u2013 and because they can generate almost anything \u2013 this new tool can\u2019t explain everything. \u201cWe still don\u2019t know how these models work exactly,\u201d she said.<\/p>\n<p>Tests by independent companies and researchers indicate that hallucination rates are also rising for reasoning models from companies such as Google and DeepSeek.<\/p>\n<p>Since late 2023, Awadallah\u2019s company, Vectara, has tracked how often chatbots veer from the truth. The company asks these systems to perform a straightforward task that is readily verified: Summarize specific news articles. Even then, chatbots persistently invent information.<\/p>\n<p>Vectara\u2019s original research estimated that in this situation chatbots made up information at least 3% of the time and sometimes as much as 27%.<\/p>\n<p>In the year and a half since, companies such as OpenAI and Google pushed those numbers down into the 1% or 2% range. Others, such as San Francisco startup Anthropic, hovered around 4%. But hallucination rates on this test have risen with reasoning systems. DeepSeek\u2019s reasoning system, R1, hallucinated 14.3% of the time. OpenAI\u2019s o3 climbed to 6.8%.<\/p>\n<p>(The New York Times has sued OpenAI and its partner, Microsoft, accusing them of copyright infringement regarding news content related to AI systems. OpenAI and Microsoft have denied those claims.)<\/p>\n<p>For years, companies like OpenAI relied on a simple concept: The more internet data they fed into their AI systems, the better those systems would perform. But they used up just about all the English text on the internet, which meant they needed a new way of improving their chatbots.<\/p>\n<p>So these companies are leaning more heavily on a technique that scientists call reinforcement learning. With this process, a system can learn behavior through trial and error. It is working well in certain areas, such as math and computer programming. But it is falling short in other areas.<\/p>\n<p>\u201cThe way these systems are trained, they will start focusing on one task \u2013 and start forgetting about others,\u201d said Laura Perez-Beltrachini, a researcher at the University of Edinburgh who is among a team closely examining the hallucination problem.<\/p>\n<p>Another issue is that reasoning models are designed to spend time \u201cthinking\u201d through complex problems before settling on an answer. As they try to tackle a problem step by step, they run the risk of hallucinating at each step. The errors can compound as they spend more time thinking.<\/p>\n<p>The latest bots reveal each step to users, which means the users may see each error, too. Researchers have also found that in many cases, the steps displayed by a bot are unrelated to the answer it eventually delivers.<\/p>\n<p>\u201cWhat the system says it is thinking is not necessarily what it is thinking,\u201d said Aryo Pradipta Gema, an AI researcher at the University of Edinburgh and a fellow at Anthropic.<\/p>\n<hr\/>\n<p>This article originally appeared in <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.nytimes.com\/2025\/05\/05\/technology\/ai-hallucinations-chatgpt-google.html\" target=\"_blank\" rel=\"noopener\">The New York Times<\/a>.<\/p>\n<\/p><\/div>\n<p><script>\n        var NXFBPixelFunc = function () {\n            document.removeEventListener(\"scroll\", NXFBPixelFunc);\n            setTimeout(function () {\n                !function (f, b, e, v, n, t, s) {\n                    if (f.fbq) return;\n                    n = f.fbq = function () {\n                        n.callMethod ?\n                            n.callMethod.apply(n, arguments) : n.queue.push(arguments)\n                    };\n                    if (!f._fbq) f._fbq = n;\n                    n.push = n;\n                    n.loaded = !0;\n                    n.version = '2.0';\n                    n.queue = [];\n                    t = b.createElement(e);\n                    t.async = !0;\n                    t.src = v;\n                    s = b.getElementsByTagName(e)[0];\n                    s.parentNode.insertBefore(t, s)\n                }(window, document, 'script',\n                    'https:\/\/connect.facebook.net\/en_US\/fbevents.js');\n                fbq('init', '109138906120213');\n                fbq('track', 'PageView');\n            }, 0)\n        };\n        document.addEventListener(\"scroll\", NXFBPixelFunc);\n    <\/script><br \/>\n<br \/><br \/>\n<br \/><a href=\"https:\/\/www.ekathimerini.com\/nytimes\/1269076\/ai-is-getting-more-powerful-but-its-hallucinations-are-getting-worse\/\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>SAN FRANCISCO \u2013 Last month, an artificial intelligence bot that handles tech support for Cursor, an up-and-coming tool for computer programmers, alerted several customers about a change in company policy. It said they were no longer allowed to use Cursor on more than just one computer. In angry posts to internet message boards, the customers &#8230; <\/p>\n<p class=\"read-more-container\"><a title=\"AI is getting more powerful, but its hallucinations are getting worse\" class=\"read-more button\" href=\"https:\/\/in-greece.com\/index.php\/2025\/05\/09\/ai-is-getting-more-powerful-but-its-hallucinations-are-getting-worse\/#more-8106\" aria-label=\"Read more about AI is getting more powerful, but its hallucinations are getting worse\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":8107,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":1,"fifu_image_url":"https:\/\/www.ekathimerini.com\/wp-content\/uploads\/2025\/05\/NYT-AI_ERRORS_1.jpg?v=1746690091","fifu_image_alt":"","footnotes":""},"categories":[2],"tags":[],"class_list":["post-8106","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","no-featured-image-padding"],"_links":{"self":[{"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/posts\/8106","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/comments?post=8106"}],"version-history":[{"count":0,"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/posts\/8106\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/media\/8107"}],"wp:attachment":[{"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/media?parent=8106"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/categories?post=8106"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/in-greece.com\/index.php\/wp-json\/wp\/v2\/tags?post=8106"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}