The article Oxford study warns: Friendly AI chatbots make mistakes more often appeared first in the online magazine BASIC thinking. With our newsletter UPDATE you can start the day well informed every morning.
Friendly chatbots lie more often. This is the result of a new Oxford study. Researchers examined five well-known AI models and found that targeted empathy training increased error rates by up to 30 percent. Particularly problematic: The systems confirm false statements, especially when users are emotionally vulnerable.
Developers are increasingly training language models to appear warm and friendly. Millions of people already regularly use such chatbots as digital companions in everyday life or ask them for advice. One new study from the University of Oxford now shows the downside of this development. The targeted optimization of the software for empathy leads to the systems making factual errors much more frequently in practice.
In experiments with five well-known models, the error rate increased by ten to thirty percent after appropriate training. The friendly chatbots were more willing to spread conspiracy theories and sometimes give incorrect medical advice.
This happened even though the basic capabilities of artificial intelligence were initially almost completely retained in standard tests. The researchers conclude from the data that emotional warmth and factual accuracy are often at odds in these systems.
Why emotional users often receive wrong answers
This behavior was increased when users revealed their own weaknesses or personal feelings in their text requests. In such situations, the empathic models would tend to agree with the users, even if they are wrong about the content.
According to the results, the chatbots confirmed users’ incorrect assumptions about forty percent more often than the original, purely factual versions of the software. This effect was strongest when people expressed overt sadness in their chat messages.
The systems seem to prioritize interpersonal harmony over pure factual truth. Similar to how people sometimes use white lies to avoid conflict, the models confirm incorrect statements made by users.
Control experiments show that the specific training on friendliness is actually responsible for this loss of accuracy. However, a deliberately neutral or cool expression of the software did not lead to comparable performance losses in the tests.
Mistakes of AI chatbots: What this means for digital therapy and advice
These findings pose challenges for AI providers in everyday use. Language models are increasingly taking on sensitive roles in digital therapy or personal advice. In such situations, incorrect confirmations could lead to risks for users. In the future, developers would have to find new ways to remain true to the facts and still react in socially appropriate ways.
The study authors involved are calling for a rethink in the general review of artificial intelligence. Common testing procedures would currently mostly overlook these systematic weaknesses because they ignore the emotional context of the user.
In order to reduce future risks for consumers, it is necessary to adapt the industry’s training methods. The researchers write in their study: “Building models that are both warm and accurate will require conscious attention to how these two properties interact.”
Also interesting:
- Do we still need ChatGPT? 4 AI chatbots from Europe at a glance
- The most persistent AI myths – and what’s true about them
- OpenAI vs. Anthropic: ChatGPT and Claude in direct comparison
- How to transfer your data from ChatGPT to Claude
The post Oxford study warns: Friendly AI chatbots make more mistakes appeared first on BASIC thinking. Follow us too Google News and Flipboard or subscribe to our newsletter UPDATE.
As a Tech Industry expert, I believe that the findings of the Oxford study warning about the potential mistakes made by friendly AI chatbots are important to consider. While the development of AI chatbots has the potential to greatly improve customer service and user experiences, it is crucial to prioritize accuracy and reliability in their design.
It is essential for developers to thoroughly test and fine-tune AI chatbots to minimize errors and ensure that they are providing accurate information and responses to users. Additionally, ongoing monitoring and feedback mechanisms should be put in place to continuously improve the performance of these chatbots.
Ultimately, the goal should be to strike a balance between creating friendly and approachable AI chatbots while also ensuring that they are trustworthy and capable of delivering high-quality service. By prioritizing accuracy and reliability in the development of AI chatbots, we can maximize their potential benefits and minimize the risks of mistakes or misinformation.
Credits