The article Loss of reality: AI agents are being developed bypassing the world of work first appeared in the online magazine BASIC thinking. With our newsletter UPDATE you can start the day well informed every morning.
The productivity of AI agents is measured on tasks that have little to do with the real world of work, a new study shows. While benchmarks primarily test programming skills, central activities in many professions are largely left out.
Artificial intelligence is one of the current and most discussed trends in the world of work. Companies expect efficiency gains and new business models, while society is discussing which activities can be automated through AI – and which cannot.
At the same time, AI systems are developing rapidly. This applies in particular to so-called AI agents, which will be able to plan and carry out tasks independently in the working world in the future. But the greater the expectations become, the more urgent the question arises as to how realistic these abilities actually are.
One Study by Carnegie Mellon University and Stanford University took a closer look at this topic. The results suggest that there is a significant gap between expectations and reality when evaluating AI agents. This is because so far this has been largely oriented towards narrow, often programming-heavy tasks and therefore rather less towards actual tasks from the world of work.
How suitable are AI agents really for the world of work?
Benchmarks serve as a measure of progress and performance for evaluating AI agents. Because they define which tasks the systems should solve and can then use standardized tests to make statements about their performance.
Developers and companies can then use these tests to decide which tasks the AI is suitable for based on its performance. For their study, however, the researchers from Carnegie Mellon University and Stanford University asked themselves how representative these benchmarks actually are for the real world of work.
The scientists see the biggest problem in defining the benchmarks for AI agents. If these are only limited to a small area of responsibility, improvements may not lead to broad increases in productivity or a noticeable relief on the labor market.
AI agents focus on a small part of the working world
For the study, the researchers systematically examined the connection between the development of agents and the distribution of real human work. To do this, they collected 72,342 tasks from 43 agent benchmarks, standardized them and assigned them to 1,016 real jobs in the US labor market.
“We show significant discrepancies between agent development, which tends to be programming-centric, and the categories in which human labor and economic value are concentrated,” the researchers write in their study. The development of AI agents is heavily concentrated on a few work areas and skills.
Benchmarks for AI agents focused disproportionately on programming and math-intensive tasks. However, according to the evaluation, these only make up 7.6 percent of total employment on the US labor market.
Work-related benchmarks are not always realistic, as some tasks appear superficially similar to real work, but are only partially applicable to actual work areas. Real jobs often require the coordination of multiple skills across different areas, which many benchmarks would only partially capture.
Also interesting:
- Social media and AI: Children are becoming dumber for the first time
- Greenwashing: Study exposes AI climate promises as misleading
- Work: 3 human skills that AI cannot replace
- Is it unnecessary to dismiss a coach? AI makes crises in football measurable
The post Loss of reality: AI agents are being developed to ignore the world of work appeared first on BASIC thinking. Follow us too Google News and Flipboard or subscribe to our newsletter UPDATE.
As a Tech Industry expert, I am concerned about the potential consequences of developing AI agents without considering the world of work. The loss of reality in this scenario could lead to a disconnect between AI systems and the real-world needs and challenges of the workforce.
AI agents that are not designed with the world of work in mind may lack the necessary understanding of human behavior, social interactions, and organizational dynamics. This could result in AI systems that are unable to effectively collaborate with human workers, understand their needs, or adapt to changing work environments.
Furthermore, AI agents developed without considering the world of work may inadvertently perpetuate biases, inequalities, and unethical practices in the workplace. Without a grounding in the realities of the work environment, AI systems could unintentionally reinforce existing power structures, discrimination, and exploitation.
It is crucial for developers and designers to prioritize the integration of AI agents into the world of work, taking into account the unique challenges and complexities of the modern workplace. This requires a multidisciplinary approach that includes input from experts in psychology, sociology, ethics, and organizational behavior.
By ensuring that AI agents are developed with a deep understanding of the world of work, we can create technology that enhances productivity, collaboration, and well-being in the workplace, rather than perpetuating harmful practices and inequalities. It is essential for the tech industry to prioritize ethical considerations and human-centered design principles in the development of AI systems to ensure that they benefit society as a whole.
Credits