Our company’s main business is developing AI agents that act as coaches and tutors for workforce training. However, this has led some clients to ask “If your team can build an AI agent to teach someone to do a job, why can’t you just build an AI agent to do the job?”

The answer to that question is twofold:

• Yes, we can build AI agents to automate work… and often do.
• Training people to do a job versus building AI agents to do a job isn’t necessarily an either/or proposition.

AI implementation isn’t just about setting up automated processes and letting them run: it’s also about reshaping roles, redefining expertise, and sometimes rethinking an organization’s entire approach to work. Meanwhile, the capabilities that make AI effective for training employees (i.e., understanding context, applying complex methodologies, adapting to individual learner needs) are also critical for effective automation.

In this article, we’ll examine four distinct levels of integrating AI and human capacity at work – and the right and wrong ways to go about each of them:

1. Automation where AI executes tasks on its own
2. Augmentation where AI raises baseline capability of less-experienced staff
3. Expert acceleration where AI enhances the value or increases the availability of experts
4. New capabilities where AI makes work economically or technically feasible that wasn’t before

1. Automating Tasks (That Naturally Lend Themselves to Automation)

2. Augmenting the Capabilities of Junior and Mid-Level Staff

As aging populations increase demand for medical tests, hospitals are investing in imaging technology like digital X-ray and ultrasound machines, CT scanners and MRIs. Yet, at the same time, there is a global shortage of radiologists to interpret the output of those machines.

This led researchers at Stanford University and Mount Sinai Hospital to conduct an experiment to see if non-specialist physicians could interpret images with AI assistance. The result? Non-specialists with AI performed about as well as radiologists without it – meaning it should be possible to partly close the labor gap by having non-specialists handle routine cases with AI support (escalating any unusual cases to an actual radiologist only when necessary.)

These types of “specialized but routine” tasks, which only require a narrow slice of expert judgement, represent the low-hanging fruit of human-AI collaboration. But AI-enabled delegation requires experts to cede a measure of control and demands a high degree of digital literacy and attention to detail from those operating the AI system. That’s why, when testing AI agents for clients, we will often have one of our own junior staff perform a task alongside the client’s staff. And if our team member is able to beat both the client’s junior team members or the client’s experts by a significant margin (e.g. 94% quality rating versus 72%), we might recommend training for the client’s staff, or rethinking the job description for junior roles to emphasize quality assurance / editorial / research experience (for the attention to detail / fact-checking aspect) plus overall digital literacy.

3. Accelerating the Work of Experts

If AI assistants can turn non-specialist doctors into passable radiologists, what happens when an expert radiologist uses AI?

Surprisingly, the answer is “it depends.”

When Artificial Intelligence Meets Lived Experience

Researchers at Harvard Medical School, MIT, and Stanford studied 140 radiologists working on chest X-ray diagnostics across 324 patient cases. Some radiologists saw dramatic improvements with AI assistance. Some actually performed worse. Some experienced no change at all.

And here’s what really surprised the researchers: the factors you’d expect to predict success – years of experience, whether they specialized in chest radiology, prior use of AI tools – didn’t reliably predict anything. Some veterans benefited enormously. Others didn’t. Some rookies improved. Others struggled.

This isn’t unique to radiology. A recent study of experienced software developers found something similar: when working in codebases they knew intimately, AI assistance actually slowed them down by 19% – even though the developers *believed* they were working 20% faster.

The culprit? Time spent reviewing and correcting AI suggestions. “The AIs made some suggestions about their work, and the suggestions were often directionally correct, but not exactly what’s needed,” the researchers noted.

Looking at yet another field, a major study published in Oxford’s Quarterly Journal of Economics tracked 5,172 customer service agents handling 3 million customer chats. Overall productivity increased 15% with AI assistance – but the gains were dramatically uneven. Workers with less than two months of experience saw their productivity jump 30%. The most experienced, highest-skilled workers? Essentially no productivity gain, and small but statistically significant declines in conversation quality.

How AI and Human Experts Can Learn to Collaborate

Recent research from the Universities of Texas and Yale offers an explanation for why experts might not benefit as much from AI support as non-experts: most AI systems are optimized for accuracy, not for adding value when advising humans.

The difference matters. A lot.

When AI offers advice that contradicts an expert’s judgment, the expert has to stop, evaluate, and reconcile the disagreement. That takes time and cognitive effort. If the AI’s advice isn’t good enough to justify that cost – or if the expert doesn’t trust it enough to accept it – you’ve just made things worse, not better.
The crux of the issue is that humans are imperfect advice-takers. We reject good advice and accept bad advice based on confidence signals, cognitive biases, and whether the advice aligns with our existing mental models. An overconfident expert might ignore superior AI recommendations. An under-confident expert might defer to mediocre AI suggestions.

The researchers found that AI advisors with superhuman accuracy can still diminish expert performance if they don’t account for:

• Novelty of the advice – Is the AI providing ideas and insights the user would not have considered on their own?
• The cost of reconciling contradictory advice – Is the improvement worth the interruption?
• How convincing the advice is – Confidence scores, explanations, and presentation matter as much as accuracy.
• The expert’s individual decision-making patterns – One approach to presenting recommendations does not fit all.

The first point – novelty – explains why top-performing customer service agents saw no quality improvement or even quality declines. The AI agent in that study was trained on aggregate “best practices” that the most experienced reps already knew (as the AI agent was trained / grounded on the experts’ own call transcripts) and could only add value for less experienced call center staff who didn’t know them.

The second point – reconciling contradictory advice – explains why experienced developers slowed down in familiar codebases. The cost of evaluating AI suggestions exceeded their value in domains where the developer already knew what to do.

Meanwhile, the researchers in the Stanford / Mount Sinai study readily acknowledged that the design of the agent did have a direct correlation to outcomes, regardless of the individual “AI readiness” of doctors.

Based on all these studies (and our own practical experience), human experts and AI work best in situations where the AI has been optimized for expert interaction and experts are given guidance on how to evaluate and leverage AI advice.

This starts with only consulting AI in situations that warrant it. The Oxford study found that AI assistance was most useful to experienced support reps when solving “moderately rare” problems that human experts might not encounter often but where the AI agent had sufficient training.

Then, there’s the level of openness to AI advice. On one end of the spectrum are users who are too easily persuaded by AI recommendations – they second-guess their own judgment and defer to the machine. On the other end are those who are too resistant – they ignore good advice because they don’t trust the source. The sweet spot? An expert who uses AI judiciously and can “take it for what it’s worth.” Someone with enough confidence to evaluate AI suggestions critically, accept what’s useful, and discard what isn’t.

On the AI end, the Texas / Yale study concluded that “Producing AI advisors that can be relied on in practice to add the most value and avoid losses in high-stakes contexts requires that they offer effective responses by accounting for the pathways through which AI advice can add or diminish value.”

That means a good AI advisor should:

• Advise selectively (only when it can actually add value)
• Provide inherently convincing recommendations (with appropriate confidence signals)
• Complement the individual expert’s strengths and weaknesses
• Account for the user’s tolerance for interruption and cognitive overhead

In short, building an AI agent that’s smarter or more knowledgeable than humans is the easy part. Building AI that makes humans smarter? That’s the hard part.

4. Introducing New Capabilities

As a project manager, I was taught that all things are possible through some combination of time, money, and effort. However, we might want to add “intelligence” to the classic time / money / effort triangle to account for all the once-“impossible” tasks that are becoming possible with AI. This can include:

• “Economically impossible” tasks: Things that were always technically feasible but never worth the labor costs.
• “Technically impossible” tasks: Things that were beyond the ability of humans using previous-generation technologies.

Realizing the “Economically Impossible”

A decade ago, when our team would sit down with an enterprise / institutional client to develop a workforce training program, the client would usually talk about their grand ambitions to create variations of the training material for different audiences, translate the materials into a dozen different languages, and regularly update the course contents to ensure they stayed relevant and fresh.

But then budget and bandwidth realities would kick in. The goal of creating “differentiated content for each audience” might get scaled back to PDF one-sheets telling people how to adapt a generic course to their role, all the planned translations might stop with a Spanish or French version, and the “regular updates” might amount to one or two find and replace tweaks over a five year lifecycle.

However today, with AI, you can create a course that simply asks the learner “what department do you work in?” and – based on their answer – tailors the content to the user’s role on the fly. And while sometimes it takes effort to optimize an AI agent for a particular language, if you ask most of the AI coaches we develop to hold a conversation in Croatian or Vietnamese, they’ll be perfectly intelligible. Meanwhile, having AI coaches and tutors connected to a central knowledge base means that, if you edit the article about anti money laundering regulations to reflect newly passed legislation, all of the bank’s AI learning tools will reference those numbers going forward.

Basically, what would have required a whole separate scope of work and weeks of effort for writers, voiceover artists, animators, and e-learning developers that can now be accomplished with a few quick keystrokes. Which is amazing but also a little bit disturbing since this type of work traditionally represented a good portion of our company’s business.

Achieving the “Technically Impossible”

In 2017 an auto parts distributor reached out to us and said “We have a knowledge management problem – one of the Big Five consulting firms already tried to solve it, but gave up.”

That sounded like a dare, so of course we took it.

The manager at the distributor explained that they had nearly a hundred suppliers, mostly small to midsize businesses who each specialized in a very specific type of automobile component, and about a dozen big auto companies as customers. This created a situation where their staff had to spend an inordinate amount of time transposing data from their suppliers’ catalogs to match the format and categorization schemas for their customers’ procurement systems. “Between the suppliers and customers, we got about 50 different ways to describe the same damn bolt!”

Our team got a little farther than the other consulting firm, creating a database structure and taxonomy that aligned the myriad inputs and outputs. However, in order to populate the database, we would have to develop complicated formulas to transform each field from each supplier catalog, then similarly elaborate formulas to reformat the fields for each customer’s system. Eventually the client decided it wasn’t worth the effort and long-term maintenance obligation to put all that in place.

But today, a lightweight Large Language Model (or a custom-trained Small Language Model) could easily handle those transformations with just a couple of exemplars and plain-language instructions for each vendor and customer. In fact, our team recently accomplished a very similar task by using AI to compile a comprehensive set of Japanese food safety regulations, drawing from the disparate policy documents of the industry associations that matinatin them (e.g. one set for grocery stores, another for restaurants, etc.) What was impossible in 2017 has since become a week and a half’s work for a mid-level technical writer with AI.

Conclusion

Effective AI implementation isn’t a contest between humans and machines – it’s a partnership that will add new dimensions to the talent equation. Rigid, hierarchical structures will be replaced by a task-based approach, where the question isn’t “Who reports to whom?” but rather “What is the right mix of human and AI capacity to solve this problem?” Organizations that proactively reshape themselves to leverage AI this way will gain the advantage over those who wait to be reshaped by it.

Hopefully this article offered some useful perspective on how AI can be implemented across an organization. If you’re thinking about how AI could help your team – whether by training your workforce or automating operations – please reach out to Sonata Intelligence for a consultation.

Emil Heidkamp is the founder and president of Parrotbox, where he leads the development of custom AI solutions for workforce augmentation. He can be reached at emil.heidkamp@parrotbox.ai.

Weston P. Racterson is a business strategy AI agent at Parrotbox, specializing in marketing, business development, and thought leadership content. Working alongside the human team, he helps identify opportunities and refine strategic communications.