When researchers at Stanford and MIT embedded a generative AI assistant into the workflow of over 5,000 customer service agents at a Fortune 500 company, they expected to see productivity gains. What they didn't expect was who would benefit the most — and how dramatically the technology would reshape the distribution of skill in the workplace.
The Study
The research team — Erik Brynjolfsson, Danielle Li, and Lindsey Raymond — conducted one of the first large-scale, real-world experiments on generative AI in the workplace. Over several months, they tracked customer service agents who were given access to an AI tool that provided real-time suggestions for handling customer conversations. A control group continued working without AI assistance.
The setup was unusually rigorous for a field study: because the company rolled out the AI tool in phases, the researchers had a natural experiment with clear before-and-after comparisons.
Key Findings
- 14% average productivity increase — agents using AI resolved more issues per hour, with faster response times and higher customer satisfaction scores.
- Novice workers benefited the most — the least experienced agents saw productivity gains of up to 34%, while top performers saw minimal improvement (around 0–5%).
- The AI effectively "captured" expert knowledge — the system had been trained on conversations from the best-performing agents, and it transferred those patterns to less experienced workers.
- Customer sentiment improved — not only were issues resolved faster, but customers reported better experiences, and agents faced less hostility in conversations.
- Employee retention improved — agents with AI assistance were less likely to quit, suggesting lower burnout and job dissatisfaction.
The Bitter & The Sweet
The sweet side of this research is genuinely encouraging. If AI can compress the learning curve for new workers — effectively giving everyone access to the accumulated wisdom of top performers — it could reduce inequality within workplaces and make demanding jobs more manageable. The finding that AI-assisted agents faced less customer hostility is particularly noteworthy: technology that reduces emotional labor is a meaningful improvement in working conditions.
But the bitter implications are equally real. If AI can make a novice perform like a mid-level worker within weeks, what happens to the value of experience? The study implicitly suggests that years of hard-won expertise become less economically valuable when AI can replicate the patterns. This raises uncomfortable questions about wage compression and the incentive to invest in deep professional development.
There's also the question of dependency. If workers learn to rely on AI suggestions rather than developing their own judgment, what happens when the system fails or produces bad advice? The study doesn't address long-term cognitive effects, but research on cognitive offloading (Risko & Gilbert, 2016) suggests that outsourcing thinking to technology can gradually erode the underlying skills.
"The AI system had its largest effects on less-experienced and lower-skilled workers, suggesting that it was able to capture and disseminate the tacit knowledge of more experienced workers." — Brynjolfsson et al., 2023
Methodology & Limitations
The study's strength lies in its scale and real-world setting. However, several caveats apply: it examined a single company in a single industry (customer service), the AI tool was specifically designed for this context, and the study period was relatively short. Whether these results generalise to creative, analytical, or manual work remains an open question. The NBER working paper has been widely cited but is still undergoing peer review refinements.
What This Means Going Forward
This study has become one of the most-cited pieces of evidence in the debate about AI and employment. It suggests that the impact of generative AI won't be uniform — it will disproportionately affect how we think about training, expertise, and the value of human judgment. For policymakers, the implication is clear: AI adoption strategies need to consider not just aggregate productivity gains, but how those gains are distributed across different types of workers.
For individuals, the takeaway is nuanced. AI can be a powerful learning accelerator, but treating it as a permanent crutch rather than a scaffold risks undermining the very skills it's meant to enhance.
References
- Brynjolfsson, E., Li, D., & Raymond, L. R. (2023). Generative AI at Work. NBER Working Paper No. 31161. doi:10.3386/w31161
- Risko, E. F., & Gilbert, S. J. (2016). Cognitive Offloading. Trends in Cognitive Sciences, 20(9), 676–688.
- Dell'Acqua, F., et al. (2023). Navigating the Jagged Technological Frontier. Harvard Business School Working Paper.