This landmark study by Dell'Acqua et al. (2023) — a collaboration between Harvard Business School and BCG — tested 758 consultants on realistic business tasks with and without GPT-4:
- For tasks within AI's capabilities: consultants with AI completed 12.2% more tasks, 25.1% faster, with 40% higher quality
- For tasks outside AI's capabilities: consultants with AI performed 19 percentage points worse than those without AI
- Two usage patterns emerged: "Centaurs" (who strategically divided work between themselves and AI) and "Cyborgs" (who deeply integrated AI into every step)
The Jagged Frontier
The "jagged technological frontier" is the key insight: AI's capabilities are not uniform. Tasks that seem equally difficult to humans may fall on very different sides of AI's ability boundary. Users who blindly trust AI across all tasks fall prey to automation bias — over-relying on AI's suggestions even when they are wrong.
Source
Dell'Acqua, F. et al. (2023). Navigating the Jagged Technological Frontier. Harvard Business School Working Paper 24-013.