The Jagged Frontier: Where AI Helps and Where It Quietly Fails
A trial with 758 consultants found AI raised quality 40% on most tasks and cut accuracy by 19 points on one. The edge between them is invisible.
The most useful study of artificial intelligence at work is not about how much faster it makes people. It is about the shape of what it can do.
In 2023, researchers from Harvard Business School, Wharton, MIT and Warwick ran a preregistered experiment with 758 Boston Consulting Group consultants, about 7% of the firm’s individual contributors. They gave one group no AI, one GPT-4, and one GPT-4 plus prompt-engineering training.
Inside the frontier
On 18 realistic creative and analytical consulting tasks, the consultants with AI completed 12.2% more tasks, 25.1% faster, and produced work rated more than 40% higher in quality.
The gain was not evenly distributed. Consultants who had scored below average improved by 43%; those above average improved by 17%. This levelling effect appears in almost every study of these tools, including the developer productivity research.
Outside it
The researchers also included one task built to defeat the tools: a business case requiring the consultant to reconcile quantitative data with qualitative interview material, where the obvious data-driven answer was wrong.
The group without AI got it right 84.5% of the time. The two AI groups scored 60% and 70%, an average drop of 19 percentage points. The consultants did not detect that they had crossed a boundary. They accepted confident, fluent, incorrect reasoning.
Why this generalises
The same failure appears in a Stanford study of security-sensitive code, where participants with an AI assistant wrote measurably less secure code and were simultaneously more likely to believe it was secure. The model in that study is now old, so it is not current evidence about today’s tools, but the mechanism is well established: fluency reads as correctness.
That is the practical lesson. The risk is not that the tools are bad at things. It is that their failures do not look like failures.
What to do about it
- Treat any task that requires reconciling two different kinds of evidence as high-risk
- Be most careful precisely when the output reads as most confident
- On anything where being wrong is expensive, do the work first and compare afterwards
For people early in a career this matters more than it does for their seniors, because catching a confident error requires knowing the material. That is the argument in what junior means now, and the practical version of it sits in the software engineer guide.
Frequently asked
What is the jagged frontier of AI?
It describes the finding that AI capability is uneven in ways that are not predictable from the outside: two tasks that look equally difficult to a person can sit on opposite sides of what the model handles well. The term comes from a 2023 Harvard Business School study run with 758 BCG consultants.
Does AI improve work quality?
On most realistic knowledge tasks in the BCG experiment it improved quality by over 40%. On a task engineered so the obvious answer was wrong, it reduced accuracy by 19 percentage points. Both results come from the same study, which is the point.
Why do people trust wrong AI output?
Because fluency reads as correctness. In the BCG study consultants did not notice they had crossed into work the model handled badly, and in a separate Stanford study participants who wrote less secure code with an assistant were more confident it was secure.