AI for Productivity: 5 Myths the Research Busts

Your feed has a strong opinion about AI for productivity. One camp swears a chatbot turned them into a 10x human overnight. The other insists it's rotting everyone's brain. Meanwhile, you're just trying to clear your inbox before lunch.

Here's the fun part: researchers have actually run randomized experiments on this, with coders, writers, customer support agents, and consultants. The results are way more interesting than either camp admits. AI productivity tools can deliver huge gains, do nothing, or even slow you down, depending on the task and on who's using them.

Let's bust five myths, one study at a time.

Myth 1: AI for productivity is a universal speed boost

Some of the early numbers are jaw-dropping. In a 2023 experiment, Sida Peng of Microsoft Research and colleagues from GitHub and MIT asked 95 freelance programmers to build a simple web server in JavaScript [1]. The group with GitHub Copilot, an AI pair programmer, finished 55.8% faster than the group without it [1]. One disclosure: GitHub makes Copilot, and the study was approved by Microsoft Research's ethics board [1].

Writing looks similar. MIT economists Shakked Noy and Whitney Zhang gave 453 college-educated professionals incentivized writing tasks tailored to their occupations and randomly gave half of them ChatGPT. Average time dropped 40%, and graded quality rose 18% [2].

Consulting, too. Fabrizio Dell'Acqua of Harvard Business School led a 2023 experiment with 758 consultants from Boston Consulting Group (BCG), co-authored with Wharton's Ethan Mollick and others [3]. On realistic consulting tasks that GPT-4 handled well, the AI group finished 25.1% faster [3]. Worth knowing: the study was run in collaboration with BCG, three co-authors worked at BCG, and co-author Karim Lakhani disclosed that he advises BCG on AI strategy [3].

Now the plot twist. In 2025, Joel Becker, Nate Rush, Beth Barnes, and David Rein at the AI research group METR ran a randomized trial with 16 experienced open-source developers working on 246 real tasks in big, mature projects they knew inside out [4]. When AI tools were allowed, tasks took 19% longer [4].

Translation: AI tends to shine on well-defined, self-contained tasks. The METR authors pointed out that the Copilot task was simple and likely resembled a lot of AI training data [4]. Their own developers had years of deep, unwritten knowledge about their codebases that the AI didn't [4]. The authors also warn against overgeneralizing: their results don't mean AI can't speed up lots of other people and tasks [4].

Myth 2: If AI feels faster, it is faster

This might be the most important finding in the whole pile. Before starting, the METR developers predicted AI would cut their task time by 24%. After the study, they still estimated it had saved them about 20%. In reality, it added 19% [4]. Outside experts were even more optimistic, predicting time savings of 39% (economists) and 38% (machine learning experts) [4].

Where did the time go? Screen recordings showed developers spending less time actively coding and more time prompting, waiting on the AI, reviewing its output, and sitting idle [4]. They accepted fewer than 44% of the AI's code suggestions, and cleaning up AI output ate about 9% of their time on AI-allowed tasks [4].

Perception can miss in the other direction, too. In the Copilot study, participants estimated a 35% productivity gain on average, which undershot the 55.8% the researchers actually measured [1]. Either way, the lesson is the same: your gut is not a stopwatch.

Myth 3: AI helps the experts the most

You'd think the best performers would squeeze the most out of a powerful tool. Across several studies, it's often the opposite.

The big field study here is literally titled "Generative AI at Work." Erik Brynjolfsson, director of the Stanford Digital Economy Lab, with Danielle Li and Lindsey Raymond, tracked 5,172 customer support agents as a company rolled out an AI chat assistant [5]. Issues resolved per hour rose 15% on average [5]. Less experienced and lower-skilled agents improved both speed and quality. The most experienced and highest-skilled agents saw small speed gains and small declines in quality [5]. An earlier working-paper version put the boost for novice and low-skilled workers at 34% [6].

The same pattern showed up in writing, where ChatGPT narrowed the gap between stronger and weaker writers [2]. It showed up again among the BCG consultants: those below the average performance threshold improved by 43% with GPT-4, versus 17% for those above it [3].

Translation: AI can work like an instant mentor for people early in a skill. If you're already great at something, the payoff may be smaller, and you'll want to review the output more carefully.

Myth 4: AI is great at everything it sounds sure about

The Dell'Acqua team coined a perfect phrase for this: the "jagged technological frontier" [3]. AI is excellent at some tasks and surprisingly bad at others that look just as hard.

On 18 realistic consulting tasks inside the frontier, BCG consultants using AI completed 12.2% more tasks, finished 25.1% faster, and produced work rated more than 40% higher in quality [3]. Then the researchers gave them a business problem deliberately chosen to fall outside what AI could handle. Without AI, consultants got it right about 84.5% of the time. With AI, accuracy fell to 60% to 70%, or 19 percentage points lower on average [3].

The scary part? AI's wrong answers didn't come with a warning label. A confident tone looks the same whether the AI is right or wrong.

Myth 5: Offloading your thinking to AI is free

Now for AI and critical thinking. A team led by Hao-Ping (Hank) Lee of Carnegie Mellon University, with six co-authors from Microsoft Research, surveyed 319 knowledge workers about 936 real examples of how they used generative AI at work [7]. People with higher confidence in the AI tended to report less critical thinking, while people with higher confidence in themselves reported more [7]. The authors also found critical thinking shifting toward checking facts, stitching AI output into the final work, and overseeing the task [7]. Since Microsoft sells AI assistants, it's notable that its own researchers flagged this risk.

Michael Gerlich of SBS Swiss Business School went broader, surveying and interviewing 666 people in the UK [8]. More frequent AI tool use was linked to lower critical thinking scores, partly explained by more "cognitive offloading," meaning handing mental work to a tool [8]. Younger participants leaned on AI more and scored lower than older ones [8]. Big caveat: this was a correlational study built on self-reports, so it can't show that AI caused the drop [8].

One more data point: in the consulting experiment, ideas from AI users were higher quality but noticeably less varied than those of people working without AI [3]. Great for speed, maybe less great when you need an original idea.

How to use AI without the traps

1. Map your frontier. Use AI first on clear, self-contained tasks like drafts, summaries, and boilerplate, where the studies found the biggest gains [1][2][3].
2. Time it, don't vibe it. Run a quick test: do a few similar tasks with AI and a few without, and clock both. Your gut was wrong in both directions in the studies above [1][4].
3. Be your own editor. The more expert you are, the more you should check the output. AI gave top performers the smallest boost and sometimes nudged their quality down [3][5].
4. Think first, then prompt. Sketch your own answer or outline before you ask. It keeps your judgment in the loop, which is what confidence in the AI seemed to crowd out [7].
5. Don't scroll while it generates. METR saw more idle time with AI allowed [4]. Fill the gap with the next step of your task, not your feed. Our dopamine detox guide has tricks for breaking the reflexive scroll.
6. Keep the "why" human. Let AI planning help you break a goal into steps, but you decide which goals matter. Our guide to setting goals that turn into daily action shows how to structure them.

How Goald AI fits in

The research points to the same sweet spot: let AI handle the structure and the busywork, while you keep the judgment and the effort that builds real skill.

That's the idea behind Goald AI. You set the goal and decide what matters. Goald AI connects it to the habits that actually move it forward, tracks your progress (workouts included), and uses smart scheduling to find real time for the work in your week. Your AI Mentor remembers your goals and history and helps you adjust your plan when life happens. It's built to keep you doing the thinking, not to do it for you.

Goald AI is coming soon to iOS. The free tier covers up to 3 goals, and Goald AI Pro is $9.99/month or $99.99/year. Join the waitlist.

Sources

1. Peng S, Kalliamvakou E, Cihon P, Demirer M. "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot." arXiv, 2023.
2. Noy S, Zhang W. "Experimental evidence on the productivity effects of generative artificial intelligence." Science, 2023.
3. Dell'Acqua F, McFowland E III, Mollick E, et al. "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality." Harvard Business School Working Paper 24-013, 2023.
4. Becker J, Rush N, Barnes B, Rein D. "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." arXiv (METR), 2025.
5. Brynjolfsson E, Li D, Raymond L. "Generative AI at Work." The Quarterly Journal of Economics, 2025.
6. Brynjolfsson E, Li D, Raymond LR. "Generative AI at Work." NBER Working Paper 31161, 2023.
7. Lee H-P, Sarkar A, Tankelevitch L, et al. "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." Proceedings of CHI 2025, 2025.
8. Gerlich M. "AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking." Societies, 2025.