Metadata
- Author: Joel Becker; Nate Rush; Elizabeth Barnes; David Rein
- Full Title:: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- Category:: 🗞️Articles
- Document Tags:: Vibe Coding,
- URL:: https://readwise-assets.s3.amazonaws.com/media/wisereads/articles/measuring-the-impact-of-early-/2507.09089v1.pdf
- Read date:: 2025-07-21
Highlights
Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19% (View Highlight)
When allowed to use AI, developers spend a smaller proportion of their time actively coding and reading/searching for information. Instead, they spend time reviewing AI outputs, prompting AI systems, and waiting for AI generations. Interestingly, they also spend a somewhat higher proportion of their time idle, (View Highlight)
we strongly caution against over-indexing on the basis of any individual pieces of evidence, as we are not powered for statistically significant multiple comparisons when subsetting our data (View Highlight)
C.1.1 Over-optimism about AI usefulness (Direct productivity loss) (View Highlight)