The weekly briefing on the future of work
Work Futures Report

Analytical, data-driven intelligence on the future of work — for HR leaders, L&D managers and workforce strategists.

Briefing · November 7, 2026

What James Shore Just Got Right About Measuring AI's Real Impact on Teams

James Shore's rigorous methodology for measuring AI delivery speed is the accountability framework HR and engineering leaders have been missing.

James Shore is a software development consultant, author, and longtime contributor to the agile community whose career has been built on one unfashionable conviction: that you should measure things honestly, especially when everyone around you is celebrating. His recent empirical investigation into whether artificial intelligence (AI) tools actually accelerate software delivery — published on O'Reilly Radar — is exactly the kind of grounded, evidence-first work that senior leaders should be reading before their next vendor renewal conversation.

Does AI actually make software teams faster?

The core finding from Shore's investigation is worth stating plainly: most practitioners believe AI is making them faster, but quantifying by how much — and attributing that speed to AI specifically — turns out to be far harder than the sales deck suggests. As Shore writes in Measuring AI's Impact on Delivery Speed (O'Reilly Radar), the question is not simply "are developers completing tasks more quickly?" but "what value are you actually getting for your money, and are there other configurations of AI use that perform better?" That distinction — between perceived productivity and measured value — is precisely the one most organizations are skipping.

This matters to HR and operations leaders because the current AI investment cycle is being justified largely on productivity claims that few organizations are actually testing. When nearly two-thirds of HR leaders expect layoffs in 2027, according to HR Dive's 2025 survey data, those reduction decisions are increasingly tied to assumptions about AI-driven efficiency gains. If the efficiency gains are real but unmeasured, the workforce math is guesswork.

Why is rigorous AI measurement so hard — and why does Shore's approach matter?

The underlying mechanism Shore is interrogating is what researchers call confounding in productivity measurement: delivery speed changes over time for many reasons simultaneously — team composition, codebase maturity, project complexity, and tooling all shift together. Isolating the contribution of any single tool requires controlled conditions that most engineering teams never create. AI-assisted development (the practice of using large language model (LLM)-powered tools to generate, review, or refactor code alongside human engineers) compounds this problem because developers adopt these tools unevenly, use them for different task types, and vary their reliance on them week to week. Without a measurement framework that accounts for this variance, any "AI made us X% faster" claim is an anecdote wearing a spreadsheet's clothing.

Shore's value is that he builds and shares that framework. He asks the questions most organizations are afraid to ask aloud: How much of the speed gain is task-type-dependent? What are you giving up when you optimize for velocity? His work at O'Reilly Radar gives leaders a template for demanding honest answers from their own teams and their vendors.

The timing is sharp. A parallel dynamic is already playing out in workforce decisions: employers who replaced workers with AI are now facing a "rebounding" problem — when they attempt to rehire, returning workers are demanding higher compensation, as reported by HR Dive. That is what happens when AI adoption decisions are made on faith rather than measurement: the costs get deferred, then amplified.

What HR leaders should do with Shore's framework this week

The actionable takeaway from Shore's work is not a tool recommendation — it is an audit prompt. Before your next AI productivity review, ask your engineering or operations leads three questions: What baseline did we establish before deploying this tool? What task types are we measuring, and which are excluded? And what would it cost us to run a 60-day controlled comparison?

If those questions generate discomfort, that discomfort is the signal. Organizations that cannot answer them are making headcount and investment decisions based on vendor-supplied benchmarks, which is a governance failure dressed up as a technology strategy. Shore is doing the work of insisting on intellectual honesty at a moment when the incentives all run in the opposite direction.

The board conversation most chief human resources officers (CHROs) are not yet having goes something like this: "We have committed significant capital to AI tooling based on productivity claims we have not independently verified, and we are now making layoff decisions downstream of those same claims." Shore's framework gives you the vocabulary to have it — and the methodological grounding to demand better data before the next round of workforce restructuring is approved.

Measurement is not a technical problem. It is a leadership choice.

Created with AI assistance. Editorial oversight: Juergen Ritzek. See our AI disclosure.

The weekly briefing for people who run the workforce

One big idea, the data behind it, and the “so what” for HR leaders — every week, free.

Double opt-in, no spam, unsubscribe anytime. See our privacy policy.