Measuring AI impact
“Organisations should measure real-world AI impact beyond adoption and usage”
8 people · 8 agree · 1 no clear side · 38 views · 2026-11-09 to 2026-11-19
Who stands where on the statement above, from their own words. A person's side is shown only where two separate readings of the quote agree; otherwise they are under "Side unclear". This page does not say who is right.
Agree 8 people
- ✓Alizee Fraudin
Most organisations investing heavily in AI cannot answer that question with confidence.
Without evidence of real-world impact, organisations risk mistaking adoption for success.
Framing the question correctly determines what needs to be measured and what evidence organisations need to make better decisions.
captures perceptions rather than necessarily demonstrating outcomes.
they cannot establish what has actually changed in people's work, communities or daily lives.
Answering these questions requires combining multiple sources of evidence: quantitative data that shows what has changed, alongside direct conversations and observations that explain how these changes are experienced in practice.
Map AI use cases and prioritise those with the greatest potential consequences for business performance, workers or communities
Combine operational indicators with independent quantitative and qualitative research to understand what has changed, for whom and why.
understanding its effects will become increasingly important.
suggesting that usage alone does not explain the outcomes people experience.
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ✓Olga V. Mack
Choose the uncertainty that matters to the next business decision. Design the pilot around resolving it.
An evaluation needs contact with the company’s actual mess.
An overall accuracy number can hide that distinction.
Those minutes belong in the calculation.
A pilot should have an end date, a decision owner, and an agreed set of possible next steps.
Agreeing on the decision rules early makes it easier to act on what the experiment actually reveals.
Follow the benefit far enough to understand it.
The pilot gives legal and procurement a chance to discover these dependencies before they negotiate around an optimistic spreadsheet.
Before expanding the tool, assign an operating owner, approve its permitted uses, establish support, and document how errors will be handled.
Preserve the evaluation findings, including the situations in which the system performed poorly. That record should survive the original project team.
- ✓Stephen Embry
I wonder if the reason we aren’t seeing the true financial payoff of AI in legal is not because AI doesn’t save time but because we aren’t good at counting all the time it takes to turn AI’s output into something useful.
Which means the hidden review work may shrink both the promised savings and the revenue losses firms fear.
the appropriate comparison isn’t drafting time versus AI generation time. It’s the total cost of producing an acceptable final product with and without AI.
The difference between promised savings and actual savings may create a credibility problem for law firms.
Across corporate America, that gap could be swallowing a lot of the promised productivity.
Firm management and partners may conclude they are taking a financial beating on expensive AI tools. That can lead to strife within the firm.
- Quote not reprinted: this page already repeats as much of this article as we allow.
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ✓Fortune
If you look closer, you’ll probably find that most of that usage is small tasks that don’t make anyone valuable to the company.
The better test is to inspect what you expect.
The AI skills worth rewarding are the ones that will keep growing in value: the strategic thinking, or the great line of questioning.
AI readiness cannot be measured by technology spend, license numbers or chatbot adoption alone.
- ✓Harshit Garg
- Quote not reprinted: this page already repeats as much of this article as we allow.
- ✓CRPE Admin
Help educators gather practical evidence on feasibility, time savings, learning outcomes, educator and student experiences, and unintended consequences.
- ✓Kelsey Piper
the benchmarks increasingly don't reflect the value of the model to users
- ✓Rob Robinson
Ask a vendor which kind of output its agent produces, then ask for the downstream measure rather than the drafting one
No clear side 1 person
- ~Fortune
It depends on the company. If you’re an organization that typically ties performance to specific skills that will grow in value, it makes sense.
Week by week
- · 3: Kelsey Piper ✓, Rob Robinson ✓, Fortune ✓
- · 6: Olga V. Mack ✓, Stephen Embry ✓, Harshit Garg ✓, Alizee Fraudin ✓, CRPE Admin ✓, Fortune ~
Built 2026-11-21 06:50 UTC from the news knowledge
graph by code. Every quote was checked against the source's own words; which side a quote takes is a model's reading,
made twice.