#blogs

Cross-Reference Validation Matrix - $644B Article Claims

Executive Summary

Independent validation of major claims in the AI Economic Vandalism article, with discrepancy analysis, confidence scores, and robustness testing.

Claim 1: AI Project Failure Rate (70-95%)

Supporting Sources

Contradicting Data

Discrepancy Analysis

The 95% failure rate has been criticized for:

1. Narrow success definition: Requires deployment beyond pilot + measurable KPIs + 6-month ROI

2. Denominator issues: 40% of companies didn't even attempt AI pilots

3. Recalculated rate: 5% of attempts succeed = 8.3% of those who tried

Confidence Assessment

• Claim: "Most AI projects fail"

• Confidence: HIGH

• Range: 42-95% depending on definition

• Conservative estimate: 50-70% failure rate is defensible

Claim 2: $644B GenAI Spending in 2025

Source Validation

Contradicting/Complementary Data

Discrepancy Analysis

• No significant contradiction - Gartner is primary source for $644B

• IDC figures use different methodology (AI solutions vs GenAI total)

• $644B is specifically GenAI; $1.5T is all AI spending

Confidence Assessment

• Claim: "$644 billion GenAI spending in 2025"

• Confidence: HIGH

• Note: Single source (Gartner) but widely accepted

Claim 3: Only 17% Attribute 5% EBIT to AI (McKinsey)

Source Validation

Contradicting Data

Discrepancy Analysis

Major discrepancy identified:

• McKinsey: 17% see 5%+ EBIT → 83% little/no impact

• Industry surveys: 74-75% positive ROI

Resolution:

• "Positive ROI" ≠ "significant bottom-line impact"

• Small ROI can be "positive" but insignificant

• Consultant surveys may have selection bias (their clients)

• McKinsey definition more rigorous (5% EBIT threshold)

Confidence Assessment

• Claim: "Most companies see no significant bottom-line impact"

• Confidence: MEDIUM-HIGH

• Weakness: Self-reported survey data, definition variance

Claim 4: Enterprise LLM Market Share Shift

Source Validation

Contradicting Data

Discrepancy Analysis

• No contradiction - Consumer vs Enterprise are different markets

• OpenAI leads consumer; Anthropic leads enterprise

• Both can be true simultaneously

Confidence Assessment

• Claim: "Anthropic now leads enterprise AI"

• Confidence: HIGH

• Source quality: Menlo Ventures survey, well-regarded

Claim 5: Consultant Capture Rate (High Fees vs Low Outcomes)

Supporting Data

Contradicting Data

Discrepancy Analysis

• High consulting revenue + high project failure = systemic capture

• Consultants profit whether client succeeds or fails

• Market growth ≠ client outcomes

Confidence Assessment

• Claim: "Consultants profit from failure"

• Confidence: MEDIUM

• Weakness: Capture rate is calculated, not directly measured

Validation Matrix Summary

Robustness Testing

If failure rate is 50% instead of 95%:

• Core thesis holds: Half of $644B = $322B still potentially wasted

• Article argument remains valid

If GenAI spending is 30% lower ($450B):

• Still largest technology investment category

• Argument about scale remains relevant

If ROI improves to 30% of companies:

• 70% still seeing little impact

• Thesis about economic destruction still supported

Overall Robustness Score: HIGH

• Core claims survive 20-50% variation

• Multiple independent sources for key figures

• Worst-case scenarios still support thesis

Weakest Claims (Flag for Hedging Language)

1. 95% failure rate - Use "70-95%" range instead

2. Capture rate calculations - Present as estimates, not facts

3. 2028-2030 revenue projections - Clearly label as company forecasts

Key Sources for Cross-Reference

1. Fortune - MIT 95% Failure

2. Marketing AI Institute - MIT Critique

3. Larridin - Only 23% Can Measure ROI

4. CFO Dive - 47% Profitable

5. Google Cloud - ROI of AI 2025

Confidence Scoring Legend