Key points
- Metric Sherpa and Sprinklr surveyed organizations running customer-facing AI: 92% said it met or beat expectations, but only 27% said they could prove it.
- Nearly all respondents (99%) named a goal for their AI, yet 26% set success metrics and 10% also captured a starting baseline.
- Among those who said AI sped up work and cut costs, only 32% had tracked costs before launch.
- The researchers could not say how 85% reported gains in metric categories nobody tracked before launch.
Why it matters for marketers: Without a baseline and a defined success metric set before launch, marketers cannot show finance that AI paid off, and vendor dashboards end up grading their own work.
A new study from Metric Sherpa, produced with Sprinklr, points to a gap between confidence and evidence in marketing AI. Among organizations using customer-facing AI regularly, 92% said results met or exceeded expectations. Only 27% said they could demonstrate that with proof.
The weak point comes early. Almost every respondent could state what they wanted AI to improve. But a much smaller share, 26%, had defined what success would look like in measurable terms, and only 10% had gone further and recorded where they started. Cost claims show the same pattern: 67% of those who said AI sped up work also said it lowered costs, yet just 32% of that group had tracked costs before launch.
For marketers, this matters because platform dashboards tend to report their own wins. A paid media tool can show better conversion, a service tool better containment, a content system better engagement, while the business result barely moves. The study also found that 85% reported improvement in at least one category they had not tracked before launch. The researchers could not tell whether those teams added measurement later or drew on another source.
The practical step is to settle the metric, the starting point and the success threshold before any AI goes live. MarTech's analysis adds that marketing ops should link platform activity to CRM, finance, customer or experiment data. For a service use case, that means looking at resolution, repeat contacts, escalations and handoffs rather than containment alone.
Sources
This brief is our own summary. Read the full reporting at the source.
