the brief
A solution chosen before the problem was understood
A third-party AI platform had been procured and was in the final stages of being signed. The solution had already been decided: an AI chat, scoped to answer common questions and point people to relevant content.
It would go live without connecting to any of the services behind it. The plan was to launch it on one service, parking, then roll the same chat out across Council Tax and the rest of the Council’s services.
In effect, the solution had been chosen before the problem was fully understood. The PM and I saw discovery as the opportunity to make sure it fitted a real, well-evidenced problem, rather than simply validating a direction that had already been set.
So we made a deliberate decision to widen discovery beyond the proposed chat, looking at AI support across the Council's services more broadly, to understand the wider opportunity, explore where AI could create value, and inform the roadmap rather than only validating the chosen direction.
discovery and synthesis
Making sense of messy evidence
The discovery drew on around fifteen different sources, including chatbot transcripts and exit-survey responses, website and contact-centre analytics, general website feedback, a residents' engagement panel, a content-designer review of the existing chatbot, a competitor review, and stakeholder and workshop insights.
To add direct evidence on how people used the current chat experience, I set up and ran an unmoderated study in Optimal Workshop, with tasks shaped around our discovery questions, then folded the findings into the wider evidence base.
The volume and variety of evidence made it hard to see the patterns. I designed a synthesis structure built specifically around this work rather than a generic template: each finding captured consistently and colour-coded by source, so we could see where evidence converged across channels, then grouped into themes, evidence-led statements, and problem statements.
The synthesis produced ten themes, which I then distilled into the patterns most important for shaping the product direction.
Insights
What the evidence revealed
Chat was not always the right interaction model
Users did not naturally use chat as a way to find information. Most tried to self-serve first through navigation, search, and content, and only turned to chat when they could not find the right route. In testing, even when participants were explicitly asked to use chat to find information, most still searched and clicked through the site themselves.
The task had not failed; the behaviour was the finding. For simple tasks or quick information retrieval, conversation could add effort rather than remove it.
Support was needed in context
Help was most valuable at the point users hit uncertainty, failure, or a blocked journey. The opportunity was not simply to create a separate chat destination, but to make support available where the problem happened.
Some problems needed service fixes, not AI
Several issues pointed to underlying service problems: unclear navigation, content that did not match user language, poor visibility of timelines, weak progress updates, and disconnected handovers. In these cases, AI risked becoming a bandage over problems that needed fixing in the service itself.
Personalised support depended on integration
The strongest opportunities involved personalised status, next steps, recovery routes, or continuity across channels. These could not be met by a generic chat returning links; they depended on access to the right systems, data, and sources of truth.
The work shifted from "how do we launch an AI chat?" to "what would AI support need to know, access, and do to genuinely help people?"
Reframing the product opportunity
From generic chat to service aware support
The synthesis made the challenge clearer. An AI chat returning generic answers and links, as originally scoped, was unlikely to create meaningful impact on its own.
The evidence pointed towards a more useful opportunity: AI support that could connect with the services behind it, understand where someone was in a journey, and provide specific, contextual help about their own situation rather than general information they could already find.
That changed the strategic conversation. The Head of Service who had originally scoped the chat asked the PM and me to play the problem statements and hypotheses back to senior leadership, with the aim of opening up conversations about the integration, mapping, and resourcing a more effective AI service would require.
what's next
Validating trust, context, and interaction patterns
The programme is still in progress. We're preparing the senior leadership playback now, and I'm using the synthesis to shape the goals and questions for the next phase of research.
That stage will focus on validating the right problems before the team commits further to a solution: how people understand AI support, what would make it trustworthy, where contextual information is essential, and what interaction patterns are needed once AI moves beyond generic answers into specific, service-aware help.
This work is ongoing, but it has already shifted the programme from validating a chat interface to asking what kind of AI-supported service would create meaningful value.