SKT
2023
Making AI Conversations Natural and Reliable
Initiating and designing Adot’s first grounded LLM experience to reduce hallucinations and deliver natural conversations based on verified, real-time data during the Hangzhou Asian Games
Overview
Adot could sound natural or stay factual. I set out to make it do both.
Adot is SKT's AI Assistant app, used by 3M+ users each month. A fully LLM-powered experience could support natural conversation, but frequent hallucinations made it too risky for topics that required factual, real-time information. To ensure accuracy, Adot relied on scenario-based conversations written and maintained by product designers. The information was reliable, but the responses were rigid and repetitive.
When I took on the Hangzhou Asian Games, I saw an opportunity to overcome this tradeoff. I approached a team researching grounding technology and initiated Adot’s first project combining an LLM with verified, real-time data. This allowed Adot to generate responses naturally while keeping the underlying information factual. I designed the grounding framework, user-facing experience, and operational tool in close collaboration with engineering, design, and operations teams.
Opportunity
Every question could be phrased differently. Adot’s answer was always the same.
For major events such as elections and the Olympics, product designers anticipated user intents, mapped out conversation branches, and wrote and maintained every response. Dynamic fields such as dates, scores, and rankings allowed the latest data to be inserted, but the structure and language of each response remained fixed.
This approach delivered reliable information, but Adot could only respond within flows that had already been designed.
When I was assigned the Asian Games as my first major project, I saw an opportunity to create an experience that could understand the many ways users naturally ask questions and respond with language shaped by their intent and context.
Initiative
I took the idea to a team researching grounding, and together we turned it into a live product.
At the time, grounding* was still being explored internally and had not yet been applied to an Adot service. I reached out to the team, shared the limitations of the existing scripted approach, and proposed using the Asian Games as an opportunity to test a new conversational experience.
We worked together to evaluate the idea from both technical and user experience perspectives. I defined the types of questions users might ask and the information each answer would require, while the engineers explored how an LLM could retrieve and use verified data. This collaboration became Adot’s first live service to combine an LLM with grounding technology.
* Grounding: A technique that connects an LLM to verified, up-to-date data so it can generate responses based on facts rather than relying solely on what the model already knows.
Solution
I designed the full system behind each answer, from grounding logic to real-time operations.
A natural conversation required more than connecting an LLM to a database. Adot needed to understand what users were asking, retrieve the right information, present it in a useful format, and keep it accurate as events unfolded in real time.
I designed the solution as three connected parts: 1) a grounding framework that linked user intents to verified data, 2) conversational and visual interfaces that made answers easy to understand, and 3) an operational tool that kept live match data up to date.
More details coming soon...
like really soon. promise.
