ROLE
UX Designer
DURATION
3 Days
TOOLS
NotebookLM, Figma
Team
Charles Meng
Tristyn Lai
David Zhang
Mayaa George
Awards
1st Place 2025 Winter Designathon hosted by UCSC & AImpower.org
Context
Current speech technologies, such as automated phone menus or computer-generated captions, often fail to work well for people who stutter due to training on fluent speech.
To make these speech technologies more responsive and accessible for people who stutter, it is important to collect more diverse stuttered speech data to improve these technologies.
Design a data collection process that empowers people who stutter to contribute speech samples on their terms for building more inclusive AI speech technologies.
Timeline
3 Days
research
Literature Review
Interviews
Ideation
Affinity Mapping
Proposal
DeSign
Wireframes
Prototypes
Research
methods
Guiding questions to inform our approach
01
What are the limitations of current speech impediment technology, according to users?
02
What ethical factors affect speech dataset construction?
03
What privacy aspects are important to PWS (People Who Stutter)?
Literature Review
Purpose
Existing research developed our foundational understanding of the current landscape in Automatic Speech Recognition (ASR) models and its effectiveness in analyzing stuttered speech.
Many articles also served as a guideline for the ethical and technical considerations of speech data.
Articles
Ashita Batra, Mannas narang, Neeraj Kumar Sharma, Pradip K Das
Bob MacDonald.,Pan-Pan Jiang, Julie Cattiau, Rus Heywood, Richard Cave, Katie Seaver, Marilyn Ladewig, Jimmy Tobin, Michael Brenner, Philip Q Nelson, Jordan R. Green, Katrin Tomanek
Lauren Wilcox, Robin Brewer, Fernando Diaz
Ethical and Technical Dimensions
Here are some important concepts we learned about speech technologies, which then fed into our design implementations.
Ethical
Technical
Informed Consent
Data Collection
Data Privacy
Variability of Speech Patterns
Fairness, bias, and diversity
Model Training and Personalization
Interviews
None of us have a speech impairment, so we wanted to hear from those who do. Given the scope, we opted for convenience-sampling - reaching out to individuals we knew who had some form of speech impairment.
Alexa's Background
Matt's Background
Has difficulty pronouncing the letter “R”.
Has prolongation, block, and word repetition speech impediment.
Attended speech therapy as a kid
Attended speech therapy as a kid
A couple insights
“Majority of the cases, 6/10 the recordings would stop before done speaking. I used to interact with [Apple] Siri a few times. Still not the best experience.”
“If I knew the data was only being processed to train a model to recognize different ways of speaking... As long as the data wont be used against me, I’m reassured”
“I think it would really help if there was record playback so I can hear how I sound and see if the data... will be helpful to the contributions”
Brainstorm Session
Purpose
We began the ideation stage by writing down our ideas on sticky notes, then sharing them out loud. After discussing each one and identifying common themes, we agreed on a direction:
Idea
Gamify a speech data-collection process.
Why?
Our goal is to create a welcoming, engaging space for PWS to contribute.
A gamified experience can promote these qualities, and open opportunities to foster its community.
Fun
Interactive
Anonymous
A gamified experience where users feed a cute creature recorded pieces of sentences.
Emphasis On
01
How we use their voice
02
Explicit consent to using their data
03
Community building
04
Option to retract their data anytime
Gamification Element (XP)
Given our project scope, we prioritized a single, thoughtfully designed gamification element to drive engagement and user participation. This decision allowed us to validate core interactions without overextending the experience.
XP is granted upon submission of recorded audio
XP can be spent on cosmetics for the user's personal creature.
With additional development time, the system could seamlessly scale to incorporate a richer set of in-game mechanics, further enhancing motivation, accessibility, and sustained user contribution.
Home Page
As the first touchpoint in the user experience, we designed the landing page to clearly communicate the project’s mission and value proposition.
Given the inherent challenges of encouraging participation within the PWS community, we also prioritized elements that fostered a sense of belonging and collective impact.
Starting the recording process
We make sure users know they have full control before they contribute. Account creation was avoided as it could’ve been a barrier.
Recording
The main flow has users read the displayed sentence, with other options including accessibility features, help, or generating a different sentence.
For each recording contributed, XP is earned for the user.
How does it sound?
Replaying and re-recording lets users take their time and make adjustments without feeling like they contributed poorly to training models.
An optional decision
Having a direct way to access and manage their data reinforces that their consent and comfort are a priority.
Below is where the user can customize their Shape character using their XP. Only if they choose to create an account.
