Your virtual buddy: UU
🌟An XR app with AI chatbot for pediatric patients who are experiencing long-term hospitalization
🏆 2023 PICO DevJam Hackathon Best MR design and 2nd prize
Team: 6 members
Me: PM/ Product designer
2 UX designers
1 Visual designer
2 Developers
Role
Product management
Product design
Visual lead
User interview
Time
5 months
research ideation
3 weeks prototype
Tools
Figma
Adobe illustrator
C4d
Unity
Background
The project was inspired by the documentary film Life Matters (2019). It depicts a dream about child patients fighting against cancer in a gamified way in a Chinese public hospital. Besides the side effects of treatments, children in long-term hospitalization suffer from unimaginable torture (fear, loneliness, loss of control, and despair). I hope that through the design, I can find a solution that could help them alleviate the pain during the treatment process.
Problem
25%
In this project, I decided to focus on leukemia because it is the most common pediatric cancer, comprising 25% of all cancer types.
85%
The 5-year relative high survival rate of leukemia is 85%, which means they have a high chance of returning to normal life after hospitalization.
According to my research, child patients with Leukemia may have a lot of problems and difficulties during long-term hospitalization in Chinese public hospital:
Long treatment:
Most patients take a total treatment about 3 years to get healed
Restrained activities:
Get treatment in an unfamiliar environment with restrictions
Psychological risk:
Long-term hospitalization may cause isolation
Social demand:
Need the company of friends to share stories and get support
Solution
We designed the virtual character and user interfaces for the XR app with design system. For the 2023 PICO DevJam hackathon, we also made a prototype implemented in VR headset using unity:
Reading time
Thanks for checking! It takes about 15 minutes to read the entire case study. Please feel free to use the navigation bar below to help you navigate more efficiently!
Research
How do pediatric patients think and feel during hospitalization?
With this question, I conducted secondary research using methods such as interviews and the examination of real medical cases. I interviewed 8 medical staff members in Chinese public hospital who are strongly associated with pediatric patients and 2 leukemia pediatric patients’ parents. Out of ethical considerations, I refrained from direct conversations with the patients.
What do pediatric patients with Leukemia need during the treatment process?
I went through 10+ papers related to leukemia treatment in China and come out with the experience map. Then, I approach to the touchpoint of each treatment stage for the design ideation.
Interview with 8 medical staff and and 2 leukemia pediatric patients’ parents
Pediatrician
Orthopedist
General nurse
Surgeon
Hematology Nurse
Anesthetist
Pediatrician
Psychologist
Patient parent
Patient parent
Screenshot of my interview notes and questions (in Chinese)
I conducted the interview with medical staff members and patients’ parents in Chinese. If you read Chinese, please feel free to check the full interview and ideation notes in my Miro board.
Empathy map
What do they think and feel
Family financial burden
Slim chance of a cure? Hope?
Fear of surgery
The pain of the treatment
Confused and out of control.
Vomiting, ulcers
Pain so bad and can't sleep
Bone piercing waist piercing pain
What is death? Where do people go
The loneliness of being in hospital
Pediatric patients with leukemia experiencing long-term hospitalization in Chinese public hospital
DO
Too boring to use Tiktok all the time
play with early education toy
SEE
Doctors, nurses, parents
Infusion port, syringe needle
Really long puncture needle
The cold and empty ward
The little shaved head in the mirror
Dolls/photos interior decoration
SAY
"Want to go back to school"
"Want to play games outside"
"Want to go to the park"
"Want to go for a walk and run"
"Want to go home"
"Want to stay in the sun"
"Want to see my friends"
HEAR
Precautions before operation/puncture/examination
Side effects
adverse reactions
Nurses' encouragement
The news of the brother's death in the next bed
"What do children know?“
"It must be retribution from past life”
"We give injections to keep bugs out of our bodies.“
"Cooperate to leave the hospital earlier.“
“Let's fight the disease together.“
Mom turned her back and sobbed secretly
Cognitive Therapy
"Get well and go home!!“
Insights
Disconnected from normal life
4-year-old starts to have social need
Restricted freedom in the hospital
Fear of medical staff/ new place
Questions and fear about illness
Learn to be brave and optimistic
Get to know other patients and make
bonded friendships
Have more time for hobbies and find
out the meaning of life
Pains
Gains
Our Design goal is to transform little patients' negative emotions into positive experiences as they go through extended hospital treatment.
Leukemia treatment experience map
Hospitalized
Remission
Induction
33d
The fear of an unknown illness needs to be calmed
The adverse reactions brought by treatment will not only bring physical and mental pain, but also have obvious adverse
effects, which need to be pacified
More intense treatment, more intense physical pain, need to be soothed
After high intensity chemotherapy and radiotherapy, the body's immunity is almost zero, requiring close care
Extremely weak, in need of close care, high sterility requirements, high diet requirements, demanding details of life
long-time Hospitalization, need to be connected with the outside world immunity is low, need to control infection
Possible positive effects of the treatment
Get pessimistic and tired of the world, think that they are incurable or painful, lose confidence in treatment, lose enthusiasm for life, and they need to regain their confidence in life
Healthy
Early Stage
Chemotherapy
Stem Cell
Transplant
Cure
Recurrent
Newly
diagnosed
7d
PRE-
treatment
7d
hrs
2wk
months
2wk
Maintenance
2yr
Consolidation
56d
Delayed
Intensification
2wk
Touchpoint
The little patient need to switch between hospital and home environment
Getting used to the treatment environment and the treatment response, needs to be distracted
Needs to be distracted from the painful treatment
Maintenance treatment requires access to the outside world, but also requires close adherence to medical advice
Transplant
Room
Touchpoint
Emotion：stubborn, impatient, fear, alienated, anxiety, isolated
Reasons：experiencing physical discomfort caused by the side effects of the drug/ stress of the diagnosed with the disease and leaving the familiar environment
Emotion: self-esteem, inferiority, anxiety,depression, reduced social adaptability
Reasons：physiological changes the use of hormones and chemo drugs
Emotion: Difficulty to adapt to normal life, anxiety, conformism, lack of social ability
Reasons：long-term hospitalization, isolation
Emotion: High risk of mental breakdown and trauma
Reasons：unable to have friends or peers
Takeaway
Looked into the treatment, we hope to design for every stage of medical treatment and explore potential features. Then we developed our main design features based on deep-down medical research on cognitive and behavior therapy.
Ideation
Reason for virtual experience
Virtual experience breaks the interior design limit and expands the possibility of the space.
Reason for virtual friend and social connections
Virtual friend provides mental support, reduce the risk of developing anxiety and psychological trauma, and have healthy social interactions.
Three main features
Companion
The virtual character is designed to be a reliable friend who will be there for the pediatric patient through the entire treatment.
Discovery
The virtual character will help pediatric patients become familiar with the environment and reducing potential fear.
Cognition
The virtual character is able to function as an AI chatbot with voice interaction, answering questions about illness.
Character design inspiration and process
Virtual character UU: persona
What kind of friends can give young patients warm company and strong support? After analyzing the characteristics of various companion characters in animation, films, television works, and our own friends, we created UU, this cute buddy.
UU
Personalities:
UU’s story:
Willing to express but better at listening
Round Body
Light and Agile
White and Clean
Bouncy and Soft
Dim Glow
Cute and Naive
Tenderly Look
Kind and Patient
“Little mysophobia”, can't stand sticky things
Adventure and Travel
Write a diary/photography/record life
Play with friends
Collect meaningful things on the journey
Magic Tricks
Teleport
Story-telling
UU is a little explorer who likes to travel around. Have been to many places, met many friends, and wrote them in the diary. This time, UU came to the earth, looking for the bravest soul.
Here, UU and the little patient met. "Let me accompany you", said UU.
Short stay turned into long-term companionship. UU wants to grow up with the child and completes a story in their life journey.
Hobbies
From Parallel Universe
Personality
“Characteristic Clouds”
Appearance
Expression
Material
Personality Traits
Color
Round Face
Empathetic Look
Always smiling
Round Head
No edges or corners
translucent
Silicon like
fluffy
Mocha like
patience
kindness
Characteristics of the companion role
Warmer tones
luminescent
Light color
Proactive
Friendly
Gentle
Being a Good Listener
Reliable
Have similar interests
Companionship
Humorous
Round Belly
Appreciated Qualities
in Friends
Characteristics of a “Reliable Friend”
When I‘m sick
I‘d like to hear my friends saying that
Communicate Effectively
Won’t get embarrassed even when no one’s talking
Soft tone
Sincere look
The Person Who Speaks To Me First
Good Temper
Won’t force me to do things that I don't like
Emotionally
stable
Quiet not
noisy
Problem
solver
Ability to Make People
Feel at Ease
Happiness maker
Similar experiences
Similar hobbies
Positive values
considerate
empathetic
Get used to the other's presence
Never worried that they would leave
Share the
happiness
Comfort and Care
Play toys
Someone’s there with me, when I wake up
Mom’s hug
dad’s shoulder
Being cared and asked patiently
do not want to talk
Diversion
during injection
Divert Attention
“I’ll always be
there for you”
Provide
Confidence
“Not feeling alone”
Virtual Friend “U”
“U” has the same pronunciation as “友”, a Chinese character stands for “friend”
Support my
decisions
“You’re already
excellent”
Encouragement
Illustrated journey map
Final output
Interface design
According to the function map based on three main features: companion, discovery and cognition, we designed the XR interfaces.
How do we combine PICO development kits
This part is important since we are implementing the design using PICO 4 VR device.
Responsive design
Interface Design
Discovery feature: Exploration in the hospital !
Cognition feature: Learn about Cell !
Discovery feature: Wall paint
Discovery feature: Choose paint brush
Discovery feature: Intro to MRI !
Companion feature: Emails from friends, and other patients
Discovery feature: Wall paint selection
Discovery feature: Paint brush
Discovery feature: Intro to nurse station !
Companion feature: Read emails
Discovery feature: Wall paint color palette
Discovery feature: Drawing a master piece!
UU’s introduction!
Choose two ways of discovery
Companion feature in the daytime: Choose a story
Companion feature at night: Idle state UU
Landing page
Gesture interaction
Companion feature in the daytime: Tell a tory
Companion feature at night: Voice trigger UU show up
Select feature
Gesture recognition
Companion feature in the daytime: Interactions in the story
Companion feature at night: Learn and explore the universe
Design System
How do we combine PICO development kits
PICO Sense Pack
Video seethrough
We use seethrough camera to allows users to see the hospital while wearing the headset.
Space Calibration
By using space calibration, we can map the assets and scenes based on the environment.
Spatial Anchor
Spatial anchors allow users to place virtual objects in the hospital and then revisit them later.
PICO Interaction Pack
Tracking Origin
We use tracking origin to mark the user’s position and orientation in space for accurate position.
Haptic Feedback
We use it to provide haptic feedback to users when certain action is triggered.
Hand Tracking
We use hand tracking to track/recognize different hand gestures to trigger interactions.
Controller HMD input mapping
By using this, we can receive user input through controller for interaction.
PICO Interaction pack: Tracking origin
Pico interaction pack: Hand Tracking
Pico interaction pack: Haptic feedback
Responsive design
We are aware that VR headset technology is still developing. Considering the weight and comfort of the device, today's VR headsets may not be suitable for continuous wear for 24 hours, not even for longer than 2 hours. Multi-platform functionality expands the user scenario and makes virtual characters much more accessible on frequently used devices.
Takeaways
Technology would change the way we think of design
Two years ago when I just have an idea about the project after watching the documentary. I couldn’t think any kind of real technology for designing a virtual character. AI chatbot seems impossible applying to normal life secenario.
Not take criticism personally and keep iteration
I remember how frustrated I was when this project was criticized as too ideal and ‘studentish’. However, with the emerging tool like chatGPT and Apple’s vision pro, the fact of having a virtual friend has came true.
Next steps
Mentored by PICO developers for future development:
Thanks to the PICO hackathon, we will have the chance to work on future development, mentored by PICO developers. The XR app may be fully developed, implemented in VR devices and available in the PICO store.
Consider about product strategy
Every product should align with the broader strategic goals of the business. Ensuring that the product contributes to the overall mission and objectives of the company, whether it's expanding market share, entering new markets, or increasing profitability.
Think about business viability:
Business viability is often tied to the expectations of stakeholders, including investors, executives, and customers. I would try to think from the PM perspective and communicate effectively about the business value and potential for success.
Contact
Let's start creating together
I am actively looking for opportunities
© Designed by Murphy at midnight with music and milk tea. All Rights Reserved.