CONTEXT
INTRO
Existing research shows that simulating stressful situations in VR can be a highly effective form of exposure therapy. However, technology limitations meant that these simulations were static, and relied on pre-scripted dialogues. Advances in genAI open up opportunities for change.
Simulus provides users with the ability to practice handling stressful situations—such as crowded parties, interpersonal conflict, and public speaking—by interacting with generative agents. These agents, powered by GPT, produce realistic, dynamic dialogue that changes with each simulation, offering fresh, personalized experiences every time.
Simulus also includes an optional breathwork guidance feature, allowing users to pause the simulation, practice calming breathing techniques, and resume when they are ready.
RESEARCH OBJECTIVE
As a brand-new research project, our focus was primarily on exploring the design space as we envisioned what Simulus might look like. We set out to understand:
The potential of immersive technologies for self-care and managing stress in everyday scenarios.
The design choices (e.g., medium, interactivity, guidance type) that would create the best user experience for Simulus.
METHODS
We conducted prototype-guided interviews to test eight design options for Simulus, focusing on:
Medium: VR vs. AR vs. text-based roleplay
Scene interactivity: Static vs. interactive environments
Breathing guidance: Text-based vs. visual guidance
With input from users and mental health clinicians, we developed three key scenarios: a public speaking Q&A, a crowded social party, and an interpersonal conflict with a roommate.
Participants were exposed to their most stressful scenario and provided feedback, which guided further design iterations.
MY ROLE
I worked in 4 primary areas on this project:
GUIDANCE SYSTEM DESIGN
One of the critical questions when designing Simulus was what the mental health guidance system should look like. Based on our literature review of validated stress management techniques, we chose to include an option to toggle on/off guided breathwork in the simulations.
Drawing from key studies such as Mindful Moments, Breeze, and Life Tree, we found that:
To design and prototype the guidance system in VR/AR, I had to learn how to use Unity for the first time! Designing for VR/AR was a very different experience, as I had to consider the spatial experience in an entirely new way.
Below is a series of screenshots illustrating the breathing bubble I designed:
GPT PROMPTS
USER FLOW DIAGRAM
The user flow diagram below illustrates how someone might experience Simulus:
Below is an example flow of what someone might experience in the 'interpersonal conflict with a roommate' scenario:
USER STUDIES
We conducted 19 user studies (1 hour) in which we exposed participants to 8 different prototypes for what Simulus could look like. We observed them interacting with each prototype, and asked probing questions.
Out of 19 participants, 17 said they would use one or more of the prototypes (VR, AR, or text-based) in real life. Participants valued the realistic “in the moment” practice of stress-relief techniques:
"The closest that I’d imagine doing before I came into the study would have been practicing something in my head or in front of the mirror, which compared to this, is way less evocative and doesn’t put you into the same mentality. So it wouldn’t be as useful or as effective as something like this." (P16)
There was an overall preference for AR, particularly when used in environments where participants anticipated real-life stressors, like their classroom or home.
Participants also appreciated the value of breathwork guidance, with several noting its impact on emotional regulation. Breathwork was triggered during uncomfortable, confrontational, or awkward moments of the simulation, such as when an audience member asked a pointed question or a roommate said something dismissive.
Participants especially enjoyed the visual aspect of the breathing bubble, as opposed to simply receiving text-based guidance. They also enjoyed having control over when the guidance appeared.
One participant stated that breathwork helped them become "more aware of the situation and [gave] me some time to actually watch what kind of emotions that I have in myself" (P14).
Overall, the best combination of design elements seemed to be a dynamic AR system in which users could choose when to trigger visual breathing guidance.
FUTURE WORK
Having settled on a design for Simulus, we are now building a high-fidelity version of the system and plan to conduct robust user studies next spring.
Planned improvements include:
Enhancing realism with natural conversations, avatar speech, and facial expressions.
Allowing users to customize avatars' appearance and conversational styles.
Dynamically regenerating scenes with diverse avatars for varied practice scenarios.