CONTEXT
Simulus is an immersive system that leverages LLM-powered generative agents to create dynamic, personalized simulations of everyday stressors.
I spent a summer at Carnegie Mellon’s Human-Computer Interaction Institute working on this project in the Social AI Lab.
Health and wellness are particular passions of mine—I was so excited to have the opportunity to do work centered around helping people combat stress and anxiety!
We just submitted our paper to CHI '25 🎉
INTRO
Existing research shows that simulating stressful situations in VR can be a highly effective form of exposure therapy. However, technology limitations meant that these simulations were static, and relied on pre-scripted dialogues. Advances in genAI open up opportunities for change.
Simulus provides users with the ability to practice handling stressful situations—such as crowded parties, interpersonal conflict, and public speaking—by interacting with generative agents. These agents, powered by GPT, produce realistic, dynamic dialogue that changes with each simulation, offering fresh, personalized experiences every time.
Simulus also includes an optional breathwork guidance feature, allowing users to pause the simulation, practice calming breathing techniques, and resume when they are ready.
RESEARCH OBJECTIVE
As a brand-new research project, our focus was primarily on exploring the design space as we envisioned what Simulus might look like. We set out to understand:
The potential of immersive technologies for self-care and managing stress in everyday scenarios.
The design choices (e.g., medium, interactivity, guidance type) that would create the best user experience for Simulus.
METHODS
We conducted prototype-guided interviews to test eight design options for Simulus, focusing on:
Medium: VR vs. AR vs. text-based roleplay
Scene interactivity: Static vs. interactive environments
Breathing guidance: Text-based vs. visual guidance
With input from users and mental health clinicians, we developed three key scenarios: a public speaking Q&A, a crowded social party, and an interpersonal conflict with a roommate.
Participants were exposed to their most stressful scenario and provided feedback, which guided further design iterations.
GUIDANCE SYSTEM DESIGN
One of the critical questions when designing Simulus was what the mental health guidance system should look like.
Drawing from key studies such as Mindful Moments, Breeze, and Life Tree, we found that:
We chose to create a breathing bubble that could be activated and deactivated by the user so that guidance would remain unobtrusive. In opening Simulus, all users undergo an onboarding session in which they learn how to use the bubble.
To design and prototype the guidance system in VR/AR, I had to learn how to use Unity for the first time!
PROMPT ENGINEERING
Designing prompts for Simulus' generative agents proved challenging, especially in socially complex scenarios. Early simulations felt repetitive, overly predictable, or lacked the natural awkwardness of real-life interactions.
We iterated by reducing personas (e.g., three instead of ten for parties), removing predefined traits, and specifying the inclusion of awkward or uncomfortable moments to improve realism.
Adjustments to conflict prompts introduced defensive and sarcastic behavior, while public speaking scenarios benefited from more varied and skeptical audience dynamics. Challenges like agent interruptions and bias in persona generation (e.g., stereotyping certain names) remained areas for further refinement.
USER FLOW DIAGRAM
The user flow diagram below illustrates how someone would experience Simulus:
USER STUDIES
We conducted 19 user studies (1 hour) in which we exposed participants to 8 different prototypes for what Simulus could look like.
Out of 19 participants, 17 said they would use one or more of the prototypes (VR, AR, or text-based) in real life. Participants valued the realistic “in the moment” practice of stress-relief techniques:
"The closest that I’d imagine doing before I came into the study would have been practicing something in my head or in front of the mirror, which compared to this, is way less evocative and doesn’t put you into the same mentality. So it wouldn’t be as useful or as effective as something like this." (P16)
There was an overall preference for AR, particularly when used in environments where participants anticipated real-life stressors, like their classroom or home.
Participants also appreciated the value of breathwork guidance, with several noting its impact on emotional regulation. Breathwork was triggered during uncomfortable, confrontational, or awkward moments of the simulation, such as when an audience member asked a pointed question or a roommate said something dismissive.
Participants especially enjoyed the visual aspect of the breathing bubble, as opposed to simply receiving text-based guidance. They also enjoyed having control over when the guidance appeared.
One participant stated that breathwork helped them become "more aware of the situation and [gave] me some time to actually watch what kind of emotions that I have in myself" (P14).
Overall, the best combination of design elements seemed to be a dynamic AR system in which users could choose when to trigger visual breathing guidance.
FUTURE WORK
Having settled on a design for Simulus, we are now building a high-fidelity version of the system and plan to conduct robust user studies next spring.
Planned improvements include:
Enhancing realism with natural conversations, avatar speech, and facial expressions.
Allowing users to customize avatars' appearance and conversational styles.
Dynamically regenerating scenes with diverse avatars for varied practice scenarios.