CONTEXT

I spent a summer at Carnegie Mellon’s Human-Computer Interaction Institute where we built Simulus, an immersive system designed to help users manage anxiety and stress. The system leverages LLM-powered generative agents to create dynamic, personalized simulations of everyday stressors, allowing users to build transferable stress-reduction techniques. 

Wellness is a particular passion of mine; I was so excited to have the opportunity to do work at the intersection of technology, health, and human-centered design.

We just submitted our paper to CHI '25 🎉

I spent a summer at Carnegie Mellon’s Human-Computer Interaction Institute, where we built Simulus, an immersive system designed to help users manage anxiety and stress. The system leverages LLM-powered generative agents to create dynamic, personalized simulations of everyday stressors, allowing users to build transferable stress-reduction techniques. 

Wellness is a particular passion of mine; I was so excited to have the opportunity to do work at the intersection of technology, health, and human-centered design!

INTRO

Existing research shows that simulating stressful situations in VR can be a highly effective form of exposure therapy. However, technology limitations meant that these simulations were static, and relied on pre-scripted dialogues. Advances in genAI open up opportunities for change.

Simulus provides users with the ability to practice handling stressful situations—such as crowded parties, interpersonal conflict, and public speaking—by interacting with generative agents. These agents, powered by GPT, produce realistic, dynamic dialogue that changes with each simulation, offering fresh, personalized experiences every time.

Simulus also includes an optional breathwork guidance feature, allowing users to pause the simulation, practice calming breathing techniques, and resume when they are ready.

RESEARCH OBJECTIVE

As a brand-new research project, our focus was primarily on exploring the design space as we envisioned what Simulus might look like. We set out to understand:

The potential of immersive technologies for self-care and managing stress in everyday scenarios.

The design choices (e.g., medium, interactivity, guidance type) that would create the best user experience for Simulus.

METHODS

We conducted prototype-guided interviews to test eight design options for Simulus, focusing on:

  • Medium: VR vs. AR vs. text-based roleplay

  • Scene interactivity: Static vs. interactive environments

  • Breathing guidance: Text-based vs. visual guidance

With input from users and mental health clinicians, we developed three key scenarios: a public speaking Q&A, a crowded social party, and an interpersonal conflict with a roommate.

Participants were exposed to their most stressful scenario and provided feedback, which guided further design iterations.

MY ROLE

I worked in 4 primary areas on this project:

GUIDANCE SYSTEM DESIGN

One of the critical questions when designing Simulus was what the mental health guidance system should look like. Based on our literature review of validated stress management techniques, we chose to include an option to toggle on/off guided breathwork in the simulations. 

Drawing from key studies such as Mindful Moments, Breeze, and Life Tree, we found that:

To design and prototype the guidance system in VR/AR, I had to learn how to use Unity for the first time! Designing for VR/AR was a very different experience, as I had to consider the spatial experience in an entirely new way.

Below is a series of screenshots illustrating the breathing bubble I designed:


GPT PROMPTS

Prompt engineering Simulus' generative agents presented challenges, especially in socially complex situations. We iterated on our prompting strategy multiple times.

Crowded Social Party

Initial Approach: We generated 10 personas to simulate party interactions, but conversations lacked the natural awkwardness of real social events, and users could only interact with one persona at a time.

Refinements: Reducing the number of personas to three and allowing guests to join and leave conversations improved the realism. We also removed predefined conversational styles (e.g., "outgoing" or "introverted") to increase diversity in interactions. We specified that there should be awkward or uncomfortable moments in the simulation, as there often are in new social scenarios.

Challenges persisted with agents interrupting one another, and bias in persona generation, with names like "Mark" frequently linked to introverted, shy engineers.

Conflict with a Roommate

Initial Approach: Early conflict scenarios (e.g., a messy roommate) were repetitive and predictable. Roommates were often overly-nice, with conflicts being resolved in a matter of a few minutes. 

Refinements: We specified once again that there should be uncomfortable or awkward moments in the simulation, leading to much more realistic conflict styles (e.g. defensive or sarcastic roommates).

Public Speaking

Initial Approach: We simulated both supportive and critical audience members, but interactions became repetitive with predictable patterns of positive or combative feedback.

Refinements: The revised prompt asked GPT to simulate more realistic, varied audience Q&A sessions with awkward or skeptical moments.

Crowded Social Party


Initial Approach: We initially generated 10 personas to simulate party interactions, but conversations lacked the natural awkwardness of real social events, and users could only interact with one persona at a time.


Refinements: Reducing the number of personas to three and allowing guests to join and leave conversations improved the realism. We also removed predefined conversational styles (e.g., "outgoing" or "introverted") to increase diversity in interactions.


Challenges persisted with agents interrupting one another, and bias in persona generation, with names like "Mark" frequently linked to introverted, shy engineers.

Conflict with a Roommate

Initial Approach: Early conflict scenarios (e.g., a messy roommate) were repetitive and predictable. Roommates were often overly-nice, with conflicts being resolved in a matter of a few minutes. 


Refinements: We specified once again that there should be uncomfortable or awkward moments in the simulation, leading to much more realistic conflict styles (e.g. defensive or  sarcastic roommates). We also refined the prompt to expand the conflict scenarios. However, the "messy roommate" scenario often resurfaced.


Biases also emerged in persona generation, with "Alex" frequently being portrayed as the messy roommate and "Jake" as the party host, indicating a need for further refinements.

Public Speaking

Initial Approach: We simulated both supportive and critical audience members, but interactions became repetitive with predictable patterns of positive or combative feedback.


Refinements: The revised prompt asked GPT to simulate more realistic, varied audience Q&A sessions with awkward or skeptical moments, and ensured that only one persona could speak at a time.

Crowded Social Party


Initial Approach: We initially generated 10 personas to simulate party interactions, but conversations lacked the natural awkwardness of real social events, and users could only interact with one persona at a time.


Refinements: Reducing the number of personas to three and allowing guests to join and leave conversations improved the realism. We also removed predefined conversational styles (e.g., "outgoing" or "introverted") to increase diversity in interactions.


Challenges persisted with agents interrupting one another, and bias in persona generation, with names like "Mark" frequently linked to introverted, shy engineers.

Conflict with a Roommate

Initial Approach: Early conflict scenarios (e.g., a messy roommate) were repetitive and predictable. Roommates were often overly-nice, with conflicts being resolved in a matter of a few minutes. 


Refinements: We specified once again that there should be uncomfortable or awkward moments in the simulation, leading to much more realistic conflict styles (e.g. defensive or  sarcastic roommates). We also refined the prompt to expand the conflict scenarios. However, the "messy roommate" scenario often resurfaced.


Biases also emerged in persona generation, with "Alex" frequently being portrayed as the messy roommate and "Jake" as the party host, indicating a need for further refinements.

Public Speaking

Initial Approach: We simulated both supportive and critical audience members, but interactions became repetitive with predictable patterns of positive or combative feedback.


Refinements: The revised prompt asked GPT to simulate more realistic, varied audience Q&A sessions with awkward or skeptical moments, and ensured that only one persona could speak at a time.

USER FLOW DIAGRAM

The user flow diagram below illustrates how someone might experience Simulus:

Below is an example flow of what someone might experience in the 'interpersonal conflict with a roommate' scenario:

USER STUDIES

We conducted 19 user studies (1 hour) in which we exposed participants to 8 different prototypes for what Simulus could look like. We observed them interacting with each prototype, and asked probing questions.

Out of 19 participants, 17 said they would use one or more of the prototypes (VR, AR, or text-based) in real life. Participants valued the realistic “in the moment”  practice of stress-relief techniques:

"The closest that I’d imagine doing before I came into the study would have been practicing something in my head or in front of the mirror, which compared to this, is way less evocative and doesn’t put you into the same mentality. So it wouldn’t be as useful or as effective as something like this." (P16)

There was an overall preference for AR, particularly when used in environments where participants anticipated real-life stressors, like their classroom or home.

Participants also appreciated the value of breathwork guidance, with several noting its impact on emotional regulation. Breathwork was triggered during uncomfortable, confrontational, or awkward moments of the simulation, such as when an audience member asked a pointed question or a roommate said something dismissive.

Participants especially enjoyed the visual aspect of the breathing bubble, as opposed to simply receiving text-based guidance. They also enjoyed having control over when the guidance appeared.

One participant stated that breathwork helped them become "more aware of the situation and [gave] me some time to actually watch what kind of emotions that I have in myself" (P14).

Overall, the best combination of design elements seemed to be a dynamic AR system in which users could choose when to trigger visual breathing guidance.

FUTURE WORK

Having settled on a design for Simulus, we are now building a high-fidelity version of the system and plan to conduct robust user studies next spring. 

Planned improvements include:

  • Enhancing realism with natural conversations, avatar speech, and facial expressions.

  • Allowing users to customize avatars' appearance and conversational styles.

  • Dynamically regenerating scenes with diverse avatars for varied practice scenarios.

Made With ❤️ & 🍵 in Canva, Figma, and Framer!

Made With ❤️ & 🍵 in Canva, Figma, and Framer!

Made With ❤️ & 🍵 in Canva, Figma, and Framer!