ICU AR/DR Assistive System that increases alarm response efficiency with enhanced situational awareness.

Cisco Systems. | Georgia Tech IMTC

M.A.C.C. is a new system design that enhances nurse’s performance dealing with alarm interruptions by utilizing team collaboration, increasing situational awareness, and facilitating seamless communication. This system allows nurses to document their interrupted tasks, manage alarms with real-time situational updates, get help from other personnel more effortlessly, and better manage their tasks with smart system suggestions.

#Novel Interaction   #AR/DR   #Participatory Design   



‍Took full charge of primary user research, including user interviews, contextual inquiries, and multiple data analysis methods.


Took full lead on developing the user flow, wireframes, and prototypes;
Co-designed core features with target users and Product Managers;


Co-designed core features with target users and Product Managers;




This is a 8-month research lab project at the IMTC Lab at Georgia Tech, Sponsored by Cisco.

Why this project?

Alarm fatigue is a severe threat to medical safety in ICUs. However, the intervention methods are limited to more technical aspects, overlooking human factors that have been repeatedly reported as key contributing factors to inefficient alarm response.


“Nurses want to reduce alarm fatigue by dealing with alarms interruptions more efficiently during tasks like handoff.” How can we increase efficiency without diminishing performance and safety?


A system that enables interruption recovery and enhances situational awareness. By introducing multimodal documentation and multiple communication features, we not only allow nurses to better evaluate alarms and teammate's actions in real-time, but also better recover from necessary interruptions with less errors.


Our design greatly reduced cognitive and physical load comparing to existing workflow. We were able to decrease the amount of self-reported workload by 20%. We also received positive feedback from participants on system usability.


“I love the enhancements, I did some user acceptance testing, and had no issues!”

Karen Peterson · Senior Vice President of Operational Control Risk Management, Bank of America

“I love the enhancements, I did some user acceptance testing, and had no issues!”

Karen Peterson · Senior Vice President of Operational Control Risk Management, Bank of America

“I love the enhancements, I did some user acceptance testing, and had no issues!”

Karen Peterson · Senior Vice President of Operational Control Risk Management, Bank of America

00   Design Process.


For this project, I took the Double-Diamond design approach, diverged from a vague and generic term, Alarm Fatigue, and converged to a practical solution. I explored many possible designs along the path, but our participatory design sessions helped me narrow down to the current one.


It has been a very unique design experience during which little pre-existing knowledge or personal experience affected the solution design. Without any pre-assumption of the final solution, I was able to truly and fully enjoy the double-diamond design procedure and have my users tell me what I should design.


Define user's goals, needs, and specific use cases.


  • Semi-structured Interviews
  • Pseudo Contextual Inquiries
  • Affinity Mapping
  • Task Analysis

Understand pain points, user attitudes, and behaviors.


  • User Interviews
  • Stakeholder Interviews
  • Affinity Mapping
  • Journey Mapping

Ideate features, user flows, and solution formats.


  • Co-design sessions
  • Brainstorming & Brainwritting
  • Storyboarding

Prototype interaction in 2D & 3D for wizard-of-oz studies.


  • Figma 2D Prototyping
  • Unity 3D Prototyping

Evaluate solution performance and usability for iteration.


  • Benchmark Tasks Studies
  • Cognitive Walkthrough
  • System Usability Scale

Reflect on design process and challenges.


  • Process Summary.
  • Key Takeaways.

01   Define.

“Nurses want to reduce alarm fatigue by dealing with alarms interruptions more efficiently during tasks like handoff.”

"Nurses want to reduce alarm fatigue."

We only knows that Alarm Fatigue is a serious issue to be solved in ICUs, but have no idea about how it actually affects nurse's job, and how it can be solved in a way that does not compromise safety and quality of care.

Refinement I.

We conducted a series of Problem Space Exploration which help us identify a typical task to focus on: Handoff, also known as shift-to-shift report where the outgoing nurse summarize the patient's status to the incoming nurse.

Problem Space Exploration

Existing User Research

Semi-Structured Interview



Affinity Mapping

Problem Space Refinement

“Nurses want to reduce alarm fatigue by dealing with alarms interruptions more efficiently.”


Even though we have grasped a more refined understanding on our user's preferred solution, it is very difficult to design without a more specific task in mind, since nurse's daily job is so complex that designing for everything is way beyond our scope.

Refinement II.

We conducted 3 Pseudo Contextual Inquiries, a virtual method we designed to comply with social distancing requirement, to learn more about 2 potential tasks that we identified from previous interviews: handoff and catheter change. We ended up selecting handoff over catheter change because it requires more informations from EMR and other care team members, making it a good example of more complex situations.

We conducted a task analysis based on the contextual inquiries.

Problem Space Exploration

Existing User Research

Semi-Structured Interview



Affinity Mapping

Task Specification

Pseudo Contextual Inquiries



Task Analysis

Problem Space Refinement

“Nurses want to reduce alarm fatigue by dealing with alarms interruptions more efficiently during tasks like handoff.”

02  Understand.

1   Research Method.

Research Goals.

  • Understand specific pain points in current workflow.

  • Understand existing technologies incorporated in the workflow.

  • Understand Available/Preferred technologies, features and interactions.


We conducted semi-structured interviews with both target users, ICU RNs, and other medical professionals who work closely with ICU care teams. Compared to our previous research, we asked more specific questions about pain points , information/communication need, interruptions and, recovery methods during handoff, as well as their desired assistive solutions.

Semi-Structured Interview - Target User

11 ICU Registered Nurses
Remote, 60 min
Recruited from Facebook, Reddit, and other social media.

Semi-Structured Interview - Stakeholder

1 Nursing Student, 1 ER Nurse, 1 Charge Nurse.
Remote, 60 min
Recruited from Facebook, Reddit, and other social media.

2   Data Analysis.

Affinity Mapping.

We organized 345 notes from all 14 participants and categorized into 4 major aspects: Workflow Pain points, Alarm Fatigue, Existing Technologies, and Preferred Features. Affinity mapping helps to identify shared feelings and preferences among nurses, which further inferred potential design opportunities.

Journey Mapping.

After identified the sample task, we created 2 journey maps showcasing nurse's response to interrupted handoff and interrupted caregiving respectively. The two journey maps share a lot similarities, which also indicates that Handoff is representative among many nurse's tasks.

3   Key Findings.


We found that although the current EMR systems have many helpful designed functions, these features are not always easily accessible. Nurses find it difficult to evaluate alarms only based on audio, and there are also insufficient passive communications across the team to enhance response efficiency. Lastly, nurses prefer wearables such as wristbands and bone-conducting earbuds.

Design Opportunity.

Since the existing EMRs are complex and sophisticatedly designed systems, our design should work with existing workflow and serve as a middle part between the full system and the end users. Our system should also assist cross-team communication and provide rich situational information to help with nurses' judgement. Also, the controls of the system need to be robust and easy-to-use in most, if not all, situations (eg. hand-free, with PPE).

Detailed Insights.

What do nurses find frustrating in current workflow?

  • While there are really great EMR systems, current handoff still mainly relies on pen & paper.

  • Short-handed due to Covid.

  • Closed doors during Covid makes it difficult to locate alarms.

  • Multiple nurses might respond to the same alarm.

What do nurses say about Alarm Fatigue?

  • Nurses are very concerned about situational awareness, making it less feasible to simply filter out alarms.

  • Physically demanding as well.

  • Alarm noise is also bad for patient’s rest.

  • It is hard to tell if an alarm is false or non-actionable.

What technologies are nurses currently using?

  • Nurses have very refined EMR systems (EPIC, Cerner, etc.) which has a lot of well designed features. While some of the hospitals are not using the most advanced systems due to budget, the solution exists.

  • However, nurses find it cumbersome to repeatedly retrieve information from the system.

What features or technologies are most wanted or more acceptable?

  • Nurse’s most accepted wearable devices is smart watch/wristband.

  • Bone-conducting headphones is another acceptable solution.

  • Nurses want to know if others have respond to the alarm, and more detail regarding the alarms for better judgement.


Our Design Should...

Be compatible with current EMR systems/workflow.

#system design

Maintain/Enhance situational awareness by facilitating care team communication and showing alarm details/response update. 

#added value
#information need

Increase interruption resilience by provide easier documentation and information retrieval methods.

#data visualization
#intuitive design

Use non-intrusive wearables to achieve robust control under different circumstances.


03   Ideate

1   Key Features.

Based On Previous Insights + Co-Design Sessions:

We co-designed 4 key features to with 3 nurses to fulfill the need for better interruption resilience, situational awareness, task management, and cross-team communication, 4 key functional needs we identified based on research insights and co-design brainstorming.

Why these features?

We want to fill in the gap between 3 pairs of entities: nurses - EMR systems nurses - other care-team members, and nurses - patients in emergency. To achieve these, we seek to provide persistent documentation and information retrieval methods, seamless multi-end communications, as well as contextual information for alarms to nurses in a non-intrusive, customizable, and easy-to-control approach..

Multimodal Documentation

Use multiple methods to document interrupted handoff or caregiving.

Image & Text: Auto-capture and scan handwritten report sheet.
Audio: Record, transcribe, and tag handoff procedure.
Video: Auto-retrieve viewport recording of the last 30s before alarm goes off.

Alarm Acknowledgement

Allow nurses to acknowledge alarms remotely to enhance floor-level efficiency.

Broadcast: Broadcast alarm details to responsible/nearby nurses.
Acknowledge: Allow nurse to accept alarms and notify others.
Transfer: Allow unavailable nurses to transfer their alarm/notification to others.

Contextual Task List

Use contextual data to show most relevant task list.

Location: Prioritize current patient’s task when entering their unit.
Urgency: Prioritize urgent task and send notifications.
Availability: Show available time slots when other nurses need help.

Call System

Improve current Vocera workflow by adding locational information.

Voice Control: Receive/make calls to others hands-free.
Availability: See availability of others with rough locations.
Request: Enable open call for nurses/technician to nearby personnel.

2   Hardwares.


In addition to the AR frames as required by the sponsors and the lab, we envisioned a smart watch to work as a complementary hardware for the system. Both parts will serve as both display and control, while providing multi-modal feedback.


We selected smartwatch for 3 reasons:

  • Smart watches are less-intrusive and burdensome to carry around.

  • Nurses are already used to wearing and using smart watches.

  • Smart watches can be used both with precision (tangible screen) and hand-free (accessible touch).

System Display

This system has two display components: AR lenses and smart watch screen. Most information will be rendered to the lenses since it has more display real estate, and is more intuitive to look at. The smart watch screen will show either simplified information, or controls and options that require precise interaction.

Because the AR frames can render information without physical screens, there are also potential to add additional virtual display into the 3D space, as long as with appropriate information density.

System Controls

The primary control for the system would be the smart watch, through either touch screen or accessible touch. However, considering intuitiveness and Fit's Law, we designed a simpler "hot-key" control with the AR frames. Inspired by the "transparency mode" on Oculus Quest, which allows users to quickly turn on and off transparency display by double tapping on the side edge of its frame, we incorporated frame touching gesture for quick control of key features.

Audio Feedback

We imagined build-in bone-conducting audio components in the AR frames, allowing nurses to hear the audio notifications clearly without blocking environmental sounds or causing discomfort after continuous wearing.

Tactile Feedback

The smartwatch will also provide tactile feedback as a supplementary modality, making it possible for modality-remapping or notification accentuation.

3   Storyboards.

04   Prototype

FLOW 01   Basic Interactions.

This section showcases the basic interface and interaction of the system. Nurses can view their patient’s information on their smartwatch, or pin different types of information onto their viewport.

The arrangement of different components will change based on context and other pinned content. For example, task lists for two patients will condensed into a full timeline to reduce screen real estate usage.

Nurses can also very easily hide different sides of the viewport display by tapping on the corresponding edge of their device, or triple type on any edge to hide all components (Figure 11). The hidden component can be easily brought back by repeating the disable action.

Flow 02  Multimodal Documentation.

This section showcases different methods of documenting and retrieving handoffs and interrupted care sessions.

Nurses can simply start their handoff session by tapping on the handoff button on their smartwatch for corresponding patients. They can also customize what system they will be reporting for each specific patient. The default is set to “All” since for most patients it is required to go through all systems, but customizations are available for more stable or comfort care patients. During a handoff session, the system will auto capture the nurse's handoff sheet and start voice  recording. A miniature view with keyword extraction from the voice recording is visible above the viewport as a weak indicator of progress.

It is worth noticing that on each side of the viewport, there is only one actionable button rendered in purple, which is the same case for the rest of the prototype. Nurses can quickly activate these buttons by double tap of the corresponding edge. This interaction is especially helpful when wearing PPE during which the smartwatch screen is not available. 

If nurses want to retrieve their handoff data, they can review their full handoff sheet and handoff recording, or select a specific system they want to check. The standardized report sheet and auto text recognition will help the system to zoom into the desired section on report sheets, and the auto keyword extraction allows the system to quickly locate the corresponding section in the voice recording. Nurses can also turn on and off report sheets or audio review respectively on their smartwatch to comply with their current workflow. In the case of alarm interruption, the system will automatically retrieve the last 30 seconds of the viewport data, and allow users to review the footage once they get back from the emergency.

Flow 03  Alarm Acknowledgement.

This section showcases how nurses can review and manage alarms using the system. The following image shows the standard view of an alarm. Alarm details will be shown on the top and right side.

Nurses will be able to accept or reject the alarm when being notified. If other nurses have accepted the alarm, their names will appear for all notified nurses, and the action buttons disappear for other people. The visual and audio alerts of the alarm are also diminished.

As for the nurse who acknowledges the alarm, the notification will stay on their viewport (Figure 18), but they will have the ability to remap the audio modality to vibration, or set a delayed notification for themselves, as shown below.

If no one acknowledges the alarm or the acknowledged person has not been in place for a certain amount of time, varies on emergency level, the alarm will escalate and broadcast again to all nurses, even if they have rejected the previous notification (Figure 21). The notified nurses can no longer reject an escalated alarm, but can remap its modality, until it is resolved.

Flow 04  Contextual Task List.

This section demonstrates how task lists and notification works with contextual data. The task list for a patient will automatically pops up when nurses approach their unit door. Task lists will also appear with time sensitive notification when approaching a scheduled care time. Although this function is a standard feature for everyday-use mobile devices, it is not yet included in many current EMR systems.

Since the system is fully voice control enabled, nurses can easily set up new tasks using voice control, or transfer tasks to other nurses. If a task is about to transfer to another nurse, their device will automatically shows their existing task list to suggest if they are available for the incoming task. 

Flow 05  Call System.

This section demonstrates the user flow of the call system, in which users can call specific medical personnel using voice control, or initiate open calls to a type of technician. Similar to existing call units such as Vocera, our system supports search by name, designated patient names and role, or unit number. Nurse can initiate an open call for a specific types of care Respiratory Technician. Instead of making a direct call to one technician, they can broadcast their help request to the entire floor or multiple floors, in a style similar to alarms broadcasting. When someone responses, the initiated nurses can check on the respondents name, estimated arrival time, and make direct calls.

05   Evaluate


We conducted a set of efficacy benchmark tasks using wizard-of-oz approach to compare performance using existing methods and our solutions. We defined 3 tasks that utilized all four features and require users to perform the tasks without and with our system. We then distributed a NASA-TLX questionnaire at the end of each task.

We also conducted usability testings with another set of users using the System Usability Scale (SUS).


For efficacy, our NASA-TLX result suggest that users find the system helpful, especially for Physical Demand, Temporal Demand, and Mental Demand. Our user's performance on alarm response is greatly improved, with less significant but still recognizable enhancement for in-team communication and handoff resume.

As for usability, our SUS result (77.5/100) shows that our users find the system helpful and organized, but recognize a deep learning curve. Users like that our system is coherent with medical professional standards (eg. color-coded Vital monitors, level of emergency), but demand more documentations and guidance.

Resume Handoff

Recollect where they left off complete handoff.

Audio of sample handoff interrupted by Alarm sound.

Respond to Alarms

Find alarm location and respond to all 3 alarms.

3 different alarms go off with short interval.

Get RT’s Help with other’s task.

Help pairing nurse’s assessment task and find a RT to help.

Request assistant with a future task in 15 min.


Walkthrough Documentation Flow



Walkthrough Alarm Management Flow



Walkthrough Task List Flow



Walkthrough Call System Flow


06   Reflect

For this project...

We explored a complex problem space and designed a solution following Double-diamond and Participatory design approach. Our evaluation shows that our solution is innovative, helpful, and usable, and our users show strong interest in the concept.

While this is still an ongoing project in the IMTC lab, my part of the project builds a foundation for the solution and is able to inspire future expansion in features and use cases. I am looking forward to seeing larger scale of in-person and in-situ evaluation after the pandemic.

In the future...

Design for vague, complicated, and highly professional problem space

It is important to incorporate the divergent-convergent cycle even at the beginning. While there are no clear path to follow, iteration on entry points and ideas is essential for a successful design. Defining a specific task or scenario to focus on is specifically helpful.

Design for accessibility

Disability is a fluid status. Everyone can be come "disabled" under specific circumstances. Just like how we considered alternative controls for nurses wearing PPE, it is important to consider how might our users be disabled when using our products.

Thank you for watching!