Life Enriching Experiences
for People Living With Dementia

Memory Care Experience Station

Overview

The Memory Care Experience Station (MCES) is a large scale interactive experience from Maria Mortati Experience Design.

Made for people with mid-to-late stage dementia, it provides engaging experiences via bespoke multi sensory content. It is meant for life enrichment.

The current iteration is deployed and being enjoyed by residents at the San Francisco Campus for Jewish Living.

My Role

I was brought on to build a research database for Alzheimer's. My role quickly grew to include developing proof of concept mock ups, low fidelity physical prototypes, and digital UI design. All of which were tested with residents and staff

Duration: 18 non-consecutive months

Experience Design
Prototyping
Research

Results

The experience station lives at the SFCJL facility with strongly positive feedback from residents about my experiences and the station itself.

I also made the digital content library more relevant to user needs and easier to develop.

Team

Hridae Walia (Me) - Interaction Designer

Maria Mortati - Principal Designer

Scott Minneman - Technical Architect

Many others! - Staff, Student Designers, & The Residents!

Deliverables

High Fidelity Wireframes

Mid Fidelity Physical Prototypes

Research Reports

Experience Diagrams

Tools

Figma

Arduino

C++, JS, HTML

Screwdriver

Hammer, probably

Awards & Recognition

Fast Company

The Station was a finalist in Fast Company’s 2022 World Changing Ideas Awards, under the experimental category.

SCAN Foundation

The foundation created an honorable mention category for the project, recognizing organizations that have embraced the principles of human-centered design.

Leading Age, California

Finalist in the Innovation Showcase, which recognizes ideas that include evidenced-informed technologies, and improving quality of life for older adults.

The Challenge

Create Life Enriching Experiences for People Living With Mid to Late Stage Dementia at a Memory Care Facility

The San Francisco Campus for Jewish Living aimed to improve the quality of life for residents living with mid to late stage dementia.

People living with mid to late stage dementia often lack engaging and stimulating experiences, leading to isolation, boredom, and other behavioral challenges, which ultimately result in a decreased quality of life.

I was hired by Maria Mortati Experience Design as an Interaction Designer for the MCES project to research, design, and prototype immersive experiences. It was an opportunity for me to design in ambiguity, and with a ton of freedom.

My Key Challenge: Create immersive, interactive, and viable experiences for the MCES, to create life enriching experiences for people living with Alzheimer's at the SFCJL.

Stakeholders

Residents

The MCES was made for the ~83 residents living with mid-to-late stage dementia at the SFCJL. They require a lot of care and close monitoring by facility staff, so we needed to be make sure the experiences we designed were safe and accessible.

Facility Staff

Residents are always assisted by the SFCJL staff who are nurses. The staff's duty is to ensure resident safety and to guide them through any activity. Staff members needed things to be as simple and streamlined as possible.

Key Constraints

Sensitive Population

Due to the sensitive conditions of people living with Dementia and their environment, we did not have immediate access to them. We tested and evaluated internally until we were able to test higher fidelity prototypes with residents.

Healthcare Regulations

Since we were designing for something within a medical care facility, experiences and prototypes needed to be designed with great care and certain regulations in mind such as the stability and form of the experiences.

Usage by Proxy

The population we were designing for were not able to setup and initiate the experience themselves, so it had to be designed to be used and maintained by staff.

My Contribution

Evolving Role

My responsibilities grew as the project progressed.

Initial Role:

  • Building an HCI research database for Alzheimer's

  • Heuristic evaluation of existing technologies

Evolved Role:

  • Developing proof of concept mock ups

  • Low fidelity physical prototypes

  • Digital UI design & design system creation

Key Challenge: Create immersive, interactive, and viable experiences for our target audience.

Foundational Research

Conducted foundational research on Alzheimer's that directly contributed to the development of the experience.

  • Understanding the age-tech landscape

  • Evaluating multi-sensory experiences and technology for people with Alzheimer's

  • Establishing design principles and best practices for mid-to-late stage Dementia

Experience Design

Prototyping engaging, multisensory experiences for people with mid-to-late stage Dementia.

  • Designing for more interactivity and immersion

  • Rapidly prototyping effective and feasible experiences

  • Balancing soothing vs stimulating experiences

  • User testing with residents and facility staff

Digital Content Library

Designing a 'Netflix-like' content library for the station, enabling personalized experiences for the residents while enabling facility staff to track sessions.

  • Designed in collaboration with CCA MDes students

  • Streamlining the information architecture

  • Establishing a design system for easier dev handoff

  • Collaborating with staff to maintain parity with paper-based forms

Result

  • Created impactful experiences that are well reviewed and consistently used by the residents

  • Formed a strong research foundation that served as a guide for the design of all experiences

  • Streamlined Content Library UI for easier development and closer adherence to staff needs

  • Over 200 sessions with residents demonstrated overwhelmingly positive feedback

Foundational Research

Understanding the Stages of Dementia

The residents who we were designing for were mostly at the mid to late stages of Dementia, I researched their quality of life, behaviors, and accessibility considerations.

On this scale, our target audience lie between stages 4 and 6. Key characteristics are;

  • Memory deficiencies

  • Desensitivity to stimuli

  • Difficulty concentrating

  • Difficulty socializing

  • Behaviors can be similar to children with Autism

Another key point to understand was that this project was meant for life enrichment, not attempting to cure Dementia in any way (which at the moment is not possible).

Researching Multisensory Experiences

Building off the insights from the stages of Dementia, I wanted to research the efficacy of (multi) sensory experiences for people with Dementia, Autism, and in general.

Key sensory stimuli were;

  • Scent - Strongly connected to memory

  • Touch - Can facilitate immersion and a sense of presence

I also explored potential implementation techniques

  • Diffusers - I researched the use of diffusers for controlling and emitting scent according to content displayed on screen

  • Ultraleap Haptics - I explored the capabilities of the Ultraleap Ultrasonic Haptic device by trying its demo content, and attempting to make my own demos. It was not a good fit because the tactile feedback was not strong enough, and the small surface area made it difficult to implement.

Slide from a tech review document I prepared
Playing around with the Ultraleap demos

Evaluating the Potential of AR/VR

The leadership of the SFCJL were also curious about AR/VR and wanted us to explore its use for the project.

As an AR/VR enthusiast, I used my expertise to research existing immersive technologies and analyzed their application through primary and secondary research.

Since we were dealing with a sensitive population, our research showed that Virtual/Augmented Reality solutions were not suitable.

People with mid to late stage Dementia have difficulty concentrating and struggle with memory. The inclusion of a virtual reality would further disconnect them from their reality and cause confusion or even harm.

I synthesized my findings into a slide deck comparing different immersive and interactive devices.

Slide comparing active/passive interaction with level of immersion

Synthesizing Research & Defining Principles

I also used our learnings of the problem space and the facility to create an Ecosystem Diagram to map out how different stakeholders would interact with our solution, and how it exists within the SFCJL facility.

Our research led us to form design principles that would serve as guides throughout the product development process.

Adaptive

Safe

Social

Rich

Screenshot of the research database I helped create & maintain
Ecosystem diagram I created

Research Outcome

My initial explorations and research findings directly contributed to the first iteration of the Memory Care Experience Station.

It used diffusers to emit scent, included haptic feedback through the use of low frequency emitters, and immersive visual driving experiences.

The experience station was successful at engaging the residents, so the SFCJL continued their investment in the project, and I got to develop some new experiences of my own

First iteration of the MCES in the SFCJL facility

Experience Design

Building off of the findings from our foundational research and insights from some of the early experiences tested with the MCES, it was my job to create new experiences that were interactive, immersive, and engaging while also being easy to setup and operate by facility staff.

The Cat Petting Experience

An interactive experience for the cat loving residents! 3 cats are set up on a board and when pet, they emit a purring vibration, while playing a video of that cat on the MCES screen.

  • Sensory stimulation

  • Social engagement

  • Residents love cats

  • Easy to setup for facility staff

Challenge

The station needed more interactive experiences, as most of the existing experiences were either static viewing experiences or actively facilitated by staff.

My goal was to create experiences that were immersive, stimulating, and interactive while still being suitable for people with mid-to-late stage dementia.

Ideation & Prototyping

Building off an existing texture stimulation toy for Alzheimer's, I set out to create an interactive, tactile, and engaging experience.

  • Experiment: Testing the sensory toy with pressure sensors to see if it can sense through a material.

  • Exploring Form: I experimented with fur-like textures attached to large arcade buttons

  • Exploring Form: I bought stuffed cat toys to see if I could embed them with pressure sensors

  • Exploring Form: I had to do horrible things to get those sensors in there… sorry kitty.

  • Implementation: Embedding pressure sensors in the cat toys

  • Implementation: Sketching the Arduino implementation

  • Implementation: Attaching small LFEs to the cat platform and testing them with an audio amplifier

  • BIG CONSTRAINT: THE DOG WANTED TO EAT MY PROTOTYPE!

  • Solidifying: Organizing all the wiring for the prototype so its ready for on-site testing.

Testing Internally

I tested the cat experience internally with team members, because our residents were sensitive we had to ensure our prototypes were safe and secure. Key outcomes were:

  • Effective and fun! The experience felt immersive and the cat petting interaction had enough potential to move forward with the concept

  • Increased haptic feedback strength

  • More sensitive activation of pressure sensors

  • More stable prototype, so it can be ready for testing

Testing With a Resident

After tweaking the sensitivity and further solidifying the prototype, we scheduled a session with a resident on site.

The resident was engaged while petting the cats and responded to the haptics and video. They also wanted to pick up the cats and pet them.

Result

An engaging experience that grabbed attention from staff and residents alike as I carried it around the campus.

It quickly became a core experience within the MCES and remains popular with the residents as they continue to use it.

If I was to work on it further, I would definitely make another version that residents can pick up and hold.

Accessible Haptic Feedback Platform

I prototyped a footrest that let residents experience the texture and haptic feedback in the MCES experiences in an accessible manner.

  • Accessible by wheelchair using residents (majority)

  • Easy to implement

  • Strongly effective

  • Immediate mainstay of the MCES

Challenge

Haptic feedback is particularly successful in sensory stimulation for people with Alzheimer's and it's ease of implementation with audio output made it one of the most important aspects of the MCES.

At the time, Scott Minneman had created a haptic platform for the residents that used tactile transducers for haptic output. The platform provided strong haptic feedback but it was inaccessible for wheelchair users (most of our residents).

I took the existing concept and turned it into a more accessible footrest form factor while retaining the haptic sensation.

The original haptic feedback platform under the chair

New haptic feedback platform

Ideation

I took the haptic platform home and explored ways to make it accessible for wheelchair users while maintaining the simplicity of the original haptic feedback platform.

  • Starting point: Back side of the original haptic platform

  • Experiment: Why don't I just stick a transducer onto a footrest and see what happens

  • Experiment: That worked pretty well so I added another, along with some gaffers tape for sound insulation.

  • Implementation: Increasing haptic strength with a larger transducer, based on testing feedback

Testing

We tested the footrest with a resident, alongside a driving experience so they could feel the roar of the engine and texture of the road. Key outomes were:

  • Stronger haptic feedback

  • Simplified wiring

  • Limit movement

This testing session also taught me a lesson in managing prototype fidelity. I used a racing simulation to use with the foot rest and the resident was fully immersed in the experience, driving and talking along the way.

At first I was elated by the resident's engagement but they wouldn't let go of the station when it was time to go.

Since I loaded an open ended simulation, it kept going as there was no 'finish line' so the resident kept driving and got frustrated when we told him that the experience was over. I'm sure if I had prepared an experience with a clear start and end, this could have been avoided.

Result

The footrest became an immediate mainstay for the MCES and is used as an integral part of every session. The form factor made it easy to implement and accessible by all residents.

This prototype was so much fun to use, I kept one for myself and continue to use it for games and entertainment.

Extras

I explored many different concepts in the pursuit of creating engaging experiences.

  • Motion controlled video games

  • Combining multi sensory effects with immersive video games like Assassin's creed's exploration mode

  • Driving simulations with haptic feedback

  • Interactive, gesture based music experiences

  • Exploring accessibility focused video game controllers for people with Dementia

Digital Content Library

With more residents becoming regular users of the system, they needed a system that could track their interests and provide a personalized experience.

Maria Mortati had consulted a group of students from CCAs MDes program to design the first iteration and lay the foundations for this part of the project. I built off their work to create a simplified IA, design system, and set of high fidelity wireframes ready for dev handoff.

Overview

A web based library that controls the experience station, includes a personalized library for each resident, instructions for running sessions, and categories for stimulating/soothing content. My contribution:

  • Set the design system

  • Simplified information on screen for more relevancy for facility staff

  • Improved personalization for resident profiles

  • Redesigned UI for consistency and usability

  • Documentation for dev handoff

Challenge

The content library was redesigned in close communication with facility staff so it would improve the session running experience and resident tracking. My challenge was to redesign the system so it's closer to the needs of facility staff while being easier to implement by our developer.

Based on communications with staff, key challenges to address were:

  1. More guidance during experience sessions

    1. Staff members wanted more direct guidance on how to run specific experiences since some of them needed multiple pieces of equipment

  2. Better understanding of resident preferences

    1. Staff members wanted more relevant information on residents, since they already take notes internally and would like to have their detailed information at hand

  3. More flexibility with MCES experiences

    1. Experiences with the MCES are often improvised, not every session uses a predetermined experience and staff often just run youtube videos that the residents enjoy

Ideation

I explored ways to address our challenges without redesigning the whole experience. I worked with Maria Mortati and facility staff to make design decisions and I did the UI design.

Old Dashboard

  • Too much information at once

  • Notification and calendar widgets were not relevant for staff

  • Session tips needed more guidance

New Dashboard

  1. Added the ability for staff to add experiences for residents [Challenge 3]

  2. Consistent UI elements [Better usability & dev handoff]

  3. Immediate guidance for running experience sessions [Challenge 1]

  4. Filtered down dashboard elements for immediate and relevant information

Old Resident Profile

  • Focuses too much on non-session related information

  • Absence of staff notes for residents

  • Lack of importance for resident content personalization

New Resident Profile

  1. Introduced playlists for each resident [Challenges 2 & 3]

  2. Emphasized staff notes within the resident profile [Challenges 1 & 2]

  3. Sidelined metadata that was not primary information for staff [Challenge 2]

Old Experience Library

  • Too much emphasis on non-session related information

  • Absence of staff notes for residents

  • Lack of importance for resident content personalization

New Experience Library

  1. Introduced playlists for each resident [Challenges 2 & 3]

  2. Emphasized staff notes within the resident profile [Challenges 1 & 2]

  3. Sidelined metadata that was not primary information for staff [Challenge 2]

Feedback & Results

The digital content library design received positive feedback from facility staff as it gave them quicker access to relevant information while also empowering them to better personalize experiences for residents. It is currently under development and will be deployed to work alongside the MCES.

Reflection

A Unique Opportunity

I love to work in a hands-on capacity, screens, toys, games, physical form, all of it. This was a unique chance to stretch my skills and have the independence to create impactful experiences from scratch, as challenging as that was.

This project came with a lot of special considerations for our users, and it changed my perspective as a designer to try to always consider the needs of all populations, not just the most common.