Face painter

Can embodied interaction help create more inclusive digital experiences?

In collaboration with Portuguese artist Rudolfo Quintas, I explored the use of embodied interaction to reactivate elderly people through digital tools. We decided to repurpose the face, our most expressive and personal communicative part of the body, as an interface.

The tool
In a series of experiments, held with the elderly inhabitants of the Casa do Artista in Lisbon, we developed the Face Painter. This interactive web application was built with Google’s body-tracking library MediaPipe. It was designed to enable users to draw with nothing but their face and to offer new accessible and playful was to express themselves digitally.

The main purpose of this tool is to reactivate elderly inhabitants, many in their 80s, who already struggle with everyday activities. They spent a week playing with the tool, providing feedback and simply having fun. The interactions we came up as a result of this are simple: open your mouth to adjust the size of a colorful ellipse, move your head to draw on the screen. You can try it yourself with this live demo.

Societal potential of creative coding
Fo me this project exemplifies how digital tools can create societal benefit. It shows how creative coding is applied speculative design. As designers and coders we can open up new fields in therapy and mental health. Inclusion, societal participation and creative expressions are core values that we can explore and contribute to. Creative coding is one method to explore this field and to open up new possibilities of how technology and society could find new ways of coexistence and symbiosis.

²⁰²³