Haptic Art-cade
A haptic-based art generation tool for affect expression and emotion regulation
Haptic Art-cade system, with the User Interface zoomed in on the left, showing the Haply device and laptop setup on the right
Role
Team Member
Date
January - April 2021 (4 months)
Team
Sri Gannavarapu
Raquel Robinson
Tools
Arduino, Processing, Adobe Premiere Pro, Haply, Miro
Methods
Affinity Diagramming, Literature Review, User-Centered Design, Prototyping, Usability Testing, Survey, Brainstorming, App Development
Overview/Context
As a course project for CPSC 543 (CanHap501), we developed Haptic Art-cade, a haptic-based art generation interactive tool, for affect expression and potential emotion regulation.
Over 4 months, we iteratively designed and prototyped the haptic tool, guided by intermediate usability and user tests. Each team member participated equally in all processes. In addition, I led the presentation and video creation for the final deliverable of the project.
So, what was the problem?
Our course project involved the Haply device, where we had to come up with an application for the device, centered around exploring the haptic capabilities of the device.
Our team decided to focus on the potential of haptics in anxiety and stress management, through the use of vibrotactile and force feedback. We were partially driven by our own experiences with anxiety and stress management, particularly on the impact of touch in soothing or stress-relief, and partially by the potential of such an application in today's world where stress and anxiety have become a part of our daily lives.
I see, so who was your target demographic?
Our target users were people looking to self-manage their stress and anxiety. While our tool could potentially be used by anyone, people familiar with haptics, or interested in the use of art or games as a stress management tool would be more likely to try this tool. A technical constraint was given our development and focus on the Haply device, target users would need access to the same.
So, what was the plan?
We followed the Design Thinking Process of Empathize-Define-Ideate-Prototype-Test.
Empathize:
We began by conducting a preliminary literature review, to assess the use of haptic feedback in affect expression and emotion regulation. Outside of haptics, several coping mechanisms for stress and anxiety exist, from indoor to outdoor activities, like journaling or exercising. Some popular activities to self-manage stress include journaling, colouring, and games like Bubble Wrap and Paper Toss.
However, we did not find any applications that combined these activities with a haptic modality.
Define:
We wanted a tool that would be portable, and could be used whenever users felt they needed to relieve their stress or anxiety. A game-based art generation format would also make the emotion regulation process enjoyable, and thus possibly more engaging, for users. We created a rough design space map (shown below) to scope our project and brainstorm the different functionalities it could have.
Design Space Mapping to define the project scope
Given the timeline and scope of this project, we decided to exclude music from this tool, instead focusing on creating a haptic tool that would provide a game-like environment for users, and generate art through gameplay, through a series of configurable environments.
Our goals for the tool were threefold:
-
Enable affect expression through customizable elements (realized through varying colour palette options and the ability to move between different environments within the application)
-
Enable users to record their current emotional states for future reflection (realized by optional timestamped snapshots in each environment)
-
Create an aesthetic experience which used multiple modalities and was both therapeutic and enjoyable (realized through the careful synchronization of visual, audio and haptic modalities, and an artistic gameplay experience)
Ideate, Prototype, Test:
In Stage 1, we design and evaluate the haptic sensations for the final application, and in Stage 2, we design the user interface of the tool.
Stage 1:
The Haply development kit we were given was a two-degree-of-freedom haptic device, that could be programmed to deliver varying levels of force feedback. To determine how best to deliver this feedback for our art generating tool, we decided to define ‘base haptic sensations’ that could be built upon later to include art-rendering capabilities:
Our Base Haptic Sensations that could be rendered via the Haply
We then conducted a round of evaluations (N=5), where our participants were our peers with previous haptics experience. Through an open-ended survey requiring users to try out the interactions and answer what they felt about the interaction emotionally and physically, we assessed which interactions could be perceived as calming, yet also interesting. Interactions 4,6 and 7 were the ones that felt the most calming and interesting, namely ‘breaking through a barrier’, ‘popping a ball’, and ‘pushing against a squishy surface’.
With these results in mind, we moved on to Stage 2, where we then thought of how these haptic interactions could be incorporated in an art-generation game-like format.
Stage 2:
We began by affinity diagramming the input and output parameters for the Haply device. This was done to gain a better understanding of the configurable parameters so we could define our scope for designing the user interface.
Affinity Diagram of the input and output parameters of the Haply
We converged on a set of four environments for the final application, three of them providing active user control of the device, and one passive. We detail each of these environments below, with ‘Guided’ being the passive interaction, while the rest are active. Each environment has a colour palette for the users to colourize their art.
1. Slingshot: An environment where users can simulate throwing paint on a wall, using the Haply end effector as a sling. A paint splatter is formed when the user lets go of the end effector, coupled with a splattering sound, for a convincing paintball explosion rendition.
The Slingshot environment with the colour palette on the right
2. Pop: An environment comprising of several varying-size colourful bubbles, that can be popped on applying force with the end-effector. Each pop creates a paint splatter as well, accompanied by sound effects.
We felt both Slingshot and Pop provided game-like environments that would be fun for users to play in, similar to games like Bubble Pop and Paper Throw, but additionally include artistic and haptic elements that enhanced the experience.
The Pop environment with the colour palette on the right
3. Squish: An environment where users can move the end effector to squish it against a bouncy surface. This provides a feeling similar to that of squishing a stressball, or slime.
The Squish environment with the colour palette on the right
4. Guided: This passive environment guides the user through a fixed route. Users can hold on to the end effector as they are guided across the device surface to create patterns.
This environment does not require the user to make any decisions which we felt would benefit users who do not want to actively play a game or so, but go through a more guided regulation mechanism.
We show the final application and the results of the final user test in the next section.
The Guided environment with the colour palette on the right
Oh…sounds good, how did that turn out?
The final application consists of a user interface controlled by and accompanying the Haply device, with four haptic environments. For each of these environments, users are provided with a colour palette, which they can choose from. There are also multiple colour palettes to encompass a variety of shades and hues. Each of these environments uses a different ‘base sensation’ for haptic feedback. Users can navigate through these environments from a central menu, and save a timestamped snapshot of the art created each time for future reflection.
The final Haptic Art-cade user interface, showing (clockwise from top left) the Paintball, Pop, Guided and Squish interactions
This video further shows the tool in use:
We evaluated the final application with 7 participants, through a survey. The survey consisted of Likert-scale questions asking participants how calming and enjoyable each environment felt, as well as how the interactions felt physically and emotionally.
All four scenes were thought to be calming as well as enjoyable to interact with, with participants saying “It felt a bit soothing, especially when looking simultaneously at the visuals”, “feel great, I love colors, and splashing color was a cool experience” and “lot of fun, enjoyed repeating the game three times”. While we were unable to explore the use of this tool long-term for the purpose of emotional regulation, our preliminary user tests show the potential of such an application.
Nice! Who else knows?
This project was completed as a course project and not pursued further. We did create a report as documentation, available here: https://drive.google.com/drive/u/0/folders/15Sb83IyhujOzLq8gvq7ArZvX7XA10hI9, as well as the demo video shown above.
Cool! Did anything get in the way?
Our main challenges with this project were:
Device-dependent user experience: Given that each Haply comprises of multiple pieces of hardware, and was assembled by each researcher themselves, each device had minute differences that led to slightly varied haptic renditions. We found rigorous internal testing and communication to be an effective mitigation strategy.
Remote testing and evaluation: Given this project was completed amidst COVID-19 restrictions and across institutions in Canada, the research team was not co-located at any point. This meant not being able to experience and troubleshoot hardware issues any team member faced, as well as having to resort to online strategies for evaluation. We overcame this by having Zoom calls for troubleshooting, and using surveys and recruiting from our social bubbles for evaluations.
Limited evaluations: Given the time and scope of the course project, we were unable to evaluate the long-term impact of Haptic Art-cade on emotional regulation.
Hmm, interesting. What’s been your biggest takeaway?
This was an incredibly fun and challenging project to take on, for someone coming from a majorly software-oriented background with minimal haptics knowledge. While my haptics knowledge certainly improved, there was another key takeaway from this experience:
The importance of other modalities, but in precise synchonization with haptics: There was a massive difference in user perception of the haptic sensations, when comparing the ‘base sensations’ to the final application. While the base sensations had no audio and visual element, the final application included carefully synchronized audio and visual elements complementing the haptic interactions which greatly improved the ‘feel’ of the interactions. Users reported how the audio and visual elements made the experience enjoyable, with improved results in the second user study on the enjoyability and calming effect of the interactions.
So, what’s gonna happen next?
This project ended here, but showed the potential of haptics in emotional regulation, through a medium of art and gaming. While we may or may not pursue this further some day, we would love to see other researchers progress in this avenue!
For now, this was it!