BEARS (VR)

BEARS is a Virtual Reality (VR) package that targets young people with bilateral Cochlear Implants to develop their spatial listening skills with both ears and improve their quality of life through gamified rehabilitation.

Cochlear Implants are electronic devices that stimulates the auditory nerve through electrodes placed in the cochlea of the inner ear, allowing some severely deaf people to perceive sounds. This is usually used in the form of medical intervention to help people to regain their hearing. The pioneer research started in UK in 1979 [1], In Year 2020/21, 433 children and 415 adults in UK received cochear implants [2]. In the past, surgeons would usually put implant on one side. As the technology matures, more and more people with hearing loss are going to receive implants for both of their ears, mostly at very young age. However, there isn't any formalised research thus no guideline to systemically support these implant receivers and their families in teaching themselves how to use CI for active listening with both ears. That's why BEARS project came into place to address the gap.

There is lots of research showing that sound localisation, as well as speech understanding in noise and music listening, can get better with training [3]. Human brain is so flexible and adaptable that it continues to develop in adults and children.

In this project, I explored the frontier of spatial audio and user experience in VR for hearing rehabilitation as a clinical intervention. My main tasks were to produce appropriate game materials, design children-friendly interactions, and outline strategies to keep players motivated using a rational level design approach. I supported both the game development team and clinical trial team in end-user research, digital assets generation, and player experience optimisation.

I embraced the iterative process of game development and participatory nature of this research/development project, which entails creating functional prototypes, collecting feedback from Patient and Public Involvement (PPI) activities, and improving interaction flows to maximise players’ satisfaction.

Using insights from our human-centred design research, I designed rich contents and improved usability of the game based on player interviews, digital playtesting, pilot trials, and extensive reviews with subject domain experts.

I proposed and implemented two new game mechanics and developed the existing ones further with contextualised interactions. I co-created various game interfaces and the narrative between the overall quest. These efforts would allow our players with delayed language perception and difficulty in hearing to understand and register the instructions embedded within our application.

Meanwhile, I communicated our team’s work effectively using design documentation, storyboards, and video demonstrations. My engagement with the general public at conferences and community outreach events extended the social impact of BEARS project.