Case Study

Safeguarding VR: Piloting safeguarding training for the aid sector with Virtual Reality & AI.

1. Background

Conversations with survivors of exploitation and abuse can be challenging, and while traditional options to practice those conversations are available, they are often not scalable, consistent or emotionally engaging.

Safeguarding VR is an innovative training experience developed by the Humanitarian Leadership Academy in collaboration with BODYSWAPS®, an immersive learning platform, and leading subject matter experts from the sector.

The experience was conceived as an alternative for learners to practice their soft skills in a safe environment and is a practical complement to a self-guided e-learning course which covers the essentials of safeguarding. That course was made available on Kaya, the Humanitarian Leadership Academy’s eLearning platform, in November 2019.

Safeguarding VR is a 15-minute experience that combines embodied Virtual Reality and conversational AI to create an interactive and realistic scenario in which the learner is immersed and has the opportunity to practice real-world skills using their voice and body-language.

2. Audience & Objectives

Objectives:

Safeguarding VR is primarily an innovation initiative designed to pilot the use of virtual reality as a training delivery alternative in the aid sector. In precis, the objectives of the pilot were to:

  1. Demonstrate the learning performance of the virtual reality format in the context of soft skills training.
  2. Demonstrate appetite from the sector to deploy virtual reality training at scale.
  3. Explore various delivery modalities to inform a cost-efficient deployment model moving forward.
  4. Gather quantitative data and feedback to inform the content and user experience (UX) format of future experiences.

Audience:

Ultimately, the primary audience for Safeguarding VR are staff, of any level, from the humanitarian & development sector.

From a learning design perspective, the experience addresses the training needs of potential observers of safeguarding incidents or people receiving a disclosure of such incidents rather than addressing specifically managers or safeguarding focal points.

3. The Experience

Safeguarding VR is a 15-minutes real-time interactive learning experience developed to train aid workers on how to handle conversations with survivors of safeguarding incidents. It can be experienced with a range of VR headsets from standalone solutions to mobile-powered goggles, e.g. Google Cardboard.

At the beginning of the experience, the learner is tasked with listening, asking questions then providing an intervention with the survivor of a safeguarding incident. The experience allows the learner to use his or her own voice and body language to practice having a conversation with the survivor following the disclosure of said incident.

The system then uses a virtual “body swap” mechanic to allow the learner to relive the intervention from the survivor’s perspective, this gives the learner the opportunity to self-reflect on what they said and how they said it.

Finally, the system provides feedback on the behaviour, word choice and body language of the learner and makes personalised recommendations to help the learner improve their soft skills over time.

Here is a break-down of the experience:

1 – The Brief

After a disclaimer about the sensitivity of the content and a short introduction to the context of the  experience, the user reads a text panel giving them information about the conversation about to happen.

 

 

 

 

2 – Observation

The learner passively observes the 6-minute conversation between Mary and Susan.

Susan (the Survivor)  is attempting to disclose a safeguarding incident. Mary (the Observer)  is not handling the situation correctly. She’s being judgemental, self-righteous, fear-mongering, .etc.

 

 

 

3 – Assessment

The learner plays a short interactive mini-game during which he is asked to link together known bad practices with specific elements from Mary’s behavior during the conversation.

 

 

 

 

4 – Questions & Answers

The learner, now embodied as a virtual character, finds themselves face-to-face with Susan as she tries again to disclose what happened to her. The learner can ask a series of questions, each time having to make a choice between various options and getting instant feedback on whether they made the right choice.

 

 

 

 

5 – Voice Response

The learner is then tasked with providing a response to Susan, explaining why they think she should report the incident and detailing the next steps. The learner provides the response using their own voice.

 

 

 

 

 

6 – Body-swap

The learner “body-swaps” with Susan. They see & hear themselves back  from Susan’s perspective. The learner is directly impacted by what they said and identify ways for improvement, both in content and delivery.

 

 

 

 

 

7 – Analytics & tips

The learner accesses an Analytics Dashboard comprising of behavioral and semantic data and gets personalised recommendations on how to improve.

 

 

 

 

 

8 – Self-reporting survey

The learner rates the experience: their engagement and understanding level, their confidence to apply learnings in their work and their likelihood to recommend the experience to a colleague.

 

 

 

4. Technology

Safeguarding VR is a virtual reality experience which requires the use of a VR headset. To accommodate various deployment modalities and be able to reach the broadest possible number of people in the context of aid sector training, we made the experience available on a range of headsets from low-cost smartphone-compatible headsets to more large scale-ready B2B solutions like Oculus Quest for Business.

Fig 1. Overview of distribution channels

 

It is worth noting that there is a trade-off between the cost of the headset and the level of immersion achieved. Crucially, Safeguarding VR looks to engage the learner on an emotional level. This simulated presence is a function of the degree of immersion.

We are currently collecting analytics to measure the difference in self-reported engagement and performance depending on the headset used.

5. Design & Production

The Humanitarian Leadership Academy worked with VR learning provider BODYSWAPS®.

Research background

The experience builds on a growing body of academic research, led by Stanford University and the University of Barcelona, which identifies embodied virtual reality as a powerful conduit for sustained behavioral change. Key findings can be articulated as follows:

  1. VR boosts engagement and creates new ways of experiencing knowledge which improves understanding and memorability
  2. VR creates the illusion of social presence and impacts one’s implicit biases
  3. Embodied VR can foster sustained attitude change towards other groups
  4. Body-swapping in VR drives self-awareness and can improve psychological well-being

For more information, this folder contains several research papers on embodied VR and behavioral change.

Partnership

The overarching Safeguarding Essentials project, which comprises of both the self-guided eLearning course and the VR experience, was initiated by the Humanitarian Leadership Academy with the support of a network of partner organisations including Save the Children, War Child, Plan International, the UN Refugee Agency (UNHCR) and the International Labour Organization amongst others.

Learning Discovery Phase

Fig 2. Desired behaviors & obstacles exercise

The project started with a Learning discovery phase during which Virtual Reality UX designers worked closely with Learning Designers from the Humanitarian Leadership Academy and a safeguarding subject matter expert.

The primary objective of the discovery phase was to identify the behavioral outcomes the training would look to achieve and to design a unique user experience to leverage the body-swapping VR format.

Following an initial learning discovery workshop, a thorough analysis of the desired behavioral outcomes and the current obstacles to the adoption of said behaviors, the team decided to focus the experience on helping observers / bystanders to:

  1. Empathise with the survivors
  2. Practice having a conversation with a survivor of a safeguarding incident
  3. Build the confidence to support the survivor in reporting incidents to the organisation

Scripting and scoping phase

Fig 3. Early character and environment design

 

The following phase was an iterative exploration during which:

  1. The full script was written with the VR UX designers and subject matter expert collaborating closely.
  2. The user experience was fully documented in order to scope the final production.
  3. The characters and environments were agreed upon.

Production Phase

Fig 4. Performance capture ses

Fig 4. Performance capture session

 

Safeguarding VR aims to engage learners on an emotional level. Therefore, a unique production pipeline was developed to maximise the impact of the script whilst not compromising on the immersion afforded by real-time simulated VR environments (as opposed to 360 videos). In other words, we needed to make the characters of Mary (the Observer) and Susan (the Survivor) as believable and relatable as possible.

We worked with professional actors in collaboration with London-based motion-capture studio Target3D. Using the state-of-the-art facility, we were able to record the subtleties of the actors’ performance, including voice, body-movements and facial expressions.

The capture data was then cleaned and integrated into the interactive project to create the final version of Safeguarding VR.

6. Results

Measuring performance

Safeguarding VR uses behavioral, semantic and self-assessed analytics in order to:

  1. Provide the learner with personalised feedback and recommendations on how to improve
  2. Give organisations an overview of their staff’s improvement on a specific competency
  3. Inform the improvement of the learning format for future iterations

The experience provides the following analytics:

  • Evidence of understanding best practices by scoring the observational assessment exercise and choices made during the Q&A phase
  • Evidence of applying best practices during the intervention by looking a the semantic content of the user’s intervention and their body language
  • Self-reported improvement in understanding and confidence captured during the survey at the end of the experience.

The anonymised data is transmitted in real-time to a secure analytics platform. It is important to point out that individual behavioral data is not made available to employers/organisations. Instead we only offer aggregated data to protect the individual users’  anonymity.

The Pilot

The following organisations are currently piloting Safeguarding VR, with the number of pilot organisations expected to grow:

  • The UN Refugee Agency (UNHCR)
  • The Norwegian Refugee Council
  • Save the Children
  • The International Labour Organization (ITCILO)

Over 200 users have experienced Safeguarding VR. The following data has been generated from 63 survey answers (up to 12/02/2020). The full results, updated in real-time can be found here.

77% of learners reported a “good” or “great” understanding of safeguarding best practices

75% of learners reported an improvement in their confidence to handle someone disclosing an incident

89% of learners reported looking to apply what they learned in the simulation to their work.

93% of learners reported being likely to recommend the training to their colleagues.

86% of Learning Managers reported looking to roll the training further into their organisation

7. Key learnings and future planning

The feedback received highlighted the following elements:

Strength:

  • The data shows that learners felt engaged by the experience with a large majority of learners likely to recommend the experience to their colleagues (Net Promoter Score of 68)
  • The data shows that the experience greatly help learners not only understand the principles of safeguarding (77% reporting improved understanding)  but also build confidence to apply them in their work (75% reporting improved confidence)
  • The data shows that learners are overwhelmingly willing to apply what they learned during the experience to their work (89% reported they will apply learnings)

Challenges:

  • At this stage of the technology adoption, it is essential that the first contact with the experience, especially given the sensitivity of the topic, is done as part of a facilitated session. This means Safeguarding VR will require logistical adjustments from the trainers or learning providers to be efficiently embedded into a session. This includes, for example, providing a safe and quiet space for learners to go through the experience but also some time for learners to debrief their experience as a group.
  • Given enough time and resources, we would look to conduct a longitudinal study, on a bigger scale, with partner organisations to demonstrate that the training has been sedimented into work practices for the long-term..

Opportunities:

  • Partner organisations surveyed indicated a strong desire (82%) to roll out the training further into their organisation. Amongst those organisations, the UNHCR and the ILO are exploring different deployment modalities (in classroom, dedicated VR space, self-service offering…) and reporting on success to the Academy. This will help create a handbook for organisations looking to deploy immersive training at scale.
  • Qualitative feedback repeatedly identified multiple other use cases for the body-swapping format which points towards the possibility of creating a platform containing several experiences. This would greatly improve the cost-efficiency or investing into an immersive learning programmes for organisations.
Atish J Gonsalves - Global Learning Director
Atish Gonsalves, Global Innovation Director, Humanitarian Leadership Academy
Christophe Mallet, CEO, Bodyswaps
Region:
Global
Themes:
AIartificial intelligenceHumanitarianInnovationKnowledgeLearningsafeguardingSkillssoft skillsTrainingvoice recognitionVRvritual reality
Share this
Search