Abstract

This research project will explore the use of Virtual Reality (VR) technology to develop standardised methods of emotional induction and to test predictions derived from the Evaluative Space Model (Cacioppo et al. 2012) (ESM).

It is further proposed that such an embodied model of positive/negative valence states could subsequently be developed (Cacioppo and Berntson 1999) to: make predictions on behavioural action (i.e. approach vs. avoidance) within a VR environment and create an adaptive VR experience designed to enhance positivity or negativity.

This article will provide examples of the conceptual and testing phase of the first negativity study and provide up to date information on the ongoing data shaping and analysis of the data acquired from the first study, with particular emphasis on the event related analysis process.

The experiment utilised custom code written in C++ in Unreal Engine 4, the engine used to run the virtual environment, this code allowed participant movement data to be captured and logged the timings of interaction with objects in the VE.

The development of programmable and highly accurate event-based analysis of participant activity has been little studied in the field and shows promise in academia and commercial areas.

The Evaluative Space Model

The fundamental elements of emotional states as generated by positive and negative stimuli was interpreted by Cacioppo and Berntson in their Affect System as the two dimensions that influence appetitive and aversive behaviour (Cacioppo and Berntson 1999).

The process of evaluation and response within ESM is conceptualised within a three-dimensional space wherein: the X axis of the graph extends towards increased negativity, the Z axis extends back from the origin towards a greater positivity and behavioural response falls on the Y axis. The Net Predisposition axis represents the resultant behaviour that emerges from the results of the positive and negative affect system and represents a balance between approach versus avoidance behaviours.

In summary ESM can serve as a conceptual framework within which to assess emotional states in an embodied form in room scale VR. The model reconciles positive and negative responses, which can be assessed via psychophysiology and whole-body behavioural responses, such as approach and avoidance.

ESM postulates abridged

Testing

{% endraw %}

It was decided upon during the conceptual phase that a Virtual Environment (VE) would be constructed which subjected the participant to a room-scale (ambulatory) task. Initial designs focused on movement across small floating platforms.

{% endraw %}

Initial piloting showed a surprising level engagement with the task, the two principal changes established at this point were to start the user at ground level and move them to height. Contextual audio was quickly established as an effective method to convey environmental changes to the user.

Study One

{% endraw %}

One of the constraints of the HTC Vive target platform is that only three points on a users body are continuously tracked. The head and the hands holding the two controllers. The falling mechanic relied on head position as the means to detect whether a user was over a solid platform. If a user be stood over a solid area and peer over the edge they would then virtually fall over that edge.

This inherent limitation of the chosen platform would have required a complete experimental redesign. Steam trackers were then pre-ordered and after their release to developers incorporated into the experiment. They allow for additional tracking points to be measured and when, in this case, attached to a users feet allowed for highly accurate measurement of those two additional points in 3D space. The “Ice-block run” was then further developed.

• Three levels with incremental increase of threat (more cracked blocks, more fall blocks)
• Gradual increase in threat, Increasing Negativity Gradient (ESM)
• Increase in expected corrugator activity as perception of threat increases

The use of foot trackers allowed for a greater level of movement fidelity and varied mechanics within the VE. User agency was increased as they were now able to physically test each ice block with one of each of their feet. The ice blocks themselves were divided over three levels, each level grid had three types of ice-block, solid (non interactive), cracking (audibly and visually cracked when stepped on) and falling blocks (cracked when one foot stepped on falls when both feet stepped on).

The variety of ice-block types created agency without diminishing threat. Splitting the experience across three levels allowed for a longer participation experience. Due to the size constraints of the room-scale VR area each participant was directed back to their start position between levels. The threat factors (cracking, falling blocks) were also increased in number per level.

Data Analysis

The sample consisted of 32 participants (19 female). Participants were selected from an age range of 18-35 years and pre-screened for conditions such as epilepsy and any physical issues that may have inhibited their ability to move freely within the VE. The study used a mixed 2x3 design, participants completed a personality scale questionnaire using the “OCEAN” (Openness, Conscientiousness, Extroversion, Agreeableness and Neuroticism) Big Five personality traits. A shorter version of the big five was chosen for ease of translation and to facilitate efficiency in the time available for the experiment. (O’Keefe, Kelloway, and Francis 2012)

Table of physiological measures

VR provides the researcher with potent tools for the induction of emotional states. Room-scale VR can directly measure approach and avoidance responses through accurately measuring the movement and posture of participants. A combination of room-scale VR and digital embodiment allows the researcher to induce embodied emotional states in a controlled environment. The base sensors and sensors worn by the participant allow us to capture approach and avoidance movements to a high degree of precision within the VE. The researcher can also produce interactive goal driven tasks that are both repeatable and standardized as well as permitting a high degree of experimental control.

Results

Per-level Analysis

The initial phase of data analysis focused on a macro “by level” analysis of the physiological data. The levels being the three stages of the VE study which increased in height and number of threat blocks (cracking, falling blocks). After this point a micro event based analysis of per user interactions with each interactive element in the study would occur. The custom C++ Blueprint code written for Unreal Engine 4 allows this level of fidelity

It was hypothesised that markers of sympathetic activation from the autonomic nervous system (heart rate, respiration rate, SCL) will significantly increase from level one to level three of the ice block task due to the increased prevalence of ‘cracking’ ice blocks and probability of falling. In addition, markers of negative emotion (corrugator) was expected to increase from levels one to three, whereas the fEMG marker of positive emotion (zygomaticus) would show the opposite pattern of reactivity.

Participants were also divided into two groups based on their levels of trait Neuroticism as collected via the OCEAN scale. All scores of trait neuroticism were rank-ordered and the median value (23) was identified (minimum value = 10, maximum value = 38). Two participants with the median score of 23 were omitted (N=32) and all participants with a neuroticism score < 23 were ascribed to the ‘low’ neuroticism group, whereas those scoring higher than 23 were placed in a ‘high’ neuroticism group. It was hypothesised that participants in the high neuroticism group would show greater autonomic activity and experience higher levels of negative emotions due to increased threat from levels one to three of the ice block task.

All dependent variables were subjected to a 2 (high vs. low neuroticism) x 3 (levels 1-3 of ice block task) ANOVA. All analyses were conducted using SPSS v. 24. Violations of sphericity were detected via the Mauchly’s Test and corrected via the Greenhouse-Geisser adjustment. Outliers were defined as any data point lying more than three standard deviations from the cell mean and omitted from the ANOVA test.

Means and standard deviations for change scores for heart rate (beats per min) (N=27) and breathing rate (breaths per min) (N=32) across three levels of ice block task.

Change score in mean SCL across all three levels of ice block task (N=30).

The change scores for SCL were subjected to the 2 x 3 ANOVA model. Two outliers were removed from these analyses (N=30). The analysis revealed a significant effect for Level [F(2,27)=3.43, p<.05, eta squared = 0.20]. Post-hoc analyses revealed that SCL activation was significantly higher during levels 2 and 3 compared to level 1 (p<.05).

In-Level Event Analysis

In order to parse and manage the large volumes of physiological data and then reference it with respect to the event data logs created during the study a GUI was developed in Matlab AppDesigner.

Critique, Deliverables and Future Work

• Analysis of Study One data must attempt to reconcile results with movement data.
• The zygomaticus data is ambiguous in the context of this study.
• Analysing pre and post event data has proved difficult and necessitated the development of custom tools

Results

Results of this study can be viewed on a seperate post

Deliverables

• Working VE framework and knowledge gained will speed up future study design
• Custom MatLab GUI can parse and manipulate complicated time series data and allow manual analysis of complex events in psycophysiological study.
• The unique combination of VR technology and data analysis tool sets appears to have genuine originality and scientific value and to our knowledge is relatively unique in the field.

References