It's immersive but is it cinema?
In the last three years I have been researching the development of interactive and immersive storytelling mediums and tech, with a specific interest in cinema. I spent time working from Digital Catapult's Immersive Lab in Newcastle, attending symposiums in Liverpool, consumer facilities in Manchester and museums in Bradford and London. Here is what I found...
Virtual Reality (VR) is here. A diverse range of VR content is being consumed, and produced through an equally diverse range of media, and has so, for multiple years. Nevertheless, primarily due to the pace at which the technology is evolving, VR lacks a fully developed language. Thus, this article contributes to a diverse range of research and textual analyses as well as the increasing methodologies, terminologies and critical frameworks for monitoring and analysing the development of VR content. This article produces a methodology which assists in the categorisation of VR content, drawing from the ludological and narratological work of game studies and William Uricchio’s experienced and authored storytelling to contribute to this growing VR language. This is a language that allows consumers, creators, academics and journalists to more effectively discuss, document and categorise the current iteration of VR content.
In the 1990s, after years of development in the fringes of technological sub-cultures, VR experienced a certain cultural peak. (LaValle, 2016). VR headsets started to appear in arcades such as a short Aladdin (Clement, Musker, 1992) experience in Disney’s Quest arcade and in motion pictures such as Lawnmower man (Leonard, 1992). Hype and expectation followed, created by the promise of complete sensory immersion, alas, the technology that was needed to code visually realistic and responsive 360-degree virtual environments, simply was not ready, and VR was confined to the outskirts of technological development (LaValle, 2016). Finally, however, in 2012 Palmer Lucky invented the Oculus Rift, an innovative VR headset, far smaller than anything which had preceded it and offering a far superior pixel resolution. In the eight years since, HTC, Samsung, Google and Microsoft would all release respective VR or AR (augmented reality) headsets, and a new wave of VR’s cultural significance and content consumption would begin (LaValle, 2016). In-home headsets from some of the aforementioned distributors were purchased in their millions (Liu, 2019). Moreover, VR content started to be consumed out-of-home at the Void, IMAX VR, arcades, film festivals, theme parks and conventions.
Nevertheless, creators, academics, consumers, and journalists are still limited in their ability to fully discuss, categorise and communicate about this new media. Despite a growing VR studies and an increasing cross-discipline interest in VR, it is still particularly difficult to - using an appropriate terminology - distinguish between VR texts. As a result of VR’s youth and ever-evolving role in popular culture, social commentators seem limited in their ability to fully comprehend the primary textual and technical differences which separate VR texts.
Theorists such as Steven Aukstakalnis, (Aukstakalnis, 2016) critics such as Mark Kermode (Kermode, 2018) and creators such as Alejandro González Iñárritu (Roxborough, 2017) together embody a popular cinephilic perception that for various textual reasons, VR is not cinema. Following this, will be a discussion of the ways in which VR creators have adopted or apposed a videogame form, preferring sometimes for a narrative format, inspired by theories of VR as an ‘empathy machine.’ (Milk, 2018). This is the theory that VR as a medium possesses an inherent quality to induce hyper-immersive experiences and thus create increased levels of empathy. (Milk, 2018). This will be followed by the creation of a methodology which will assist in the placement of VR content within the spectrum of these two media. The article builds primarily on the work of narratology and ludology studies drawing from the research of Gonzalo Frasca and Janet Murray as well as William Uricchio’s (Uricchio, 2018) authored and experienced storytelling to produce a methodology, which will ideally assist consumers, creators, academics and marketing consultants to more effectively understand how the content is consumed, and which pre-existing media it will most likely be aligned to. The methodology suggests that there are four possible components in a VR text, the presence or absence of which will resultingly effect the extent to which the content is experiential/ludological and, will thus, share many characteristics with the videogame format or authored/narratological, and will therefore share more characteristics with narrative cinema. The methodology focuses on the analysis of primarily four VR texts, Dear Angelica (Unseld, 2017), TreeHugger: Wawona (Steel, Ersin, McNicholas, 2016), Super Hot (Bączyński, T. Kaczmarczyk, 2016) and Clouds over Sidra (Arora, 2015). The analysis does draw from other texts where appropriate, but these are the texts which form the basis of the analysis and were selected for their ability to represent the diversity of VR content, and diverse range of the possible characteristics highlighted by the methodology.
VR studies as a field of research is still young and evolving at an equally fast rate as the technology which it reports on. Therefore, to fully comprehend the textual and technical differences between texts, it was helpful to draw from three distinct areas of research. As well as VR studies, the methodology utilises work on authorship and environmental immersion.
The empirical work in my article consisted of primarily textual analyses, supplemented by participant observation. I carried out textual analyses on approximately 115 VR and 360-degree texts. These were entertainment-orientated texts, which offered various levels of interactivity, and various levels of system agency. They were consumed on the HTC Vive, Oculus Rift, Oculus Quest and through a google cardboard. I then documented any and all technical and textual moments when I felt - from an autoethnographic perspective - immersed or removed from the world of the content. I also documented any features that felt native or foreign to film or videogames. The proposed methodology was also informed by extensive participant observations and informal interviews carried out between 2018 and 2020.
I attended an Immersive Storytelling research Symposium in 2018 where I conversed with creators, engineers and academics, whilst also viewing various tech demonstrations. Furthermore, I worked from Creative XR’s Northern Immersive Lab where I trialled a range of content on the Oculus rift, HTC Vive and HoloLens, as well as observing research talks and touring their facilities. I visited consumer-accessible facilities such as Virtual Hideout and trialled Stereoscopes and other immersive content at the Royal Albert Museum and Bradford Media Museum. Finally, I purchased a Google Cardboard and viewed a range of 360-degree/VR content, as well as 2D (two dimensional) CYOA (Choose Your Own Adventure) material remotely. I took field notes and viewed such a vast amount of VR content that significant differences between the available content started to become apparent. I thus used these differences as the basis for the proposed methodology.
VR as Cinema
Director Alejandro González Iñárritu has been particularly vocal in his shunning of CVR as film. CVR is a term used sparingly in existing research to refer to narrative-based VR or VR with innately cinematic characteristics, of which this essay attempts to categorize. Whilst Iñárritu has worked successfully in VR with his narrative VR text Carne Y Arena (Iñárritu, 2017), he is adamant, as are many others, such as Guardian critic Mark Kermode, that VR and film are inherently distinct from one another. Iñárritu claims that ‘Cinema is frame, cinema is length of lens, cinema is editing, the position of images that create time and place’, ‘virtual reality, is exactly all what cinema is not’ (Roxborough, 2017). Kermode also proclaims VR as a new medium, in addition to many of Iñárritu’s issues, referring also to the significance of his preference for ‘strong narratives’. (Kermode, 2018). None of these particular issues alone should discount VR from being considered film. Detailed and rich narratives can, and do exist in VR, wide angle lenses exist and are slowly being introduced which can create a 360-degree view, as well as focal range, and the concept of cinema as what exists within the frame has been problematic ever since the move from celluloid to digital. As D.N Rodowick exclaims ‘an image recorded digitally no longer creates causality or photo-realist contingency, it creates only illusion.’ (Gaudreult, Marion, 2015). It is the perspective of Rodowick, as well as Paolo Cherchi Ursai and Francois Amy de la Breteque amongst others which problematize Kermode and Iñárritu’s conclusions. Although VR is more than ever a marriage between engineers and artists, with so much more of the heavy lifting done by - and creativity lost to - the ones and zeros, cinema as a concept is far less binary and temporally static than either Kermode or Iñárritu acknowledge (Doyle, 2018).
The indescribable element which they are searching for however, and what omits VR from being neatly placed into cinematic sub-categories, is the language of cinema. (Aukstakalnis, 2016). It is 130 years of cinematic history, and the sociological, technological and cultural contract between an individual’s previous contextual knowledge, and their understanding of cinema. Steven Aukstakalnis builds upon this notion, affirming that ‘changing the very medium, whether it is simply surrounding the viewer with scene action or even allowing some form of interaction, directly impacts the art and science underpinning the language.’ (Aukstakalnis, 2016). Whilst it is true that incorporating VR content into film categories would impact the language of cinema, there is nothing about the 360-degree view alone that should omit VR content from being considered a film or being incorporated under the umbrella term cinema. Every cinematic term which may have been regarded as film law has been broken and reinstated time and time again, the concept of film is fluid, incorporating technological and sociological developments.
Nevertheless, as mentioned, various technical components have been utilized to exclude VR from the gates of cinema and obviously none more so than the 360-degree view. Despite technological and sociological developments, feature length and short form motion pictures have almost entirely existed within a rectangular frame, somewhere in-between an academy ratio of 1:39 and an ultra-wide screen ratio of 21:9 (LaValle, 2016). The frame is the chosen shape of the majority of art created within the last millennia. It is the shape of the Mona Lisa, the shape of the Bible and the formatting shape for Citizen Kane (Welles, 1942). The rectangle has been quite possibly one of the few constants throughout cinematic history, and any divergence from that, concerns individuals interested in or working in the film industry. Theorist Leon Alberti saw the frame in painting ‘as a window into the world’. (Friedberg, 2019). Consequently, to look through the window is to observe the painting or the film, meaning that when a viewer consumes a VR text, they step through the window into the world and instead experience it. This is then, perhaps a further reason why so many cinephiles and film enthusiasts have some indescribable issue with VR, and more imperatively the 360-degree field of view which it offers.
VR as a Videogame
It is the industrial convergence between VR and other media that has led to a diverse range of distribution methods, with out-of-home originating from cinema and in-home from the videogame industry, the success, as the article argues, currently lies with and in the future will almost certainly be occupied by, the latter (LaValle, 2016). Consequently, it may seem that VR is actually a natural successor or divergent of the videogame industry, nevertheless it appears the sheer diversity and immersive potential of VR is what omits VR content from being solely incorporated into the videogame industry, and potentially any other such singular medium or industry.
It is this immersive potential that has cinephilic commentators such as leading VR creator and enthusiast Chris Milk to argue that ‘Virtual reality is not a videogame perifocal’. (Milk, 2018). It is Milk’s view that VR possesses an innate ability to become an ‘empathy machine’ and should be utilised to create genuine humanist connections between viewer and subject, and thus, its use as a less poignant technology simply to entertain would be wasted potential. (Milk, 2018). This is a relatively popular perspective especially when VR is used for factual content.
In Racing the King Tide (Chadwick, 2018), the 360-degree camera is placed at various locations on an island in the Philippines, which is undergoing hard times due to consistent flooding. In a particularly harrowing moment, the camera is placed at the back of a classroom which is half-submerged underwater. Halfway through the shot, the rest of the child protagonists turn to look at you (the audience), and you feel completely and clearly empathetic for their struggle, and here is why. In 2019 Ni Ding et. al. recorded an increased level of immersion between a scene from Disney’s Jungle Book (Favreau, 2016) when watched in VR compared to a 2D viewing, measuring cognitive emotional feedback using ECG recorders (Ding Et al, 2019). Ding et. al. listed four reasons; the 360-degree field of view, the first-person perspective, how new the media is and the sense of place-taking (Ding Et al, 2019). Consequently, it is the increased sense of place-taking which VR offers, and its concluding link to a heightened empathy as sustained by Mark Davis (Davis, 1980), which results in a scene where at one single moment the struggle of the film’s subjects instantly feel like a personal struggle, or at least one which is not abstracted by spatial distance.
Nevertheless, when it comes to VR and its ability to induce empathy, there is not the consensus that Milk may desire. Helen Kennedy and Sarah Atkinson - whilst stating clearly that there are limitations to this relationship - are largely in agreement that VR’s ability to induce empathy is promising (Kennedy, Atkinson, 2020). Whereas, Si Mitchell is doubtful of these ‘empathy machine’ claims and - perhaps more importantly - shares concerns over whether the pursuit of inducing empathy should be a goal in the first place (Mitchell, ). He is partly building on the disappointment many key figures in VR such as Janet Murray and William Uricchio feel in the pursuit of VR creators to prioritise narrative and empathy inducing content (Mitchell, 2020).
Nevertheless, Milk may well be correct that VR does inherently possess the ability to convey a heightened level of empathy to its users, but that does not mean that it should necessarily be solely used to do so. A complete variety of VR texts exist, offering narrative-fiction, narrative-factual and ludological experiences. Creators of VR narrative fiction or ludological experiences are, much to the disappointment of creators such as Milk, using VR’s immersive potential to entertain rather than inform or induce empathy. It is unrealistic to expect such a versatile medium to be used in a singular way. Therefore, reductive affirmations which force VR content into a singular medium, and which demand that it be used in a specific manner, or omit it from some other pre-existing medium, are limited in their ability to fully account for the diversity and versatility of VR, both in potential and present existence.
The VR Methodology
As concluded then, VR content often doesn’t neatly fit into pre-existing mediums. Consequently, this paper offers a methodology that will ideally assist individuals in the placement of VR content into existing categories or at least assist them in more greatly understanding such content. The methodology builds on the work of William Uricchio (Uricchio, 2018) and to a greater extent the narratological/ludlogical debates of game studies (Frasca, 2004). William Uricchio listed three types of storytelling, authored, experienced and algorithmic. (Uricchio, 2018). Authored storytelling is where the narrative is predetermined, and the viewer has no ability to alter it. Experienced storytelling is where the viewer does not have the narrative delivered to them but is in fact an active part of its production. Algorithmic storytelling refers to a narrative that responds and learns from a user’s inputs. Whilst algorithmic or AI storytelling may well define how we view VR content and perhaps the content of many other media in the future, presently, it is with experienced and authored storytelling where this article’s concerns lie.
Similarly, in games studies, like Uriccho’s authored storytelling, narratological games allow users to consume a pre-planned narrative (Murray, 1997). Whereas ludological refers to the structuring of videogame content around rule making, allowing users relative agency within the boundaries of these rules, resulting in a much more interactive experience between user and text, similar to Uriccho’s experienced storytelling. The methodology thus lists four possible features of a text, the presence or absence of which will determine how closely the content will likely be aligned with a narratological/authored format more present in cinematic content, or ludological/experienced components which are more likely to be present in a videogame format. Consequently, in turn, allowing individuals of all relative disciplines to more suitably understand, discuss, and categorise the VR text in question, contributing to the language of VR.
The initial, and primary factor is whether, and the extent to which users can alter the course of the narrative. Due to the - as discussed - heightened immersivity and autonomy of VR, narrative alteration is often available, narrative alteration however, likely moves the content further away from an authored/narratological text, instead towards a ludological videogame format. The second component is whether the viewer of the content is made aware of their existence within the narrative, and the extent of this existence thereafter. To exist within the narrative is to an extent to experience the narrative, as opposed to watching the story unfold from an external position. The third factor is the varying ‘degrees of freedom’ (DoF) that the content and its associated headsets or exhibiting technology offers. The industry term ‘degrees of freedom’, which will be discussed in more detail, refers to the amount, and variety of movement offered within the virtual environment. (Bouw, 2016). The greater the movement offered, the closer the experience is to how humans experience life, and thus, closer to experienced storytelling in contrast to the standard static authored consumption of 2D film. The final component is the number of senses that are stimulated. Cinema has almost always been a duo-sensory form of entertainment allowing visual and audible sensory stimulation (Cook, 2007). However, because of the culture of innovation and experimentation that exists within VR development, as well as the increasingly popular experiential and immersive content type, creators are increasingly interested in involving some of the other five main senses, as will be discussed.
This is not a prescriptive process that allows marketing consultants to know precisely how to market their content, nor will it allow consumers to confirm and communicate indefinitely the groups of content or mediums that the particular VR text falls into. It will, however, allow consumers, academics and creators to more effectively contextualise the ways in which VR content will be consumed, defined and understood. For instance, a creator or producer of VR content might learn through this process that their three DoF duo-sensory content which offers no narrative interaction and does not place the consumer within the narrative is, but for the 360-degree perspective, almost identical to the primary expectations of a cinematic text and would be suitably marketed as so. Less imperatively, they may just understand and learn more about their content having understood more effectively the characteristics that impact and alter VR consumption. Alternatively, this method of analysis, when applied academically will help monitor the development of VR as a medium, whilst also enabling the documentation of changes in content type based on region, technology, genre and other subcategories. Most significantly, however, the analysis allows entry level VR enthusiasts to understand more effectively the content they consume, and for professionals and academics to more succinctly dictate and apply a coherent language and terminology when discussing VR content. The intention is perhaps summarised more suitably by Stolz, Kennedy and Atkinson in their 2020 report, when they sate, if storytelling is to thrive in this virtual age, individuals must understand what the world is ‘what the boundaries are and how to tell the best story tailored to the platform.’ (Stolz et. al. 2020).
All four of the factors impact where the content can be placed on a spectrum, where one end is experienced/ludological content and the opposite authored/narratological content. However, perhaps no factor moves the content closer towards the experienced/ludological end of the spectrum than the ability to alter or interact with the narrative. User narrative alteration or impact can be achieved in many different ways, the most obvious would be to allow the participant to select with a joy stick certain narrative options, but, can nevertheless be as passive as selecting which response to give to an AI character, similar to in CYOA content and walking simulators. Alternatively, it can be as active as having a regular and open impact on the content such as is offered by the popular VR game Beat Saber (Beat Games, 2019). Beat Saber allows viewers to move virtual light sabres (which are linked to physical handheld controls) in various directions and motions in accordance with instructions and cues in the VR headset, to the tune of different songs. The user’s actions almost constantly dictate the nature of the resulting content, for this reason, amongst others (existing within the narrative, and being afforded with six DoF), Beat Saber, Super Hot and much of VR content would likely be placed on the experienced or videogame end of the aforementioned spectrum.
However, being able to impact or alter the narrative can be more covert. This is the point where this factor can become slightly more problematic, because, in VR, the viewer has the ability to direct their visual attention to within a 360-degree, or 180-degree area. Nevertheless, this is not an issue unique to VR, a viewer placed in front of a 22 metre curved IMAX screen has a similar level of autonomy as to where they direct their gaze, and even viewers of standard 2D film have a base level of agency over what part of the frame they view (Mateer, 2017). Whilst of course this issue is heightened in VR, creators are starting to develop the necessary attention controlling devices, in the shape of AI characters acting as virtual tour guides and binaural audio cues to direct attention. (Jerald, 2015). However, limiting the delineation from an authored narrative and offering the ability to alter and impact an experienced narrative are quite different. For instance, in Clouds over Sidra (Arora, 2015), the story of a Syrian refugee, at no point can you choose where you move, what subjects you directly interact with or what content you consume. Yes, you can choose which area of the 360-degree frame to observe, but attention guiding devices ensure, with relative success, that you consume the single narrative that Gabo Arora intended. This is not a democratic viewing process, whilst you have a greater sense of autonomy, you still primarily consume a pre-planned narrative just as the creators had intended.
In BBC’s Bear Island (Jones, 2017), you have a limited ability to alter the narrative. For example, at a certain point in the content after listening to some information about a particular bear, you are given the option to select either the ‘move up stream’ or ‘move down stream’ options, effecting undoubtedly how the narrative will play out henceforth. This is not as radical as having a constant and open ability to interact with the text, but it is still significantly more participatory than Clouds over Sidra, or other such VR films. This, I would describe as an ‘interactive narrative text’, where you are watching an authored story with a limited and structured ability to interact. The majority of VR content, however, is at the extreme other end of the spectrum, offering a complete ludological videogame format where participants have a much greater control over the narrative, and can almost constantly dictate how and when the narrative will progress (Frasca, p. 223). An example of this is the VR game Star Wars: Droid Repair Bay (ILMxLAB, 2017), where users are carrying out maintenance work in a repair bay for droids, set on an aircraft in the Star Wars cinematic universe. The content provides hints of what is expected of users and then provides a sizeable location to manoeuvre around and complete these tasks, whilst also offering enough autonomy for the user to entertain themselves by interacting with other elements around them. Only occasionally are you passively watching a story unfold, during the majority of the content you are actively participating and contributing to the progression of the narrative, you are in fact an integral part of this progression.
An audience’s inability to alter cinematic narratives is one of the primary ways in which we define cinema. Although there has been CYOA films, and films with interactive elements, largely, throughout film history, the vast majority of films are observed passively by viewers, that is not to say that audiences are not able to interact with or be immersed in a cinematic text, but rather that they cannot alter the course of the narrative. Currently, the vast majority of VR content has at least some viewer participation, with the majority offering a complete videogame style/ludological experience (Frasca, 2004). The closed-off, singular experience of VR and its increased level of immersion results in a medium which is tailored to offering experiential content. Nevertheless, it is problematic to propose whether in time all VR content will start to offer some audience interaction, or whether the portion of content that does not offer any active narrative alteration will solidify, and a VR film culture and market will materialise. However, currently, when considering how VR content may be categorised and discussed, the most obvious way content will be viewed and communicated as something familiar to film, something familiar to the videogame format, or something entirely unique, is likely to be whether the content allows the ability to alter the narrative, and the extent to which it does.
A similar consideration as to whether the viewer has the ability to alter or interact with the narrative, is whether the viewer exists within the narrative. Throughout film history there has been devices of cinematic self-consciousness, from as covert as POV (point of view) shots or protagonists looking straight down the barrel of the lens, to as overt as characters directly addressing the viewer or referencing the film that they are in (Cook, 2007). These were made increasingly popular in counter cinematic movements of the 60s such as the French New Wave and the New Hollywood period (Cook, 2007). They are, however, used sparingly within traditional 2D film, the very nature of their radicalism is suggestive of the emphasis film culture places on the passivity of film viewing. Existing within a narrative ultimately creates content where the viewer is closer to experiencing, rather than witnessing or passively consuming. Consequently, content without narrative existence will often be closer to widely held definitions of film, and content with narrative existence closer to videogame formats. As with the previous factor there is an initial binate question of the viewer existing or not existing within the narrative, however, if the viewer does exist in the narrative a spectrum exists thereafter.
In Abe VR (Jelley, 2016), the viewer is bed bound in a dystopian hospital ward as a nihilistic robot directs his entire monologue to the viewer and at the end starts to operate on them, the viewer watches the whole interaction from a POV angle shot. The viewer is thus integral to the narrative, and the narrative could not exist without them; they are an active part of it. Of course, as mentioned, in 2D film, POV shots existed frequently and some dialogue might even be directed to the audience, but rarely do either take place during the entirety of the picture, and it would be problematic for the content to attempt to simulate the audience being attacked by any character. Almost all VR content that offers a ludological experience ensures that the viewer exists in the narrative, but Abe VR is an example of the many narrative-based VR content which also allow this. Once again, some of the content is more covert in their inclusion of the viewer in the narrative. For example, in Racing the King Tide the viewer is not an integral part of the narrative, but as mentioned, in an attempt to induce empathy by simulating place-taking the creators direct the subjects of the narrative to look to the camera in a particularly poignant scene. Consequently, the breaking of the fourth wall here does momentarily place the viewer within the narrative, as the subject acknowledges the viewers existence, but no more so than is often seen in a 2D film of a similar genre or tone. Therefore, this element alone would not likely limit the content from being included in definitions of or discussed alongside narrative 2D film.
Dear Angelica (Unseld, 2016) is a short piece of VR content where the viewer does not in any way exist within the narrative, whilst you can move around the world in six DoF, you watch the authored narrative of Angelica dealing with the loss of her mother from an external narrative position. Dear Angelica, however, is an example of content that, whilst it does fall on the narratological/authored end of the spectrum, it does not feel from an autoethnographical perspective relative to a traditional 2D film viewing experience. Rather than attempting to compose and direct this 3D space by directing attention with visual and audible cues, the creators surround the viewer with text, colours, shapes and images, that, wherever the viewer looks conveys the creators emotional and contextual intentions. (Robertson, 2017). This is an example of VR content which is entirely unique. The content utilises the increased sense of immersion and place taking, whilst also ensuring viewers consume an authored narrative from an external perspective. Due to the increased sense of immersion, place taking and the singular experience of VR, it is sensible to assume content which places viewers in the narrative will also increase their already-existing majority within VR over the coming years. What is not clear though, is what will happen with content such as Dear Angelica, which is clearly different from 2D cinematic storytelling, and different, even more so from experiental/ludological videogame formats. What will become apparent is whether this particular type of storytelling will develop and grow into its own medium, or whether this is how CVR can exist and thrive in VR.
Degrees of Freedom
Degrees of freedom refers to the amount of movement a particular headset or exhibiting technology and/or piece of content offers. Three DoF means that an individual can look right and left, up and down and on an axis (Bouw, 2016). Six DoF - which is what high-end and almost all new VR headsets offer- refers to the above parameters as well as the ability to move forward and back, left and right and up and down. (Bouw, 2016). Most often these extra set of parameters are limited to a certain amount of space, limited quite simply by walls, furniture or tethered lead lengths, but can be extended with larger surface areas, untethered headsets and specially designed treadmills or other similar rigs which mimic a larger surface area.
This particular factor builds on the work of Slater et. al. who found that the ability to walk in a way that mimicked reality, as opposed to selecting destinations with a remote, increased a sense of presence, and thus, resembled something closer to reality (Slater et. al., 1995). Resultingly, as human beings almost always consume all sensory information with at least six DoF, meaning they can move freely during every interaction and event, content that provides a greater number of DoF will, consequently, provide an exhibiting process that feels experiential. Whereas content that is limited to three DoF places the audience in a static position, where they cannot roam around their environment as is the manner through which 2D film is consumed.
Narrative VR text Fresh Out (Wey & Tao, 2018) limits viewers to three DoF, resultingly, you can look fully around the 360-degree space, with binaural audio cues ensuring that the main content is observed. However, as you are embodying a carrot in the film it would be entirely futile to offer six DoF and would limit the emotional impact of the final act of the narrative when the antagonist seizes you helpless from the ground. Whereas in content such as Super Hot VR (Bączyński & Kaczmarczyk, 2016), the narrative is dependent on your movement around the 360-degree space. The virtual protagonist’s defence and offence against the oncoming antagonists speeds up and slows down depending on the movement of the participant. Consequently, this fundamentally human capability to move freely in every direction leads to content that feels more experiential and thus closer to the ludological/videogame end of the spectrum.
In the previous years and in fact decades, VR has progressed to involve more frequently six DoF and it should be expected that content, in the future will start including an even greater range of movement, such as climbing stairs, and negotiating a range of obstacles. The ability for viewers to move freely around the virtual environment will inevitably introduce further attention retaining issues for creators of VR content, or perhaps even lead to a greater number of content similar to Dear Angelica which takes an abstract approach to attention retention. Nevertheless, although the ability to move around the narrative does lead to a more experiential content consumption, it is far less significant in achieving this than the previous two contextual issues and should not solely omit VR content from discussions or conversations of what cinema encompasses.
The final factor to consider is also technical, it is the number of senses that the content directly stimulates. Until 1927 motion pictures did not include audible dialogue, during the 90 years since they have stimulated both audio and visual senses (Cook, 2007). There was, however, a brief experimentation with smell, with Sensorama and other 4D theme park attractions. Moreover, there has also been infrequent experimentations with touch or haptic feedback with DMAX and other 4D attractions, and the argument could also be made that reverb from speakers and sound systems does in a sense create a touch sensation which is in accordance with the content (Recuber, 2007). Furthermore, certain tastes and smells can almost certainly be associated with the cinema, though they are not created or stimulated by the content itself (Hediger, Schneider, 2005). Nevertheless, in summary, experimentations with touch, taste and smell are in the extreme minority, with the overwhelming majority of cinematic content stimulating solely audible and visual sensations.
Hunter Hoffman et. al. found that involving the other senses, in this case touch and taste resulted in an increase in the realism of virtual environments, consequently, the more senses which are incorporated, the closer the content is to feeling experiential (Hoffman et. al. 1998). Just as humans experience life with the freedom of movement in all directions, they too experience life through all of the five standard senses. Therefore, to experience stimuli, and content which move away from films duo-sensory genetic make-up towards the penta-sensory manner in which humans exist, is to consume content in a way that is likely to be increasingly experiential.
To affirm, the majority of VR content does solely stimulate audible and visual sensation, consequently, three of the four primary texts, (Super Hot, Dear Angelica, Clouds over Sidra) do not stimulate anything more than aural and visual senses. However, over recent years, more and more creators of VR content are attempting to capitalise on these hyper-immersive experiences by stimulating more senses. Production companies such as Marshmallow Laser Project are beginning to push the sensory boundaries of VR as a medium (McNicholas & Jowers, 2018). In TreeHugger: Wawona (Steel, Han Ersin, McNicholas, 2016), viewers were able to smell a damp, musky smell when the virtual protagonist was in the undergrowth of a forest, whereas when the camera tracks up to the canopy the smell slowly becomes fresher.
Moreover, haptic suits, haptic gloves and other haptic adornments which replicate touch sensation are also becoming increasingly popular (Hurst, 2013). In summary, in an experience when a virtual antagonist punches the viewer, they would feel some touch stimulation in the same area as where the punch landed on the virtual protagonist. Birdly is an experience which combines many innovative factors to create an immersive sensory experience (Birdlyvr, 2019). Participants are reclined and harnessed into a horizontal lying rig that is designed to simulate the general disposition of a large bird. Viewers can lean forward, back, left, right and flap their ‘wings’ at varying paces in doing so impacting the speed and direction of the virtual bird, which the viewer is embodying. During this process reactive fans mounted to the rig blow at varying rates depending on the bird’s simulated speed and trajectory. Furthermore, some more niche experiments are also underway using small electro cardio-graph attachments to simulate chewing and taste (Ranasinghe et. al. 2013).
Of course, taste, touch and smell can be harnessed far less easily than visual, and audible sensory data, due to the fact that presently the digitization of chemical data is yet to be made possible. (Spence et. al. 2017). Nevertheless, a technology that offers such unique and immersive experiences inspires innovation. Thus, recent trends would suggest we are likely to see an increasing amount of senses being stimulated in creative ways in VR. Moreover, VR creators have not just sought to involve the other senses but also develop and adapt how audio and visual senses are stimulated. Obviously, the 360-degree view immerses the viewer, but binaural audio created through a spatial audio engine which operates as if in a physical space responds to and immerses the viewer further still. (Haywood, 2018). As with DoF, I do not believe that increased sensory information negates VR from inclusion in film definitions, but it does, in accordance with the other factors, likely move the content further away from that which is understood and discussed as film and towards something experiential.
A great deal of attention, when discussing VR and cinema, is actually dedicated to virtual apps such as Irusu which allow individuals to watch content in a virtual cinema. Many apps in fact allow interfaces where viewers can watch 2D content in these virtual spaces. It is hard to see how this will be successfully grown, when the cinema itself is built on event-isation and VOD (video on demand) on convenience (Cook, 2007). Alternatively, many commentators and academics such as Ralph Schroeder believe the future of VR will be as a one stop virtual Hub, similar to the iPhone or android where entertainment, work and every day activities are carried out in a VR, AR or MR (Mixed reality) space (Schroeder, 1996). This may well be a potential home for virtual cinemas, however, it is near-sighted to expect narrative content not to develop and find some sort of consistency and format in such a versatile and promising medium.
Researchers such as Eryn Parker and Michael Saker have started to ask some interesting questions around the spatial considerations of VR from a museological perspective, however, I believe, as do they, that there are some other useful questions that need exploring, such as, how does the exhibition technology itself impact the consumption of the content? (Parker, Saker, 2020). How does the surrounding location impact the consumption of the content? What is the prominence of environmental priming on viewers of VR content and how can environmental objects and factors be combined with virtual environments and experiences to create more immersive and layered content?
Moreover, just as it is necessary for an increase in textual analyses, it is also fundamental that we see an increase in autoethnographic, and audience led studies. Participant observation is perhaps the most common method of choice for VR-orientated research, but such is the pace at which virtual reality is evolving that even these could see to increase. VR, perhaps more than any other medium, invites the audience behind the curtain, thanks to CYOA and AI-authored storytelling and a general increase in agency, viewers have never had so much autonomy. Consequently, methodologies such as participant study-led research that fully acknowledges the participants significant role in the creation of VR texts are fundamental to understanding the medium and this relationship further. Similarly, as a result of VR’s closed off singular exhibition and ethereal reputation in popular culture, autoethnographical approaches must play an important role in the progression of VR studies.
However, what must remain of primary importance and is the single purpose of this study, is to contribute to the gradual development of a VR language. In summary, to, in time, create a terminology, a method of communicating, a culture with its accompanying capital, a documentation, an archive and a categorisation of all that exists within VR currently. We cannot idealise, and wait until VR stops, stagnates or finally resembles something familiar. VR possesses by the very fabric of its DNA something truly original, overwhelmingly immersive and perhaps more versatile than any medium before it, but without a language, humans – without whom the technology is utterly worthless – cannot effectively communicate with one another about what VR, and its accompanying content is.
The analysis thus contributes ever so slightly to this struggle by helping viewers to consider broadly whether the content is primarily narratological/authored and thus, likely aligned with a cinematic form or ludological/experienced and resultingly, more likely aligned with a videogame form. It is the intention that the methodologies creation and existence is useful in contributing to this critical framework, for creators, innovators, commentators and VR consumers to question the aforementioned key areas of the content. Whilst the four factors are separate and refer to different components of the content, they share the commonality of separating the texts as experiential/ludological or authored/narratological. This is the primary delineation, and the most important area to consider, after this consideration the other factors show the micro-elements that can all be separated by this primary question. It is the hope of this paper, that it may act as a rallying call for an increase in research across a spectrum of disciplines and methodologies and contribute to the slow introduction of a VR language. A language that consists of numerous methodologies and terminologies that enable this emerging media to be understood and encourages any whom might share any interest in VR to communicate about the technology and its accompanying content in a more cohesive manner.
Aukstakalnis, S. (2016). Practical Augmented Reality: A guide to the technologies,
applications, and human factors for AR and VR. 1st ed. Massachusetts: Addison-Wesley
Birdlyvr.com. (2019). Birdly. [online] Available at: http://birdlyvr.com/. [Accessed 8 Sep. 2019].
Bouw, K. (2016). The Comprehensive Guide to Getting Started in Augmented Reality and
Virtual Reality. [online] Medium. Available at: https://medium.com/@KristianBouw/thecomprehensive-
guide-to-getting-started-in-virtual-reality-c6a6419cf8cf [Accessed 7 Oct. 2019].
Chadwick, C. (2018). Racing the king tide; using immersive storytelling to highlight the real impacts of sea-level rise on small island communities in the Philippines. Presented at the Immersive storytelling symposium: Liverpool.
Cook, P. (2007). The Cinema Book. British Film Institute: London.
Davis, M. (1980). A multidimensional approach to individual differences in empathy. 1st ed. Texas: University of Texas.
Digital Catapult (2018). Immersive Making the UK the best place in the world to create immersive content and applications. [online] Digital Catapult. Available at: https://www.digicatapult.org.uk/technologies/immersive/. [Accessed 11 Nov. 2019].
Ding, N., Zhou, W. and Fung, A. (2019). Emotional effect of cinematic VR compared with traditional 2D film. Telematics and Informatics, 35(6), pp.441-460.
Dooley, K. (2019). A question of proximity: exploring a new screen grammar for 360-degree cinematic virtual reality. Media Practice and Education, 19(2).
Doyle, R. (2018). Making a New Reality: Ruthie Doyle - Alternate Realties Summit. [video] Available at: https://www.youtube.com/watch?v=vSeMIv_eDk4&index=4&list=PLhKbY6Z_pXkHFfLPMr13KT2SC1vIdML8D. [Accessed 15 Jul. 2019].
Frasca, G. (2004). Simulation Versus Narrative: Introduction To Ludology. In: M. Wolf and B. Perron, ed., The Video Game Theory Reader, 1st ed. New York: Routledge, pp.243-258.
Friedberg, A. (2019). The Virtual Window: From Alberti to Microsoft. Journal of Aesthetics and Art Criticism, 67(3), pp.351-353.
Gaudreault, A. and Marion, P. (2015). The End of Cinema?: A Medium in Crisis in the Digital Age. 1st ed. Columbia: Columbia University Press, pp.05-16.
Golding, D. (2020). Far from Paradise: The Body, the Apparatus and the Image of Contemporary Virtual Reality. Convergence, 25(2), pp.340-353.
Haywood, G. (2018, December). Audio in Immersive experiences. Presented at the Immersive storytelling symposium, Liverpool.
Hediger, V. and Schneider, A. (2005). The Deferral of Smell: Cinema, Modernity, and the Reconfiguration of the Olfactory Experience. In: V. Re, ed., The Five Senses of Cinema, 1st ed. Udine: Forum, pp.241-264.
Hoffman, H.G., Hollander, A., Schroder, K. et al. (1998). Physically touching and tasting virtual objects enhances the realism of virtual experiences. Virtual Reality, [online] 3, pp.226–234. Available at:
https://link.springer.com/article/10.1007/BF01408703 [Accessed 23 Dec. 2019]
Hurst, N. (2013). Can You Feel Me Now? The Sensational Rise of Haptic Interfaces. [online] Wired. Available at: https://www.wired.com/2013/02/haptics/ [Accessed 11 Oct. 2019].
Jerald, J. (2019). The VR book: Human-centred design for virtual reality. 1st ed. California: Morgan & Claypool.
Kennedy, H. and Atkinson, S. (2018). Virtual Humanity: Empathy, Embodiment And Disorientation In Humanitarian VR Experience Design. Refractory, [online] 30. Available at: <https://refractory-journal.com/30-kennedy-atkinson> [Accessed 15 February 2020].
Kermode, M. (2018). Kermode and Mayo's Film Review with Anna Boden and Ryan Fleck. [podcast] Kermode and Mayo's Film Review. Available at: https://www.bbc.co.uk/sounds/play/m00030hq [Accessed 23 Oct. 2019].
Kinicho (2019). Making Binaural Spatial Audio Easier. [online] Kinicho. Available at: https://www.kinicho.com/. [Accessed 24 Jul. 2019].
LaValle, S. (2016). Historical Perspective. [video] Available at: https://www.youtube.com/watch?v=f-vUixm-YlQ&index=2&list=PLbMVogVj5nJSyt80VRXYC-YrAvQuUb6dh. [Accessed 6 Sep. 2019].
Liu, S. (2019). Global virtual reality device shipments by vendor 2017-2019. The Virtual Reality Market. [online] Statista. Available at: https://www.statista.com/statistics/671403/global-virtual-reality-device-shipments-by-vendor/ [Accessed 2 Feb. 2020].
Mateer, J. (2017). Directing for Cinematic Virtual Reality: how the traditional film director’s craft applies to immersive environments and notions of presence. Journal of Media Practice, [online] 18(1), pp.14-25. Available at: https://www.tandfonline.com/doi/full/10.1080/14682753.2017.1305838 [Accessed 10 Jan. 2020].
McNicholas, R. and Jowers, M. (2018). Art in the Age of Distraction & How to Bring the Squidge to the Sterile. Presented at the Immersive storytelling symposium: Liverpool.
McMullan, T. (2018). Enter a psychedelic virtual forest in Treehugger: Wawona. [online] Available at: https://www.alphr.com/art/1004973/enter-a-psychedelic-virtual-forest-in-treehugger-wawona. [Accessed 17 June. 2019].
Milk, C. (2018). Milk. [online] Milk. Available at: http://milk.co/ [Accessed 1 Oct. 2019].
Mitchell, S. (2020). The Empathy Engine: VR Documentary And Deep Connection. Senses of Cinema, [online] 83. Available at: <http://sensesofcinema.com/2017/feature-articles/vr-documentary-and-deep-connection/.> [Accessed 24 January 2020].
Murray, J. (1997). Hamlet On The Holodeck: The Future Of Narrative In Cyberspace. 1st ed. New York: The Free Press, pp.1-15.
Parker, E. and Saker, M., 2020. Art Museums And The Incorporation Of Virtual Reality: Examining The Impact Of VR On Spatial And Social Norms. Convergence,
Ranasinghe, N., Cheok, A., Nakatsu, R. and Yi-Luen Do, E. (2013). Simulating the sensation of taste for immersive experiences. ImmersiveMe '13: Proceedings of the 2013 ACM international workshop on Immersive media experiences, [online] pp.29-34. Available at: https://dl.acm.org/doi/abs/10.1145/2512142.2512148 [Accessed 4 Jan. 2020].
Recuber, T. (2007). Immersion Cinema: The Rationalization and Reenchantment of Cinematic Space. Space and Culture, 10(3), pp.313-330.
Robertson, A. (2017). Dear Angelica might be the most beautiful virtual reality I’ve ever seen. [online] The Verge. Available at: https://www.theverge.com/2017/1/23/14345316/dear-angelica-oculus-vr-story-studio-sundance-2017-review [Accessed 25 Aug. 2019].
Roxborough, S. (2017, 05 24). Alejandro G. Iñárritu Embraces Virtual Reality (But Don't Call It Cinema). The Hollywood Reporter. Retrieved from https://www.hollywoodreporter.com/news/alejandro-g-inarritu-embraces-virtual-reality-but-dont-call-cinema-1006881
Schroeder, R., (1996). Possible worlds: the social dynamic of virtual reality technology. 1st ed. Boulder, CO: Westview Press.
Slater, M., Usoh, M. and Steed, A. (1995). Taking steps: the influence of a walking technique on presence in virtual reality. ImmersiveMe '13: Proceedings of the 2013 ACM international workshop on Immersive media experiences, [online] 2(3), pp.29-34. Available at: https://dl.acm.org/doi/abs/10.1145/210079.210084 [Accessed 10 Dec. 2019].
Spence, C., Obrist, M., Velasco, C. and Ranasinghe, N. (2017). Digitizing the chemical senses: Possibilities & pitfalls. International Journal of Human-Computer Studies, [online] 107, pp.62-74. Available at: https://www.sciencedirect.com/science/article/abs/pii/S1071581917300885 [Accessed 6 Nov. 2019].
Stoltz, A et al. (2020). The Future of Film Report. Future Film, London: Kings College London. Accessed 24 Apr 2020.
Uricchio, W. (2018). Stories Are Changing: William Uricchio - Alternate Realties Summit 2018. [video] Available at:
https://www.youtube.com/watch?v=hoY_aBaRfF0&list=PLhKbY6Z_pXkHFfLPMr13KT2SC1vIdML8D&t=0s&index=2. [Accessed 18 Oct. 2019].
Abe VR. (2016). [video] Produced by S. Jelley. UK: Hammerhead VR.
Aladdin. (1992). [DVD] Directed by R. Clements and J. Musker. United States: Disney.
Bear Island. (2017). [video] Produced by C. Jones. UK: BBC Earth.
Treehugger: Wawona. (2016). [film] Directed by B. Steel, E. Han Ersin and R. McNicholas. UK: Marshmallow Laser Feast.
Dear Angelica. (2017). [film] Directed by S. Unseld. USA: Oculus Story Studio.
Carne y Arena. (2017). [film] Directed by A. Iñárritu. USA: LMxLAB.
Clouds over Sidra. (2015). [Video file]. Directed by G. Arora. USA: VRSE. Retrieved
October 16, 2018, from https://www.youtube.com/watch?v=mUosdCQsMkM.
Jungle Book. (2016) [film] Directed by J. Favreau. USA: Disney.
Racing the King Tide. (2018). [video] Directed by C. Chadwick. UK: Hatch.
Superhot VR. (2016). [video Game] Published by M. Bączyński and T. Kaczmarczyk. USA: SuperHot.
The Lawnmower Man. (1992). [DVD] Directed by B. Leonard. USA: Allied Vision.