The challenge is to keep the app’s media files small so that they can be served on-demand from a content delivery network (CDN). This ensures only content that is relevant to the current user is downloaded and the main app size is kept small. Instead of using a mp4 or other type of movie file to convey the town 360 scene I am using the new Google webp format which I have previously validated within the 360 Flutter component. My aim is to keep this file as small as possible so it can be quickly served from a remote server. Other content such as voice and 3d characters which are part of the experience but not specific to a town may be compiled as assets with the build as they will not need to update as often as other interactions. There is also the option to stream the non-town specific video over the underlying 360 animated webp image file.
Using an Adobe Media Encoder plugin I was able to export a short section of my 360 movie into a webm file. The movie version of the webp format. However, this format turned out not to be supported by the 360 component I am using. So, I am looking to convert to webp as I will not need embedded audio; which can be played from a separate file.
I found that Google provides decent documentation for webp as well as a number of command line programs to help convert.
The conversion worked nicely. I am now able to open up my town scene with movement; in this case rainfall. However, (there is always a however…) the 360 plugin supports the standard Flutter Image widget which in turn support webp animated images, but I am so far unable to loop. So, the animation stops after the final frame.
As the 360 visual content of the spatial narrative is town based; the relevant media file is served from the CDN. Its important that the image files are as optimized as possible do they transfer quickly. Sound effects will be embedded within the app as they are common to all users, whereas narrative speech is also, like the town based visual content, based on the user’s language but also the path they take through the adventure based on their decision making. So, it also makes sense to serve this content from CDN. Speech files do not need to load instantly as they are usually triggered through an interaction. So, the final recipe involves playing embedded sound effects and an embedded visual effect while the main 360 media loads. The media is then cached so the slight delay is only noticable on the first playback. The narrative sound is then played on top of the 360 visual content at the appropriate moments.
For the sound I have opted for an AI generated voice; albeit with intonation and a Hollywood accent.
While, in itself, the narrative of Noirscape is to be interpreted at will by it’s participants, the structure of the environment in which the experience will take place will of course provide the underlying props and backstory along with the opportunity for interactivity with these props, people and different types of spaces – fictional, semi-fictional and realworld.
To help anchor my own understanding of the interactive aspect I have put together a flowchart to represent a snapshot of the app’s experience flow. This diagram illustrates how the narrative interactivity moves between levels of fiction, semi-fiction and realworld spaces while providing the participant with the props, events and actions that make the app ‘playable’.
While this diagram only illustrates a section of the app’s world it would represent a satisfactory achievement if I am able to build the functionality over the next four to six weeks. From then on I would be able to focus on adding new interactive items, 360 events and so on. The functionality to power the above snapshot is to be fed by cloud based data; hence I am able to add new elements on-the-fly; without having to bake them into the app itself and have many releases.
A common theme throughout the experience is the aquisition of Noirscape objects. These objects can be found in the Augmented Reality (AR) part of the app, the immersive 360 world as well as outside in the local town.
The objects will each have their own unique properties and actions. Actions represent what a participant can do with each item. For example, as per the flow chart, the participant will experience the old fashioned realsized doorway appear in their own home within the AR part of the app. They will learn that the door is locked as each item has a description. A common action in choose your own adventure is ‘examine‘. The participant may examine the door. The door object will be pre-configured to possess this ‘examinability‘ quality along with a consequence of carrying out the action. In the case of the door, the participant will find a keyhole as a consequence of the examine action. At some point they will surely find a key with which to ‘open‘ the door. Until they find the key, though, it remains shut. The keyhole is also treated as an interactive object which the participant has found. Albeit, it cannot be removed from the door, which is the parent object. However, the participant can ‘look through‘ the keyhole and this will reveal a keyhole view of a fictional space on the other side of the door.
This transitioning through nuances of fiction and reality is an important aspect to Noirscape and very much inspired by my research into the success of Bandersnatch but also the creative influence of other filmmakers and indeed videogame makers and thinkers who are keen to explore and expoit these blurred bounderies.
The Noirscape app narrative design features several conceptual layers.
Present The present narrative is a choice based narrative loosely based on CYOA Quest Pattern. The pattern uses small, tightly-grouped clusters of nodes allowing many ways to approach a single situation (Ashwell 2015)
“This mode [Quest Pattern] is well-suited for journeys of exploration, focused on setting; the quest’s structure tends to be organised by geography rather than time.”
Sam Kabo Ashwell, Standard Patterns in Choice-Based Games, These Heterogenous Tasks, 2015
Following on from my reflection and study-notes about the porous boundary between the fictional [or synthetic] world of electronic narrative platforms and the realworld, I would like to now switch my focus to the role of realworld within the scope of interactive networked narrative; I shall focus particularly upon the idea of spatialised narrative i.e. the intersection of space and narrative; with the aim of exploring the synergistic potential of combining space, place, computing and interactive narrative.
Cyberspace or Cyberplace?
“Whereas film is used to show a reality to an audience, cyberspace is used to give a virtual body, and a role, to everyone in the audience. Print and radio tell; stage and film show; cyberspace embodies”
Randall Walser, 1990
When Walser was writing about cyberspace some 30 years ago, he did so in terms of both a medium – enabling humans to gather in virtual spaces and a phenomenon – analogous to physical space filled with virtual stuff.
However, Aarseth, a decade later, quoting the philosopher Anita Leirfall, argued that cyberspace theorists confuse the concepts of space with place.
“Leirfall, following Aristotle and Kant, does not accept the notion that Cyberspace, virtual spaces and, implicitly, computer games, constitute an alternative type of space of autonomous qualities. By being generated, cyberplaces are ”regions in space”, and cannot exist as parallels of real, three-dimensional space. This is an important point. ”Cyberspace” and other such phenomena (e.g. computer games) are constituted of signs and are therefore already dependent on our bodily experience in, and of, real space to be ”hallucinated” as space.”
Aarseth 2001, p162
The arrival of smartphones, which are effectively pocket sized computers, has opened up new ways to experience a crossover between different types of spaces, places and narrative. In my present research driven project, the narrative is anchored in real space through mobile technology superposed with alternative representations of the associated place using immersive photography and special effects.
“The development of mobile technology, global positioning systems (GPS), and augmented reality counters the tendency of computers to lure sedentary users into virtual worlds by replacing simulated environments with real-world settings and by sending users on a treasure hunt in the physical space”
Ryan, Foot & Azaryahu, 2016, pp102
Embedded Narrative Design
Drawing inspiration from Henry Jenkins’ (2004) narrative categories, Ryan, Foot & Azaryahu identify and provide detail about several narrative structures, including epic, emergent and embedded . The epic narrative involves a character who steadily progresses over an endless journey of levels, episodes or sequels. Whereas the emergent structure is one in which the narrative emerges depending on the actions of the player and other events and interactions within the gameworld – i.e. there is no fixed predetermined plot. I am, for the interest of my present project, mostly interested in the third structural type : embedded narrative which, explains Ryan, Foot & Azaryahu, combines a predefined story about events which took place in the past superposed with the real time narrative of the player investigating those past events.
Duality : place/space & story world/physical world
“The search for the hidden story takes advantage of the visual resources of digital systems by sending the player on a search for clues hidden in the storyworld”
Ryan, Foot & Azaryahu, 2016, pp108
The consequence of this approach leads to interesting duality of the player’s search for hidden clues in the fictional story world and a GPS treasure hunt in the physical world. This is further echoed by the dualistic nature of the embedded narrative experienced through the augmented immersive representation of the place of interest featured at the given space.
“In embedded narrative, space is there to be searched, since it contains the clues to the story that need to be retrieved”
Ryan, Foot & Azaryahu, 2016 pp110
Narrative quality of Toponyms
“Embedded in and evocative of stories of different kinds, street names are deeply imbued with narrativity, even though they rarely comprise complete narratives by themselves”
Ryan, Foot & Azaryahu, 2016, pp158
[work in progress… ]
“A storyline becomes an option whenever a chronological or a thematic sequential structure is introduced into a spatial arrangement of coesistent elements in the form of routes and paths that direct movement in space”
In my previous post I compiled some of my thinking and study notes around the notion of implementing interactive, branching narrative using the smartphone medium. This got me thinking more about the uniqueness of the smartphone app as a creative interactive medium as opposed to book or video. I considered how the Choose Your Own Adventure books I played as a kid were in fact book-games just as games I played on a tv screen or monitor are video-games. More recently I played the 2018 Netflix movie-game called Black Mirror: Bandersnatch.
Bandersnatch requires the player to makes decisions for the main character using the TV remote control. An intriguing twist [spoiler alert] within the Bandersnatch narrative, to me at least, is when you, the player, are able to inform the main character about the nature of what’s actually going on. The narrative leaves the confines of the inner story and blurs with the player’s own reality when the main character demands some kind of sign from the invisible entity [you!] they feel is controling their actions.
“I am watching you on Netflix. I make decisions for you.”
“[it’s] a streaming entertainment platform from the early 21st century,”
“it’s like TV, but online. I control it.”
At this point, should you reach it in the gameplay, the main character’s father enters the room and sees his son in a state of shock. When he tries to explain to his dad about how he is being controlled by someone from the future (the game is set in the 1980’s) using Netflix; his dad invites him to take a trip to the psychiatrist – the main character agrees and the game ends.
What I find most intriguing about this sequence is the subtle blurring of boundaries between the overarching fictional narrative of the main gameplay and the player’s own reality. You, the player, become directly implicated in the narrative and the worlds on either side of the screen bleed into one another when the main characters learns about you.
Furthermore, Bandersnatch demonstrates how interactive narrative content continues to adapt to new platforms – Netflix invested in specific technological innovations for Bandersnatch to enable the branching nature of the narrative and for players to go back, try different choices within gameplay. But, that which interests me most of all is the way this innovative work bridges the domain of video with the real world. Or, at least it hints at this; as set out above in respect to the player revealing themself to the fictional character.
Edward Castronva, writer of Synthetic Worlds : The Business and Culture of Online Games (2005), considers that there is a kind of boundary between the real-world and electronic synthetic one which is a kind of protective membrane between the rules of either world. He describes this membrane as the Magic Circle – a term borrowed from Jogan Huizinga, 1938 who used the same term to mean “a state in which the player is bound by a make-believe barrier created by the game” (Huizinga, 1938)
For Castronva, the synthetic world is:
“an organism surrounded by a barrier. Within the barrier, life proceeds according to all kinds of fantasy rules involving space flight,fireballs,invisibility,and so on.Outside the barrier, life proceeds according to the ordinary rules.”
Castronva, 2005 p.147
But, Castronva admits, this membrane is not sealed completely. In fact it is quite porous and people are “crossing it all the time in both directions, carrying their behavioral assumptions and attitides with them.” concluding that the result of which is that:
“The valuation of things in cyberspace becomes enmeshed in the valuation of things outside cyberspace”
Castronva, 2005 p.147
Coming back to Bandersnatch, a further blurring of the boundary between the synthetic and realworld is the addition of a hidden QR code during the final sequence of one of the possible endings. The code, leads the player, should they scan it with a mobile device, to a website bearing the name of the fictional software company from the game’s narrative on which you can play the 1980’s style computer game that another character within the movie-game had been building within the fictional world of Bandersnatch. The secret website and the Netflix production are from fictional worlds allowing the player to further experience elements of the story across different multiple platforms.
References & Resources
Homo Ludens, Johan Huizinga, 1938 Penguin Random House
Like so many other kids growing up in the 1980’s I spent a heck of a lot of time playing Choose Your Own Adventure (CYOA) books as well as playing/making text based adventure games for my Commodore and Sinclair home computers. I recall spending hours upon end staring at a tv screen with a flashing cursor, writing simple commands within a text game : go west, sit down, pull chord, enter hole. Often I’d be stuck in a seemingly neverending sequence of typing the same commands and just not being able to unblock new content. Even if I should finally unblock something new, it would only be a simple line of text. But, it was utterly thrilling when it did happen. Why? Jen Doll, explains in a 2012 Atlantic article, that back when when home computing was only just getting started, text based adventures, whether in book or screen form, were often our first dose of narrative type interactivity.
Edward Packard, a pioneer of the CYOA genre, says that what made this type of book so original was that:
written in second person, yet about you;
it’s you makes the decisions which lead to multiple plot lines;
and these lead to multiple endings.
Packard revisited some of his original books and released mobile app versions in the 2010’s when smartphones were becoming increasingly widespread. Of this format Packard said:
“Of course there’s a lot you can do in this format that you can’t do in a printed book,” he says. “There is no limit to the number of pages, so in a scene where you are swimming, trying to get to shore, you can keep swiping pages and you only encounter more scenes of the sea. Your frustration and uncertainty mimic what you’d feel in reality. In one scene you have to make a repair on our spaceship and the computer says there will be a catastrophic failure in 20 seconds. The reader has to solve the problem in real time: The countdown of time remaining is shown on the screen. We have light and sound effects. The computer remembers where you’ve been, which can affect what you know and what happens when you reach a certain locale and if and when you come back to it.”
In my current project, an app with a CYOA branching narrative, I aim to take this format further by combining the player’s experience within the medium (smartphone/book) with the experience of the realworld, using geographical positioning, augmented reality and other mobile technologies to interconnect the narrative with real things, places and people.
“The thing about choose your own adventure games is that even when they’re not so good, you feel a sense of ownership and therefore a connection to the decisions you made for your character. It’s a lot more compelling to be able to feel this ownership over a so-so game rather than not having any power to change the path of any other mediocre title.”
Agency, is one of the three elements identified by Douglas Brown in his thesis, the Suspension of Disbelief in Video Games. A section of which, I reviewed last year as part of my ongoing study of creative app development. While a person can choose to watch a film or read a book, in order to engage meaningfuly with a game does require wilful and active participation i.e. interaction or as Brown states:
“the gamer is something other than an audience“
This is why I would argue that CYOA books are not books at all in the traditional sense; they belong more to the game genre.
A further element, identified by Brown, that differentiates video games from books, film and other media is [player-]authorship. Through interaction with media a game-player actively contributes to the authorship of the narrative. Whereas, traditionally, film, music and books merely allow the user to experience narrative play out the same sequence of events everytime.
Along with the notion of player skill, Brown successfuly demonstrates that video games are a distinct media from other narrative genres or entertainment.
But as I hinted at above, I believe there is a space somewhere between video games and traditional media where one genre bleeds over into the other. CYOA books are actually perhaps better described as book-game just as games which use the medium of video are described as video-games. Eitherway, this wilful participation required of games with the media ultimately enables player-authorship – the player is part-viewer/reader and part-author. The early smartphone versions of some of the CYOA books introduced an element of skill, too, using timed sequences and other software features. Might it be worth thinking about app-games as belonging to a further genre?
“Skill, and skilled readers suggest quite the opposite of any kind of ‘suspension’, agency is almost diametrically opposed to ‘disbelief’ and the ‘will’ to empower theatrical proceedings is challenged by the possibility of actual, meaningful audience authorship.”
An early version of the interactive narrative is Raymond Queneau’s Un Conte à Votre Façon, 1967 (A story of your making) partly inspired by an earlier work Morphology of the Tale by Vladimir Propp written in Leningrad in 1928. Propp write extensively about theories of narrative forms. Queneau, according to a french article by Carpentier, 2018 was highly influenced by the following simple structure: