Noirscape uses NFC (Near Field Communications) technology to add some fun to the signing in process. A detective ID card is among the items that will be included in the boxed version of the game but they’ll also be available via the future website and onlin merchandise store.
NFC is a tiny, low-cost, batteryless radio transmitter which allows app developers to embed the featherweight, waferthing device into clothing, toys, posters and just about anything else. In my case, I am using a business card format. The actual NFP cpmponent is inside the plastic and is only about an inch in diameter. For the purpose of my testing I have created my own version using card and a stick’on NFP emitter.
Near Field Communication (NFC) is a standards-based short-range wireless connectivity technology that makes life easier and more convenient for consumers around the world by making it simpler to make transactions, exchange digital content, and connect electronic devices with a touch. NFC is compatible with hundreds of millions of contactless cards and readers already deployed worldwide.
The ID card for Noirscape contains a unique reference code that may only be used once and is used in conjunction with anonymous sign in through Google Firebase integration. It is a gamification feature rather than a security element. ALthough there are plenty of possiblities for future feature development; inlcuding ‘friending’ other participants in the realworld by allowing one another to scan each other’s card.
Noirscape experiments with a cross reality approach to narrative. Participants search for and find fictional objects in their own physical home through the AR (augmented reality) feature. One of these objects is a door and it’s keyhole a doorway between an augmented view of one world and the entirely fictional world of another. Furthermore, Noirscape binds the narrative with physical spaces in the nearby town. In my case and for the purpose of the pilot version of Noirscape, this is the French town of Bourg-en-Bresse.
I previously carried out fieldwork collecting 360 panoramic photographic content in and around the town; at over twenty locations selected not neccesarily for their prominence, but also for their intrigue; whether that be a curious street name, a bygone brewery turned warehouse or the site of a house where a celebrated artist and philosopher one lived. The noirscape experience will take participants into their town where segments of narrative are reveals through what I call flashbacks – these are a staple component of film noir; where the protagonist, who is usually since deceased or condemmed to a life of prison, recounts events from the past which played an important role in their current whereabouts [or absence of].
My challenge is to take my 360 content from the town and combine it with fictional Noir narrative to give an augmented or combined immersive experience whereby the content is triggered only by visiting the place in the physical world and at which point, a flashback from a fictional past occurs. To achieve this I decided to work with digital 3d character creation and animation. I had previously arranged to work with a friend who is also an actor; but, it’s complicate right now with the pandemic, to meet up and spending enough quality time to get something filmed; I was planning to use my green screens and then take the content into the 360 editor using Adobe After Effects and Premier Pro. One thing lead to another and I opted for digital characters. I initially hoped I’d be able to use Adobe software but they have discontinued their Fuse product which was a character designer app that could be used with Mixamo, their recently acquired character animation service. I decided to use Reallusion’s Character Creator instead due to the vast amount of resources available. I used Headshot, their AI face generator to base character on my own face (although I’ve reworked it somewhat since!) and I imported custom objects like a Fedora hat and set up the character in a black coat.
Next I took the character into iClone, Reallusion’s 3D animation suite. The challenge with iClone was to be able to bring in my 360 photo and create my own scene within the panorama. However, I ran into problems with this at first. While export to 360 panorama format is suported in iClone, I couldn’t achieve this using photography without experiencing problems with the way the image was being wrapper; due to distortion at the poles of the sphere if the Skybox object. The Skybox object in iClone and more generally in 3D design, is the imagery used to define the farthest most visible details; this would normally be the sky, hence the name; but may also be a distant mountain range. Usually this would only be thought of as a backdrop, with far more focus on the foreground and midground detail. In my case the Skybox would be represented by a complete 360 photo, in which I would place 3D assets like a person, a vehicle, etc.
I discussed the issue in the Reallusion support forum; and one solution put forward was to create my own 3d sphere object and set my 360 image as the texture. This did produce a slightly better outcome but not satisfactory enough for what I need. The Reallusion is fantastic nontheless; what I am seeking to do is certainly not a typical user-case by any means. One really good feature with iClone, and one of the key reasons for settings a photo as the Skybox, is for calculating light within a scene. The iClone software will identify from the image, in my case the 360 photo, which direction light is coming from and therefore where to cast light and shade within the 3D assets added to the scene. So, although I chose not to use iClone with the 360 photo visible, I still used it for the lighting work.
Within iClone I applied some subtle animation to my character; his tie blows in the wind and he blinks and moves a little while he waits for his rendez-vous. I applied rain effects with splashes and flickering light effects. In order to export my animation without the Skybox image so that I could bring it into Adobe After Effects I would need to export as an image sequence so ensure a transparent background. The sequence is 30 seconds long and 30 frames per second; so the software rendered 900 images in total which I then imported into After Effects.
Within After Effects the first challenge was to align the two-dimensional representation of my sequence within a 360 environment. If I place it as-is then it will be foreably bent into a banana shape as it is interprated through a 360 viewer. So, to avoid this, it’s important to alter the curvature of the 2d assets to align with the 360 image in equirectangular-panoramic format.
I’m generally pleases with the outcome and although it took quite a bit of time to get what I wanted, I now have a documented workflow for the process; I have a character ready to deplot to new scenarios and the knowhow to create others much more quickly. A small issue I have with the end result is that the animation is too subtle to really see properly on a mobile device; but this is easily tweaked. For now, I’m going to settle with what I have for the purpose of integrating with the app. The next step is to create a looping image based version of the scene in webp format as I have shown in a previous post. I will then play the audio channel, with the voice narration and sound effects via the app/device rather than the media file itself. This will keep the size of the media file down and allow me to serve the localised element (the view using footage from a specific town) and the global content – the spoken narrative.
I added some final sparkle to my vintage rotary dialler. Using various calculations I am able to predict which number has been dialled in a way this quite closely emulates the real thing. For example, a digit is only registered when the dial is turned fully to the catch.
I also added sound effects. Each digit has its own sound file to correspond with the length of time of the rotation which of course varies for each number. The sound effect also has to respond to when the dial is released too early and therefore has an irregular return rotation timespan.
Lastly, I added some texture to the background and some text using an style typewriter font which shows the participants name and their own tel number within the experience.
I had great fun putting this into the hands of my two childre, aged 9 and 12; they had no idea how to work a rotary dial phone. Who needs to invent enigmas and puzzles when we can just emulate the trials and tribulations of analog technology!
I also shared a demo with friends and colleagues as well as on social media and I was quite surprised by the number of people suggesting this could become an app in it’s own right. I haven’t researched what’s already in the app stores. But, certainly an interesting idea would be to make a standalone rotary dialler app which interacts with the phone’s call api to initiate real calls. I am already thinking about the fun that could be had creating different version; the 1970’s, 1890’s, 1950’s.
Anyway, here is a video I put together to demonstrate the feature in action.
While, in itself, the narrative of Noirscape is to be interpreted at will by it’s participants, the structure of the environment in which the experience will take place will of course provide the underlying props and backstory along with the opportunity for interactivity with these props, people and different types of spaces – fictional, semi-fictional and realworld.
To help anchor my own understanding of the interactive aspect I have put together a flowchart to represent a snapshot of the app’s experience flow. This diagram illustrates how the narrative interactivity moves between levels of fiction, semi-fiction and realworld spaces while providing the participant with the props, events and actions that make the app ‘playable’.
While this diagram only illustrates a section of the app’s world it would represent a satisfactory achievement if I am able to build the functionality over the next four to six weeks. From then on I would be able to focus on adding new interactive items, 360 events and so on. The functionality to power the above snapshot is to be fed by cloud based data; hence I am able to add new elements on-the-fly; without having to bake them into the app itself and have many releases.
A common theme throughout the experience is the aquisition of Noirscape objects. These objects can be found in the Augmented Reality (AR) part of the app, the immersive 360 world as well as outside in the local town.
The objects will each have their own unique properties and actions. Actions represent what a participant can do with each item. For example, as per the flow chart, the participant will experience the old fashioned realsized doorway appear in their own home within the AR part of the app. They will learn that the door is locked as each item has a description. A common action in choose your own adventure is ‘examine‘. The participant may examine the door. The door object will be pre-configured to possess this ‘examinability‘ quality along with a consequence of carrying out the action. In the case of the door, the participant will find a keyhole as a consequence of the examine action. At some point they will surely find a key with which to ‘open‘ the door. Until they find the key, though, it remains shut. The keyhole is also treated as an interactive object which the participant has found. Albeit, it cannot be removed from the door, which is the parent object. However, the participant can ‘look through‘ the keyhole and this will reveal a keyhole view of a fictional space on the other side of the door.
This transitioning through nuances of fiction and reality is an important aspect to Noirscape and very much inspired by my research into the success of Bandersnatch but also the creative influence of other filmmakers and indeed videogame makers and thinkers who are keen to explore and expoit these blurred bounderies.
The short Youtube presentation below charts the app customer, Kevin’s ste-by-step process from purchasing the boxed product to completing the initial investigation/story.
Prototype of Noirscape
The boxed product is sold on the high street in selected specialised stores.
The product’s cover design will be highly stylised to the ‘Noir’ look and feel.
It is clear from the cover illustration that this is a hybrid board-app game.
The localised nature of the product holds special appeal as it provides a bespoke look and feel and a personal connection through the relationship with the consumer’s home town.
FEATURES / FUNCTIONALITY
The boxed product is attractively illustrated and crafted
The product includes an NFC ID card which allows a radio signal to be detected by the user’s phone from the ID card. This is used to activate the app, so that only people in posession of a card can play. It, is also used to provide hints to the player when they are stuck on an enigma.
The smartphone application is available in English and French.
The app uses GPS tracking to anchor gameplay in geographic places.
360 degree video with special effects is used to create a sense of immersiveness
All features are widely supported by smartphones including budget models
Technical experiments have been validated in the following areas:
Applying special effects to 360 degree film
Applying superimposed footage in a 360 degree film
Accessing the 360 footage via Flutter, the cross-platorm app development platform
DESIGN & THEORETICAL JUSTIFICATIONS
Familiarity Hypothesis – building a product which is tailored to a user’s home town generates increased curiousity, intrigue, a sense of personal associated; but, also a level of respect and appreciation for the product innovators for acknowledging their town.
Thematic Approach – allows for the possibility of future releases of new editions of the product along different themes : Cyberpunk, Medieval, Victorian, etc. I decided to use Film Noir for the prototype primarily because I am personally inspired by the genre but also because it lends itself so well to the embedded narrative of investigating past events, but also the notion of searching and solving enigmas, which is synonomous with escape style games.
GPS/Geo – The arrival of smartphones, which are effectively pocket sized computers, has opened up new ways to experience a crossover between different types of spaces, places and narrative.
“The development of mobile technology, global positioning systems (GPS), and augmented reality counters the tendency of computers to lure sedentary users into virtual worlds by replacing simulated environments with real-world settings and by sending users on a treasure hunt in the physical space” [Ryan, Foot & Azaryahu, 2016, pp102]
Embedded & Spatialised Narrative Design – an innovative approach to combining storyworlds with the realword using recent smartphone technologies.
“The search for the hidden story takes advantage of the visual resources of digital systems by sending the player on a search for clues hidden in the storyworld” [Ryan, Foot & Azaryahu, 2016, pp108]
“In embedded narrative, space is there to be searched, since it contains the clues to the story that need to be retrieved” [Ryan, Foot & Azaryahu, 2016 pp110]
SHORTCOMINGS OF DESIGN
Some older, lower-end devices may not support NCF card reading functionality
App requires above average drive space due to 360 degree video media
Potential safety/responsability issues concerning public interaction aspect of app (places)
Product can be passed on from one user to another potentially without purchase
Additional releases for other towns including major cities, in France & worldwide
New episodes can be developed and added via in-app purchases
More Augmented Reality (AR) features
Opportunity for users to ‘leave their mark’ with Geo stamped AR.
DESIGN PROCESS NEXT STEPS
Build working cross-platform prototype of Noirscape
User test in my home town
Prepare Crowdsourcing / Kickstarter campaign for furthet towns
“A storyline becomes an option whenever a chronological or a thematic sequential structure is introduced into a spatial arrangement of coesistent elements in the form of routes and paths that direct movement in space” [Ryan, Foot & Azaryahu, 2016, pp158]
The app itself, will be built using Flutter, a cross-platform app development framework created by Google. As an app developer, I have been using Flutter for about a year. I have experimented with a range of concepts including machine learning driven games and narrative based educational apps.
The app will require quite advanced video editing skills as it uses 360 degree film and special effects. To address this requirement, I have taken a number of professional Adobe training courses which lead to industry certification by Adobe:
Adobe Illustrator – graphic design/icons, etc
Adobe Premier – 360 film editing
Adobe After Effects – 360 film special effects & animations
Adobe Photoshop – photo editing and effects
I recognise shortcomings in respect to my knowledge and skills in respect to the physical boxed product design & production. To address part of this weakness, I have been working with 3D design software to envisage visual aspects to the boxed product. I have also registered for an Adobe InDesign course in January to help with packaging design of the physical product.
I still need to research how to source and manufacture the box and some of the included items. However, I have already purchased samples of NFC cards that will be used for the detective’s ID interactve card. I have researched printing equipment which can be used to illustrate these plastic cards, too.
The project is being managed using Agile methodology via Trello, a popular Kanban app for organising the development into stages while integrating with the user centered design approach alongside personas and the storyboard.
To design an immersive and interactive smartphone application using 360 video and GPS to send users on a film noir themed investigation into the physical space of their local town.
To incite people to engage with the narrative of spaces and places of their town.
To encourage them to visit their town centre.
To promote physical excercise through walking and exploring.
Noirscape Industrial Design
Noirscape is a physical boxed product that includes several items for gameplay as well as a GPS/AR smartphone application.
Type of Person
Residents who frequent less, or not at all, their local town; instead, favouring peripheral commercial centers and who are therefore rapidly losing contact with their local town’s character, economic & heritage value.
They are likely:
Ages 20 – 60
Active Smartphone users
Comfortable with technology
Yearning for adventure
Persona 1 – Product Consumer
Earns €3k per month
“I seem to spend my whole life shopping, at work or sleeping”
“I hear there’s a new bar opened in town but I haven’t been down the high street for ages”
“I’d take a walk around town this weekend; but, no one else will join me”
Persona 2 – Product Retailer
Earns €4k per month
Sells Board Games
In a couple
“I’d love the opportunity to work with a local designer and cut out the middle-man”
“I sometimes feel like I’m just selling fancy boxes. There’s so much scope for something new & innovative on the board game scene”
“I love the idea of something local with a bespoke feel.”
I am currently working on a conceptual design document inspired by BJ Fogg, founder, and director of the Stanford Behavior Design Lab.
I am looking to emulate the graphical feeland mood of classic film noir as much as possible; studying posters and film titles from the era (1930s – 1950’s) and within this genre.
The title for my product is Noirscape. This is a term often used to describe a typical cityscape, townscape or even interiorscape that carries the mood and feel of Film Noir. My concept involves spatialised narrative and is also inspired by my own enjoyment of ‘escape’ games. Thus, Noirscape, is a title that can be interpreted in the sense describe above – a Noirscape (a type of place) or as Noir[e]scape (Noir Escape).
The concept is primarily an app based experience. However, it will be sold as a boxed product. The box will include a number of elements which form part of this experience – a mysterious bundle including an old newspaper clipping, an ID card which is used to acivate the app aswell as provide hints during the game via it’s NFC (Near-field Communication) functionality and several other items, pencil, notebook and a set of abstract puzzle pieces. I have used 3d software to create an early concept design of what this might look like.
Early Concept Design for Noirscape
I created a simple visual to convey what form this concept will take from a user’s perspective. Although, this is a boxed product, the main experience will by via a smartphone’s innovative features including GPS anchors, immersive 360 degree video with special effects as well as the NFC feature described earlier.
“Getting this visual into people’s heads early helps them start thinking about your concept in concrete ways”
BJ Fogg, pp 203
I have begun building a user story board to illustrate the user experience in a step by step manner. Rather than use doodles, I decided to make use of my Photoshop skills combined with 3D object design and renderings to produce a photo-montage type story.
Following on from my reflection and study-notes about the porous boundary between the fictional [or synthetic] world of electronic narrative platforms and the realworld, I would like to now switch my focus to the role of realworld within the scope of interactive networked narrative; I shall focus particularly upon the idea of spatialised narrative i.e. the intersection of space and narrative; with the aim of exploring the synergistic potential of combining space, place, computing and interactive narrative.
Cyberspace or Cyberplace?
“Whereas film is used to show a reality to an audience, cyberspace is used to give a virtual body, and a role, to everyone in the audience. Print and radio tell; stage and film show; cyberspace embodies”
Randall Walser, 1990
When Walser was writing about cyberspace some 30 years ago, he did so in terms of both a medium – enabling humans to gather in virtual spaces and a phenomenon – analogous to physical space filled with virtual stuff.
However, Aarseth, a decade later, quoting the philosopher Anita Leirfall, argued that cyberspace theorists confuse the concepts of space with place.
“Leirfall, following Aristotle and Kant, does not accept the notion that Cyberspace, virtual spaces and, implicitly, computer games, constitute an alternative type of space of autonomous qualities. By being generated, cyberplaces are ”regions in space”, and cannot exist as parallels of real, three-dimensional space. This is an important point. ”Cyberspace” and other such phenomena (e.g. computer games) are constituted of signs and are therefore already dependent on our bodily experience in, and of, real space to be ”hallucinated” as space.”
Aarseth 2001, p162
The arrival of smartphones, which are effectively pocket sized computers, has opened up new ways to experience a crossover between different types of spaces, places and narrative. In my present research driven project, the narrative is anchored in real space through mobile technology superposed with alternative representations of the associated place using immersive photography and special effects.
“The development of mobile technology, global positioning systems (GPS), and augmented reality counters the tendency of computers to lure sedentary users into virtual worlds by replacing simulated environments with real-world settings and by sending users on a treasure hunt in the physical space”
Ryan, Foot & Azaryahu, 2016, pp102
Embedded Narrative Design
Drawing inspiration from Henry Jenkins’ (2004) narrative categories, Ryan, Foot & Azaryahu identify and provide detail about several narrative structures, including epic, emergent and embedded . The epic narrative involves a character who steadily progresses over an endless journey of levels, episodes or sequels. Whereas the emergent structure is one in which the narrative emerges depending on the actions of the player and other events and interactions within the gameworld – i.e. there is no fixed predetermined plot. I am, for the interest of my present project, mostly interested in the third structural type : embedded narrative which, explains Ryan, Foot & Azaryahu, combines a predefined story about events which took place in the past superposed with the real time narrative of the player investigating those past events.
Duality : place/space & story world/physical world
“The search for the hidden story takes advantage of the visual resources of digital systems by sending the player on a search for clues hidden in the storyworld”
Ryan, Foot & Azaryahu, 2016, pp108
The consequence of this approach leads to interesting duality of the player’s search for hidden clues in the fictional story world and a GPS treasure hunt in the physical world. This is further echoed by the dualistic nature of the embedded narrative experienced through the augmented immersive representation of the place of interest featured at the given space.
“In embedded narrative, space is there to be searched, since it contains the clues to the story that need to be retrieved”
Ryan, Foot & Azaryahu, 2016 pp110
Narrative quality of Toponyms
“Embedded in and evocative of stories of different kinds, street names are deeply imbued with narrativity, even though they rarely comprise complete narratives by themselves”
Ryan, Foot & Azaryahu, 2016, pp158
[work in progress… ]
“A storyline becomes an option whenever a chronological or a thematic sequential structure is introduced into a spatial arrangement of coesistent elements in the form of routes and paths that direct movement in space”
In my previous post I compiled some of my thinking and study notes around the notion of implementing interactive, branching narrative using the smartphone medium. This got me thinking more about the uniqueness of the smartphone app as a creative interactive medium as opposed to book or video. I considered how the Choose Your Own Adventure books I played as a kid were in fact book-games just as games I played on a tv screen or monitor are video-games. More recently I played the 2018 Netflix movie-game called Black Mirror: Bandersnatch.
Bandersnatch requires the player to makes decisions for the main character using the TV remote control. An intriguing twist [spoiler alert] within the Bandersnatch narrative, to me at least, is when you, the player, are able to inform the main character about the nature of what’s actually going on. The narrative leaves the confines of the inner story and blurs with the player’s own reality when the main character demands some kind of sign from the invisible entity [you!] they feel is controling their actions.
“I am watching you on Netflix. I make decisions for you.”
“[it’s] a streaming entertainment platform from the early 21st century,”
“it’s like TV, but online. I control it.”
At this point, should you reach it in the gameplay, the main character’s father enters the room and sees his son in a state of shock. When he tries to explain to his dad about how he is being controlled by someone from the future (the game is set in the 1980’s) using Netflix; his dad invites him to take a trip to the psychiatrist – the main character agrees and the game ends.
What I find most intriguing about this sequence is the subtle blurring of boundaries between the overarching fictional narrative of the main gameplay and the player’s own reality. You, the player, become directly implicated in the narrative and the worlds on either side of the screen bleed into one another when the main characters learns about you.
Furthermore, Bandersnatch demonstrates how interactive narrative content continues to adapt to new platforms – Netflix invested in specific technological innovations for Bandersnatch to enable the branching nature of the narrative and for players to go back, try different choices within gameplay. But, that which interests me most of all is the way this innovative work bridges the domain of video with the real world. Or, at least it hints at this; as set out above in respect to the player revealing themself to the fictional character.
Edward Castronva, writer of Synthetic Worlds : The Business and Culture of Online Games (2005), considers that there is a kind of boundary between the real-world and electronic synthetic one which is a kind of protective membrane between the rules of either world. He describes this membrane as the Magic Circle – a term borrowed from Jogan Huizinga, 1938 who used the same term to mean “a state in which the player is bound by a make-believe barrier created by the game” (Huizinga, 1938)
For Castronva, the synthetic world is:
“an organism surrounded by a barrier. Within the barrier, life proceeds according to all kinds of fantasy rules involving space flight,fireballs,invisibility,and so on.Outside the barrier, life proceeds according to the ordinary rules.”
Castronva, 2005 p.147
But, Castronva admits, this membrane is not sealed completely. In fact it is quite porous and people are “crossing it all the time in both directions, carrying their behavioral assumptions and attitides with them.” concluding that the result of which is that:
“The valuation of things in cyberspace becomes enmeshed in the valuation of things outside cyberspace”
Castronva, 2005 p.147
Coming back to Bandersnatch, a further blurring of the boundary between the synthetic and realworld is the addition of a hidden QR code during the final sequence of one of the possible endings. The code, leads the player, should they scan it with a mobile device, to a website bearing the name of the fictional software company from the game’s narrative on which you can play the 1980’s style computer game that another character within the movie-game had been building within the fictional world of Bandersnatch. The secret website and the Netflix production are from fictional worlds allowing the player to further experience elements of the story across different multiple platforms.
References & Resources
Homo Ludens, Johan Huizinga, 1938 Penguin Random House