Creative Development Journal
Welcome to my journal! My name is Mat Wright and I’m a creative app developer. I use this journal to document my research and development.
Noirscape uses NFC (Near Field Communications) technology to add some fun to the signing in process. A detective ID card is among the items that will be included in the boxed version of the game but they’ll also be available via the future website and onlin merchandise store.
NFC is a tiny, low-cost, batteryless radio transmitter which allows app developers to embed the featherweight, waferthing device into clothing, toys, posters and just about anything else. In my case, I am using a business card format. The actual NFP cpmponent is inside the plastic and is only about an inch in diameter. For the purpose of my testing I have created my own version using card and a stick’on NFP emitter.
Finding hidden objects in an augmented reality [AR] experience is fun; especially when the items have interactive qualities which trigger new content, clues and narrrative. Furthermore, given the quality of 3D assets, AR lighting, shadow and reflection prediction and generation; coupled with a the Noirscape black & white filter, it’s a great little feature to be able to capture an augmented view within the home and share with others on social media or messaging.
Augmented Reality (AR) provides a way to place realistic looking virtual objects into a realworld scene. While the object may merely exist upon the screen of a phone; there are features to AR which combine the worlds; fiction and reality; beyond the two dimensional surface of a smartphone. For example, when a 3D object is placed within the AR scene; it may not be physically present upon the targetted surface; but certain qualities of the said surface are projected into the machine generated final composition (horizontality, width, height and distance of the plan) and ultimately this data is processed within the mind of the person who momentarily accepts the presence of a fictional item within their immediate realworld environment.
This is a demo of an interactive scene using 360 photography, 3D character design and animation pulled together in Adobe After Effects to produce a WEBP image file which then has interaction through sound and buttons using the Flutter mobileapplication development framework.
Noirscape experiments with a cross reality approach to narrative. Participants search for and find fictional objects in their own physical home through the AR (augmented reality) feature. One of these objects is a door and it’s keyhole a doorway between an augmented view of one world and the entirely fictional world of another. Furthermore, Noirscape binds the narrative with physical spaces in the nearby town. In my case and for the purpose of the pilot version of Noirscape, this is the French town of Bourg-en-Bresse.
While the allure of Noir is an assumption I hold about the target audience; it is just as important that the audience is interested in role-playing, interacting with other participants and within the context of their local town. I made assumptions about attitudes towards a product which is tailored to a person’s home town as well as the concept of leaving one’s mark withing the fictional world of the experience. In order to test and respond further to these assumptions I carried out some primary research which I have detailed below.
A demo of my vintage rotary phone dialler build with Flutter with sound effects and number dialling recognition.
I created a telephone dial using Flutter’s CustomPainter API. This permitted me to draw the various elements onto the screen relative to the size of the screen. The problem now is : how to detect which number has been dialled.
The Noirscape participant collects fictional objects in the Augmented and Immersive reality parts of the experience. Beyond the fun of finding these items and seeing them appear in-situ on your living room table; they also provide important props and gameplay. The objects are associated with actions. Sometimes an object will have multiple actions, sometimes they’ll depend on other objects to work together. Here are a couple of examples of interactive fictional objects I’m currently developing.
While, in itself, the narrative of Noirscape is to be interpreted at will by it’s participants, the structure of the environment in which the experience will take place will of course provide the underlying props and backstory along with the opportunity for interactivity with these props, people and different types of spaces – fictional, semi-fictional and realworld.
I was able to scale my 3D object correctly using a combination of GLTF settings and ARCore config. With some Shader work within Flutter I’ve created a Noiresque look in which the vintage 1940’s 3D telephone I got from Sketchfab (see link in video description) is positions consistently in the AR of ‘Mixed Reality’ world based on the detected horizontal Plane, the Pose of the object and of course the World Coordinate Space.
In order to get to the bottom of the issues I have been experiencing with ArCore for Flutter I decided to test and document different combinations of factors. Given that there is a multitude of possible test cases with different types of 3d object file, different versions of Sceneform and heaps of other stuff; I wanted to test each case one by one and record my findings, as I’m convinved this is doable with Flutter and ARCore.
I encountered a number of issues with getting complex 3d objects into a Flutter AR app. The Flutter AR components takes either a remote glft file, a format designed for loading 3 objects in a web browser; or sfb a format specific to Sceneform, a 3D rendering component for smartphones.
The Noirscape app narrative design features several conceptual layers.
The boxed product is sold on the high street in selected specialised stores. The product’s cover design will be highly stylised to the ‘Noir’ look and feel. It is clear from the cover illustration that this is a hybrid board-app game.
To design an immersive and interactive smartphone application using 360 video and GPS to send users on a film noir themed investigation into the physical space of their local town.
I am currently working on a conceptual design document inspired by BJ Fogg, founder, and director of the Stanford Behavior Design Lab.
Following on from my reflection and study-notes about the porous boundary between the fictional [or synthetic] world of electronic narrative platforms and the realworld, I would like to now switch my focus to the role of realworld within the scope of interactive networked narrative; I shall focus particularly upon the idea of spatialised narrative i.e. the intersection of space and narrative; with the aim of exploring the synergistic potential of combining space, place, computing and interactive narrative.
In my previous post I compiled some of my thinking and study notes around the notion of implementing interactive, branching narrative using the smartphone medium. This got me thinking more about the uniqueness of the smartphone app as a creative interactive medium as opposed to book or video. I considered how the Choose Your Own Adventure books I played as a kid were in fact book-games just as games I played on a tv screen or monitor are video-games. More recently I played the 2018 Netflix movie-game called Black Mirror: Bandersnatch.
Like so many other kids growing up in the 1980’s I spent a heck of a lot of time playing Choose Your Own Adventure (CYOA) books as well as playing/making text based adventure games for my Commodore and Sinclair home computers. I recall spending hours upon end staring at a tv screen with a flashing cursor, writing simple commands within a text game : go west, sit down, pull chord, enter hole. Often I’d be stuck in a seemingly neverending sequence of typing the same commands and just not being able to unblock new content.
During my recent experiments I have been working with Adobe After Effects to apply special effects and animations to 360 degree video footage. My first experiments were quite successful. I was able to do what I set out to do, with a couple of areas that would need some further attention. One such area was the exportation format from After Effects which was producing an unusual effect whereby the 360 image was not wrapping correctly.
360 video files can be quite big, the short ones I have been experimenting with are around 10Mb and even on a fast wifi connection there was a significant delay while the video is downloaded in its entirety before the 360 video player starts playing. This would provide a poor user experience in terms of the wait. I could potentially add some gameplay functionality into this waiting time; but it’s still a big ask – to download large files; especially if the user is outdoors and using mobile data, which may be limited or even expensive to the user.
Following on from my 360 video experimentation in Adobe After Effects I wanted to test out options for embedding my 360 videos within a a mobile app using Flutter.
I would like to explore the possibility of shooting studio footage – for example of character acted scenes, using a green screen which would be integrated into the town footage. This would mean I could reuse the same studio footage in multiple towns and therefore offer the app to a greater number of users. As my theme for the app is Film Noir, I could also use other props and footage such as period vehicles and so on.
The aim would be to create a ‘Pilot’ version of the interactive experience at a local town, to which I have easy access and which I know well.
I would be looking to pitch the pilot to the local population as well as the tourist board and local newspaper.
Although I consider this app to be foremostly an artistic endeavour, this doesn’t mean I can simply ignore user research. A creative app is quite pointless if no-one ever interacts or engages with it. My hypothesis is that that people react with greater curiousity and desire to engage with a creative piece when they feel […]
Over the last 6 weeks I’ve spent much time ideating for a new app. I know that I want to make good use of the technical skills I’ve been developing over the last couple of years in augmented reality, digital illustration, applied machine learning and somehow bring it all together with my gamification research as well and my philosophical interests.
It’s been a couple of years since I became keenly interested in cross reality (XR) app design and development. To begin with, I was experimenting with Google Daydream VR (virtual reality) and had also recently acquired an Oculus Go headset; which had become widely available and popular. At this time I began thinking about ideas […]
Although I’ve built several mobile apps previously using web technologies Cordova and Ionic, I was never fully satisfied with the quality of the end products. In most cases, this was because of the un-native ‘look’ of the web based ‘hybrid’ apps. I recall experiencing much frustration using native plugins too.