BRIEF: 3. A self-initiated project that critically and imaginatively explores an aspect of media design / network media.
Short Proposal: I intend to create an app based locative audio experience which will attempt to change perspectives and challenge expectations.
Long Proposal: I intend to create an app based locative audio experience which will attempt to change perspectives and challenge expectations. The app will be similar to an audio guide of the city of Bristol, but not told through the voice of a narrator in an educated position of influence. It will instead, be told by members of the public, the people that interact with locations in Bristol the most, often on a daily basis. In my opinion, they are in a better position to tell a history of a place through their own personal experiences and memories, than one person with a fact sheet of a place. I also think it’s a more interesting historical document than a personal, general history of a location.
The locations used in the app will be in Bristol, specifically selected along a designated, predetermined route for the user of the app to walk. You will hear from people who have stories to tell from the past and present. I am interested in the dialogue between a varied cross-section of different people in society, expressing their personal viewpoints, memories, histories and stories of the same location, whether that’s the cleaner of the building, the managing director and owner of the building or the homeless person outside who’s never been inside the building. All having a dialogue with one another, inside the app audio and consequently inside the ears of the user, painting a picture inside their mind, hence naming the locative app ‘Mind’s Eye’.
To collect the audio to use in the creation of the soundscapes for the app, I am considering building a method of digital submission of MP3’s, where participants can record their responses and address a few questions about a location featured in the app along the route, which they may have a personal experience, history with or a story to tell. Then sharing this open call for responses on social media to encourage people with stories to respond. In order to not be at risk of excluding certain unrepresented minorities in society, for example, those who do not have access to social media and the internet, like the homeless. To get this audio, I will personally go out and manually collect audio interviews and stories of my own. This combined with the internet submissions should be more than enough content in order to construct a cohesive audio narrative for the app to be immersive and function successfully.
I will make the app using onsen.io as a framework for the app along with Google’s firebase server service.
RESEARCH – Sources of Inspiration:
Wings of Desire – Wim Wenders: The 1987 German made film Wings of Desire by Wim Wanders was one of the main sources of inspiration for this project. The film revolves around two angels, Damiel and Cassiel, who glide through the streets of Berlin, observing the bustling population, providing invisible rays of hope to the distressed but never interacting with them. There is a key scene in the film (see below) where the camera pans along a Berlin subway car and you can hear the thoughts of all the passengers on board. I like the idea that you hear all the passengers thoughts and opinions, and there is no hierarchy or preference. You hear what they are thinking, no matter how seemingly meanless or trivial it may be. It made me think of the idea of producing an audio guide but told by the people that experience the space first hand. As they are probably in a better position to tell the history than a historian or ‘expert’. I also thought this more candid form of storytelling would be more interesting for the listener, as they would feel like they were being let into local secrets and inside knowledge.
The Memory Dealer – Rik Lander: The Memory Dealer: Another source of inspiration for this project is The Memory Dealer by Rik Lander, it is an interactive drama that uses a smartphone app, live performers and installations that take place in several places around the center of Bristol, in an attempt to immerse the audience in the story. The Memory Dealer is a form of interactive theatre where you become an integral part of the story, I had a chance to experience this project first hand and I think that is where the real immersion lies, feeling like a fundamental piece of the storyline. Unlike most plays and novels where you are not a rudimentary part of the story, you are just a third party watching the story unfold. In The Memory Dealer, you have to walk to certain places and communicate with characters in the story and carry out tasks, which makes the whole drama much more immersive and captivating because you have a vested interest in the outcome and feel a sense of responsibility.
It Must Have Been Dark by Then – Circumstance: Last year I was fortunate enough to experience this project first hand and meet the creator Duncan Speakman. The project uses an audio soundscape embedded in an app and a book which you read in tandem with roaming the city. It is extremely immersive and very well made the soundscapes and narration are beautiful. It was a big inspiration for this project.
They Live – John Carpenter: I’ve chosen the 1988 cult film They Live by John Carpenter as a source of inspiration because the main character Nada retrieves a box of sunglass that reveal a hidden reality: the media and advertising hide omnipresent subliminal stimuli to obey, consume, reproduce, and conform, thus explaining humankind’s passive attitudes towards progress and obsession with the banal, while many of the elite are actually grotesque aliens who look like animated corpses. I think the sunglasses are a great metaphor for the changing of one’s perspective which is what I am trying to achieve with my app based immersive audio experience.
The Cartographer’s Confession – James Attlee:
Post-Truth Guide – Peter Bennett:
Transgressing Boundaries – Radley Cook, Levi Giles & Will Grant: Transgressing Boundaries is an audio documentary investigating the perspectives of; skateboarders and members of the public about how spaces used for street skateboarding in Bristol impacts the community in a social, political and cultural way. As well as, examining where the boundaries lie between public spaces and skateparks.
ACADEMIC RESEARCH – Sources of Inspiration:
Guy Debord – The society of the spectacle: The ‘Situationist International’, was a relatively small Paris-based group of influential, avant-garde artists and intellectuals, best known for their radical political theory and their influence on the May 1968 student rebellion and worker revolts in France. Situationists increasingly applied their critique not only in culture but to all aspects of capitalist society. Guy Debord emerged as the most important figure. Debord applied Marx’s ideas to mass communication, showing how capitalism has penetrated not just what we produce and consume, but how we communicate. The Situationists characterised the whole of modern capitalist society as an “organisation of spectacles” this is clearly a cynical statement, I believe he is suggesting that the ‘spectacle’, as manifested in mass entertainment, news, and advertising, alienates us from ourselves and our desires in order to facilitate the accumulation of capital and is used as a smokescreen, that can be manipulated and leveraged to deceive us. I believe we can harness these elements and use them to captivate and immerse users in digital experiences.
Marshall McCluhen – Understanding Media: Marshall McCluhen is famously quoted in his seminal book Understanding Media, ‘The Medium is the Message’. He is suggesting that the medium affects the society in which it plays a role, mainly by the characteristics of the medium rather than the content. This couldn’t be more relevant to my project, because I don’t believe that the project could exist in any other medium and still have the same effect, it’s not the history of the city as told by the people that is important, it’s the way that it’s delivered. You could transcribe everything that people said and put it in a book, but I don’t believe that it would have the same effect. So despite McCluhen writing in 1964 way before any of the modern technologies I plan on implementing, his sentiments are still very applicable to this day and age.
Charles Baudelaire – The Painter of Modern Life: One could argue that no one’s writings would be more relevant to this project than Baudelaire’s writing of flânerie in his essay, ‘The Painter of Modern Life’. Flânerie is the art of strolling and looking, commonly associated with the shopping arcades of late nineteenth century Paris. He describes the anonymous man on the streets of Paris, drifting through an urban crowd, strolling as a detached observer, part of the crowd yet also removed from it. This is exactly how I would describe being a participant of an immersive audio experience if you are doing it in a built-up metropolitan area. You feel part of the crowd because you are present in the space but removed by what you are hearing, it’s like you are in a haze or a bubble. I think this text and writing around the idea of flânerie, by Walter Benjamin, for example, is extremely relevant to my project. Whilst researching flânerie, I discovered a new term called, ‘cyberflâneur’, which is a version of the word flâneur, for the digital world in which we live in today. Cyberflâneur, is a term for surfing the (Geocities) arcades of the world wide web, with no particular place to go. The idea of the flâneur, for Baudelaire was a man who could “reap aesthetic meaning from the spectacle of the teeming crowds, the visible public of the metropolitan environment of the city of Paris”, for me that is what audio can do for a user, it can inject context and meaning in to the environment around us. I think the ideas around the flâneur are more relevant today than ever before, I think we are all capable of being immersed in a crowd whilst also being detached and removed from it observing the environment around us.
WIREFRAMES & UX PROTOTYPING: Below you can see a screenshot of my artboard in Sketch, with a series of interconnected wireframes I used to develop the user experience for the Mind’s Eye app. This was a very helpful process, I asked friends, family and lecturers for feedback based on the interactive wireframes and this helped me develop the user experience for the app which proved very beneficial. You can see the live version of the wireframes and test them out for yourself here.
GRAPHIC VISUALISATIONS: Below you can see a series of graphic visualisation mock-ups of the interface design for each app screen, as you progress through the audio experience. I made this to primarily assist me with the CSS styling inside the Onsen framework and also to have a consistent aesthetic to work towards. The design only consists of 3 colours white, black and orange (#F7A70B). I wanted to keep it simple to match the experience design of the app which is also simple, I also like the challenge of forcing myself to only use 3 colours that are very contrasting. I used the basic wireframes I made in Sketch as a starting point to add detail and colour.
PROJECT CHANGES: It is around this point where there were significant changes to the project idea. I realised that it would be too complex and ambitious to attempt to build the recording functionality into the app as well as positioning the sounds on the map with pins for users to find. The initial idea development was to enable users to record memories and stories about the location at the location. But the more I thought about it, going out myself and collecting the content and authoring it, was actually a better idea and it enabled me to collect a much wider variety of content for different people. Because, if I made it so you could only upload to the app if you have the app, that is creating a prejudice towards those who don’t have smartphones because they can’t afford one if they are homeless for example. I would be imposing a bias towards those who have smartphones, therefore, the submissions would be less diverse, and the point of the project is to hear a cross-section of societies history of urban space. Another element that I wouldn’t have to create is separate user logins, profiles and databases, allowing them to upload their own recordings in the app, would have been totally unachievable in the given timeframe, I would also have to monitor submission and then accept them and push them back to the app.
CODING PROCESS: The coding process was lengthy, it involved making a series of separate prototypes and then combining them together to create the final project. As you can see from the image below each integral element of the project I made as an individual prototype. For example: placing multiple pins on a map, creating custom pins, panning the map based on movement, drawing a geofence on a map, triggering an alert when you enter a geofence, creating multiple geofences, playing audio, randomly playing audio, play and pause button image change, randomly playing an audio file when you walk into a geofence, styling the look of the Google map, plotting a route on the map, creating the onsen navigation, styling the default onsen elements, creating and styling a slide up dialogue box and button styling. Each individual prototype had to be fully working and tested before they were combined together and often combining them together was the most difficult part of the process as incorporating one prototype would break something else like the page navigation system.
LOGO ANIMATION: Below you can see the process involved in making two Mind’s Eye logo animations using Adobe After Effects. I exported them out of After Effects as .mp4’s and then opened the video files in Photoshop and exported them out as endlessly looping gifs for use in the app. The main lengthy process was producing the artwork for each frame of the animation in illustrator, once that was complete the process was quite quick inside after effects.
APP TESTING & DEVELOPMENT: Below you can see evidence of user testing at different stages of the project development. This kind of testing, feedback and iterating was integral to the success of the project. I found that people instantly understood the concept of the app and what to do, and the overall feedback was positive, however, some users closed the app to listen to audio soundscape consequently closing the app and cutting off the soundscape, this could be something to explore for version 2.0 of the app development.
AUDIO: Collecting good audio was integral to the success of this project. Before I collected the final audio clips for the app, with members of the general public around the Harbourside area. I wanted to test the idea and see if it would be as effective as I thought it might be. To do this I made a concept prototype (you can listen to below), which consisted of me and some of my friends describing stories or memories from the harbourside area specifically outside the Arnolfini art gallery, the stories were approximately one minute long. I edited the clips I collected together using Adobe Audition, I applied a crossfade between the clips to make it sound like one person was walking away into the distance and the next was walking towards the listener. Once I had this edited version, I put it on my phone to test it in the location. I went down to the harbourside area and played the audio. I think that the concept worked really successfully, it definitely made me look at the space around me differently and really painted a picture in my mind as the story was being described and being in the space they were describing really helped with that. There was one thing however that I didn’t think worked as successfully with the concept prototype, it clear to me listening to the recordings in the space in which they were describing, they were clearly not recorded in the space, they were recorded in a quiet room and not standing outside the Arnolfini whilst they were describing the story. I, therefore, decided it was integral from that point onwards that all the recordings must be captured in the location, so that the listener can hear the natural background hustle and bustle, that they will hear in the space anyway, it also attempts to not take them out of the place they are standing in too much. The idea of Mind’s Eye is to add another layer or perspective to the location that the user is standing in, not take them out or away from that location with the audio.
Below: You can listen to a 20-minute long audio file I created by combining all the audio I collected outside the Arnolfini gallery and around the Harbourside area. I approached everyone who was willing to talk and asked them: ‘What does this area mean to you?’, ‘Do you have any stories or memories about this area?’ and if they have known the area for a long period, ‘Could you describe how the area has changed over time?’. I tried to get a very wide spectrum of participants involved in the project whether they were; buskers, homeless people, ice cream salesmen, old, young, locals or tourists. I like the contrasting opinions, knowledge bases, insight, and depth in the stories that participants told. To me, it doesn’t matter how seemingly interesting the stories are at face value. The important thing is the relevance, that all the stories are tied together by a single location in which they were recorded and that people were describing, that’s what’s important. I was really pleased with the responses I captured, people were much more candid, honest and open to talking about big important subject ideas than I thought they would be. I almost felt that they wanted to be given a platform to share them. I think when you listen to the audio below, you get an accurate picture of the place, in the same way, you would through an audio guide, which is exactly what I intended. The only thing I would like to improve about the audio I collected, would be going back to the location and collecting some more audio on a less sunny day than when I made the recordings to see if the general mood amongst people would be different and consequently a change in responses to questions.
REFLECTION & EVALUATION: I set out to create Mind’s Eye, an app based locative audio experience which attempts to change perspectives and challenge expectations about urban spaces. The app is comparable to an audio guide of a city. However, unlike most audio guides, Mind’s Eye portrays a genuine and comprehensive recent history of a city, told by the people experiencing it first hand. Individuals who may be unfamiliar with the city or have an established history with the area. I believe that everyone’s perspective and opinion is important and should be listened to. I also think that these individuals are in a better position to tell its history, over an academic because they have personal experiences, emotions and memories attached to the place.
I collected stories from people across the city for users to stumble upon. Through experiencing Mind’s Eye, users might discover something new about their city, hear from people with different backgrounds and conflicting perspectives. Users will listen to vastly different stories, varied knowledge and expertise about shared urban spaces, all of which will be unfiltered and unstructured. I believe this makes learning about the modern history of a place fundamentally more interesting. I think that Mind’s Eye is a true historical document of modern day society, as it gives users a real untouched stream of instant raw emotion and perspective on the environment around them.
The Mind’s Eye app went through many iterations these iterations mainly involved simplifying the app and considering the user experience over the user interface. I think initially, the app had too many pages. I learnt that for audio experience apps, in general, you do not want the interface design of the app to confuse, over complicate or distract the user from the audio itself. The audio should be the main focus and the design should facilitate this. As a result, a lot of the app design is in the user experience and the code; the construction of the audio triggered by entering geofences using the user’s geolocation involves a lot of work behind the scenes. I wanted to encourage the user to look up at the environment around them, not down at their smartphone which is difficult and unintuitive when designing an interface, you generally want to do the opposite. I think this method of enabling the user to appreciate the context around them and not encouraging them to interact with anything directly on the app interface, allows them to focus more of their attention on the audio and its relationship with the scene around them. In this day and age, the attention economy is big business, most apps, especially social media are competing for our undivided attention. I think there is a gap in the market for an app that has the opposite approach, an app that still adds another layer to our lives through technology, but isn’t doing this through encouraging us to look down at our screens. I believe this is the reason that podcasts have become so popular in the last few years, they are a relatively old concept, Apple added podcasts to the iTunes store in 2005. I believe we have seen this resurgence in radio, products and long-form audio as a result of people wanting to consume content whilst appreciating the world around them, and not being distracted by their phone screens. The great thing about audio is that it allows you to continue living your life and appreciate your surroundings whilst also learning, consuming and changing your perspectives and challenging your assumptions.
In the final version of my app, the user simply has to walk towards a marker on the map, they are usually arranged along a predetermined route. As they walk towards the map marker, an audio soundscape will automatically play when you enter the area around the marker. The audio soundscape they receive is randomly selected and regularly updated with new audio as stories are collected, so it is likely that each user’s experience from one another. Therefore if you run the app again it is probable that the soundscape you hear may be different. In an earlier wireframe iteration, I thought the user should travel to the pin using the map, then tap the pin once they arrive at the location to trigger the audio. For example, I could have added the formation slide up the panel as a separate page but I felt this is unnecessary. I think a lot of the experience design came out from the wireframing process and using interactive wireframes from within sketch to test the user experience with friends and family and myself at the location with the test audio.
I believe that the core idea of the app is successful. As I said previously, the success of the project was heavily dependent on the quality and variety of interviews I collected from members of the public in urban spaces and the stories and memories they had to share. I think that the audio I collected for the app is a great start, I plan to collect more to increase the apps repository of audio interviews along with their variety to build upon what is already there to make the project more interesting for users and hopefully increase users returning to the app. The app is still very much in its infancy as it is only the first version, but I believe the app has scope to grow, evolve and expand. The cornerstones are there, from building this first version of the app, it is clear that the core functionality of the idea works as an immersive and captivating audio experience. Ideally, I would like the app to work without it being open on the user’s smartphone. I think it would be most effective if when you walked into a geofence it triggered a push notification on the users home screen and users could just play it by unlocking their phone, which would open the app and trigger the audio to play. Instead of actively seeking out the hotspots indicated by pins on the map across the city. I can imagine a time where there are lots of hotspots across all cities, this would increase the likelihood of entering one. Most people walk around cities today with Airpods or headphones in. Therefore they could easily switch from listening to music or a podcast to experiencing temporary immersion in their daily commute to work for example through Mind’s Eye.
On the other hand, however, there are a few improvements and a few small bug fixes to be made to the app. If I were to make a second version of the app, instead of writing the code so that it randomly selects just the one clip at each hotspot. I would write the code so that it randomly selects a few audio clips and plays through those clips. The reason for this being that the clips vary in length quite massively this means that some clips are 15 seconds and some are 5 minutes long. I discovered through testing that it is much better is the audio track gets you from one hotspot to another which works very well, but occasionally you get a short audio clip and you are left to walk the rest of the way to the next marker in silence which is a bit disconcerting. Another example is the pinpoint location button which doesn’t function properly. I would also like to make the pins clickable, with a small popup dialogue that has some information about each location and how many audio clips were captured in the area. Finally, I would like to develop the website and the submission element of the app, to engage further with the community and make it easier for the public to join the conversation. Mind’s Eye is a community focused app and my initial objective was to involve as many people as possible and give them a platform to tell their story, personal history and collaboratively make their own audio city guide of the urban place in which they live. Therefore I want to make it as easy as possible for participants of the app to do this. I believe that this starts by firstly, spreading the word that the app exists and then producing a clean easy to use website with a simple submission system, accompanied by a social media campaign and a physical advertising campaign, by pasting up posters across cities for example, would be a good start in encouraging app adoption and getting the word out.
DEGREE SHOW: For the degree show exhibition, I will produce a short cinematic film demonstrating how the Mind’s Eye app works in a clear visual way. I think this is important because I imagine most of the people at the degree show won’t go outside to test the app straight away. I want to give people an idea of how the app works in its intended environment, I hope the degree show encourages them to download the app, which will be available on both the Apple App Store and the Google Play store for them to download at the show. However, I feel that users get the most out of the app when using it in its intended environment. Therefore, I want to give the viewer an idea of its indented function of use and encourage them to do that. To overcome this hurdle, as well as the promotional and instructional film that I will produce I will also create an iteration of the app especially for use at the degree show, which will be without about the locative elements of the app and just showcase the audio I collected from members of the public. The idea is that visitors to the degree show can put on some headphones and press a button on the iPad to randomly listen to one of the 20 different audio recordings I collected from different Bristol visitors and locals, to give visitors a taste of the kind of audio that is used in the app. Furthermore, if viewers do not want to listen to the audio on the iPad, there will also be display boards behind the iPad stand, with a selection of quotes taken directly from the recordings used in the app which they can read. I believe that all of these elements combined should encourage visitors to download the app and participate in the whole Mind’s Eye audio experience. In addition, to make the ap download more efficient, I will install an NFC tag on the plinth supporting the iPad stand, with a quick link to download the app straight to degree show visitors smartphones.