Augmented and Mixed Reality Design for Contested and Challenging Histories: Postcolonial Approaches to Site-Specific Storytelling

Published paper: Augmented and Mixed Reality Design for contested and challenging histories

This paper presents a series of mobile Augmented Reality Experiences focusing on the postcolonial history of various sites in Copenhagen, Denmark. The applications—Bitter&Sweet & Finding Alberta—are part of a university-led research project, situated at the now closed Royal Cast Collection of the Copenhagen National Gallery and in the city itself. Using site-specific archival material for contemporary experiences in an urban landscape, the design process deals in part with contested sites and sensitive histories, while the application tells the story of colonial pasts in a Danish context. We will discuss the process of designing for a locative, embodied experience; the challenges of providing an embodied and compelling experience for visitors both inside and outside of buildings; and the challenges of designing for the wider urban landscape, given the constraints of a small screen and the limitations of the mobile device.

Bibliography:
Günter Alce, Klas Hermodsson, and Mattias Wallergård. 2013. WozARd: a wizard of oz tool for mobile AR. In Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services (MobileHCI '13). ACM, New York, NY, USA, 600-605. http://dx.doi.org/10.1145/2493190.2494424
Bill Ashcroft, Gareth Griffiths, and Helen Tiffin (eds). 2009. The Post-Colonial Studies Reader. Routledge, London.
Ron Azuma. 1997. A survey of augmented reality. In Presence: Teleoperators and Virtual Environments. 6,4 (August 1997), 355-385.
Evan Barba and Blair MacIntyre. 2011. A scale model of mixed reality. In Proceedings of the 8th ACM conference on Creativity and cognition (C&C '11). ACM, New York, NY, USA, 117-126. http://dx.doi.org/10.1145/2069618.2069640
Evan Barba, Blair MacIntyre, Rebecca Rouse, and Jay Bolter. 2010. Thinking inside the box: making meaning in a handheld AR experience. IEEE International Symposium on Mixed and Augmented Reality--Arts, Media, and Humanities, Seoul, 19-26.
Jeannette Allis Bastian. 2003. Owning Memory: How a Caribbean Community Lost its Archives and Found Its History. Libraries Unlimited, Westport, CT.
Steve Benford and Gabriella Giannachi. 2011. Performing Mixed Reality. The MIT Press, Cambridge, MA.
Mark Billinghurst, Adrian Clark, and Gun Lee. 2014. A survey of augmented reality. Foundations and Trends in Human-Computer Interaction. 8, 2-3, 73–272. http://dx.doi.org/10.1561/1100000049
Bogen, M., & Krauss M. (2010). Conveying Culture Heritage and Legacy with Innovative AR-based Solutions. (Trant, Jennifer, David Bearman, Ed.).Museums and the Web.
José M. Braz and João M. Pereira. 2008. Tarcast: Taxonomy for augmented reality casting with web support. The International Journal of Virtual Reality. 7.4, 47–56.
Bill Buxton. 2007. Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann, San Francisco, CA.
Clive Dilnot. 2015. History, design, futures: Contending with what we have made. In Design and the Question of History. eds. Tony Fry, Clive Dilnot, and Susan Stewart. Bloomsbury Academic, London, UK.
Judith Marlen Dobler. 2013. Lines & models. Embodied drawing acts. In Nordes 2013: Experiments in design research. 5, 484-485.
Paul Dourish. 2001. Where the Action Is: The Foundations of Embodied Interaction. The MIT Press, Cambridge, MA.
Paul Dourish. 2013. Epilogue: Where the action was, wasn't, should have been, and might yet be. ACM Trans. Comput.-Hum. Interact. 20, 1, Article 2 (April 2013), 4 pages. http://dx.doi.org/10.1145/2442106.2442108
Maria Engberg and Jay David Bolter. 2015. MRx and the aesthetics of locative writing. Digital Creativity. 26:3-4, 182-192. http://dx.doi.org/10.1080/14626268.2015.1100120
Katherine J. Goodnow and Haci Akman (eds). 2008. Scandinavian Museums and Cultural Diversity. Berghahn Books, New York. http://doi.org/10.1007/s00779- 003-0238-7
Mads Hobye. 2014. Designing for Homo Explorens. Open Social Play in Performative Frames. Ph.D Dissertation. Malmö University, Malmö, Sweden.
Lilly C. Irani and Paul Dourish. 2009. Postcolonial interculturality. In Proceedings of the 2009 international workshop on Intercultural collaboration (IWIC '09). ACM, New York, NY, USA, 249-252. http://dx.doi.org/10.1145/1499224.1499268
Lilly Irani, Janet Vertesi, Paul Dourish, Kavita Philip, and Rebecca E. Grinter. 2010. Postcolonial computing: a lens on design and development. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 1311-1320. http://dx.doi.org/10.1145/1753326.1753522
Susan Kozel . 2012. AffeXity: Performing Affect with Augmented Reality. In The Fibreculture Journal issue 21, 2012
Susan Kozel. 2017 (forthcoming). Devices of Existence: Contact Improvisation, Mobile Performance and Dancing through Twitter. In Improvisation and Social Aesthetics, eds. G. Born, E. Lewis and W. Straw. Duke University Press. http://twentyone.fibreculturejournal.org/fcj-150-affexity-performing-affect-with-augmented-reality/#sthash.kA5q0Zmy.dpbs
Dominick LaCapra (ed). 1991. The Bounds of Race: Perspectives on Hegemony and Resistance. Cornell University Press, Ithaca, NY.
James A. Landay and Brad A. Myers. 1995. Interactive sketching for the early stages of user interface design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), Irvin R. Katz, Robert Mack, Linn Marks, Mary Beth Rosson, and Jakob Nielsen (Eds.). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 43-50. http://dx.doi.org/10.1145/223904.223910
Minkyung Lee and Mark Billinghurst. 2008. A Wizard of Oz study for an AR multimodal interface. In Proceedings of the 10th international conference on Multimodal interfaces (ICMI '08). ACM, New York, NY, USA, 249-256. http://dx.doi.org/10.1145/1452392.1452444
Jonas Löwgren. 2004. Animated use sketches as design representations. interactions 11, 6 (November 2004), 22-27. http://dx.doi.org/10.1145/1029036.1029048
Henry Mainsah and Andrew Morrison. 2014. Participatory design through a cultural lens: insights from postcolonial theory. In Proceedings of the 13th Participatory Design Conference: Short Papers, Industry Cases, Workshop Descriptions, Doctoral Consortium papers, and Keynote abstracts - Volume 2 (PDC '14), Vol. 2. ACM, New York, NY, USA, 83-86. http://dx.doi.org/10.1145/2662155.2662195
Elena Marquez Segura, Laia Turmo Vidal, Asreen Rostami and Annika Waern. 2016. Embodied Sketching. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16). ACM, San Jose, CA, USA, 6014-6027.
Michael McEachrane (ed). 2014. Afro-Nordic Landscapes: Equality and Race in Northern Europe. Routledge, New York.
Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 77.12, 1321–1329.
Temi Odumosu. 2015. Blind Spots (A Traveller’s Tale): Notes on Cultural Citizenship, Power, Recognition and Diversity. Museums – Citizens and Sustainable Solutions. Danish Agency for Culture. 110-119.
Ross Parry, ed. 2010. Museums in a Digital Age. Routledge, London and New York.
Mary Louise Pratt. 2008. Imperial Eyes: Travel Writing and Transculturation. Routledge, London.
Rebecca Rouse, Maria Engberg, Nassim JafariNaimi & Jay David Bolter. MRx design and criticism: the confluence of media studies, performance and social interaction. Digital Creativity, 26, 3-4 (2015/10/02 2015), 221-227. http://dx.doi.org/10.1080/14626268.2015.1100124
Margriet Schavemaker and Hein Wils, Stedelijk Museum; Paul Stork and Ebelien Pondaag, Fabrique, The Netherlands. “Augmented Reality and the Museum Experience.” 2011. Museums and the Web. April 6-9, 2011.
Johannes Siapkas and Lena Sjögren. 2014. Displaying the Ideals of Antiquity: The Petrified Gaze. Routledge, New York.
Smørdal, O., Liestøl, G. & Erstad, O. (2016) 'Exploring situated knowledge building using mobile augmented reality', in QWERTY 11, 1 2016, pp. 26-43.
Gheric Speiginer et al. 2015. The Evolution of the Argon Web Framework Through Its Use Creating Cultural Heritage and Community–Based Augmented Reality Applications. 17th International Conference, HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Proceedings, Part III. New York: Springer International Publishing
Nick Starbuck and Carsten Junker (eds). 2014. Postcoloniality - Decoloniality - Black Critique: Joints and Fissures. Campus Verlag, Frankfurt.
Alex S. Taylor. 2011. Out there. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 685-694. http://dx.doi.org/10.1145/1978942.1979042
Jakob Tholander, Klas Karlgren, Robert Ramberg and Per Sökjer. 2008. Where all the interaction is: sketching in interaction design as an embodied practice. In Proceedings of the 7th ACM conference on designing interactive systems. (DIS’ 08). ACM, New York, NY, USA, 445-454. http://dx.doi.org/10.1145/1394445.1394493
Leena Ventä-Olkkonen, Maaret Posti, Olli Koskenranta, and Jonna Häkkilä. 2014. Investigating the balance between virtuality and reality in mobile mixed reality UI design: user perception of an augmented city. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI '14). ACM, New York, NY, USA, 137-146. http://dx.doi.org/10.1145/2639189.2641201
Ina Wagner, Wolfgang Broll, Giulio Jacucci, Kari Kuutii, Rod McCall, Ann Morrison, Dieter Schmalstieg, and Jean-Jacques Terrin. 2009. On the role of presence in mixed reality. Presence: Teleoper. Virtual Environ. 18, 4 (August 2009), 249-276. DOI=http://dx.doi.org/10.1162/pres.18.4.249