Virtual reality and archaeological reconstruction: be there, back then

Albert Sierra, Agència Catalana del Patrimoni Cultural, Spain, Gabriel de Prado, Museu d'Arqueologia de Catalunya-Ullastret, Spain, Isis Ruiz Soler, Agència Catalana del Patrimoni Cultural, Espanya, Ferran Codina, Museu d'Arqueologia de Catalunya, Spain

Abstract

Keywords: archaeological site, virtual reality, storytelling, immersive room, archaeology

Introduction

Inside every museum visitor, there is an explorer trying to get out. Faced with an ancient object in a display case, or observing the ruins of a former palace, we can not help but wonder what they must have been like in their day. We wonder who made them, and what their owners used them for. We dream of traveling back in time to see these objects and buildings in all their splendor. Museums have used texts, drawings, videos, physical reconstructions and actors to help us imagine what the past was like. We now have a new medium, an extraordinarily powerful one, to achieve this aim.

Three-dimensional modeling has been with us for decades, but over recent years it has become more democratic, with programs that are much more accessible from the economic and technical point of view, and with the irruption of game engines such as Unity and Unreal Engine, which provide access to resources which were previously difficult to reach.

Heritage museums and institutions can now realistically consider having complex 3D reconstructions for a fraction of what it would have cost only a few years ago, and with infinitely better results; indeed, with a final image that is almost cinematic.

Moreover, over the last two years there has been a further development in the field of historical reconstruction: the ability of these game engines to easily create 360º videos and virtual reality experiences.

For those who might not know, virtual reality (VR) is becoming a mass phenomenon, with the appearance of VR headsets such as HTC Vive, Sony PlaystationVR, Oculus Rift and Samsung Gear VR. They all have similar features, and by means of a headset, which incorporates a screen, and headphones and sensors to detect the movement of the head, they take over the senses. When it is turned on, viewers are immersed, no matter where they look, in an entirely virtual environment, which envelops them and which can be incredibly realistic.

With the Ullastret 3D and VR project we have tried to take full advantage of this new medium’s potential, so that visitors to the ruins of the Iberian archaeological site can fully imagine what it was like, comprehend its architecture, and understand how people once lived there.

Ullastret, a 2,200-year-old Iberian Town

The Iberian town of Ullastret (6-2 B.C.) is located on today’s Empordà plain in Catalonia, Spain, and is one of the most well-known and outstanding Iron Age archaeological sites of the northwest Mediterranean (figures 1 and 2).

This large urban settlement was formed by two inhabited centers, one being the Puig de Sant Andreu  (Saint Andrew Hill) and the other the Illa d’en Reixac (Reixac Island), separated, one from the other by a distance of only 300 meters (figure 3).

Between them, the two settlements eventually extended over a walled area exceeding 15 hectares, and they constituted the capital of the Iberian tribe known as the Indigetes, or Indiketes, who were mentioned by such Classical authors as Avienus, Avieno, Ptolemy, and Strabo.

From a scientific point of view, the site was discovered around 1930 and was continuously and systematically excavated from 1952 onwards. Nevertheless, it is the research that has been carried out over recent years, based in many cases on the application of the latest generation of geophysical prospection techniques (figure 4) (Codina et al. 2015; Garcia-Garcia, Prado, & Principal 2016) that has produced the greatest knowledge with regard to its urban and defensive structures and the surrounding area (Codina, Martin, & Prado, 2012).

This has enabled us to devise and support a project for the virtual 3D reconstruction of the entire settlement based on very sound scientific knowledge which, in turn, has benefitted from the important advances made in the study of Iberian culture over recent years.

Because it is the best known period from a scientific point of view, one that therefore affords the best chances for producing the most accurate representation, the reconstruction focuses on a specific moment in the settlement’s history, around the year 250 B.C. It would have been interesting to produce a sequence that would have included the years prior to this, but knowledge of these periods is only partial and, in order to produce an accurate virtual representation, it would have been a very complex task to delineate such basic features as the settlement’s urban structure, for example.

The process of virtual reconstruction

The first step in the reconstruction consisted of an exhaustive compilation of all the available archaeological information. Nevertheless, bear in mind that, on occasion, we have had to employ comparative, theoretical and/or experimental models and that, in some cases, these have not been archaeologically proven. While they are based on well founded hypotheses, they will always be subject to future revision. Consequently, in addition to compiling already interpreted archaeological data, another important task during this initial phase was to reach a consensus about how to recreate something that, from what we might consider to be a tangible point of view, did not exist.

Once these data had been validated from a scientific point of view, they were thematically categorized and passed on to the design studio responsible for their virtual 3D texturing and modeling in accordance with a pre-established work sequence; the guidelines were inspired by the London Charter (http://www.londoncharter.org/). In this regard, the first aspect under consideration was the geomorphological structure of the settlements and the paleo-landscape, given that it was of fundamental importance to locate the settlement within its geographical context.

Some of the important aspects to bear in mind when modeling the landscape were the kind of vegetation that would have existed at that time and, especially, the transforming effect of human activity arising from the creation of access routes, agricultural exploitation, the extraction of mineral resources, and so forth.

The next consideration was the structure of the settlement itself, firstly based on recreating the defensive structures (wall and trench), and then by considering the urban layout to determine the position of the streets and thoroughfares, so as to recreate the individualized three-dimensional volumes of the various kinds of buildings and other constructions in the settlement (figures 5 and 6).

Finally, movable elements of the buildings’ interiors were reconstructed and incorporated, in addition to other kinds of artifacts, based on real items of Iberian culture found during excavations, such as the amphorae, pitchers, tools, swords and shields which now form part of the collection at Ullastret Museum (figure 7). Similarly, on the basis of comparative research, other organic elements, of which there are consequently no archaeological remains but which we know existed via other means, were also reconstructed.

At a technical level the model was built using 3D maps, provided by the Institut Cartogràfic i Geològic de Catalunya (Cartographic and Geological Institute of Catalonia), which make it possible to generate a geometrical mesh to represent the lay of the land. However, the lake of Ullastret, which was drained in the 19th century, had to be reintroduced virtually (figure 8). The topographic surveys carried out by the archaeological community over recent years were also used in a complementary way.

Three-dimensional outlines were made using the Cinema4D and 3D Studio Max applications, mainly in order to reproduce the landscape and its architectural features. Objects were mainly reproduced from vectorized drawings exported to a 3D format from which Maya software generated the model, adding texture and color.

All these models of terrain, vegetation, pathways, the lake, buildings and objects were then imported into the Unreal Engine 4 game engine which is outstanding for its ability to create landscapes and for the photorealist graphic quality it gives to images, in this case by means of textures created from photographs of real things (Making of the 3D Reconstruction: https://www.youtube.com/watch?v=Xtq94zHkPmw).

This software also enables the relatively simple export of the entire project to immersive virtual environments; for example, for multiscreen projection, 360º video, and especially, for virtual reality headsets.

As the virtual models were being produced they were checked for scientific accuracy from the outset, when only the volumes themselves were being represented, and then throughout the entire working process up to the production of the hyperrealistic image.

This procedure made it possible to introduce any necessary corrections along the way, and some of these corrections meant that some of the preconceived ideas about Iberian settlements, which until now were widely accepted and disseminated, had to be revised. In effect, what Ullastret 3D has enabled us to do is to stay up-to-date with the most recent research, create a new image of the Iberian town, and make it available to the general public in a very powerful and accessible way.

The outputs: Internet, immersive room, and virtual reality experience

After almost a year and a half of intensive work, the first phase of the project concluded with the creation and dissemination, via social media, of a video of the reconstructed Iberian town (https://www.youtube.com/watch?v=73HOao7W7oA&t=76s). It was very well received and had more than 30,000 visits.

Nevertheless, the aim of the modeling was not limited to the creation of this visual representation. After evaluating various options from the technical and economic viewpoint, we decided to develop two practical applications; an immersive room in which to project an audiovisual of the virtual reconstruction, and an adaptation of that reconstruction for viewing with virtual reality headsets.

The inspiration for the room comes from Cave Assisted Virtual Environment (CAVE) technology, which has been used in some museums for some years (Barceló, 2001), and the more recent Magic Box (for example the State Grid Pavilion at the 2010 Shanghai Universal Exposition). Both systems require an audiovisual projection room conceived in order to create a virtual reality immersive experience that can be viewed collectively.

The project undertaken at the Ullastret branch of the Archaeological Museum of Catalonia consists of a multi-screen projection visible on three of the room’s four walls, which is accompanied by a three-dimensional audio system synchronized with the movement of the images. Together they create an enveloping effect for viewers. A great deal of work has gone into the production of the sound so that it achieves a sonorous texture that matches the various virtual spaces that appear in the projection.

The aim is for the room, which covers an area of approximately 10 square meters, to be able to reproduce the dimensions and characteristics of an ordinary Iberian dwelling, such as those which appear in the projected audiovisual, as accurately as possible (figure 9).

Attention was also paid to the flooring, which is covered with recycled extended rubber that simulates the flooring and paving of the settlement’s dwellings and thoroughfares. Furthermore, this material has properties which, in combination with that of the walls and ceiling, create optimum acoustic conditions within the room.

The room is used to project a six-minute-long audiovisual that consists entirely of a journey through a digital model, filmed with virtual cameras. Viewers make a dream-like journey through the Iberian settlement, passing through its streets and empty houses. The story is narrated by a former inhabitant, a member of the elite, who remembers dramatic moments in the settlement’s history. In this regard an attempt was made with the concept and narrative structure to avoid the traditional format of a linear, descriptive audiovisual with a description and contextualization of the settlement. The Ullastret branch of the Archaeology Museum of Catalonia already has an audio guide service for this purpose, as well as a mobile phone application, which allows visitors to tour the site and discover the essential archaeological information as they go.

The audiovisual is therefore conceived as a complementary tool, a much more congenial one, that is especially useful for enhancing people’s knowledge about this settlement, as well as their ability to interpret it (Making of the Immersive Room:  https://www.youtube.com/watch?v=ys_X0XZ6_8U).

The audiovisual created for the immersive room was adapted, in collaboration with Visyon, a VR and 360 production company, to create a version for the HTC Vive Virtual Reality headset. These devices, unlike the immersive room, make complete and absolutely realistic immersion possible because viewers can navigate within the virtual space in a natural way and at a life-size scale.

In this case, and within certain limits, the device enables interaction with the virtual environment such that no two viewers ever have an identical experience.

The current version of this virtual experience is not intended to be permanently available at the museum itself, given that the original concept is identical to that for the immersive room and, although the experience is perceived in a different way and is more immersive, it is basically a duplication of the same resource.

Thus far it has been used to present Ullastret3D at specialized congresses such as Arqueológica 2.0, Valencia; Archeovirtual, Paestum, Italy; and so forth, and at popular events to raise people’s awareness about their cultural heritage, such as the Museum’s Night held in Barcelona and the Iberian Weekend, held in Ullastret, to name just two (figure 10). During 2017 it will become a permanent feature at the Archaeological Museum of Catalonia in Barcelona.

Narrative

As work started on the immersive outputs, a detailed study was undertaken to determine what kind of narrative would be most appropriate for this medium. A thorough study was made of existing 360º videos and available virtual reality experiences. It soon became clear that a traditional narrative would not be the most suitable. In our case it was essential to create a story that would justify the spectator’s presence in the most important places in the settlement. It was also necessary to use visual and auditory resources to guide the spectators’ gaze so that they might best explore this virtual environment. The script also had to contain key elements of Iberian culture such as the inhabitants’ totemic animals, the social prevalence of warriors, and the collapse brought about by the Roman Empire.

In the end, the story embroils visitors in the dream of a member of the settlement’s elite. Every night he returns to the settlement and remembers fragments of his life such as the first time he saw the walls of the town, the children playing in the street, the courtyard where he learned to fight, and the empty city after the downfall.

This narrative device makes it possible to use the protagonist’s voice to guide visitors around each of the scenes included in this dream-like environment.

For the immersive room it was decided that the 3D effect would be best appreciated if the main motif were to be the constant movement seen in the streets and buildings. For the virtual reality experience, in contrast, it was important to establish an exact point at which visitors would appear, and always in places that would encourage them to move around and explore, such as on the corner of a building; so that by moving sideways, an interior courtyard would become visible, or besides an open door beckoning people to enter.

Great attention was paid to the audio for both of these media, with a complex soundtrack perceived three-dimensionally by visitors whereby the sources of the different sounds can be located in the surrounding scene, the overall sensation being one of perfect and total immersion.

Public response

People have responded enthusiastically to both the individual VR and the group immersive room experiences. With regard to the latter, the impact created by some of the scenes, such as the flight over the Iberian town, and the appearance of objects which visitors can later see in the museum display cabinets, demonstrates that this medium is highly effective.

For most people this was the very first time that they had used a VR headset, and the reaction to it was incredible. The first thing to note is the fact that it is a medium suited to all ages, warmly received by children of twelve years of age, to people in their 90s. It is particularly interesting to note the impact it had on older people. Their experience was particularly vivid and emotional. Furthermore, it was a particularly striking discovery for people with restricted mobility. People confined to wheelchairs, for example, experienced a sensation of freedom of movement that was a revelatory experience for some of them, and this has led to us establishing contact with hospitals specializing in rehabilitation to offer such a service to them.

Virtual reality and cultural heritage

Is virtual reality a tool for disseminating cultural heritage? The answer to that question is, without doubt, a resounding “Yes.” It enables us to go where we have never been before with regard to the sensation of immersion. But let us make no mistake about it: it is still an absolutely new tool going through the first phases of development and for which, together, we are still drawing up the basic guidelines for its use. It would be very easy to get carried away by its current appeal and try to use virtual reality to create an experience that might be better approached with a more traditional medium, or it might only reproduce the real-world limitations of museum facilities, but now in a virtual environment.

In contrast with that, the way forward may be to immerse our visitors in worlds which would be impossible to visit in the real word. There are many possible examples. We could be present at an underwater excavation contemplating the remains of a sunken Roman ship. We could be inside a pyramid where access is prohibited. We could fly over a large building to fully appreciate its grandeur. All this implies traveling through space.

But it is also possible to travel through time and, as we have done in Ullastret, walk around the streets of a settlement that existed 2,200 years ago. Indeed, we can do this for any period of time we need to recreate. Dinosaurs? Ancient Egyptians? The Mediaeval world? The attack on Pearl Harbor? The sky is the limit.

But let us pause for a moment. What is the function of such journeys in the context of a museum, an historic or an archaeological site? What can be achieved?

We believe that there are two very different approaches. One is to recreate the experience of being in a place where people are denied access. Places, for example, that may not be visited on account of considerations of safety or conservation, such as pyramids, cave paintings, and so forth. This can be extended to include groups of people with limited mobility or restricted transport, such as patients in hospitals or the residents of care homes for the elderly. Many such experiences are possible, and the individual experience can be astonishing. In fact, the dissemination of such experiences online means that, not just these groups of people, but anyone with a virtual reality headset can enjoy them.

The other approach is to integrate these experiences into the museology of historic buildings, archaeological sites and museums in a way that enables us to provide context and emotional depth to spaces and collections of objects.

This second approach is not an easy one, because virtual reality is a very different medium than those we are familiar with. Basic cinematographic tools, such as framing and montage, have no meaning in this new world because the relationship between the work and the spectator is a new one. Users are not just spectators. They are not passively watching content that is taking place in front of them. They are present in a place that surrounds them, in which there are multiple options to choose from. With virtual reality what we achieve is presence (Milk, 2016), the sensation of being there, and the ability to explore that environment.

The narrative language for videos in virtual reality is currently being established by pioneers such as Felix & Paul Studios (https://www.felixandpaul.com)  and Jessica Brillhart (Brilhart, 2015-2016), who are discovering how to achieve 360° films that really connects with viewers. What we learn from them is that what was formerly the job of a film director is now something more like that of a scenographer or choreographer who creates and arranges spaces and actions around those viewers.

The procedure in 3D worlds is substantially different because users in these environments can move around and perform all kinds of actions. The narrative is not, therefore, cinematographic, but interactive and therefore much more like a video game in which the spectator ceases to be a spectator and becomes an actor, while the narrative moves forward thanks to the successive actions of that actor (Burford, 2016).

Virtual reality: basic concepts for cultural heritage managers

Equipment

At the present time there are three kinds of devices that produce a virtual reality experience:

Cardboard: Simple cardboard glasses with two lenses in combination with any mid-range smartphone. They provide limited quality but are accessible for almost everyone.

Mobile VR headsets: Samsung Gear VR and Google Daydream. Plastic glasses, in combination with a high-end mobile phone, with sensors that provide users with a very good quality VR experience. Google has incorporated a new, very important detail; a remote control, which allows users to perform actions within the digital world.

High-end VR headsets: HTC Vive, Oculus Rift and Sony Playstation VR. When connected to a powerful computer, or in Sony’s case, to a PS4, these kits produce an experience of total immersion in the virtual world. Users can, through their movements, modify and interact with this world thanks to two controllers that make it possible to perform any actions that are possible in the real world, as well as some that would be impossible.

How to access content

Each platform (Samsung, Oculus, HTC Vive and Sony) has an online store. Daydream and Cardboard can, furthermore, access content available from YouTube and other open platforms such as Sketchfab. Open Web access for VR content is currently being developed.

Video vs. 3D

There are two main ways of experiencing virtual reality: 360º videos and 3D experiences.

Videos are produced using groups of between two and 16 cameras which film around 360º. There are huge numbers of 360º videos on YouTube and Facebook. Spectators occupy the place of the camera and can observe their surroundings, which they see, but cannot interact with. The spectator is passive.

3D Experiences: A virtual world is generated using 3D creation tools, and often also with game engine software such as Unity and Unreal Engine. Objects and characters can be introduced with which to interact. Viewers can move around and act in this 3D world.

Installing VR in a museum

Installing virtual reality equipment in museums might be a very attractive idea, but it is not easy. It is a medium that has just arrived and though it does undoubtedly have a wow factor, there are problems associated with the immaturity of the technology. For starters, it involves equipment that most people have never seen before, so users need a certain amount of instruction on how to use it, and on an individual basis. The equipment is used individually and is therefore not suitable for large numbers of visitors. In the case of mobile devices, quite apart from questions of safety, the batteries run out very quickly, so they need to be plugged in. In the case of the high-end devices such as HTC and Oculus, they need a powerful PC for each station, and each user needs a swivel chair, or even better, a space 3x3m in order to be able to move around inside the virtual space.

Given these limitations, there are three realistic possibilities for installation: guided group visits with a mobile device for each visitor; the installation of various stations (10, 25, 40) in a room with scheduled sessions; and renting the mobile equipment out to visitors on their arrival at the museum so that they can use it during their visit and return it as they leave. The logistics involved in this latter option are more complicated than the previous two. Some examples of these kind of installations are:

• Historium VR (http://historiumvr.com/) in Bruges;
• Natural History Museum in London, two experiences: “First Live” and “Great Barrier Reef Dive”  (http://www.nhm.ac.uk/discover/david-attenborough-great-barrier-reef-dive-trailer.html) both with David Attenborough as the guide;
• The upcoming “Space Descent Vr” in the Science Museum in London (http://www.sciencemuseum.org.uk/visitmuseum/plan_your_visit/simulators/space_descent_vr);
• Museo dell’Ara Pacis “L’Ara comm’era” in Rome, (VR and AR) (http://www.arapacis.it/l_ara_com_era/progetto);
• The Domus Aurea visit with VR  (http://www.coopculture.it/en/heritage.cfm?id=51).

There is, however, another difficulty—a conceptual and communicative one, it concerns content and visitor care. At an archaeological site virtual reality is useful for providing context to the real world—for providing a complete vision of what is being observed, and every action taken in virtual reality can strengthen and support the appreciation of the contents of, and emotions stimulated by, the real remains at the site. In a museum, however, when would such complete immersion be really necessary? From the visitor’s point of view it is not an insignificant experience. How do we modulate it with respect to the route taken? We cannot expect visitors to move out of one reality and into another in every room of the museum. Every object tells a story, every display case has a context. Either we choose an object or group of objects that merits or requires this creation of context or we run the risk of dividing visitors’ attention with multiple stimuli, with the end result that we have made the resource work against the intended aims, not in favor of them.

This is the challenge we face. To experiment with this new medium and discover the ways in which it can help us to create a better connection between heritage and the general public.

Acknowledgements

This project was carried out within the framework of the “Patrimoni en Acció” (Heritage in Action) program arising from the collaboration agreement between the Departament de Cultura de la Generalitat de Catalunya (Catalan Government Ministry of Culture) and Obra Social “la Caixa” private foundation. The aim is to ensure that, through the creation of the tools necessary for its comprehension and enjoyment, all citizens have access to the Catalan cultural heritage.

To all the team: (http://patrimoni.gencat.cat/en/ullastret-3d-credits)

References

Barceló, J.A. (2001). “Virtual reality for archaeological explanation beyond ‘picturesque’ reconstruction.” Archeologia e Calcolatori 11, 221-44.

Brilhart, J. (2015-2016) “The language of VR.” Medium.com. Last updated July 6, 2016. Consulted February 12, 2017. Available https://medium.com/the-language-of-vr

Burford, G.B. (2016). “There is no such thing as a cinematic video game.” Medium.com. Last updated November 29, 2016. Consulted February 12, 2017. Available https://medium.com/@DocSeuss/there-is-no-such-thing-as-a-cinematic-video-game-977741cc1a50#.rjmkmbtdb

Codina, F., E. Garcia-Garcia, A. Martin, G. de Prado, G., R. Sala, & R. Tamba. (2015). “Metodologia de treball i resultats preliminars de la prospecció geofísica multisistema realitzada a l’Illa d’en Reixac (Ullastret, Baix Empordà).” Tribuna d’Arqueologia, 2012-2013, 236-43. Consulted November 17, 2016. Available http://calaix.gencat.cat/handle/10687/123243 .

Codina, F., A. Martin, & G. Prado. (2012). “La recerca arqueològica al conjunt ibèric d’Ullastret en els darrers anys (1995-2010).”Tribuna d’Arqueologia, 2010-2011, 63-99.

Garcia-Garcia, E., G. de Prado, J. Principal. (eds.). (2016). Working with buried remains at Ullastret (Catalonia). Proceedings of the 1st MAC International Workshop of Archaeological Geophysics. Ullastret: Museu d’Arqueologia de Catalunya-Ullastret (Monografies d’Ullastret, 3).

London Charter for the computer-based visualisation of cultural Heritage. Consulted October 18, 2016. Available http://www.londoncharter.org/.

Milk, Ch. (2016). “The Future of Virtual Reality (and why it’s causing us to change our name).: Virtual Reality Pop. Last updated June 16, 2016. Consulted February 12, 2017. Available https://virtualrealitypop.com/futureofvr-8be30f0fca6a#.rupvwmefi

Rae, J. & L. Edwards. “Virtual reality at the British Museum: What is the value of virtual reality environments for learning by children and young people, schools, and families?” MW2016: Museums and the Web 2016. Published January 28, 2016. Consulted February 12, 2017. Available http://mw2016.museumsandtheweb.com/paper/virtual-reality-at-the-british-museum-what-is-the-value-of-virtual-reality-environments-for-learning-by-children-and-young-people-schools-and-families/

Ullastret 3D, walk through an Iberian town that existed 2000 years ago. Patrimoni. Gencat. Consulted February 12, 2017. Available http://patrimoni.gencat.cat/en/stories/ullastret-3d-walk-through-iberian-town-existed-2000-years-ago.