A proposal for a virtual reality museum for virtual reality art

Michael Fischer, Stanford, USA

Abstract

Keywords: virtual reality, art, design, brainstorming, museums, social interactions

Introduction

A museum houses things. Things that exist. Typically, these things have existed for hundreds of years. The tension in this paper is that it is about designing a museum for things that don’t exist. It is an odd duality. This paper is about museums for virtual reality art. Virtual reality art doesn’t exist in multiple ways. Virtual reality art doesn’t exist in the physical sense. Unlike a typical work of art, there is nothing that reflects light, there is nothing that physically wears out over time, and there is nothing that can be broken.

Virtual reality art can’t be broken because it doesn’t exist in an existential way. Virtual reality art doesn’t exist as a discipline. Yet. Today, there are only a handful of artists that do art work in virtual reality. Of these artists, most attempt it out of curiosity or for publicity.

To add to the indirection, a virtual reality art museum doesn’t need to physically exist in the brick and mortar sense. If virtual art is composed of 0s and 1s, there doesn’t need to be a reason that the museum shouldn’t also be composed of 0s and 1s.

It may seem strange to consider what a museum might look like to house an item that doesn’t exist. That is what we shall be examining in the paper. We shall examine the technology behind virtual reality, the effects virtual reality will have on art, and an overview of a virtual reality museum for virtual reality art.

In this paper, we will focus on virtual reality museums for virtual reality art. There will be demands from consumers to build other types of museums in virtual reality. For example, a natural history museum in virtual reality would be amazing. Instead of looking at a wooly mammoth exhibit from behind a pane of glass, one could go inside the exhibit and everything would be interactive. This exhibit would need a person to create a 3D model of the exhibit. The exhibit would in turn be a replication of the original animal. In this example, there are many levels of abstractions away from the artifact. In this paper we focus on the subject of art, and more specifically, art that is not a replication of art that already exists in the real world but art that is made for a virtual reality art museum.

Content is (still) king

The most important part of a museum is what is inside. There will be many differences between traditional art museums and virtual reality art museums. However, in both the content is king. Any medium without interesting content isn’t going to be interesting. Technology can’t save a platform if the content isn’t there. The reverse is also true. We read newspapers today because the content is good. Words on paper beat a holograph with nothing interesting to say any day. After all, you’re reading words right now.

To understand what a digital art museum could look like, we first examine the most important part of the museum, the art. Technology is only a tool for creation. Like a paint brush, technology can’t (yet) create a creative spark, intention, or story.

Digital art can take many forms. With most digital art, the artist uses digital technology as a tool in creating or presenting the work. This type of artwork is displayed in a physical venue typically. The digital artwork that we are interested in this paper is art that is built in virtual reality for display in virtual reality.

What makes virtual artwork different

What makes virtual reality artwork different from physical art is that it can be copied exactly. It is not unique. It is easy to make an exact replica. With traditional art, the digital representation is different from the original (Benjamin, 2008). In person, the scale, brush strokes, color, and finish of the art are easy to see. These attributes are hard to convey in a reprint in a book or in a digitally scanned version online (Proctor, 2011; Smith, 2011).

Artwork created in virtual reality is always the same no matter how many times it is copied. The tools that the art is created in are the same tools that are used to view the art. The viewing of digital art is a direct representation of what the artists intends, namely, a connection from the artist’s studio to the viewer’s eye. With virtual reality art fidelity does not decrease with reproduction. All bits are transmitted without loss.

We define art that is “native” to virtual reality to mean art that is created in virtual reality for viewing in virtual reality. For example, an artist can create a sculpture made of smoke built inside a levitating case. This art is not bound to the laws of physics and cannot be created without virtual reality. The art cannot be displayed in a physical museum. The art can only exist within virtually reality.

Virtual reality art should be confident in its virtual form and not seek to unintentionally replicate physical objects. Faux-physical objects always come up short. A digital copy of the Mona Lisa won’t match the original. For virtual art to work and to be relevant, it needs to rely on its natively virtual qualities. New surfaces and textures that don’t exist in real life need to be created. Virtual reality art that uses craftsmanship that is only possible with a lack of physical constraints.

Virtual reality

Virtual reality is technology that creates an immersive environment that is controlled by software. With a desktop computer or mobile phone, we peer through a computer display onto a software-controlled programmable surface. The surrounding physical world sets the context for the interaction. The computer becomes a tool within our physical environment.

In virtual reality, instead of a computer being used in a context, the computer sets the context. If content is king, then context is god. The computers set the context by controlling all the aspects of the interaction using software. In the physical world, we interact with the world abiding by the laws of physics. In virtual reality, all interactions are intermediated by computer software, and thus we interact with the world abiding by the laws of software.

Virtual reality software intermediates our interactions using three hardware components. The first hardware component is called the head mounted display, shown in figure 1. It looks like a pair of googles. Each eye has a separate screen that displays slightly different content to give the illusion of depth.

Figure 1: a virtual reality head mounted display

The second hardware device is the controllers, as shown in figure 2. Virtual reality would be less interactive if there wasn’t an intuitive way of controlling the content. It would be like watching a movie or looking at an image of a sculpture. Typically each hand gets a controller. On each controller, there is a touchpad and a number of buttons. The controllers are equipped with a vibration module for haptic feedback. The position and rotation of the controllers are tracked using a motion capture system.

Figure 2: a pair of virtual reality controllers

The last element of the hardware setup is the motion capture system shown in figure 3. It is the secret sauce that brings the hardware together and makes everything come to life. The motion capture system is responsible for tracking the movement of the head mounted display and the controllers. Accurately knowing where the headset is and the controllers are is essential so that the software can display the right rendering. Small errors in tracking the headset can lead to motion sickness (LaViola, 2011). This causes a mismatch between what our inner ear feels and what our eyes see, which in turn leads to motion sickness. These three hardware components together create a platform on which virtual reality software can be built.

Figure 3: the system used to track the position of the head mounted display and controllers

Software to make virtual art

Virtual reality art is different. There aren’t any materials that need to be moved around a canvas. The artist’s studio is built of software. Tilt Brush is the most popular application to make virtual reality art today. The program works by having one controller be the brush and the other controller be the palette. The application describes itself as follows:

“Tilt Brush lets you paint in 3D space with virtual reality. Unleash your creativity with three-dimensional brush strokes, stars, light, and even fire. Your room is your canvas. Your palette is your imagination. The possibilities are endless.” (http://store.steampowered.com/app/327140/)

To draw using a pen and paper, the pen is activated by putting the pen in contact with the paper. In Tilt Brush though, you don’t want to always be drawing in space. There is a trigger on the bottom of the controller that the artist holds down while drawing. What the artist sees when they are in virtual reality is shown in figure 4.

Figure 4: the virtual brush used in Tilt Brush

In the other hand is the palette, shown in figure 5. The palette is in the shape of a box. On it are different brush styles and colors to choose from. The user selects a tool by pointing the brush toward a tool on the palette and then pressing the trigger on the controller.

Figure 5: the virtual palette of tools

The palette contains brushes that an artist would be familiar with as well as many new ones that are only possible in virtual reality. One brush paints lines that move in response to sound; another brush creates tubing that has glowing neon pulses going through it. There are volumetric brushes to build large objects quickly.

There are several tools to make working in virtual reality easier. For larger movements around a room there is a teleportation tool to allow people to move around the studio to other areas. The user aims the controller to where they want to go and then presses the trigger. For smaller movements, the user can walk around the room to get to the right location.

Tilt Brush is a technical work of art. However, what I have been most impressed by are the reactions people have after using Tilt Brush for the first time. When I first tried it out, the power and intuitiveness of the software was obvious. It requires no instructions and can keep you busy for hours. When the headset comes off, it takes a few moments for the transition from being in a fully virtual world to the physical world to set it. Everyone who I have witnessed using Tilt Brush has had a very similar reaction. I can’t think of a person who didn’t enjoy it. The current rating for Tilt Brush on the Steam app store says that “99% of the 119 user reviews for this software are positive.” (http://store.steampowered.com/app/327140/#app_reviews_hash)

For most museum art, we are on the outside and we peer at the art. In virtual reality, though, art can be much more immersive. Artwork today is small because it is easier to build. Virtual reality art doesn’t have limitations on scale. Building a large statue is almost as easy as building a small statue in virtual reality. There is no gravity in virtual reality. There are no limitations of physical materials. Instead of us looking at virtual reality objects, art in the future will be more immersive. It’ll be easier for the artist to set the context in which their art is viewed. It will be like a digital video game or like a digital vacation.

Art in a virtual reality can also be less static and more active. The movement of a work of art will be a tool that artists can use to express their intention. Unlike physical artwork, the piece will never break down or have to be repaired. People will also find moving art to be more interesting than static art in the same way that, on average, videos are more interesting than images.

Tools will continue to develop for artists using virtual reality. We’re still in the sticks and stone age of developing tools for creating virtual reality art. Tilt Brush has open-sourced (https://github.com/googlevr/tilt-brush-toolkit) some of their software so it will now be easier for other people to build on their impressive work. Overall though, the future is very bright for tools for digital artists in virtual reality.

Storing digital art

A key function of a museum is to store artifacts. The way physical art and virtual reality art is stored is quite different. Physical works of art need to be handled carefully and kept in secure and temperate environments. The fundamental reason for this caution is that physical art is unique and can’t be replaced. Virtual art doesn’t have the limitations (and benefits) of being one-of-a-kind. Virtual art that is 0s and 1s can be copied, moved, and backed up. Reliable storage of large amounts of data is a solved problem (https://aws.amazon.com/s3/).

The floor plan of a museum influences how people use the space (Pearson & Colin, 2003). In a similar way, the way that data containing virtual art is stored will influence how people view and use it. In this section, we examine the possibility of using version control software to store art in the virtual reality museum.

Version control software manages the changes of data over time. Typically, version control software is used by groups of software programs when they work together to build an application. Git (https://github.com/git/git) is the very popular version control systems available today. One person will “check-in” their code to the main project. The entire team will then be able to “check-out” their changes. If two people “check-in” code that conflicts with each other, the software notifies them that they need to “resolve” their conflict to proceed. If at a point the projects develops a new bug, the project can be “reverted” to an older version. Overall, version control software allows for large teams to work together on projects.

One of the most interesting features of version control is “forking.” Forking duplicates all the code under one project and allows another team or developer to make changes independently of the original project. The goal of this is to create an independent piece of software. If a digital museum used version control software as a means of storage, there would be many benefits. The first benefit would be that novice artists could see a piece of artwork as it was made. Each time an artist “checks-in” their art it is a snapshot of the current work. These snapshots would reveal the what order each artist worked in, how long different parts took, and areas that were reworked.

Secondly, artists could build off the work of others. No one expects an artist to build their own paint brush, make their own easel, or engineer their own paints. These are all tools that we use to get started more quickly. Time that is saved not building these blocks is spent developing the idea that makes an artwork one’s own.

Today, people go to great lengths to see paintings firsthand and to study them. A common sight in museums is students setting up easels in front of art. By using version control software, with a few clicks people could see how a painting was made and then “fork” it and build on that image. Artists today are influenced by the works of their peers. Software will make this existing process easier.

Version control software will also make collaboration between artists working on the same piece easier. Artists today and throughout history have worked collaboratively. A piece is conceptualized by “the” artist and built by other artists. Often finding the right artist with the right skills can mean shipping components of art around the world. Collaboration with artists around the globe is simple and can be done in real-time using virtual reality.

With a purely virtual museum we can explore radically new ideas. Virtual reality art that anyone can fork, also known as open-source (Wikipedia, 2016), will require that we reconsider how intellectual property is held. Open-source art would have beneficial impacts to society by lowering the barrier of entry to creating art. If it is easier to create art, we must re-evaluate how we consume art, which we will do in the next section.

Data as a tool for curation in a virtual reality museum

More content is only good if we have better curation. The museum experience today is impersonal. Every person sees the same artwork. People can visit different parts of a museum but it is hard to describe a museum visit as being personalized for the patron.

Data will be useful in helping museums curate their works and personalize exhibits. Data is used as a tool for curation already. Museums try and understand how many people will come to their exhibit through ticket sales. But this is a coarse level of understanding. With fine grain data it will be possible to have a better understanding of what people will find interesting.

Data will be abundant in a virtual reality art museum. The data will come from sources that at first might seem unexpected. One of the most interesting data will be eye tracking. In virtual reality, it easy to not only figure out what a piece someone is looking at but where they are looking. Everything the viewer sees in virtual reality is intermediated by software. As the software displays what the person is looking at, it can add a log to an analytics engine. From this data, we will be able to get a deeper understanding of how we consume art and what we find interesting. How long do we look at a painting? How long do we look at certain types of work? Which types of work are we not interested in looking at? Which types of work do our friends find interesting? Which types of work do people in our cities (or cyber cities) find interesting? Which artists do we like? What order do we want to view art in? A heat map can be constructed at the level of each piece to be shared with the viewer, the artists, and the museum.

This data should be used as a tool for a human to provide insights. No one wants a completely algorithmically generated art museum. It is an interesting experiment to consider what it would look like for a museum to be completely run by computer. All artwork would be algorithmically generated. Different permutations and combinations of existing work would be shown to humans. It would be a Darwinian evolution of what is the best art. There would be no outliners as each piece would be an incremental exercise in hill climbing to the next best thing.

But until that point, the artist will still need to have the nucleus of the idea from which the artwork is created. The core idea is more important than the data or technology that is used in creating it. Great artwork will lead and the analytics will follow. In addition to using implicit data collected from museum goers, a museum can solicit explicit data. In the next section, we will explore how social interactions can take place in a virtual reality museum.

Social

Museums and virtual reality naturally bring people together. Social sites like Facebook, Instagram, and Twitter developed interfaces to enable the sharing of content with other people. In this section we will examine how we can use some of the ideas these websites have developed to enhance the experience of a virtual reality art museum.

It is hard to implement social features that go beyond a face to face conversations in a physical art museum. Logistically it is hard to do. Let’s take the example if a physical museum wanted to implement a system to allow people to “vote up” the artwork that they liked, a feature that is common on many websites. The museum would have to put a box next to each piece of art. Or everyone would need to install an app on their phone. Both solutions are possible; however, it is logically hard to implement and keep up to date.

However, implementing this system in virtual reality is easier. To do so, one of the hardware controllers can be given a software interface to make it into a tool for voting. People can show appreciation for a piece by pointing their controller at the art and voting it up, voting it down, or not voting at all. This up and down vote data could be shared to create personalized museum experiences for other people. If there is a person’s taste that you particularly admire, you can see all the pieces of art that they voted up. Friends will see what their friends find interesting. In this way, the curation process takes on a new form. Instead of curation being a top-down process, each person is empowered to be a curator.  People can find people with similar tastes. Art doesn’t have to be limited to one instance of a curation.

There will also be more traditional social features in a virtual reality art museum. Chatting with another person will be as seamless as talking with another person in real life. The people will be from all over the world, bringing different cultural experiences with them.

People will want to share virtual reality art with people that don’t have a virtual reality system. It will be possible to use a virtual camera that captures a 2D representation of the 3D virtual environment. This 2D photo can then be shared on other social sites. If someone with a virtual reality headset clicks on the images later, they can be transported to the same position and viewpoint where the picture was taken. It’s like a hyperlink on a webpage.

A popular feature online allows people to comment on other people’s posts. Commenting systems have a varying degree of utility depending on how they are designed and the audience that is using them. In museums today, there are no mechanisms to allow people to comment on an artwork. In a virtual reality museum, there could be a system where people can fill out a virtual Post-It note for pieces that are particularly impactful for them. The Post-It note could be placed next to the art or even on a specific area of art. People could opt to show or hide the Post-It notes feature. We each bring our own personal experiences to the museum and the art makes us react in different ways. The Post-In notes are a way to capture these different reactions.

There will be other social features to allow people to remix art. Snapchat, a popular photo messaging application, has a feature to apply sophisticated filters to images. For example, if a person takes a picture of themselves they can apply a filter to the image that will transform them into an animal. The filter uses artificial intelligence to calculate how human features should be mapped to animal features. In a virtual reality art museum, filters could be applied to change how a piece of artwork looks, too. They will be different, though, because a filter doesn’t have to be based only on data from the camera. With a 3D model of the art the filter can apply a structural transformation to the rigging and texture of the art.

Monetization

There are many ways to make sure that everyone who puts work into creating a virtual art museum gets paid for their work. Firstly, operating a physical museum is expensive. Operating a digital museum will be orders of magnitude less expensive. In a virtual world, there are no buildings to maintain, no art to restore, and no utility bills.

Secondly, while the art can only be truly viewed in virtual reality, the art can be reproduced in other mediums. Merchandise in the form of customized posters, t-shirts, and 3D printed models are possible. When art is natively virtual, it is easy to make reproductions by sending it to a manufacture, and this can be a vital and viable source of funding to support both the artists who create the work and the virtual reality art museum.

Conclusion

In this paper we described what a virtual reality art museum of the future would look like, and how one can be set up. Art is constantly changing and adapting to new technologies. However, the role of artists will remain constant: to be the carrier of the creative spark, to be the messenger of truth, and to have intention in how experiences are consumed.

Virtual reality art will make it easier for novices to get into art. The only thing someone wanting to get started with virtual reality art will need is access to a virtual reality system. Physical art supplies today can be expensive, especially for someone who is unsure if they really want to get into art. With a virtual paint set, getting started will be easy.

Getting great ideas and learning about art will be easy, too. People will be able to import and remix existing art. Starting with a blank white sheet of paper will be outdated. “Standard” background images can be imported and modified. This will leave more time for an artist to focus on their own creative offering.

In a few years, virtual reality hardware will increase in quality and will be standardized. To give an example from another industry, mobile phones only hit their stride after the release of the iPhone. The iPhone was the right combination of features, price, processing power, battery life, and screen resolution. Once the iPhone was released, other manufactures adopted a similar set of features and a de-facto system was created. A similar thing will happen with virtual reality. A manufacture will release a system that is sufficient for most customers and at the right price. A de facto standard system will be created and virtual reality will become wide spread.

At a future point, virtual reality hardware has the potential to be even more interesting and immersive for artists. It is possible to allow for virtual reality to use special gloves to provide the sensation of touch. Other senses to be incorporated into art could include smell, taste, and temperature.

These new modes of experiencing virtual reality art will help make it unique and differentiate it from traditional art. It is rare for a message to transcend its medium. We don’t read newspapers on TV. We don’t listen to the radio at the movies. We don’t send e-mails over Snapchat. New mediums produce new types of content. The same will be true for how we experience virtual reality art in virtual reality museums.

Will virtual museums be a thing of the future? Even if museums aren’t on a path to virtual reality, having interesting and amazing content will always be a necessity for virtual reality to succeed. The items that we discussed above will be applicable to curating content in virtual reality, even if they do not take the form of a virtual reality museum.

Virtual reality art museums will be a tool for empowerment. There are many people that are not able to go to a metropolitan center to see a master work of art. For these people, virtual reality is a tool of cultural empowerment. Access will be democratized and anyone with access to a virtual reality headset will be able to see the master works of the artists of tomorrow.

References

Benjamin, W. (2008). The work of art in the age of mechanical reproduction. Penguin UK.

LaViola Jr, J.J. (2000). “A discussion of cybersickness in virtual environments.” ACM SIGCHI Bulletin 32(1), 47-56.

Pearson, M.P., & C. Richards (eds.). (2003). Architecture and order: approaches to social space. Routledge.

Proctor, N. (2011). “The Google Art Project: A new generation of museums on the web?” Curator: The Museum Journal 54(2), 215-21.

Smith, R. (2011). “The work of art in the age of Google.” New York Times.

Wikipedia contributors. (n.d.) “Open-source software movement.” Wikipedia, The Free Encyclopedia. Consulted December 2016. Available https://en.wikipedia.org/wiki/Open-source_software_movement