Artwork Identification from Wearable Camera Images for Enhancing Experience of Museum Audiences

Paper
Rui Zhang, Hubei University of Arts and Science, China, Yusuf Tas, Australian National University, Australia, Piotr Koniusz, Australian National University, Australia

Published paper: Artwork identification from wearable camera images for enhancing experience of museum audiences

Recommendation systems based on image recognition could prove a vital tool in enhancing the experience of museum audiences. However, for practical systems utilizing wearable cameras, a number of challenges exist which affect the quality of image recognition. In this pilot study, we focus on recognition of museum collections by using a wearable camera in three different museum spaces. We discuss the application of wearable cameras, and the practical and technical challenges in devising a robust system that can recognize artworks viewed by the visitors to create a detailed record of their visit. Specifically, to illustrate the impact of different kinds of museum spaces on image recognition, we collect three training datasets of museum exhibits containing variety of paintings, clocks, and sculptures. Subsequently, we equip selected visitors with wearable cameras to capture artworks viewed by them as they stroll along exhibitions. We use Convolutional Neural Networks (CNN) which are pre-trained on the ImageNet dataset and fine-tuned on each of the training sets for the purpose of artwork identification. In the testing stage, we use CNNs to identify artworks captured by the visitors with a wearable camera. We analyze the accuracy of their recognition and provide an insight into the applicability of such a system to further engage audiences with museum exhibitions.

Bibliography:
Rui Zhang, Angelina Russo. Towards Comparative Methods for Evaluating Cross-cultural Digital Creativity in Museum Exhibitions. Conference: MWA2015: Museums and the Web Asia 2015.
Rui Zhang, Angelina Russo. Digital Creativity for Children on the Exhibition Design of Museum. Journal of Digital Creativity (2016).
International Journal of Design
International Journal of Museum Management and Curatorship
International Journal of Computer Vision