Open innovation: Open movements and the role of a museum in the 21st century.
Brian Dawson, Canada Science and Technology Museums Corporation, Fiona Smith Hale, Canada Science and Technology Museums, Canada, Sandra Corbeil, Canada Science and Technology Museums Corporation, Canada
AbstractOpen movements such as open data, open information, and open participation offer important opportunities for museums to allow stakeholders to engage with institutions' missions', and to reposition the role of the museum for the 21st century. This session will explore how museums can use these initiatives as a site of open innovation, inviting participants as well as academic, enterprise, and cultural collaborators to engage with visitors on new applications. By exploring the opportunities and challenges presented by being open, the session will look at some of the practical examples on the ground, inviting a discussion on the role of museums in contemporary society and how we can allow more participants to help fulfill the mission of the museum.
Keywords: open data, open information, open innovation, participatory museum, digital strategy, culture change
Internationally, museums, libraries, and archives have been embracing open movements, from open data to open source. This trend is also mirrored in other sectors, such as open government. Many observers have noted that openness “is of great societal value” (Maxwell, 2006). But what does it mean to be “open?” Does being “open” invite participation and engagement with a museum and its mission? How can participation in open movements be a force for change within an organization? Do open initiatives help a museum be more relevant to its visitors/participants, and to society in general? What do they mean for the role of a museum in the 21st century?
The Canada Science and Technology Museums Corporation (CSTMC), a family of three national museums, has embarked on a process of sharing cultural heritage data and assets as open data, open information, and open archives through a set of related portals. But more significantly, it has used “open” as an opportunity for engagement and a site of participation: from joining community meet-ups, to supporting in-class university projects, to connecting with hobbyist and academic researchers, to participating in national hackathons. In doing so, it is undergoing a strategic shift, repositioning its role as a museum within broader society. It has embarked on a process of turning the museum inside out, inviting client participation to hack and remix culture. The museums are working to establish infrastructure to support digital cultural collaboration as a site of open innovation, inviting academic, enterprise, cultural, and governmental organizations to apply research and engage with visitors on new applications, including digital interactive storytelling media, machine learning, and cultural analytics, all while exploring the broader impacts of these technologies on society.
This paper explores the opportunities and challenges presented by being open, focusing particularly on CSTMC’s experiences with open data, open information, and other open assets, and how these can support open participation and open innovation. It examines how models of open government and open innovation can provide context for museum innovation. It looks at some of the practical examples on the ground, and invites a discussion on the role of museums in contemporary society. It traces the evolution of open thinking within museums more broadly, and draws on the experience of the CSTMC to explore how museums can become sites of collaborative, participatory innovation, allowing more participants to help fulfill the mission of the museum.
Open movements and museums
Openness is not new to the museum community. One can argue that it is part of the DNA of our sector, since many museums explicitly have a dual mandate to preserve and to share knowledge (such as CSTMC). More concretely, museums have been engaged in a range of open projects and activity through the decades. We will provide a brief overview of open movements as they relate to museums, first exploring the question, “what does openness mean?”
There are several key domains in which openness may apply, including open standards, open source, open hardware, open data, open information, open participation, and open innovation. These are certainly not isolated realms—in fact, most build upon each other—but we will do a quick survey of each, with a view on relevance in a museum context.
“The spectrum of openness is very broad” (Maxwell, 2006); but between the poles, “most works fall between the two extremes. Thus characterizing a work as open or closed is rarely a binary decision; it is generally a question of ‘how open'” (Maxwell, 2006). While not all projects in this overview are purely “open,” they fit within the broader spectrum of openness that many museums are trying to achieve.
Open standards are a foundation for most of the digital initiatives undertaken by the museum sector, if one considers the enabling standards of the Internet such as TCP/IP and HTML. In this light, “the growth of the Internet and the World Wide Web may be the most obvious arguments for open standards.” (Maxwell, 2006). “A key benefit of open standards is that they foster interoperability, allowing disparate devices, applications, and networks to communicate.” (Maxwell, 2006). They “help to maximize access, provide application- and device-independence, provide long-term access to resources and services, and support interoperability” (Kelly, 2008). Museums have been grappling with standards for some time (Kelly, 2007, Kelly 2008).
While the term “open source” dates back to the 1990s (Perens, 1999), the concept is much older, dating back to the 1950s and 1960s when people in university settings would share software in academic pursuits (Maxwell, 2006). This culture of sharing was also present in corporate research labs, like AT&T’s Bell Laboratories (Maxwell, 2006). Open source gained momentum with the release of Linux in the mid-1990s, and by 2000, had attracted the support of major technology corporations.
The use of open source and other free software within museums is a significant area of activity (for example, David and Oberoi, 2016). But museums are doing more than deploying Linux, Apache, and Drupal as part of their technology infrastructure. Museums are also engaged in open source projects specific to the museum domain. Jim Spadaccini (2008) outlined some of the opportunities around open source software in 2008, and Ethan Wilde and Laura Mann (2010) clearly outlined the benefits of broader sector open source collaboration two years later. Many of these projects continue to this day.
While the spirit of tinkering with hardware can trace its roots to the low-cost home computers of the 1980s (Dennis, 2013), the concept of open hardware also dates back to the 1990s (Perens, 1999). However, it is the success of the Raspberry Pi and the rise of open hardware platforms such Arduino in the 2000s (particularly in the last several years) that has seen the open hardware movement come of age (Dennis, 2013). (Yes, technically, the Pi’s hardware and default firmware are not “fully” open, but a global open community has developed around this low-cost platform.)
Museums have more than taken note. For example, Miriam Langer and Jason Alderman (2016) have outlined some of the potential and opportunities in open hardware for museums, with several examples from current practice.
Open data and open information
Open data is also a major movement. Linked open data also has significant activity within the library, archive, and museum sector (see, for example, Voss, 2012).
One major stream of global activity is that of open government, which encompasses open data, open information, and open dialogue (see, for example, Treasury Board, 2016). Open information also has significant sector-based activity, such as open science initiatives (for example, Owens, 2016).
Open participation is a significant trend for museums; that is, to be open to the meaningful participation of their stakeholders. Nina Simon’s Participatory Museum (2010) is perhaps the best known call for museums to become more open to participation, outlining how museums can become more audience centered, allowing users’ voices to both inform and invigorate projects and programs, and allowing visitors to construct their own meanings with the institution.
Museums can be open to many forms of participation, not just traditional “visitors” or “users,” but stakeholders, such as academia, industry, government, and other cultural organizations.
Note that open participation is analogous to the concept of “open dialogue” used in open government frameworks.
“Open Innovation is the use of purposive inflows and outflows of knowledge to accelerate internal innovation, and expand the markets for external use of innovation, respectively” (Chesbrough, 2006). Haitham Eid (2016) specifically looked at the application of open innovation in a museum context, and proposed a Museum Innovation Model that encompasses the concepts of social enterprise, open innovation, and social innovation.
An “open” case study at CSTMC
The CSTMC mandate, in part, is to foster scientific and technological literacy throughout Canada. Like many other museums, engagement is central to our vision to inspire Canadians to celebrate and engage with their scientific, technological, and innovative past, present, and future. Delivering on our mandate—and engaging with our visitors—means acknowledging that traditional stewardship of knowledge is breaking down. Today, there are changing expectations for how we deliver our messages and information along with a growing demand for participation. These changing expectations have prompted us to shift towards new ways of delivering on our heritage mandate, such as structuring content for creativity, and the creation of new content and new knowledge, as opposed to just its consumption.
CSTMC: a timeline of open initiatives
CSTMC information is living in an increasingly open ecosystem. It is evident that people’s expectations are changing, and that their desire—even need—to have immediate access to information and knowledge is growing. Beginning in 2014, CSTMC embarked on a multi-streamed program to address these changing expectations, initiating a range of open access and participatory initiatives as part of a fundamental, strategic shift for the organization. These incrementally build on prior work (for example, the CSTMC library catalogue has been accessible on our websites and through LAC and other union catalogues for more than a decade).
- October 2014: CSTMC became the first national museums in Canada to release their artifact collections as open data (CSTMC, 2014). CSTMC is actively planning to make its data available as linked open data, in collaboration with the Canadian Heritage Information Network (CHIN);
- Spring 2015: CSTMC’s beta artifact collection online went live (http://techno-science.ca/en/collection-research/collection.php);
- June 2015: CSTMC launched a major national crowdsourcing initiative. The Canada 150 digital storybook (Canada is celebrating its 150th birthday this year) is crowdsourced and includes stories of Canadian innovation from all across the country from a variety of sources, including universities, individuals, and community groups (Corbeil, 2017);
- February 2016: CSTMC launched an ambitious open information initiative called Open Heritage. Open Heritage is a portal that works in near real-time, where the the workings documents of the museums are “open by default” and released via an unrestricted Web portal. As an exercise in radical transparency, we have moved from a traditional closed by default system, where content is hidden behind a firewall, to an open by default one. Guided by the principle of open by default, employees have been empowered to share their working documents in the portal. (http://documents.techno-science.ca);
- Fall 2016: CSTMC created a beta Open Archives portal, sharing photographic collections from the museums’ diverse archives. Currently in Alpha, this portal will “officially” launch in 2017. (http://archives.techno-science.ca);
- November 2017: the renewed Canada Science and Technology Museum will re-open to the public. The museum is adopting a more participatory approach, incorporating elements such as a “maker space” and a “living lab” for childhood research.
Collectively, these initiatives are part of a strategy to openly share and provide access to the information and assets of the museums as elements of digital cultural infrastructure that can support cultural innovation. The corporation is actively repositioning itself as a site for open innovation; open to client participation to hack and remix culture, it is also actively inviting academic, enterprise, cultural, and government organizations to apply research and engage with visitors on new applications, including digital interactive storytelling media, machine learning, and cultural analytics, while exploring the broader impacts of these technologies on society. In doing so, this is fundamentally changing the organization itself.
Open heritage: a deep dive
Open Heritage is about openness and transparency, and proactively sharing our museums’ information and knowledge. Open Heritage puts the current, unclassified working documents in CSTMC’s Electronic Document Records Management System (EDRMS) within public reach. These documents are openly and freely available in an accessible Web portal with minimal barriers for their re-use and consumption.
This project is about leveraging our existing technical infrastructure, which, for example, works directly with EDRMS. Documents and changes to documents are uploaded to the Open Heritage portal every two hours. This means that as staff work, edit, and save their documents, the portal is updated.
CSTMC identified several “personas” as potential higher-runner stakeholders that would benefit from and engage with the museums’ open information. These initial personas included researchers, curious Canadians, teachers, exhibition planners, and authors.
The researcher persona was considered a strategic target audience and given particular focus during the development of the portal. During the very early days of the project, we reached out to various professors, librarians, and students about the research documents we hold in our EDRMS, and asked about their desire to have unmediated access to these documents; there was great interest.
Open Heritage vision
Our vision is to give real-time access to the majority of our unpublished, working documents, as well as the following goals:
- To provide open access to our research and information assets in order to foster new knowledge;
- To encourage participation across the museums and beyond;
- To share our lessons learned with other organizations;
- To be a leader in sharing our information openly.
Open Heritage strategy
The strategy for Open Heritage project at CSTMC encompasses following four main elements:
- We will share authority and continue to work with our networks and develop new networks and new partnerships. In other words, we will use the world to co-create new, meaningful content.
- We will continue to develop and nurture a digital culture at CSTMC which empowers staff to share their knowledge and to appreciate the value of their knowledge beyond their day to day tasks. Furthermore, this culture will encourage staff to share authority, expertise, and embrace different perspectives.
- We will continue to measure the success of our Open Heritage strategy by monitoring our dashboard regularly to observe the almost logarithmic pattern of documents added to the portal by staff, and the growing access to the portal by the public.
- We will continue to look ahead and to learn from what we’ve done. We will also incorporate feedback that we receive into our next steps; we realize that this is a work in progress and change is expected.
The Open Heritage project has first and foremost been an exercise in culture change, in much the same way as the implementation of the EDRMS was two years earlier. Managing the change in culture and recognizing the huge shift in perspective it represented to staff was critical for the success of this project. As a result, CSTMC undertook a very active change management program once launching the portal at the end of February 2016. We worked with managers at the executive level and the access to information officer to complete a “threat risk assessment.” Once this was complete, we were ready to engage employees in their working teams and encourage them to think of what they could share and how this could benefit their work and their collaboration with others. Multiple forms of training took place, including basic rules for what staff could and couldn’t share. Workshops were held in which staff could begin to share documents and mark folders as “open by default” on the spot. Follow-up sessions for groups or individuals are ongoing, and use of the Open Heritage “flag” is monitored to ensure participation across the organization.
Again, the important message to staff is to see the information they hold and manage as having value beyond their own department. It gives staff a platform to engage with others and to create new knowledge.
It became clear that once staff across CSTMC were trained and understood the possibilities and potential of the portal, they were much more willing to jump on board and add documents. Staff also felt empowered and encouraged to share their draft documents internally and to unknown external audiences without fear of repercussion. In fact, employees were encouraged to “let it go” and “just do it” from the very top of the organization. And, despite the initial concerns of potentially negative impacts (such as employees not using the EDRMS and driving the management of information underground), these fears have not been realized in practice.
The statistics demonstrate the success to date. Document sharing is on a path of exponential growth, and the numbers of documents on the portal have almost doubled each month since the launch. Use of the document management system, far from decreasing, has shown a steady increase.
This is still a work in progress, but one year in, the project is showing signs of traction with staff. The conversations that have happened have been positive, and staff have started using the portal as a way of communicating and sharing documents with external colleagues and researchers.
Assessing the effectiveness of open initiatives
Vanessa Williamson and Norman Eisen (2016) have outlined a framework that can help assess whether an open government initiative has the necessary foundations for success; we suggest that this framework can be adapted to consider key success factors for open cultural projects.
|Key features of effective open government programs||Potential cultural heritage applicability||Observations from CSTMC open heritage experience|
|1. Have the proponents identified the specific principals (e.g. segments of the public, civil society, media, and other stakeholders)?||Does your project have a target audience/ group of participants?||Target segments/personas identified at outset:
(e.g. researchers, teachers, curious Canadians, etc.)
|2. Is the information revealed by the initiative important to the principals?||Is your project of value to your target group?||Demonstrated interest within the academic segment (both professors and students)|
|3. Is the information accessible and publicized to the principals?||Is your initiative shared in relevant open portals?
Is the content indexed and searchable via search engines?
Is there active outreach to potentially interested parties to inform them of the project and engage with them?
Readily available in portals
Tuning search in Open Heritage portal
Ensure Google search indexing
Active awareness campaign
Letters sent to university deans and colleges nationally and internationally
Working through existing relationships with academic institutions
|4. Can the principals respond meaningfully as individuals?||Is the project usable by individuals?
For example, is open data provided in formats that are easily used by a range of individuals?
|Student and enthusiast projects
Student entries in hackathons
|5. Are governmental agents supportive of the reform effort?||Are museum staff and senior management supportive of the open release of information?||Staff enthusiasm regarding open project
Exponential growth of open documents
Continued growth in use of document repository
Curators embracing open data in class projects
|6. Can the principals coordinate to change their governmental agent’s incentives?||Are museum staff responsive to participation?
Are “corrections” used to improve the primary source data repositories?
|Work to enhance participatory features|
Table 1: framework for establishing effectiveness of open initiatives (adapted from Williamson 2016)
Partnerships and collaborations
As we continue to roll out the Open Heritage platform and look at ways of making our information more accessible by improving the functionality of the site, we are finding that in some circles, we have become almost a poster child for openness and transparency. From advocates and “change agents” on the ground to more senior officials, other departments in Canada’s provincial and federal governments are looking to us to learn best practices from our open by default process. Some departments want to learn about what it means to be “open” by using our technology and contributing their own documents. Other departments want to collaborate with us and use our experience to shape government policy and attempt to develop practical open government implementation tools and guidance for use across government.
For example, Canada’s Treasury Board Secretariat (TBS) has recognized that CSTMC’s Open Heritage portal is significantly advanced in terms of its scope and impact compared to other departmental open government initiatives. TBS is partnering with CSTMC on a study to explore how CSTMC’s Open Heritage pilot project, with it’s approach to operationalizing the principle of “open-by-default,” can help develop innovative and practical guidance, tools, and resources to support other Canadian Federal departments in their open government initiatives, as well as inform government policy modernization in general.
Further, other individual federal and provincial departments are looking for ways to leverage CSTMC’s open portal infrastructure in order to pilot their own open information initiative, and as a way of sharing their own information assets that would otherwise be locked behind their institutional firewalls. Some of these partner pilots may come online as early as April 2017.
It is of interest that these public sector agencies all project a strong sense of urgency to be open. Many public sector jurisdictions are finding that decades-old frameworks that were originally intended to provide access to information have in fact become barriers to openness and transparency in the modern digital age, not just in Canada, but abroad as well (Stewart, 2016). Clearly, many organizations are experiencing a range of pressures on this traditional stewardship of information, and this is forcing tight timelines for delivering on open. It’s a new world!
Open cultural innovation and the role of the museum
While others have written about the nature and significance of open innovation (for example, von Hippel, 2005; Maxwell, 2006), we have found Chesbrough’s Open Innovation framework (2003) particularly useful. Open Innovation is “a paradigm that assumes that firms can and should use external ideas as well as internal ideas, and internal and external paths to market,” and “that internal ideas can also be taken to market through external channels, outside the current businesses of the firm, to generate additional value…Open Innovation suggests that valuable ideas can come from inside or outside the company and can go to market from inside or outside the company as well.” (Chesbrough, 2006).
Haitham Eid (2016) recently explored Chesbrough’s model in a museum context, incorporating it into his Museum Innovation Model.
Figure 1: open innovation (Chesbrough, 2003, via Eid, 2016)
Many aspects of CSTMC’s open initiatives can be seen as inflows and outflows (or bridges) across an increasingly porous enterprise boundary in the open innovation model.
- Adoption of open standards and open source technology;
- Involvement of partners and stakeholders in the development of open initiatives;
- Collaborations based on shared assets;
- Prototypes created at hackathons and by open data enthusiasts;
- Feedback on open resources informing museum information holdings.
- Open data,iInformation, and other assets as material for other party cultural production and innovation;
- Active use of open portals by partners;
- Informing best practice and public sector policy innovation to help other organizations;
- Joint ventures based on museum content, such as Illumination Games
The example of Illumination Games (https://illumination-games.com/) is a concrete example of open innovation at work. CSTMC’s joint project with SEED Interactive created a structure within which “to produce titles that explored the relationship between entertainment and education, while creating games that are competitive in the cluttered mobile landscape” (SEED, 2016). This represents a substantial and growing channel for digital engagement in cultural forms that are relevant for the audiences of today; it also produces revenues that help make the initiative sustainable, and in turn generates assets such as 3D models that have been brought back into the museum for use in other initiatives, such as educational resources.
Opportunities and future directions
CSTMC’s open initiatives are still at their early stages. Building on successful pilots, CSTMC plans continued engagement of partners with open resources: government, cultural, academic, and industry. Further, it plans to develop additional projects with partners that leverage open assets.
CSTMC plans to further expand the sharing of information. Information shared in the Open Heritage portal is growing exponentially, and other data sources may be of value. (For example, milk data from the operations of the Canada Agriculture and Food museum could have potential unforeseen uses.) CSTMC will also collaborate with the Canadian Heritage Information Network (CHIN) to help expand CHIN’s Linked Open Data pilot to encompass technological object collections.
Further developing participatory features remains a priority. This may involve fleshing out user stories for additional key personas, and following up with representative stakeholders on how the portal can be more useful for them.
CSTMC will continue to serve as a pilot project for other public sector institutions. As noted earlier, active exploration of the applicability of the approach to other organizations is underway.
Participatory heritage will be disruptive. But CSTMC believes that as members of the creative sector, museums can act as an engine for economic growth and innovation, establishing an environment for open innovation by making its information and knowledge holdings widely available and accessible, and being open to engagement with this content.
CSTMC’s open initiatives represent a fundamental change in how the museums engage with their wide array of stakeholders; but engagement in these initiatives continue to change the institution itself.
Chesbrough, H.W. (2003). Open Innovation: The New Imperative for Creating and Profiting from Technology. Boston: Harvard Business School Press.
Chesborough, H., W. Vanhaverbek, & J, West. (eds.). (2006). Open Innovation: Researching a New Paradigm. Oxford: Oxford University Press.
Corbeil, S., C. Jaja, & F. S. Hale. (2017). “Crowdsourcing a nation.” MW2017: Museums and the Web 2017. Pre-published version. Consulted February 15, 2017.
CSTMC. (2014). “The Canada Science and Technology Museums Corporation Joins the Open Data Movement.” Newsroom, Canada Science and Technology Museums Corporation. Consulted January 30, 2017. Available http://techno-science.ca/en/newsroom/news-cstmc-open-data.php.
David, A. & S. Oberoi, 2016. Museums & Free/Open Source Tools: Cost-effective Best Practices. Professional forum at MW2016: Museums and the Web 2016. Presented April 8, 2016. Consulted January 30, 2017. More info available http://mw2016.museumsandtheweb.com/proposal/museums-freeopen-source-tools-cost-effective-best-practices/
Dennis, A. K. (2013). Raspberry Pi Home Automation with Arduino. Birmingham: Packt Publishing.
Eid, H. (2016). “The Museum Innovation Model: A museum perspective on innovation.” MW2016: Museums and the Web 2016. Published January 14, 2016. Consulted January 30, 2017. Available http://mw2016.museumsandtheweb.com/paper/the-museum-innovation-model-a-museum-perspective-to-innovation/
Kelly, B., A. Dunning, & M. Guy. (2007). “Addressing The Limitations of Open Standards.” In J. Trant and D. Bearman (eds.). Museums and the Web 2007: Proceedings. Toronto: Archives & Museum Informatics, published March 1, 2007. Consulted January 29, 2017. Available http://www.archimuse.com/mw2007/papers/kelly/kelly.html
Kelly, B., M. Ellis, & R. Gardler. (2008). “What Does Openness Mean To The Museum Community?” In J. Trant and D. Bearman (eds.) Museums and the Web 2008: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2008. Consulted January 29, 2017. Available http://www.archimuse.com/mw2008/papers/ kelly/kelly.html
Langer, M. & J. Alderman. (2016). “Open hardware belongs in your museum.” MW2016: Museums and the Web 2016. Published January 15, 2016. Consulted January 30, 2017. Available http://mw2016.museumsandtheweb.com/paper/open-hardware-belongs-in-your-museum/
Maxwell, E. (2006). “Open Standards, Open Source, and Open Innovation: Harnessing the Benefits of Openness.” Innovations, summer 2006, 119-76.
Owens, B. (2016). “Montreal institute going ‘open’ to accelerate science.” Science. Published Jan. 21, 2016. Consulted January 29, 2017. Available at http://www.sciencemag.org/news/2016/01/montreal-institute-going-open-accel-erate-science
Perens, B. (1999). “The Open Source Definition.” In C. DiBona, S. Ockman & M. Stone (eds.). Open Sources: Voices from the Open Source Revolution. Sebastopol: O’Reilly & Associates, Inc., 171-88.
SEED. (2016). Illumination Games Case Study. SEED Interactive. Consulted January 30, 2017. Available http://www.seedinteractive.co/case-studies/ace-academy.html?ref=wp2
Simon, N. (2010). The Participatory Museum. Santa Cruz: Museum 2.0.
Stewart, D. R. & “C.” & C. N. Davis. (2016). “Bringing Back Full Disclosure: A Call for Dismantling FOIA.” Communication Law and Policy 21(4), 515-37.
Treasury Board. (2014). Canada’s Action Plan on Open Government 2014-16. Consulted January 29, 2017. Available http://open.canada.ca/en/content/canadas-action-plan-open-government-2014-16
Wilde, E. & L. Mann. (2010). “Open Source Collaboration: New Models for Technology Development in the Museum Community.” In J. Trant and D. Bearman (eds). Museums and the Web 2010: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2010. Consulted January 30, 2017. Available http://www.archimuse.com/mw2010/papers/wilde/wilde.html
von Hippel, E. (2005). Democratizing Innovation. Cambridge: MIT Press. Available http://web.mit.edu/evhippel/www/books/DI/DemocInn.pdf
Spadaccini, J. (2008). “Open Source Software: New Possibilities for Museums.” National Association for Museum Exhibition’s Journal 27(2). Available http://ideum.com/2008/11/18/open-source-software-new-possibilities-for-museums/
Voss, J. (2012). “Radically Open Cultural Heritage Data on the Web.”MW 2012: Museums and the Web 2012. Published March 25, 2012. Consulted January 30, 2017. Available http://www.museumsandtheweb.com/mw2012/papers/radically_open_cultural_heritage_data_on_the_w
Williamson, V. & N. Eisen. (2016). “The Impact of open government: Assessing the Evidence.” Working Paper, Centre for Effective Public Management at Brookings, December 2016.
. "Open innovation: Open movements and the role of a museum in the 21st century.." MW17: MW 2017. Published February 16, 2017. Consulted .