Spatial Interface for Museum Collection Exploration

Fall 2017

Invertebrates make up the majority of species on Earth, with millions of documented species and likely millions more yet still undiscovered. 

Documenting and understanding the taxonomic relationships of these animals is a colossal task. Thus, there is need for vast collections and correspondingly powerful learning tools to help us comprehend this branch of life.

Current Situation

Museum collections are extraordinary, beautiful, and valuable physical resources in this quest for knowledge. However, they have limitations. Carnegie Museum of Natural History’s massive insect collection has not been fully documented and mapped, rendering it less than surveyable.

In its current form, the collection is extensive and highly varied, but remains essentially inaccessible to the general public and unexplorable to scientific researchers outside the institution. 

We aim to open the collection as a publicly explorable resource. 


My team (Adrian Galvin and I) designed a virtual reality interface for the collection in order to

  • foster observation and support practices of seeing

  • increase taxonomic knowledge production and learning

  • foster public engagement with the CMNH insect collection and the work of scientists in the invertebrate zoology section

  • provoke scientific curiosity and exploration. 


After extensive research of currently available in-VR authoring tools, we settled on Gravity Sketch as our canvas.

vector full 2.png

Visualization Space - Lab

A robust visualization space allows CMNH researchers to view and explore their insect collection with multiple representations, moving beyond what is currently possible with visual observation of physical specimens, into the phylogenetic and morphological which our interface favors, into novel paradigms such as geographic, locomotive, chemical, behavioral, and extra-EM patterning.

The lab presented is a propositional sketch that shows some of the ways that spatial computing can support taxonomic research. This portion of the interface is specifically tuned to support scientists, although it is available to all users.




Naturalist's Notebook

The notebook serves as a record keeper and communication device. It is always attached to the user’s body, available in any scene, invokable through a gesture.

Users can pin any insect from their explorations through the collection into their notebook, along with associated information like geographic distribution or physical characteristics, building up a record of their interests and findings, essentially curating their own subcollection.

For museum scientists, their virtual notebook can be output as a web document that the museum community can follow as a public-facing communication tool, facilitating greater engagement.

From the notebook, users can dive back to the lab to explore an insect more deeply. 



Explorable Biome for Children

This engaging three dimensional environment provides an onboarding experience for first time users. It is an explorable representation of Earth’s major biomes in which the public can find distinctive insects. The user can select from forest, desert, grassland, mountain, rainforest, and aquatic.

In each location, the system prompts the user through their naturalist’s notebook to seek out insects living in their home environment. As iconic species are found, they can be collected in the notebook and highlighted in the lab for further learning.

The biome provides a fun, visually rich experience for onlookers who can track the explorer’s progress on large monitors in the exhibition space. This encourages an atmosphere of communal exploration and play as groups learn about insects and taxonomy.

This portion of the interface is tuned to the needs of younger users who are learning the fundamentals of science. It can be used as an instructional tool by teachers and more experienced users as well. 



Dependencies + Constraints

The explorable representation is dependent on the digitization of the insect collection which alone is a massive hurdle. Either an automated robotic or carefully crowdsourced solution would be needed to accomplish this, and likely over a long period of time.

We are aware of these hurdles, and have chosen to constrain our design to the interface and interaction mechanics in order to provide a vision of what how massive scientific collections could be experienced in the future.

Novice users may be unfamiliar with more exotic VR interfaces, especially as scientific representations grow more complex. We considered scientists’ mental models of the collection and taxonomic organization when spatially representing it. We aimed to create a system which is intuitive and useful for users ranging from world class scientists to young people exploring the fundamentals of observation for the first time.

Advantages + Applicability of VR

VR is a powerful and highly flexible medium for spatial representation. Since it leverages our intuitive sense of body and location, it more directly engages viewers and is itself a curiosity.

The collection demands versatile representations due to its magnitude and numerous categories and interrelationships. Being unconstrained by physical law, renderings in VR can rearrange and transform in ways matter can’t, approaching the versatility required for the collections multivariate representations. 

Research - Interviews with Subject Matter Experts

Dr. John Rawlins is the curator and head of the Invertebrate Zoology Section at the CMNH. He helped us to understand the needs of those who are producing taxonomic knowledge.

He said “Being able to visualize and survey their entire collection is every curator’s dream.” We talked to him about the state of massive museum collections and current attempts to digitize them. He noted that this is an ongoing challenge for museums across the country and any progress made would be invaluable to many organizations. 

Marti Louw, director of the Learning Media Design Center at CMU HCII, spoke with us about the potential for our interface to be a learning and identification tool to support observation and practices of seeing. She helped us to see how our tool to be useful to specific knowledge seekers such as people who need an unknown insect specimen identified. 

Vince Villela, a graduate researcher at University of Pittsburgh’s Department of Education, spoke with us about making our tool engaging to the public and relevant to primary school learning objectives. He noted that an explorative game allows for learning objectives to be measurable and testable, crucial aspects of any learning experience. Our biome environment came out of our conversations with Vince. He helped us to see how our tool could be useful to young learners and explorers.

"Being able to visualize and survey [the] entire collection is every curator’s dream."