A team with two PhD students from CWI won the Polyvocality Award (Meerstemmigheidsprijs) at the annual hackathon organized by Netwerk Digitaal Erfgoed. They stayed up all night to build an interactive 3D interface that gives users a unique exhibition with artworks that are seen through the eyes of different cultures.
Every year programmers, historians, designers and other heritage and data experts take on a challenge in different teams: can they use linked open data (LOD) from heritage institutions to make new, creative applications? And can they connect that data to the public's experience in surprising ways? 8 teams participated in this years hackathon – called HackaLOD – at the Domtoren in Utrecht. After 20 hours of developing they pitched their project to a jury and a general audience. There were 3 prizes from which the Polyvocality Award of 1500 euro went to the Cultural AI team with Savvina Daniil and Andrei Nesterov from CWI and 4 other PhD students from the Cultural AI Lab.
More about the project
The Cultural AI team explored how symbols depicted on artworks have different meanings in different cultures. “Let's take Vermeer's Girl with a Pearl Earring, one of the most famous paintings in European culture. What does a pearl symbolize here?”, explains team member Andrei Nesterov. “In general, across multiple cultures, it means ‘femininity’. And in Chinese culture, it means ‘immortality’. We know this from a knowledge graph about symbolism called HyperReal developed by our team member Bruno Sartini, a PhD student at Bologna University. So we connected this knowledge graph to artworks available online and discovered that an object on one artwork may have several symbolic meanings depending on cultural context. Our idea was to broaden our understanding of symbols in artworks because we are often limited by the cultural views we grew up with. How would a Chinese person interpret Vermeer's art?”
Girl with a Pearl Earring by Johannes Vermeer. Photo: Mauritshuis, Den Haag
The team connected HyperReal to one of the largest knowledge graphs on the web, Wikidata, to retrieve objects that artworks depict. Then they searched for related objects in the collection of the Dutch National Museum of World Cultures which contains more than 400.000 objects originating from 2.000 cultures. Such rich cultural information allowed them to contextualize symbolic meanings.
Interactive 3D interface
To let a broader audience explore these symbolic connections, the team developed an interactive 3D interface. There, a user can select a symbolic concept they are interested in (for example: ‘femininity’) and three cultural contexts (for example: ‘Chinese’, ‘Christian’, ‘Islamic’). As a result, the 3D visualization gives a user a unique exhibition with artworks sharing symbolic associations. Nesterov: “We called this a ‘multivocal exhibition’ because it gives artworks ‘voices’ from different cultures.”
