Interact is a research and development project created through a partnership between the National Holocaust Centre and Museum (NHC), Bright White Ltd, and the University of Huddersfield.
It demonstrates how technology can be applied to preserve the powerful and engaging experience of listening to, and interacting with, a Holocaust survivor: allowing future generations of museum visitors access to these unique stories.
Based on this work, The National Holocaust Centre and Bright White Ltd will be creating a special experience at the Centre in 2016: turning the vision into reality.
Through the Interact project, the partners aimed to conduct research and development into the following proposition:
Generally, what aspects of the application of audiovisual and computing technology are most powerful in conveying first hand testimony and how, specifically, can these technologies be used to help audiences engage with key individuals who have unique knowledge or experience?
The research looked in detail at one key aspect of the technological solution; the most powerful way to match questions to answers. The basis for the work was informed by research into conversational natural language interfaces.
The Project: Design and Technical Delivery
The challenge of the project was immense; without our key witness (the Holocaust survivor) being physically present (or even alive), enable audiences to ask questions and receive answers from them, providing as true an impression of the appearance and sound of the witness as is possible.
This project tackled two primary challenges:
Create a repeatable, production-level method for generating all reasonably-predictable questions that could arise after hearing a person speak about episodes in their life.
Create a software / hardware machine that can accept a spoken question, and play back a spoken answer.
How the Team Approached Challenge 1
The team invented visual tools that enabled large groups to collaborate together and exhaustively generate as many reasonable questions as possible. The best results were achieved when the generation teams were broad and diverse, representing the profile of the target audience. Working in large teams requires structure and a common vision, and the tools we created enabled this.
How the Team Approached Challenge 2
When we peer deep into human communication, we see incredible nuance, subtlety and flexibility. In some ways, when people speak naturally, successful communication is as much about the mental agility and tolerance of the listener as it is about the clarity of the speaker. We developed a way to think about natural language that meant that natural variations in terms and ways of approaching sentences could be predicted, recorded, and utilised to allow a wide tolerance of terms and approach taken by the questioning party. This was achieved in a repeatable and production-level manner.
The team made a very convincing series of prototypes that demonstrate a strong success rate in terms of providing satisfactory and quality answers to the questions. Evaluations with test subjects improved the performance of Interact over the series and ensured the extensibility of the system.
Digital R&D Fund for the Arts
The Digital R&D Fund for the Arts is a £7 million fund that supports collaboration between organisations with arts projects, technology providers, and researchers. The Digital R&D Fund is supported by Nesta, Arts and Humanities Research Council and public funding by the National Lottery through Arts Council England.
For more information about its projects and digital R&D stories from around the world, visit Native: Magazine of the Digital R&D Fund for the Arts at artsdigitalrnd.org.uk or connect with us on Twitter @digitalrnd or using the hashtag #artsdigital.
Photographer : Katie Blake, Bright White Ltd
Survivor : Steven Frank
Interact involves a collaboration with The Shoa Foundation, and the Institute of Creative Technologies, University of Southern California and Conscience Display, Los Angeles.
The project is supported by:
Association of Jewish Refugees
Audrey and Stanley Burton 1960 Charitable Trust
Department for Communities and Local Government
Heritage Lottery Fund
The Rachel Charitable Trust
The Charles Wolfson Charitable Trust