Back to selection
Culture Hacker
Bridging the gap between storytelling and technology. by Lance Weiler
The Alchemy of THINGS
This weekend at the New York Film Festival, Sherlock Holmes & the Internet of Things will convert Lincoln Center into a massive crime scene. Come step into an immersive storytelling experience that enables you to become Sherlock Holmes. A few slots remain — get your free ticket while they last.
FADE IN:
INT: Theater — Lincoln Center
It’s pitch dark. A faint chatter spills across the theater. Someone clears their throat and everything goes still. A ringing cuts the silence. It has a crispness, a tone that signals the sound of actual metal vibrating. Just on the edge of piercing, it’s an old school ring, one that now only exists in classic cinema or reruns of pre ’80s TV shows. The last gasp of what was once known as a rotary phone —pre-pulse, pre-push tone, pre-touch screen. A spotlight flickers on revealing the iconic object. Nervous laughter fills the air. A figure steps out of the shadows and picks up the receiver.
A crackling sound rises from the phone followed by a faint voice. It begins to grow in volume. Quickly it becomes clear that it is not human but instead synthetic – a text to speech simulation.
“Welcome to Sherlock Holmes & the Internet of Things. Are you ready to play a little game? I have a case for you, Mr. Holmes. You have 90 minutes to re-create a murder scene, populate it with objects and clues and then craft a crime story that unlocks the mystery. You will work as a team and you will work as an individual. Now you’ll find a bag, a body and a rotary phone in the four corners of the room. Take them all. I will be watching your every move. The clock is ticking… “ — Yours Truly the KILLER
In a sense, Sherlock Holmes & the Internet of Things is akin to a radio play but re-imagined for the 21st century. Similar in the way that it leaves room for participants’ to imagine what the characters and setting look like but at the same time challenging the very authorship of the work itself. Instead the project becomes a more collaborative model that moves participants from passive viewers to engaged co-creators. The experience has two roles: the KILLERS who are online and the SHERLOCKS who are on the ground at Lincoln Center. It is through both their contributions that a whole story is created.
Over the course of the last year, Nick Fortugno and I have worked hard to establish a foundation that supports open collaboration. Through a series of meetups, we’ve invited participants to jump in and help to design, challenge and refine a prototype. The analog version contains four steps as participants move from team to individual activities and back again.
The current prototype and design principles form the basis for the MOOC, our Connected Crime Scenes and the Global Sherlock Challenge
Over time this analog prototype has shifted from us creating the crime scene, placing the objects/clues and letting participants “solve” what already existed, to a version where everything is instead crafted in the moment by the participants themselves. As the experience became more emergent we saw the level of participant engagement greatly increase. The goal is to not have a win state or even one right story. Instead participants are encouraged to create freely and collaboratively.
EXT LINCOLN CENTER — DAY
Teams of 6 people make their way out of the building. They disperse in different directions as they navigate with maps in hand. In addition they have a Rotary Phone and a bag containing the following…
- A number of NFC (Near Field Communication) tags that serve as objects/clues that are used to populate the crime scenes
- A small book for teams to record their findings and shape their crime stories (in a sense evoking Dr. Watson)
- A themed map which transforms Lincoln Center into London in the 1870s, Los Angeles in the 1930s or another setting of the KILLER’s choice. Later in the experience this map will transform into a magnifying glass that can scan the NFC tags that are placed around the bodies.
The play space at Lincoln Center makes use of the 13 square acre campus
Following slowly behind the teams as they make their way to Lincoln Center Plaza is a “body,” an actor dressed all in black with a mask covering their face. They are nondescript so that participants can easily cast their imaginations on them. Once the team reaches the destination depicted on their map they stop and determine how they wish to position their body as they begin to build out 5 separate crime scenes.
Each team is given an actor dressed all in black. They become the team’s “Body” which is placed and positioned at the crime scene.
As participants step into a “cat and mouse” game with a KILLER, they are also entering a connected crime scene that links the play space of Lincoln Center to a larger global community in real-time. A collaborative storytelling experiment where the KILLER is online and the team on the ground at Lincoln Center represents SHERLOCK HOLMES.
GOING FROM ANALOG TO DIGITAL
As we transition from our analog prototype to a version that can scale, we decided to create two core smart storytelling objects that can be deployed in any future connected crime scene that is staged throughout the project. We see these objects as a kind of platform for the experience. Both are cloud- enabled so that elements of the experience can be local and global. Plus we intend to provide documentation that can help others to easily build them.
1. A Rotary Phone for the 21st Century
IoTBLR put out a call to their community of 6,000 members to help build the Rotary Phone based on specs developed by the NYC Meetup group. 50 people expressed interest and from there a team of 12 was assembled. Within two weeks we went from a spec to a working prototype. The result is a smart device that is driven by a Raspberry Pi.
The role of the Rotary Phone is to guide participants utilizing the narrative voice of the KILLER. The phone is aware of its surroundings and can trigger media instances (music, dialog, sound effects). This is done through three methods:
- Proximity to a beacon
- Triggered by the system clock
- Reacts to the dialing of the phone (aka “pushing a button”) by participants
The guts of the phone contain a Raspberry Pi and a speaker. Its current features are…
Beacon Aware
Can record/playback audio
Wifi & Bluetooth enabled
Can convert text to speech
Is Cloud-enabled
Handset activates a speaker
Has a working rotary dial
Designed to take on the facilitation of the experience, the Rotary Phone guides the participants from the beginning to the end. In addition, the phone is intended to record the teams’ final stories/theories of the crime while the teams’ bodies pantomime those theories in an effort to recreate the crime.
2. A Magnifying Glass
We’re working with the folks at Conducttr to enable two core elements of the project. One is the magnifying glass utilized by teams to scan objects and clues. The second is a clue database that enables anyone on the web to submit, assign and vote on which clues are connected to an object.
Teams at the NYC Meetup on Sept 12th experiment with ways to create a case for the magnifying glass that can embrace the project’s design principles
Each of the SHERLOCK teams at Lincoln Center will make use of a Magnifying Glass to help them scan the NFC tags which represent objects/clues within the crime scenes. The Magnifying Glass is designed to aid the the placement and reading of objects/clues.
Below is a the first demo from the Conducttr team. After testing we determined that aesthetically we wanted to create the illusion that Magnifying Glass did not have typical functionality. Meaning that a behavior like typing would make the Magnifying Glass feel too much like a phone. So for Lincoln Center, we’re looking at utilizing the NFC scanning functionality, which enables the viewing of images of objects and reading simple descriptions aka clues.
This past weekend at our NYC Meetup we prototyped the case for the Magnifying Glass which will hold the phone. A team of three strangers where tasked with coming up with something that would enhance the experience, make it social and provide room for interpretation which would hopefully fuel the creativity of teams at the Lincoln Center event.
A team prototypes the magnifying case using cardboard, tape, and markers
It was interesting to give such creative freedom to a group who were new to the Meetups. At first they were reserved, but once they realized that their work would be used directly by a live audience, they rose to the occasion. The team did a number of iterations on the prototype and would share with the larger group through a process that we often use called non-judgmental feedback. Basically what that means is that the group presents their work and then everyone else provides feedback in the form of a question that is intended to build upon the ideas. The presenting team is not allowed to answer the questions. Instead they write down the questions and any thoughts they have. Part of the goal is to remove the opportunity for a team to defend their ideas. To allow for the open exploration of what an idea can become. The method embraces “Yes and…” thinking which is often found within improv. Scenes thrive with “Yes and…” and they stop short when someone uses No or But.
A circular design enables the whole team to participate during key moments in the experience.
A circular design was developed to enable all the team members to participate in key moments during the experience. It was also decided that the case for the Magnifying Glass could evolve. So for the first part of the experience it acts as a map, contains a number of clues and knockouts that lets participants see elements of the experience in a different way. Then the map leads each team to a hidden Android phone. When the phone is placed within the map the device awakens and begins to have an interplay with the Rotary Phone, the participants and the objects and clues scattered around the crime scenes.
“Any sufficiently advanced technology is indistinguishable from magic.”
— Arthur C. Clarke
The Alchemy of THINGS empowers creative collaboration
Sometimes it’s difficult to try to explain what we’re doing with Sherlock Holmes & the Internet of Things. The project is emergent. It is a layered project that works to explore collaborative practice at the same time as it’s experimenting with a new grammar for storytelling. The idea that an inanimate object can evolve over time, be aware of its surroundings and tell compelling stories that are personalized, seems counter to how we view the “things” that surround our daily lives. Objects or things usually have utility and address a core need. But as more of them become connected, the notion of a smart thing becomes a complex one. With Sherlock Holmes & the Internet of Things, we’ve found that the more we place human experience in the core of the design, the more magical and meaningful the IoT portion of the project becomes. These enchanted objects end up sparking creative exploration in the sense that they enable large groups of strangers to come together with a shared desire to be connected and be creative. Out of this exploration, a global project emerged that lets participants know that their voice can be heard and that its welcome.
INT: Theater — Lincoln Center
Everyone has returned to what they come to realize is Scotland Yard. As the teams gather they take turns sharing theories to the who, what, where, when, how and why of their crime scenes. There are bursts of laughter as each teams’ body attempts to reenact the crime ending with them settling into their final resting positions.
The SHERLOCKS and the KILLERS have woven a collective tale built from objects found within the works of Arthur C. Doyle. Lincoln Center has doubled for a number of exotic locations around the world. And a Rotary Phone at the heart of the experience has provided a peek into a new form of collaborative storytelling powered by the alchemy of IoT.
As the lights come back on the team resets the room in preparation for the next show consisting of 5 more crime scenes designed by participants from the MOOC.
FADE TO BLACK
Sherlock Holmes & the Internet of Things (Sherlock IoT) is an ongoing prototype developed and run by the Columbia University Digital Storytelling Lab that explores new forms and functions of story. Developed out of the Film Program at the School Arts and powered by Columbia University School of Continuing Education (SCE), Sherlock IoT is designed to be an open R&D space that experiments with shifts in authorship and ownership of stories, the massive collaboration also uses a detective narrative to examine the policy and ethical issues surrounding the Internet of Things.
The goal of Sherlock Holmes & the Internet of Things is to build a massive connected crime scene consisting of smart storytelling objects. This fall teams will create, design, build and test prototypes that will be plugged into a number of crime scene locations around the world.
For more details please visit Sherlock Global Challenge
