Back to selection
Building a Monster in Two Months: On Frankenstein AI: a monster made by many‘s Race to Sundance
Frankenstein AI The lobby is packed. Within minutes of opening, a line has formed for Frankenstein AI: a monster made by many, and we’re completely booked for the day. Festival-goers curious about emergent forms of storytelling weave their way through the New Frontier section of the festival, a labyrinth of VR, AR and AI. Now in its 11th year, Sundance’s New Frontier is hosted in two venues and has expanded to over 30 exhibitions ranging from individual experiences to a social VR theater for close to 100 participants.
The last few months have been a whirlwind. On November 13th I received an email from Shari Frilot head programmer for New Frontier — Frankenstein AI: a monster made by many had been officially invited to exhibit. I immediately contacted my fellow conspirators in the project, Nick Fortugno (Interactive Narrative Designer) and Rachel Ginsberg (Creative Strategist/Experience Designer). The prospect of showing at Sundance was both exhilarating and rather daunting. Unlike an official film selection – which is typically winding down post-production when accepted — projects in New Frontier often need to ramp up in order to have a completed project in time for the festival.
With the clock ticking, we had only two months to assemble a team, refine our design, build creative and technical systems and raise the funds needed to produce our ambitious project — all of which rested upon an AI that needed to be built, trained and integrated into the immersive theater experience we envisioned. With close to $350,000 in-kind contributed by our core team and various partners, I raised over $30,000 to help with hard costs associated with production, equipment, and technology.
Tight deadlines and ambitious projects require talented people with diverse backgrounds and skills. Our team consists of the following:
AI Visualizations – Klip Collective
Sound, Score & Instrument Design – Peter English & Jeff Gregorio
Choreographer & Producer – Brandon Powers
Data Design – Sarah Henry
Machine Learning Engineer – Hunter Owens
Creative Technologists – Kaho Abe, Ramsey Nasser, Nick Fortugno
Producer – Miriam Grill
Production Designer – Jennifer Nasal
Set Designer – Dale Worstall
Performer – Jacinda Ratcliffe
Video and stills – Rebecca Murga
Given our tight deadlines, multiple teams worked in tandem to bring Frankenstein AI to life. Nick Fortugno designed an integration plan that centered on OSC (open sound control), a networking protocol that passes messages between machines/computers. These “show controls” are powered and triggered by the AI that sits at the center of the experience. The OSC system facilitated communication between all of the following technologies:
– Machine Learning Algorithm (we rolled our own AI based on an assembly model)
– Surface app with physical controllers
– Internet of Things (IoT) powered 1920’s typewriter
– Artificial Intelligence (AI) vat – a 5’x4’ tank consisting of plexiglass and circulating smoke display to visualize the AI via rear projection
– Algorithmic Choreography System that enabled the AI to control a dancer
As festival-goers made their way into the Frankenstein AI experience, the technology became invisible. The AI interfaced with a number of physical objects, such as 18th century-styled tables that contained Microsoft Surface units. It communicated with participants verbally, sonically and visually respectively through speech-to-text, internet of things (IoT) enabled drums and hologram-like projection within a physical tank filled with circulating smoke. During one of the post-show debriefs, a festival-goer described the experience as “digital alchemy.” I laughed to myself and reflected on how pulling together a project like Frankenstein AI in two months was a feat of magic unto itself.
