As the first major festival to return in person as the pandemic recedes, Tribeca gave us one more sign that New York is coming back. In the Heights, which opened the festival at the United Palace on June 9, was a joyful celebration of community (even for those of us who watched at home), and even in a reduced capacity the festival was a wonderful opportunity to reconnect with the movies. It also seemed that after shuttering the 2020 festival, this year’s event was fairly bursting at the seams with new types of content—of course the short and feature films were still the center of attention, but the festvial expanded with a new category of video games, stand up comedy, storefront art, installations throughout lower Manhattan, and other work.

In fact, it may be largely due to the expansion of its immersive works that the Festival changed its name this year, dropping “Film” to become the more enigmatic—but, presumably, more expansive—Tribeca Festival. Cinephiles may bemoan the change—it still doesn’t sound right to my ears—but it does reflect Tribeca’s growing perview as it grows into something like a SXSW-esque umbrella of festivals.

The original film festival was eleven years old when it first introduced Storyscapes in 2013, a collection of five interactive films, no VR or AR in sight (which I reviewed at the time). Before long festival organizers began differentiating this core program, which now primarily consists of high-end virtual reality pieces, from its broader immersive and interactive offerings. Everything has always been housed in the same space at Spring Studios on Varick Street, however, making the split nominal and arguably unnecessary. But because of the pandemic, this year it was clear as day: the spacious hall that normally hosts everything was now limited to just five experiences—Storyscapes—while other pieces were divided between “Outdoor and Interactive Experiences” and the “Virtual Arcade,” which this year truly was virtual, primarly through an agreement with the Museum of Other Realities to host VR films for remote users of Vive and Rift headsets.

Limiting the number of pieces in Storyscapes, while obviously occasioned by social distancing requirements, allowed for some other innovations as well. In a pandemic virtual reality presents a potentially even bigger problem than a movie theater, as in past years Tribeca patrons have crammed into tight queues to each take a turn in a headset. The solution this year was not only to reduce the total number of attendees, but to admit them in periodic waves of four, so they could move together from one piece to the next. The order was curated to create a kind of narrative flow, and the extra space allowed creators an opportunity to create large visual artworks, essentially, casting the moods of their pieces, which in turn helps patrons begin mentally onboarding onto a new project before even reaching for a headset.

Tribeca’s divisions into three categories still allowed for different technologies in each program; for instance, there were two AR pieces in Storyscapes. Rather than proceed through the pieces in the way that the festival arranged them, I’d like to look at all the immersive pieces that I was able to experience in like-categories of two-dimenional films, audio works, AR, and VR. One drawback of the ever-expanding slate of projects is that it’s increasingly difficult to see everything, especially compared with 2013, and there were several this year that I wasn’t able to view. But here’s what I was able to get to.

TWO-DIMENSIONAL FILMS

Republique is a surprisingly ambitious interactive film from French writer Olivier Demangel and director Simon Bouisson, a team that’s collaborated on multiple films before. For Republique Demangel took the November 2015 terrorist attacks across Paris as a model, while shifting much of the action underground into the Metro, thus drawing parallels with subway attacks in London, Madrid, Tokyo, and elsewhere. The film begins with a young couple livestreaming as they giddily explore the labyrinth of the Metro’s abandoned foot tunnels, but, rather than sticking with them exclusively, the movie soon provides the option to watch a second group of office workers about to catch a train, and then a third group of friends who are above ground near la Place de la République, just north of the city center. As the attack rolls out we can see how each of these three groups is affected, piecing together the larger action of the attack and the police response while learning about personal connections between these people who are caught up in the action—a young woman on the street is worried about her missing husband, who viewers know is wounded and being cared for by two of the office workers, while at another two characters set off on their own, only to arrive at another group.

We’ve seen this type of film before, though perhaps without the sprawling scale or apparently large budget that Bouisson had to work with. What feels most innovative about Republique is the mechanism through which viewers select what to watch: where most interactive films have break points where viewers can choose the next segment, here each of the three branches plays out simultaneously and without stopping, while viewers are able to toggle between them by swiping up or down, like three stacked video tracks in any editing software (by the midpoint I was thinking of them in terms of “the top story,” “middle story,” and “bottom story”). This is a smooth and intuitive way to move between the stories at any point, but it also stirs up a great deal of FOMO, as viewers may wonder what they’re missing in the other streams and constantly switch back and forth between them. I realized that this is quite akin to watching three shows on TV in the pre-pausing/streaming days, where we would click back and forth and do our best to put the fragmented stories together in our minds. The filmmakers are quite aware of this, however, and thus as the story wraps viewers are presented with a map of the story (three horizontal bars, essentially) with the offer to go back and watch parts that they missed.

Creating three simultaneous/linked films is obviously an ambitious undertaking, raising the film’s total running time from 40 to 70 minutes, and it actually outperformed my expectations. Having accepted that I couldn’t see everything, I enjoyed the chaos of jumping into each group’s story in medias res, without any context of what just happened (“Oh, they’re in the street now,” “Okay, now they’re hiding in a closet,” etc.). In the end the three groups come together and the branches are clipped, so to speak, but still the mental chaos that comes from only having a partial knowledge of what’s going on remains—which is exactly how each of the characters in the film would have experienced it.

Unfortunately it also meant that the timing was sometimes off when the groups were meant to intersect. At one point the wife calls her husband’s mobile and I switched to his video to watch the conversation from his end, but no phone rang. At another point I caught one character in two places at the same time. The difficulty of planning and timing everything in production must have been immense, so these slight anachronisms weren’t terribly distracting. Quite more distracting, to my mind, was the conceit that each of the three videos were presented as being filmed by the characters themselves on their cell phones, a 2020 version of The Blair Witch Project, which is here updated for the era of Facebook Live. Each video also included a stream of comments (and emojis) from viewers that felt out of place for such serious subject matter and mercifully could be muted. The film’s website touts the comments as another character that interacts with the onscreen actors, and I did have the thought that my muting them may reflect more on me and my age than on the film. Still, the idea that characters are constantly filming themselves, especially in a life-and-death crisis, does stretch verisimilitude for me—wouldn’t they want to turn off their phones and save their batteries while they’re barracaded in a dark room?—and I don’t think the film would have lost anything if it had just presented the action as a traditional movie, with no justification for the cameras that recorded it. Still, this is a quibble, and one that would have made a rather different film than the one Demangel and Bouisson wanted, which even in this form is a towering achievement and intuitive new way to move through interactive films.

AUDIO PIECES

Current, which won the festival’s Best Creative Nonfiction Competition Award, is a walking audio tour by multidisciplinary artist Annie Saunders, with contributions from Andrew Schneider and others. Beginning and ending in Zuccotti Park—ten years after the Occupy Wall Street movement centered there—it was commissioned by Brookfield Properties’ arts development arm Arts Brookfield to connect their buildings at One Liberty Plaza and One New York Plaza. Saunders took this rather corporate commission and created an engaging meditation on the history of lower Manhattan, reminding you that it’s an island with a waterfront and a human history stretching back centuries before the Dutch arrived. She employs imagery of air and water to concretize the notion of Manhattan as a living, breathing entity, its shoreline artificially growing past Pearl and Water Streets, its buildings rising and falling to make way for ever newer, taller constructions. The analogy of the living island is reflected back on listeners by explaining how to take your own pulse and having them attend to their own breath and heartrate; when my tour arrived back at Zuccotti Park my walking partner and I both automatically checked our pulses without even being asked. Sometimes the changes are cataclysmic, as in Schneider’s memories of the neighborhood when Hurricane Sandy hit (and, though not mentioned, the shadow of September 11th hangs over the entire piece), but more often it reflects the gradual evolution of what’s now the United States’ most urbanized square mile. Colonial-era buildings like Fraunces Tavern and Trinity Church are now landmarks dwarfed by glass and steel skyscrapers, relics of past centuries that Saunders wants listeners to stop and notice.

But this isn’t just about the landscape. Again and again she and Schneider repeat the theme that “it won’t always be like this,” and they’re talking about our lives and relationships as much as the buildings downtown. Time is relentless and, for better or worse, everything constantly changes. The subtext to be present in the moment, to take in the past, and move toward the future is reinforced by the actual buildings around you as you listen. The name Current, of course, is a pun, obviously including the currents of water and air that flow in and around lower Manhattan, but also implying the flow of time and the preciousness of the current moment.

The piece is at its best when it brings in outside voices: a man from whom Saunders simply asks directions but who then goes on to explain the history of Pearl Street and the waterfront; a Native American man who eloquently discusses how to connect with the island and how its tunnels and streets are like airways that help it breathe—before adding that he also likes to take the subway out to Mets games at Citi Field. The sound design is also superb, particularly in the final moments. It lasts about an hour and covers nearly one and a half miles, which is not very far, but at times the pace goes very quickly and listeners may find themselves jogging to catch up. At one point early in my tour we were interrupted by a man asking for money, so, with no way to pause the audio, we missed the cue to turn off of Wall Street and spent about ten minutes figuring out where we should be and catching up. Of course, that unpredicability is what makes New York New York: the currents of vehicular and foot traffic make a prerecorded walking tour in downtown Manhattan very hard to time, but they do reflect the lifeblood of the city today.

Current is free and will be available through September (it begins every half hour from 5:30 to 8:30pm, to encourage groups to go together); look for the large QR codes to scan in Zuccotti Park.

Knot: A Trilogy takes place, as its name implies, in three distinct segments, each lasting around 20 minutes. Like Current, the audio is designed to be site-specific, but unlike Current the listener has complete control over what those locations specifically are. The first episode is designed to be listened to while sitting on a park bench; the second, in the passenger seat of a car; and the third in a living room or home. The desired effect, of course, is to relate the audio to the listener’s physical environment, and it can be worth the effort to add touch (the wind, for instance), sight, and real-world sounds to the audio mix. It does require a bit of effort on the listener’s part, of course, although what film critic would complain about having to go to a theater to watch a movie? Knot‘s creators at the audio-centric theater/technology company Darkfield are simply asking the same attentiveness from their audience. Of course, it also introduces an element of variability: in episode one I used a bench in Inwood Hill Park far from any road, so the moment when a car pulls up slightly breaks the verisimilitude, and, more notably, since I didn’t have access to any cars (not wanting to, say, get a Lyft) I had to listen to episode two in my apartment and simply imagine the situation. This limited the effect to just the audio, but that was still an engaging experience.

The title Knot could have many meanings, but most obviously it refers to the story itself, such as it is. It’s probably no spoiler to say that the plot loops back on itself, so episode three ends where episode one begins, with lines of dialogue and small events recurring in different orders and settings, or coming from the mouths of different characters. It’s like peeling through layers of the Gordian Knot to find that you’re not any closer to unraveling it, and this is heightened by the stilted language and purposely flat, formal readings of the lines (think of Bresson “ironing out” his actors’ emotions). Amnesia pervades: no characters can remember where they’ve been or what they’ve done, and one can’t even remember his name, defining himself by his lone “character trait” of smoking. Another woman bemoans, “I don’t know what I represent.” At no point does anything feel naturalistic—we’re in the realm of the absurd and the surreal.

The resulting tone feels like Waiting for Godot or, even more strongly, something out of David Lynch. Is this a dream sequence? Is any of it real? Is that character—or am I, as a character in the drama—simply hallucinating? This surreal sensation is strongest in episode two when a character recounts a dream in which a little boy terrifies a grown man before leaping inside him, to presumably destroy him from the inside out. The boy is in fact a motif throughout all three episodes, and if there is a logical explanation to the narrative it revolves around the trauma caused when he is struck and killed by a car while chasing his ball (he was looking for it in the park earlier). Episode three is largely a group seance in which the boy dies again and again, the other participants assert that they’re all fragments of the listener’s own shattering psyche, and a mysterious box (that the listener has been transporting in the previous episodes) has the solution to break the knot and the time loop. This of course is straight out of Mulholland Drive, and offers just as much closure, as in this case the box’s disappearance from the apartment is what causes the need to return to the park and retrieve it (again). If Knot doesn’t fully rise to the level of Lynch, it does create a wonderfully dark atmosphere in which to spend an enjoyable hour. Through all of its urgent and ominous sound and fury, in the end there’s really nothing left but silencio.

In addition to pieces from Tribeca Immersive, the festival has now expanded to include 14 fiction and nonfiction podcasts (listen to them here). This was an addition that was planned for the aborted 2020 festival, meaning that some pieces that were unable to premiere then are included now. There’s a broad topical range, from a documentary look at a police raid of a tantric temple in Mother of Maricopa to a fictional Christmas teenage love story in the projects in Brooklyn Santa to the hallucinogenic comedy The Imperfection. One podcast, Un(re)solved, is connected with an AR piece, about unsolved murders of African Americans, from the Immersive lineup.

 

AUGMENTED REALITY

After the trauma of 2020, Breonna’s Garden was probably the most emotionally satisfying piece at Tribeca this year. Breonna, of course, is Breonna Taylor, a victim who quickly became a martyr and an icon as the nation’s grief and rage boiled over last summer. Breonna’s Garden is an augmented reality memorial to her memory, crucially not just as an icon, although that’s unavoidable, but as a human being. The piece was created by her sister Ju’Niyah Palmer in collaboration with artist Lady PheOnix, and it bears the fingerprints of those who knew and loved Breonna long before her murder.

The piece, essentially a 3D sculpture, breathes tranquility from beginning to end. When you turn on the app it begins with a photo montage of family photographs of Breonna, which I had not seen before in the news, accompanied by mournful music. This transitions to the AR portion, a fully fledged garden surrounding Breonna’s animated figure. Palmer appears next to her to pay tribute to Breonna as she remembers her—missing their grandma, snuggling on the couch, “not like the media has portrayed her”—and then as she disappears the attention shifts to the garden itself. As the viewer moves their handheld device close to each flower, it triggers an audio recording. In my viewing the first few were also from friends and family who knew Breonna well, showing that this is a personal tribute from her closest loved ones, as they address her directly to recall how annoying she could be, road trips, and other experiences. The proximal audio works well in prompting viewers to move around and explore, getting close to each flower. Soon we move to voices of people who didn’t know Breonna but who have been moved by her death, and then there are others who take the opportunity to pay homage to their own lost loved owns who influenced them. Viewers can, of course, record one of these messages themselves, thus planting their own flower, and making Breonna’s Garden include everyone who has mourned. With the simple graphics of the flowers, this documentary audio of people simply speaking their grief and their gratitude is immensely powerful, far more than a handheld AR app might be expected to be.

During Tribeca the piece was housed at the WarnerMedia Innovation Lab on west 21st Street, where it premiered with Palmer in attendance. Another space, however, was set up and remains open in Wagner Park, which is the northwest corner of the Battery, south of the Museum of Jewish Heritage and near where Little West Street hits Battery Place. Large QR codes invite passersby to download the app and watch the experience there; I took the photo above facing south toward Pier A, the first pier as you round the Battery’s waterfront. But the app is also available online for anyone who wants to view the experience, making this a permanent, if digital, memorial to Breonna and everyone else who is remembered here.

While most of this year’s AR pieces were designed for handheld devices, Critical Distance was developed with Microsoft for use on their HoloLens 2 headset. Co-created by Adam May and Amy Zimmerman, the piece deals with a pod of Southern Resident orcas that are residents off the coast of Washington. This pod is both high endangered and thoroughly studied by biologists like Ken Balcomb, with each of the roughly 75 individuals named and chronicled. And while there’s insufficient status to determine the conservation status of orcas globally, this pod has been shrinking for years as they deal with issues like the busy shipping channels off the Salish Sea and the diminishing supply of salmon that are resident orcas’ chief food supply (I interviewed director Josh Murphy on his documentary Artifishal about depleting salmon at Tribeca two years ago). The main environmental issue of Critical Distance is the underwater noise pollution caused by ships and how it interferes with the whales’ echolocation and communication with each other. If they can’t echolocate they can’t hunt the few salmon that remain. And engine noise affects them in other ways: Zimmerman told me that in the quiet of the COVID-19 shutdown three calves were born, indicating that ships may interfere with mating as well.

The piece as housed at Storyscapes, and how it will appear on a tour of science museums like the Smithsonian and London Natural History Museums, consists of a large circular space with white walls that allow for video projection. This two-dimensional screen (which reminded me slightly of Jon Favreau shooting The Mandalorian) allows for the communication of information, like the family tree of the entire pod and the lifespans of all the known whales that have lived and died there, and the see-through nature of of the HoloLens allows for these dual levels of display. The truly mesmerizing content, of course, happens within the headset as first some fish, then a few individual orcas, then the whole pod swim around you. This is interrupted first by a small fishing boat then by a large vessel, and the audio and visual scramble to convey to human viewers how disruptive this noise is for creatures that rely so much on their hearing. Viewers are syncronized, so they can point out animals to each other, even from different places in the space, and this social aspect of the piece is what made Zimmerman, a five-year veteran of Unity, want to create it in AR rather than the more visually siloed technology of VR. While catalyzing action is always the hardest task for environmental films and art, she hopes that if viewers have a social experience it can best foster social change.

The animals in the piece are ethereal and ghostly, somewhat monochromatic translucent beings that glide past or through you and seem to respond to your touch. This is thematically appropriate, as the entire piece is about the precarious nature of these whales’ very existence. But there’s something about knowing that these are digital recreations of actual animals that are living in the Pacific right now—each has a name and a history—that makes it even more haunting. These ghosts of the whales may soon be all that’s left of them, and, as close as they feel in this piece, the physical whales’ absense, from, say, a building in Manhattan, uncannily echoes their potential future in the water. It may have something to do with the ontology of the photographic (or digital) image, or the work of art in the age of mechanical (or, again, digital) reproduction, but there is something about seeing the orcas’ image in this way that was different than, say, watching a traditional documentary about them. There’s not nearly as much information conveyed in Critical Distance as in a book or feature film, but there’s a different emotion to it. You reach out to touch them but feel nothing. Far away, so close! Perhaps that’s an unintentional meaning of the title, that even while we’re distant from these animals we can somehow feel—or be—close to them. And it’s critical that we do.

Jupiter Invincible tells the story of a light-skinned slave in the antebellum South. The most deceptively simple piece at Tribeca, it’s told through a comic book written by the Pulitzer Prize-winning poet Yusef Komunyakaa, with art by the accomplished artist Ashley A. Woods. On its own as a printed work it’s a tremendous addition to Black-created graphic novels, and its timeliness in focusing on the heart of the African-American historical experience, in the evils of slavery, couldn’t be better. But Komunyakaa and Woods also enlisted documentary filmmaker and AR artist Ram Devineni to enhance the book: by downloading an app onto a handheld device readers can hover their phone over each page to see Woods’ artwork come to life. The variety of Devineni’s work is impressive, with things like the drawings bursting out of the page in three dimensions, animation of the characters—a horse rearing, a man walking with his dogs—and live-action footage—a brook in the woods, for instance, bubbles to life. Audio matches all of Devineni’s visuals, increasing the immersiveness of the work.

Nine years ago I read (saw?) the book and app The Fantastic Flying Books of Mr. Morris Lessmore from Moonbot Studios, in one of my first encounters with augmented reality (Filmmaker subscribers can revisit that article here). While their work was groundbreaking at the time, it’s immensely gratifying to see how the quality of book-AR combos has improved, in both the ease of triggering the app to display the AR and in the impressive depth, range, and resolution of Devineni’s work. It’s not his first foray into printed AR, as he previously used his nonfiction film background to include documentary interviews with rape victims in his comic book series Priya’s Shatki. At Storyscapes Devineni told me that the next step in his work will be to use green screen technology to create a full film-comic book hybrid through AR. Like a movie where you literally turn the pages, this will be a delightful new medium as it’s perfected, especially when AR glasses become common enough to remove the handheld device between readers’ eyes and the page.

With all these exciting advances in the technology, it might be easy to overlook the power of Jupiter Invincible‘s story. Komunyakaa has created a magical realist tale that infuses superhero elements into a runaway slave story, as Jupiter gains the power to heal from any wound. This makes for a thrilling character itself, but there’s more at work here than a Black Wolverine or an antebellum Luke Cage. Central to this is a vision Jupiter has halfway through the book, after he’s beaten nearly to death and is struck by lightning, in which he visits his mother in Africa and learns of the majesty of his people. When he wakes, his skin, initially light enough from albinism that I didn’t initially realize he wasn’t Caucasian, has become a rich brown, and he is now a proud African man. It’s the same pride that the broader public discovered with Black Panther in 2018, and indeed Woods has drawn Shuri for a Marvel comic cover. And it’s an appropriately actionable pride for the current moment, as Jupiter then puts his head down and gets to work freeing his people. The story remains unfinished at the end, so hopefully more issues, with or without AR, will follow.

Filmmaker‘s Lauren Wissot spoke in depth with Komunyakka and Devineni at Tribeca, which readers can find here.

Procession is a new work by artist Dustin Yellen which builds upon his sculptural practice by adding an AR component for handheld devices. Yellen’s work, while multidisciplinary, is based largely in sculpture, utilizing stacked sheets of laminated glass to embed two-dimensional images and other objects to create frozen tableaus; thematically, his work often centers around the Anthropocene and how humans are manipulating the environment, generally to catastrophic ends. In the case of Procession, a small vertical piece, about the height of a human being, has been installed on the sixth floor of Spring Studios, one flight up from the Storyscapes gallery. The sculpture itself features a tottering skyscraper built essentially of open columns and floors, covered with vegetation in a way that invokes post-apocalyptic scenarios. Populating this building is a host of miniature anthropomorphized animals—a sea lion wearing a green jacket and blue jeans, a rat in a blue shirt carrying a giant mushroom, a kangaroo in a work jumpsuit with wooden boards across its back—all constructed from photographs and collaged together between the sheets of glass.

An augmented reality app animates these characters, not creating a linear narrative but giving life to the frozen images of the sculpture. This is available onsite in Tribeca but, more of a companion piece than a linked component to the physical work, doesn’t actually require the sculpture at all. The audio brings the aural world of nature, filtered through tonal music, to the piece, while the movement of the creatures increases their visual interest. Unfortunately, when I tried to run the app on my own phone later it repeatedly failed to load, a hazard of handheld AR pieces. The piece is described as implicating viewers in the havoc wrought by human-created climate change, so I’m left wondering if there is more of a narrative structure to the AR if it’s left running longer than I watched at Spring Studios—the entire piece is timed at fifteen minutes. In any scenario, it will be interesting to watch as more traditional artists like Yellen incorporate AR into their work, opening new creative avenues for them and their AR-centered collaborators.

It would have been easy to miss Un(re)solved in its installation in the northeast corner of the Battery, behind the Netherland Monument. There a large but unassuming wooden sculpture resembling a roofless pagoda held up sheets of colored glass, each bearing rows of names. It was a memorial to African Americans killed by Whites during the civil rights years, and it tragically contained 151 names. The sculpture, of course, was contemplative on its own, but the augmented reality portion brought each of those names to life. A work created by multiple partners under the direction of creative director Tamara Shogaolu, Un(re)solved in this format represents some of the best AR sculpture there is today. Still, viewers who missed it there can watch an online version via web browser here.

In the sculpture, each name had a small QR code to the side, and scanning this would bring up that person’s data. It didn’t launch straight into their story, however; first the viewer had to speak the person’s name aloud three times. This ritualized “saying of their names,” loud enough for others to hear, felt like a respectful way to embody them once again and remember who they were and how they died; since this is a crucial part of memorializing them, the online version contains this element as well. It’s similar to Procession in that it’s a sculpture that triggers a handheld AR component, but the key difference of course is that where Procession shows just one large, intricate animation, Un(re)solved contains a legion of small ones. Perhaps the greater  similarity is with Kusunda, as noted above, because these are people who are in danger of being forgotten, whose deaths may slip into the fog of time, and the act of saying their names brings them back and makes their memory that much stronger.

Once this was done, the app would present any information that was available about the person. I saw information presented in text, audio, and video, and, moving around the sculpture, I found names that were remarkably different in their stories, but all wrenchingly similar as well: a fifteen-year-old boy shot by a White police officer while waiting to go inside his New York City school; a girl killed in a drive-by shooting by three White men on the day she graduated from high school; a middle-aged couple, active in the civil rights movement, killed by a bomb in their home on Christmas night, which was also their anniversary. Fifty-eight-year-old Bessie McDowell was asleep in bed on June 14, 1956, when two White men, a father and son, came to collect on a loan they’d made to Bessie’s nephew, who was staying with her. The young man told them he didn’t have the money, and one of the men reportedly slapped him, causing him to run inside for safety. The father then shot a handgun through the window, striking Bessie in the face. He later claimed his shot was in self-defense—he thought the boy was going for a gun—and he didn’t know anyone else was home, so he only served 12 months in prison and paid a $500 fine. Of course this story, which I chose randomly, bears uncanny resemblance to Breonna Taylor’s, and it was fitting that this piece was so close to Breonna’s Garden in the Battery as part of Tribeca’s celebration of Juneteenth.

Un(re)solved is deceptively simple, but I suspect that the more time you spend with it, the more similarities you’ll see in the stories, causing you to not only honor these victims but wonder how far our society has actually come. In that way, it bears a striking resemblance to the VR piece The Changing Same, discussed below.

 

VIRTUAL REALITY

I wrote about The Changing Same, which won Tribeca’s Best Immersive Narrative Competition Award, when it premiered at Sundance in February. This time, however, I was able to view it on a proper headset that the film was designed for, rather than on my own Quest 2 at home, followed by a wonderful in-person conversation with the creative team (a small joy of the vaccine roll-out). Focused on the Black American experience, the title riffs on the fact that the more things change the more they stay the same; for all the country’s progress in race relations and civil rights, many parallels with the past remain sadly ironclad. Realistically animated (seemingly from volumetrically captured performances) in six degrees of freedom (or 6DoF, meaning viewers can move around the space and change their perspective), the piece starts in what seems like a southern bayou, where the viewer meets the film’s narrator, a young Black man, before flying through the air in a beautiful interlude and landing on a peaceful suburban street at night, where a police squad car pulls up, lights flashing, and the cops, one White and one Black, proceed to arrest you and your guide because of a match to a vague description. The situation escalates as the cops assault the young man, but as the scene shifts to the precinct the blame is falling on him, throwing him into a racist prison system through no fault of his own. A flashback to a slave auction house—where the same actor is protesting that he was born a free man and is not a runaway slave—drives the parallel between chattel slavery and modern policing home. It ends on an optimistic note, however, as viewers leave that reality and envision a future—or a present—of African-American equality, beauty, and pride.

For me, a white male, VR pieces like this and Roger Ross Williams’ Traveling While Black (2019) are incredible tools for building empathy. As with Critical Distance, there is a definite educational component, but it’s not the facts as much as the emotion of going through those situations that reveals new insight—insights that are sadly far too common for so many Americans today. Again, something about being embodied in the VR space, helplessly facing a police officer who could destroy or even end your life for no reason, feeling yourself getting sucked into the system, has a different impact than watching films, reading, or engaging in real-world activism—at least for me.

And that’s the goal, the kind of empathy it’s meant to catalyze. The creators Michèle Stephenson, Joe Brewster, and Yasmin Elayat want to lay bare the systemic roots of modern racism, and they want to affect how people feel about it so that they become involved in ending it. This is the first of three Changing Same films; future episodes will deal with topics like conditions in contemporary/1825 New York City and the horror of spectacle lynchings. And they plan on releasing the films not just online but through controlled venues and events like tent shows, museums, festivals, and community gatherings. They want viewers to see the films in public so that conversation can immediately ensue, community can be built, and societal progress made. Perhaps their most important outlet will be schools, where young people can be exposed to these issues with more emotional power than they would get from a textbook. At a time when many pundits and lawmakers are making hay out of critical race theory, works like this will provide educators a welcome counterweight to provide their students with accurate, impactful information. I can’t help but think of The 1619 Project (and the recent political pushback over Nikole Hannah-Jones’s tenure) in conjunction with The Changing Same, in terms of how both elucidate the connections between the present and the past, and I hope that as the films are finished they can help countervail those who would rather bury the United States’ racist past. As many have said, it’s only by understanding our history that we can alter our future.

Inside Goliath, a British-French coproduction led by co-creators Barry Gene Murphy and May Abdalla, is an animated documentary about mental illness. An anonymous narrator recounts his  story about living with schizophrenia and psychosis and his descent into alcohol and drug addiction in London’s seedy streetlife. Meanwhile animation in 6DoF illustrates his morphing flat that grows increasingly claustrophobic, or his isolation at a club as his addictions surge. One fun sequence near the middle is an ironic video game, presented as an arcade machine, that the viewer plays; a side-scroller that approximates a 16-bit game at best, you navigate the narrator through the streets as he gathers booze and drugs and avoids the violent gangs that, in his real life, were far too common. This was a surprisingly engaging moment in the piece and could point the way for two-dimensional pieces that could convey a similar educational message.

The title Inside Goliath makes me think of a giant that not only stands in front of someone with severe mental illness, but that has fully enveloped them so that there’s no way out. Another apt metaphor might be Kronos devouring his children, leaving them helplessly inert in the darkness. In fact, the narrator’s despair, leading to indifference about the inevitability of his fate, is perhaps the most chilling part of his story; while there obviously is help for those battling mental illness, Inside Goliath does a wonderful job conveying how difficult it can be to reach out through the fog and try to reach it. Like The Severance Theory: Welcome to Respite, discussed below, this piece conveys less factual information about mental illness than it re-creates what it’s like to live with it and the trauma that can result; in that way it resembles The Changing Same and some other of this year’s pieces as well.

Kusunda, which won the festival’s Storyscapes Award, is a wonderfully timely document.ary about disappearing languages, a global phenomenon that’s receiving far too little attention. Kusunda is a language so endangered that not only have most people never heard of it, fluent speakers probably barely reach into the double digits. The people all live in the mountains and jungles of Nepal, where Nepali has subsumed it as a dominant language. VR filmmakers Felix Gaedtke and Gayatri Parameswaran, co-founders of the Berlin studio NowHere Media, saw a chance to shine a light on the language, and perhaps help rescue it, through a woman named Gyani Maiya Kusunda who taught it to younger members of her community. Initial filming took place, but she died just as they were about to journey together to film the majority of the production in the backcountry; filming was then further hindered by the COVID pandemic. Parameswaran recounts all of this in voice over, and she told me that they decided to include the making of the film within its narrative to show how tenuous the language’s position is. Her narration describes the difficulty of setting up a volumetric capture camera in a remote hillside village, for instance, and at one point their camera array overpowers the electrical supply and causes a blackout. This illustrates the remoteness of Kusunda speakers as much as any information conveyed in the film.

Despite all these difficulties, they succeeded in making an intriguing interactive film, with wonderful visuals and 6DoF; at Tribeca viewers were seated, but nevertheless they could physically lean and move around the space slightly to take in new perspectives, an effect that always increases a VR subject’s verisimilitude. The Kusunda speaker who became their new subject, a shaman named Lil Bahadur Kusunda, has nearly forgotten it. This fills him with regret, and he seems delighted with the film crew and their interest in the language. He recounts some stories from his life, which are animated in TiltBrush, and is pleased with his granddaughter Hena Kusunda, a young woman who is determined to learn the language and keep it from extinction. In the end she performs a song for him in Kusunda, making it the only song that exists in the language, and it’s a quietly moving moment that gives a dose of optimism in the face of such harsh odds. Capturing her singing with a 3D volumetric image enhances the presence of both her and her song.

Since Kusunda deals so much with a spoken language, it’s appropriate that one of its most compelling features is in its audio. A language dies when it’s no longer spoken, so at various moments in the narrative the viewer has to learn a Kusunda word and speak it out loud in order for the story to advance; in two cases this determines which animated story you see—choosing between a story about a bear or a tiger, for instance—which adds an element of interactivity to the film. But the emotional function is much like that in Un(re)solved: by speaking these words out loud, you become a partner in ensuring that this language and the people who spoke it are not forgotten, are still in a way present today. In this case it’s also rather fun, and made me think more about how audio prompts can be used for interactivity in the highly visual medium of VR.

Parameswaran also told me some of the things their team is doing to preserve the language. The VR film itself will be available on Steam for the general public, and they’re working to get it into museums as well for those who wouldn’t encounter it online. Perhaps more importantly, they’ve made a traditional 2D film and are working in the community in Nepal to educate people and spread the use of Kusunda among those whose ancestors may have spoken it. This outreach, which is in the best tradition of Griersonian documentary, may be the most important element of the entire project, because if they succeed in exciting more young people like Hena, then Kusunda is in good hands for the future.

Lovebirds of the Twin Towers is the latest work from filmmaker and VR documentarian Ari Palitz, who is known for work like producing Clouds Over Sidra (2015), about the Syrian refugee crisis, and co-directing The Last Goodbye, about the Holocaust. This latter piece, which I spoke with him about when it premiered at Tribeca in 2017, used volumetric capture to record the testimony of survivor Pinchas Gutter at the Majdanek Concentration Camp, creating a physically palpable presence of the place as Gutter guides viewers through the various rooms (even though it was stitched together in postproduction).

The DNA of The Last Goodbye is present in Lovebirds of the Twin Towers, but the technology has only improved—and the tone is remarkably lighter. Created for this September’s twentieth anniversary of the 2001 attack on the World Trade Center, it focuses on a couple, Carmen and Arturo Griffith, who both worked as elevator operators there. They met, fell in love, and had innumerable wonderful experiences in the towers and the surrounding plaza; they also lived through the 1993 car bombing—Carmen was right there when it happened—and the September 11th attack, when they were in different buildings and, both injured, didn’t know the other’s fate for several days. Their reunion made the news in 2001, and Palitz told me that he wanted to focus on an uplifting story like this for the anniversary because there was so much goodness and joy that came out of the place but that has been neglected in the face of the immense tragedy. While this is obviously appropriate, for this piece Palitz wanted to focus on the positive human stories of the WTC rather than just the attacks and their aftermath.

The piece at Storyscapes largely consisted of an interview with Carmen, who, like Lil Bahadur Kusunda, is seated but filmed volumetrically. She sits against a black background, allowing her words to carry the weight of her story, but while she talks the scene shifts to the outdoor plaza, the view from the roof, and other locations from her memory that no longer exist. The photorealism of these animations is remarkable, and, as someone who first came to New York in 2002, a year after the attack, it was a wonderful way for me to experience this part of the city’s heritage, being fully immersed in a space that I had only ever seen in photographs and films. Visually, then, the accomplishment here arguably exceeds that of The Last Goodbye, and hearing Carmen describe her story is as emotionally fulfilling as anything else at Tribeca this year.

Arturo’s interview remains to be completed by September, and the piece will be installed at other sites. As amazing as the finished VR piece should be, the piece also includes 2D interactive interviews with Carmen and Arturo, with innumerable questions that promise to be just as enlightening and educational; this was how I heard the details of her story about the 1993 bombing. Palitz is excited by the educational opportunities that this format presents, telling me that he’s recorded his largest oral history yet with William Shatner, covering his entire life and career. If the topics are already ranging from 9/11 to Star Trek, the interactive video interview may be one of museums’ prime tools in the next few years.

The Severance Theory: Welcome to Respite is an immersive theater piece presented in VRChat. Designed as a social space where users can don animated avatars and then physically speak and interact with each other, VRChat has recently been adapted by theatrical innovators to create interactive theater pieces, where audiences and live performers alike inhabit the same world. This not only allows for geographically dispersed people to come together into a shared online space, but for actors to speak directly with viewers to create unique experiences with each performance; with titles like The Under Presents: Tempest and Finding Pandora X, this is a surging collaborative space between theater and VR professionals that filmmakers would do well to be aware of.

In The Severance Theory, writer and director Lyndsie Scoggin and her team address childhood trauma, specfically dissociative identity disorder. One participant takes the role of Alex (a name that fits multiple genders; during the setup the viewer has a chance to identity their pronouns), while others observe unseen. After the initial introduction Alex enters the narrative world by thinking back to their childhood, circa 1993, and an evening when they had just returned home to their parents after staying with their aunt for some time. Life at home varies wildly between loving and fraught, with both parents exerting obvious effort to make Alex feel welcome and cover over the cracks in their marriage: the mother involves Alex in making macaroni and cheese, and praises their drawing—in my performance the actress did a commendable job incorporating a picture of a unicorn I drew—while the father takes them up to the attic to experience a star projector and recount how he named a star after Alex the night they were born. But despite these efforts something more sinister than their marital struggles lurks in the shadows, literally, and revolves around an unspoken trauma that Alex experienced with their aunt. When the mother plays a music box that was a gift from the aunt, a dark mist oozes down the staircase and toward the viewers; in two subsequent episodes this cloud becomes more brazen, attacking Alex’s self-worth through a disembodied voice and finally attempting, apparently, to become physically violent. These moments allow the rest of the audience to participate, as they come to Alex’s aid as unseen voices, chanting that they must protect Alex and fight off the shadow. The visual metaphor is poignantly done, although it’s broad enough that it could serve as a symbol of many types of mental illness and distress. After the piece ends—happily but ambiguously—a final salon includes information about dissociative identity disorder specifically for those who are interested.

There was enough locomotion between different rooms of Alex’s house that by the end I was developing a good case of motion sickness, a common ailment in VR, but there was much more to praise about the piece, and I hope it has a post-Tribeca release as well. Scoggin uses her background in escape rooms to create a kind of metaphorical escape room here, with the feeling of claustrophobia and entrapment permeating even the piece’s gentler moments, and some simple puzzle-like work to push the narrative forward. I saw the piece near the beginning of its Tribeca run, and even then the actors were tremendous in their adaptability and improvisation to my own actions, their clarity in helping shepherd a non-professional actor/VR performer through the process with them (including in the group onboarding process), and their vocal and physical performances—acting is no easy task when one’s face is no longer visible. While in the role of Alex it was hard for me personally to not think like a screenwriter or actor performing for the rest of the audience, but with interactive VR “theater” set to grow exponentially it’s something I think audiences will grow increasingly accustomed to in the next few years (it could be argued that we’re seeing the emergence of an entirely new art form, not film, theater, or video game but something that uses parts of each). And hopefully in a few years improvements in photogrammetry, volumetric capture, and similar technology will allow us to see actors’ actual bodies rather than animated avatars. That’s the future of VR theater that I’m most looking forward to, and in the meantime pieces like this are developing the narratives that will get us there.

We Are At Home comes from the married team of Michelle and Uri Kranot, experimental animators and founders of the Danish animation studio TinDrum. An international co-production with Floréal Films in France, Late Love Production in Denmark, and the National Film Board of Canada, it’s a VR adaptation of their 2D film The Hangman at Home, which in turn is based on Carl Sandburg’s oft-adapted 1922 poem. Sandburg muses about the home life of a man whose occupation is to kill, and whether it’s a typically domestic scene, the horrors of his day buried or even blithely laughed off. The Kranots focus on a few key phrases—”play horse,” “bonfire”—and spin a series of tableaus, which, remarkably for a film based on a poem, are poignantly wordless. Presented in miniature within a proscenium arch, these moments are drawn mostly in inky, flowing black and white, and the hangman and other characters have a listless, uncomfortable feel to them, like a hybrid of Robert Crumb and Munch’s The Scream—especially when they break the fourth wall and stare directly at the viewer. Topics go beyond the poem’s literal content to evoke its tone of unease, with the hangman performing tasks like mourning a woman dying in bed, masturbating under his own covers, or arriving home to a bombed-out apartment with a view of a warzone through the missing wall.

In between these moments of miniature theater are full-scale 6DoF interactive scenes, where the viewer has to open and pass through a door (drawn in a chalky outline), or duck down to climb through a cabinet. At Storyscapes this meant that the four participants circled around a large open space, as they moved clockwise from one scene to another (and requiring a fair amount of wrangling by the staff to avoid collisions). Knowing this made these moments of walking forward somewhat uncanny, like a Kierkegaardian step into the dark, as I’ve rarely had license to move quite so far in a VR headset before. But it was also quite clever and entertaining as the four of us somewhat simultaneously figured out the tiny puzzle to move to the next scene.

The final scene breaks this pattern, as the proscenium is gone and the viewer is now within a full-scale drama in which a pregnant woman contemplates burning a stack of books and committing suicide by jumping out the window. This scene ends on a final moment of interactivity as the viewer now holds the matches and can decide whether to light up the books or not. The ending is enigmatic—it could literally go either way—but still encapsulates the tone of the entire piece, that just beyond these tranquil scenes everything is about to catch on fire. The Kranots, in fact, seem to have abandoned Sandburg’s last line, “Anything is easy for a hangman, I guess,” in favor of its exact opposite: everything seems fraught for this character, on the brink of annihilation.