In looking for sustainable ways to tell stories in 360° video – examining what is cheap, easy and repeatable within a self-sustaining business model – we have focused at immersiv.ly on experimenting with one-to-one formats. Where the viewer is addressed directly by the subject, whether it be a poet, an entrepreneur, a cook, a singer, an artist or a historian. This kind of near-field approach – with the subject between 1m and 1.50m from the camera – has worked well for some of the leading pioneers in virtual reality (VR). With the advent of consumer cameras and free VR plugins to industry-standard editing software, it is an ideal way for content-makers to experiment in what is still a nascent format.
And the question of sustainability is important, as the industry needs to make it easy and affordable for makers from other media to transition into immersive storytelling. We advise first timers in 360 to start small, to experiment, and to keep an open mind: to resist limiting thinking on what might make a good subject in VR or 360° video.
The power of near field in 360° video
Since 2014, there has been much discussion about what content, what subjects, whether in news or in other areas, work well in virtual reality. One of the most useful answers is “content that takes the viewer to somewhere they could not otherwise go”. To a truly virtual world, created digitally (whether in video or computer generated narrative).
But it is important to keep an open mind when defining that “somewhere”. In my experience it is either spectacular, and panoramic, or intimate, and near field. And that near field can be enormously powerful.
A lot of the early-stage news documentaries in 360 video, from 2014-15, covered news stories of global importance or the work of overseas charities and non-profit organisations.
- The Polar Sea by arte.tv. Explores the drastically changing Arctic landscape as seen through the eyes of the Inuit.
- Harvest of Change from the Des Moines Register. A multimedia narrative of life on an Iowa farm, opening with helicopter footage over the open plains.
- Clouds over Sidra, by Gabo Arora of the United Nations, and Chris Milk and his VRSE team (now rebranded Within). The story of a young girl and her family’s life in a Syrian refugee camp.
- The Million Man March, for police accountability in New York, by Spike Jonze and Chris Milk for Vice News.
- Hong Kong Unrest. Filmed and edited by Edward Miller for immersiv.ly in September 2014 (above).
The common factor among these pieces looks, at first glance, to be one of grand scale. The panoramic view of big events, big topics or big spaces – oceans, plazas, deserts, wheat prairies and the skyscraper streets of Hong Kong or New York. But looking back, taking these stories together, what stand out now are the moments of intimate scale, of incidental detail seen through daily life. Sometimes it is in plain sight – leading the narrative. Sometimes it is on the opposite side of the spherical video, away from the “big” action, waiting to be discovered.
- Harvest of Change is about the dynamics of a family farm. The daily personal challenges.
- The Million March is a case study in the close-quarters challenge of a big demonstration.
- The compelling discovery for us in editing Hong Kong Unrest was the incidental discovery of everyday life (shopping, catching buses) that continued even as police faced off against large demonstrations, yards away (on the opposite side of the 360 sphere).
- Clouds over Sidra has garnered headlines for the success that the UN has shown from fund-raising associated with showing it at special events or as part of fundraising on the streets of big cities. The Sidra in question is not a place, but a young girl. And the power of the piece is in being an engaged witness of her family’s life, acquiring incidental information from the viewer’s 360-degree encounter with their harsh existence in a refugee camp.
The power of Clouds over Sidra – Chris Milk memorably associated this piece with his vision of virtual reality as an “empathy machine” – is in part the experience of being placed at the heart of family life, translated to an unforgiving environment by world events. And in part it is down to Gabo Arora’s integrity as a journalist, winning the trust of his subjects in a refugee camp by spending time with them, when much refugee coverage was the product of brief press events or news conferences. He describes the process in a fascinating interview with Kent Bye in his Voices of VR podcast.
That intimate, near field, explored by Arora in Clouds over Sidra, has gone on to host some of the most powerful VR experiences in succeeding years, whether in 360° video or fully immersive computer-generated narratives. And these models are important now because the availability of consumer cameras and consumer binaural sound recorders means it is a format easily envisioned and captured by solo film-makers and guerrilla story-tellers. Those who need to travel light and film and edit against the clock.
What the standard-setting film-makers in 360 say
Some of the most beautiful 360° video work of the past two years has been done by the Felix and Paul studio, derived from their cutting-edge custom camera technology (where they have pushed the envelope on the effective use of stereoscopic video) and their artistic vision and finesse. They had some interesting things to say about near-field 360° video during a session at the SXSW conference in Austin, Texas, in 2017. And here I am indebted to the reporting on Twitter of my friend Robert Hernandez of the University of Southern California. Professor Hernandez tweeted notes from the Felix and Paul master class:
Near field is most important part of a #VRshot for presence. Composing depth is more important than 360. @felixandpaul
The key terms here are “near field”, “presence”, “composing depth” and “360”.
Presence is the most sought-after quality in any virtual reality experience. Giving the viewer the sense that they are “present” in another world.
And it is valuable that Felix and Paul, whose technical prowess allows them be “pixel perfect” at the grandest scale, should so value “near field” (within three to six feet of the camera) when they are composing their shots. They highlight an important principle for anyone looking to storyboard a 360° or VR piece: to envision first what is happening in the close field and then to document how that relates to what is happening in the outer fields of the 360° camera’s field of view. Whether it is lights, props or action. The film-maker is composing in spheres.
Felix and Paul also noted that “composing depth is more important than 360”. This “more important than 360” counters a particularly limiting belief that all 360° experiences “must” somehow be fully alive, fully populated with prompts, graphics and incident at all times in every nook and cranny. Or that they somehow lose validity.
It’s all a question of degree of what is and isn’t important: there is a clear focus (a main subject) that a 360° film maker can have in mind in any spherical shot. If the content of the remainder of the sphere adds context, information or visual richness, so much the better. There’s great value for viewer interest and viewer retention in the space.
But the received opinion (and it is out there) that the viewer must somehow be “compelled” to look around a 360° space for it to be valid – rather than merely having an interesting and fulfilling opportunity to do so – that they must physically turn their neck in a full circle, is a case of distorted priorities. That received opinion, that invented sense of compulsion, can lead to a point where an over-populated sphere becomes a distracting carnival rather than an immersive space. Where even a convinced consumer of 360° video cries “less is more”.
The simple message of big-budget pieces
Two noteworthy big-budget pieces of 2016-17 show the power of the performative gaze in 360° video, used in a straightforward way. Each has a simple editorial purpose. One that is clear to the viewer.
1) New York Times noir: The New York Times’s LA Noir season in 360-degree video, produced in late 2016, and featuring some of Hollywood’s biggest stars, was shot on a specially built set, where the exaggerated lighting design of the noir genre play particularly well in 360 shots and lead-ins. The stories are cleverly related in tone and concision. It’s a big budget piece, beautifully scripted and executed by serious talent.
But the most memorable thing is the straightforward approach to shooting, and (importantly) the simple approach to explaining the experience to the audience.
You the viewer (who are represented by the 360° rig) are told (but unobtrusively) that you are the barman listening to an agitated, fugitive played by Don Cheadle; or you are the estranged husband of a noirishly overwrought character played by Natalie Portman. You are assigned a role.
And in both cases the star performers look straight down the lens. They are addressing you. You are with them in the shot. It’s compelling and intriguing. And, fundamentally, a very simple storytelling approach. It works. You are prompted and you believe.
2) Hallelujah: The Lytro light-field camera is one of the technical wonders of the video world. Capturing multiple shots and composing them into a physical matrix where the viewer can get a sense of moving around within that volume. In their showpiece film of a concert, Hallelujah (made with Chris Milk’s VR studio, Within), the power is in your direct address by the solo singer, part of a church choir. And of the intimate sense of presence. The tech fidelity is awe-inspiring but the example of the power of one-to-one address is a simple, graspable one.
The editorial lessons to be learnt from the high-fidelity LA Noir or Hallelujah are just as valuable to someone wielding a mobile or lightweight consumer 360-degree camera and a lavaliere mic.
You the camera: The challenge of acting ‘around’ rather than ‘to’ the audience
At immersiv.ly our first experiments with having characters work “to”, or rather “around”, the camera involved a performance of Shakespeare’s Twelfth Night. We filmed the passage, which had been prepared by two of the actors – Richard Leeming (who plays Malvolio) and Alex Hope (who plays Andrew Aguecheek) – in Mount Pleasant Studio (green-screened for the occasion) in London in September 2014. The performance was directed by Edward Miller and the stitched video then green-screened into a spherical photo of a garden with a suitably Shakespearean backdrop. It is the famous gulling scene, where Malvolio is tricked by a forged letter into imagining that Olivia, the object of his hopeless admiration, in fact, loves him, and where those who mock his being “gulled” do so by talking “over” his soliloquy while concealed behind a nearby hedge.
We shot the performance in three repeat takes with the camera (representing you, the on-stage witness) in three different positions (to examine the varied effect the different positions had):
- Position 1: You are at the heart of the performance, with the actors all around you, but treating you the camera as a “ghost” (the actors are not working to the camera).
- Position 2: You are a third party, watching the scene at a close remove. You are again treated as a ghost.
- Position 3: You are a third party, but close to the protagonist, Malvolio. A close party to his unwitting embarrassment. There is a sense of his engaging with you, the camera.
Our first takeaway was that putting the camera at the very heart of the action (Position 1) created a physical challenge to the viewer, who sometimes has to spin left and right to pick up on the audio cues around them (we placed the actors’ individual audio tracks in the sphere, using Unity game engine). Position 2, with you the third party at a remove, felt like a natural viewing position, but somewhat distant from the heart of the matter. Position 3 (where the viewer is close to Malvolio) gives the greatest sense of “presence” with his plight.
You can watch the published scene, which takes elements from each of the three camera positions, on Shakespeare 360, an immersiv.ly app in the App Store.
Our next discovery of how powerfully a subject can work with a 360-degree camera came with a later project: the filming of our first travel documentary series, at the home of cinema.
The journalistic power of the auto-portrait
In 2015, immersiv.ly was lucky enough to collaborate with Jean Yves Chainon and Carole Chainon of JYC VR in co-producing our first series of travel documentaries, set in Los Angeles. At first, we played with working with an interviewer or with two “characters” talking to each other. But given the contingency of filming on the LA streets we found ourselves increasingly asking our subjects – art gallery directors, fitness models, social entrepreneurs, restaurant managers, beachside flaneurs – to tell their story straight to camera. Straight to the 360 rig. And the results we discovered in post-production process (in the days before live previews on 360 rigs) were compelling. We found that we had captured a powerful patchwork of personal testimonies, of auto-portraits, where the subject takes the viewer into their confidence. That is the power of looking straight into a 360° rig or camera – where the subject is alone with camera, with no crew close by. We had a 360° version of the city patchwork narrative approach so beautifully essayed by Brandon Stanton at Humans of New York.
This discovery opened our eyes to other pieces of 360 journalism where the reporter allows or encourages, the subject to tell their story direct to the 360 camera.
360 Case Study 1: Disfellowshipped
One of the most interesting of these is Disfellowshipped by Trey Bundy for the Californian publisher Reveal, in collaboration with the German Web VR start-up VRagments.
Bundy builds the interview around audio interviews with his main subject, a survivor of abuse in a group of Jehovah’s Witnesses in the United States. This audio is the backing to everyday 360 images of the subjects hometown, and a mix of newspaper cuttings and case records from over the years. And the really powerful moment comes when audio and 360° video become one, and the subject tells her story, straight to 360° camera, in her living room. And the special power of the scene is the incidental information we gather by witnessing the testimony told by the speaker in the home where she has somehow rebuilt her life. The small details of domesticity and order really tell.
This is how Trey Bundy describes the shot, and his whole first experience with 360.
So, what’s the point of telling this story in VR? The answer is to give the viewer a more intimate understanding of a character and her experience. The technology allows us to put you in the reporter’s shoes, to feel what it’s like to sit with people as they look you in the eye and tell you their story, to visit their towns and the places that affected their lives. In some instances, it becomes a window into a person’s emotional memory.
“Look you in the eye” says it all.
In flat video you might ask the subject to look past the camera, to address the interviewer or a point of focus past the camera. In 360, you ask the subject to look down the lens, and in doing so you find the power of the format.
360 Case Study 2: The Euronews experience
Euronews, based in Lyon, has led a ground-breaking programme (organised by Thomas Seymat) of reporting with 360° video, training their video journalists to add the format to their skill set. And their experience of reporting has shown the power of 360 in capturing incidental location information in a non-editorialising manner. (Declaration of interest: I have done a consulting report for Euronews on their first set of news videos. It has been published on LinkedIn.)
After more than a year of the programme, the leader of Euronews’s digital team Duncan Hooper posted a brief cri de coeur on LinkedIn that summarises the power of 360 as a journalistic tool, and that counters the received thinking (the one that I mentioned above) that demands that all 360 settings should be spectacular, exotic or unworldly. Hooper writes:
There’s a school of thought that says 360° video only works if you are either showing fantastic images, or you are going places your audience couldn’t otherwise. This line of thinking says a 360° interview will almost never work unless you do it on top of a mountain or in the middle of a tomato-throwing festival. But sometimes the context of an interview does matter. Sometimes putting the audience in the same place as the interviewee can help them understand what that person is thinking.
I would argue that interviewing someone inside their home, for instance, can be compelling in 360. Because as a witness to this interview I am interested in the casual details around someone’s house that tells me about them. From the books on the bookshelf to the Ikea lamp in the corner or the snow boots left lying about in the middle of the room. You cannot really show this in a flat video interview without endowing them with unwarranted significance.
Hooper is right.
And then there is Trey Bundy’s succinct comment on what made his first 360 journalism experience memorable: “to feel what it’s like to sit with people as they look you in the eye and tell you their story”.
That summarises why we at immersiv.ly have been working on one-to-one pieces with a variety of artists, creators and thinkers. To experiment with how people look you in the eye and tell you their story.
This post is adapted from a guest blog written by Louis Jebb for the Raindance Film Festival, London, in June 2017.
