TransMissions: The Journal of Film and Media Studies 2018, vol.3, no. 1, pp. 59-77.
Maciej Stasiowski
Jagiellonian University
Boundaries are (but) a blur: Computer-generated imagery and the formation of seamless filmic space
Ever since cinema’s inception, the physical creation of fictional settings has been the province of architects, set designers, cinematographers, and – nowadays – of artists in art departments, ardent in bringing to life often impossible spaces. As far as optical illusions go, in order to exist they need not a restless eye, but a static one. But what if the space being rendered is itself fluid, dynamic by default? Braiding computer-generated imagery into live-action film footage has become progressively more robust in revealing its non-material base through texture, light reflexivity, and the way these additions interact with the pro-filmic space. Nevertheless, they too are just as reliant on a suspension of disbelief in their striving for a photographic verisimilitude. Preceded by scale models, montage juxtapositions and painterly optical illusions, digital scenography has become the next logical step in enhancing filmed footage; boosting an impression of reality, going so far as to sacrifice (or ‘dematerialize’) the physical in favour of the hyperreal.
With the help of software enabling motion tracking (to merge 3D visuals into filmed scenes), picture correction, and digital composition in the post-production stages, as well as completely digital animated previsualizations, filmmakers are now able to come up with radically new spatial environments. In this way, the innovative concept of cinematic screen space that blurs or even nullifies material borders is introduced. Seamless transitions link contradictory settings into homogenous environments, whereas uninterrupted long takes can now arise through digital ‘stitching’ aimed at achieving near-to-experiential involvement. Through this, contemporary spectacles postulate a new kind of viewer – one who absorbs visual and acoustic effects viscerally, and allows himself to become engulfed by and pulled into the filmic space.
Along with films that not only employ computer-generated imagery (CGI), but are realized with techniques characteristic of animated film (at nearly every stage of their production), a general approach in treating the representational emerges. The digital space of representation outruns traditional matte paintings with its moving, dynamic descendants, if not entire 3D scenes/environments reconstructed digitally, wrapped in photographic textures. This article sets out to investigate the poly-sensory quality of represented spaces. In their surrogate of an out-of-body experience through cinematographic strategies, editing becomes quite reluctant to tie down visual spectacle to a specific point-of-view or point-of-audition narrative. Films embroidered with CGI put forward a new mode of ‘navigating’ filmic space. They reposition their audiences in a represented space, making them willingly succumb to a multisensorial ‘flow’ of diegetic events. Eventually, even Daniel Dayan’s notion of suture, explaining the process through which the viewer is positioned in filmic locations, becomes replaced by a sensation of fluid environments, intangible settings, and floating worlds inextricable from our perceptual cues, as reproduced by digital cinematography.
Bringing Maurits Cornelis Escher back to life through the power of CGI and a lack of humble decency, if that was ever an option, might have brought about the digital revolution in cinematography we are witnessing right now. But nothing of note happened in this regard, and while Escher’s grave in Baarn is rarely frequented by production designers or digital matte artists, let alone Hollywood executives, the branch of visual special effects in contemporary productions develops precisely along the lines of his architecturally-accurate optical illusions in their progressive conquest of photographic mimesis. Drawings resembling photographs, 3D models indistinguishable from material objects, abstract graphics thoroughly intercepting indexical veracity, and – apparently – promoted animation as a main mode of filmic expression, and all in the guise of photographic textures wrapped around objects, places, and actors alike. This is the cinema of the future, and the future is now.
What this article postulates is an emerging concept of representational space in films which not only employ computer-generated imagery (CGI), but are realized with techniques characteristic of animated film. These, in turn, steer nearly every stage of their production. As digital visual special effects (DVFX), with time, have come to emancipate themselves as a category, the industry standard nowadays sees feature films often driven by the use of computer graphics integrated with the live-action footage, or substituting for portions of it. Quite often what we see on screen are not just lifelike, moving matte paintings created in Maya or 3D Studio Max, but entire 3D scenes and environments reconstructed digitally, though covered with photographic textures. “Over the past twenty years, the Visual Effects (VFX) and Art Departments have worked more and more closely, bridging the gap between real life and digital environments. Sets are now often built to incorporate green and blue screens so that they can be seamlessly extended in post-production.”
[1] With the dematerialization of generic film sets comes the introduction of a virtual camera whose weightless, continuous gliding over modelled landscapes presents the viewer with a novel way of taking in visual information – immersing them into the action and scene of events. Instead of resorting to the shot-reverse shot mechanism of narrative subjectivity – regarded by Daniel Dayan as the base principle of cinema, according to which the viewer projects him/herself into filmic space[2] – we are instead drawn into represented space in a cinematic version of an out-of-body experience, in which editing is quite reluctant to constantly tie the visual spectacle down to a specific diegetic perceiver. Rather, repositioning the audience in represented space forces them to succumb to a multisensorial investment in the ‘flow’ of events on screen. As all transitions are rendered seamless and digital stitches concealed, films embroidered with CGI put forward a new mode of navigating the filmic space; Dayan’s notion of suture, standing in as a means of situating ourselves in imaginary yet veritable locations, is replaced by a sensation of fluid environments, intangible settings, and floating worlds inextricable from our perceptual cues, as reproduced by digital cinematography.
Illusionism applied
Take any one of Escher’s lithographs and you’ll immediately see that creating optical illusions involves a spatial imagination and a knowledge of visual cues based on human perceptual habits, even more so of their shortcomings. Ascending and Descending (1960), for example, lures us into believing that the circular staircase is a buildable three-dimensional object, even though it remains an impossible figure, in the realms of the Möbius strip. On the slightest disruption of that accurately constructed three-point perspective – reprised under the guise of a magician’s sleight-of-hand in the Penrose stairs scene[3] from Inception (2010, Christopher Nolan) – the last step, that seemed to be conjoined with the first, suddenly drifts apart with a single swift movement of the camera crane, thus revealing a gaping fissure, previously non-existent due to the advantageous position of the assumed vantage point. Observations can be made as follows: one – the space is non-existent, because an infinite staircase is an impossible object, and two – the picture’s representational space is possible as long as we maintain the vantage point posited by Escher or Wally Pfister, Nolan’s cinematographer. The way we look at space becomes space itself – three.
This tension, which arises between the scene and the observer, or speaking more scientifically, that turns space into a (mathematical) function of the gaze, has been one of cinema’s prime features from the very beginning. Set designers, with cinematographers, are often ardent students of classical art. This pertains to ‘proper’ construction of perspectives for matte shots, just as much as it later translates into the relationship established between the represented space and the viewer. In classic cinema, as Daniel Dayan noted, “[c]amera lenses organize their visual field according to the laws of perspective, which thereby operate to render it as the perception of a subject.”[4] In order to deepen that impression of subjectivity, a variety of visual cues are being employed. Among them are “…forced perspectives [which] created the illusion of great depth. In resorting to this technique, modern art directors joined company with writers on perspective from the Renaissance and Baroque eras, whose schemes were routinely taught in American art schools.”[5] Building an optical illusion first of all requires another illusion to back it up – namely, an optically-biased environment, disguised as a space extracted directly from real life that would give no reason to the onlooker to question what they see. Since cinema’s inception, architects, set designers, cinematographers, and now art department virtuosos have become skilled in creating fictional scenes that would be impossible to encounter in real life, because in order to exist, they involve not a restless, but a static eye.
Beyond expressionist ‘Caligari’s cabinets’, filmic scenery, encompassing physical locations, set decorations and painted backdrops, has undergone a long journey to the point of redefining the entire approach to film design, brought about with the CGI revolution. Not yet at the stage when the first computer artwork was being introduced into practice (namely, John Whitney’s opening sequence to Hitchcock’s Vertigo (1958)), what was considered ‘computer art’ was still, for example, largely the analogue re-filming of console displays in Tron (1982, Steven Lisberger). This eventually led to over-stated claims of the arrival of the digital age, with a mere 6 minutes’ worth of CGI in Jurassic Park (1993, Steven Spielberg). In fact, it was Toy Story (1995, John Lasseter) that arrived on the scene as the true binary Prometheus, disrupting the balance of computer-generated special effects shots versus regular footage, expanding the category of digital FX into full-length 3D animated features. In this way:
[a]gainst the backdrop of the wider proliferation of digital technologies, media and communication networks, digital or digitised practices have found their way into almost every aspect of filmmaking, including sequence pre-visualisation, blue and green screen shooting, face and body motion capture, compositing of image elements and digital rotoscoping, non-linear editing and sound mixing.[6]
Computer-generated imagery braided into live footage is reliant on a suspension of disbelief, according to which any film is assumed to be lifelike as long as it presents us with objects whose photographic verisimilitude – its indexical value – seem undeniable. As parametricism in architecture evolves into yet more advanced and elaborate forms, and with CGI as a standard in movie production, what was formerly regarded as merely a new tool to aid the design process has not only reconfigured both practices, but also introduced an innovative concept to the cinematic screen space, which blurs or nullifies material borders. With the help of software enabling motion tracking – facilitating the merging of filmed scenes and 3D visuals – picture correction, and overall digital compositing at the post-production stage, as well as completely digital animated previsualizations (being ‘storyboards 2.0’), filmmakers have been able to come up with radically new spatial environments (albeit mainly in the science-fiction and superhero genres). And while the surface might be misleading, resembling classical cinema, the ‘engine’ of present-day productions is purely digital, driving all production phases in contemporary filmmaking.
What prevails, then, is a desire for seamless visuals that reinforce the illusion that we are indeed looking at cities full of Marvel’s superheroes, and that flying beyond the galaxy’s farthest edges appears as if they’ve really been put in front of an actual camera. But both presumptions are wrong. Digital scenography – preceded by scale models, montage juxtapositions and painterly optical illusions – has come to supersede or enhance the practice of filmmaking and the usual modus operandi, subsequently dematerialising the physical borders of pro-filmic reality, blending interior and exterior barriers, and transforming the spaces depicted and the camera’s gaze into a seamless whole. Contemporary cinema has emancipated itself from the rules of classic montage. We are already being flooded with over-invested blockbusters that tend to rely very little on the viewer’s critical awareness. Instead, immersive narrative strategies (Gravity (2013, Alfonso Cuarón), Birdman or (The Unexpected Virtue of Ignorance) (2014, Alejandro G. Iñárritu)) are preferred. These eliminate the cognitive stage of mental ‘stitching’ – along the lines of Dayan’s suture theory – of the filmic narrative, requiring from the viewer complete involvement and naïve insertion into the events depicted, rather than any intellectual distance:
Narrative cinema presents itself as a ‘subjective’ cinema. […] These films propose images which are subtly designated and intuitively perceived as corresponding to the point of view of one character or another. The point of view varies. There are also moments when the image does not represent anyone’s point of view; but in the classical narrative cinema, these are relatively exceptional. Soon enough, the image is reasserted as somebody’s point of view.[7]
Of course, filmmakers have always excelled in hiding the ‘stitches’, just like the brush strokes that might have made one suspect the painterly nature of an end credits’ sunset, or the true storage capacity of the Hangar 51 warehouse in Raiders of the Lost Ark (1981, Steven Spielberg). With digital image processing, these directorial dreams have come true, although not without affecting our relationship with on-screen spaces – formerly a montage of fragments,[8] now a seamless environment in which the architecture of the material, illusory and computer generated all converge.
Pro-filmic space in pre-digital cinema
The emerging filmic space’s uninterrupted nature often (unsurprisingly) finds its central inhabitant in a character who exists on the borders between fantasy/delusion/mental illness and concrete reality. The procession of visual information – in such features as the aforementioned Birdman, Gravity, or The Revenant (2015, Alejandro G. Iñárritu) – postulates a manic, restless and slightly neurotic subject, whose incessant daydream we are drawn into, all the more to experience it first-hand. We are used to gazing at cinematic space as an imagined, otherworldly reality on a screen in whose wilderness the characters are meant to wander, struggle, or simply interact. Inside a bluescreen environment this task becomes much harder to accomplish, as far more unknowns about the represented space are introduced into the equation.
It is no longer a case of catching hold of and restoring a slice of pro-filmic reality, but rather of encoding the ‘data’ seized by the device. With the digital, to record reality is already, and simultaneously, to reconstruct it. We know of course that any representation, however slavishly recorded it may be, is always-already a (re)construction.[9]
It is a spatiotemporal collage conceived by editing, within-the-frame montage, compiled from a variety of sources, chiefly pro-filmic space, stage sets, scale models and matte paintings. Their juxtaposition creates the setting for the plot. Graphic artists, set designers and cinematographers have always been preoccupied with hiding from the viewer’s gaze any inconsistencies in the composited image, a practice originated with artisan-come-artists such as Robert…
…Mallet Stevens [who] had discovered the empirical approach of the professional designers. He began to study camera angles, which varied according to the focus of the lens employed. Intrigued by these studies, [Jean] Perrier took them up as well and developed a rational concept of film set design as a function of the position of the camera and the lenses. The graphic method that he worked out enabled him to determine which plan and dimensions of a set would produce the image desired and drawn by the designer.[10]
Such views can only propagate themselves. As Michael Tawa writes, “[t]he cinematic image is […] a manner of penetrating space. It constitutes the way in which a look perforates and advances into space.”[11] Therefore, represented space arrives on the ‘silver screen’ as an entity that has already been manipulated, enhanced and infused with digital hyperrealism, whose: “…architecture changes the sizes and proportions of real architecture. Even though buildings constructed in the studio were usually made smaller than life-size, their physical diminution was not noticeable when they were filmed with actors.”[12]
Fiction film beguiles us into accepting spaceship interiors and alien temples as instances of ‘probable’ architectural typologies. Documentaries make us alert, as they strive for verity, even though throughout history truth-seeking has been achieved through quite diverse means. Animated and experimental films are unique in this manner, as they present us with spaces that, even when originating in real life, have been intercepted in order to test the borders of our cognition; the limits of our perceptual capabilities, as in structuralist film. But apart from generic convention, CGI facilitates the coming (or designing, rather) into graphic existence of any environments of pure abstraction, the digital kin of Douglas Trumbull’s stargate-corridor in 2001 (1968, Stanley Kubrick) – a perfect example of a purely ‘retinal’ space that is brought to life on an inherently Dayanian basis of shot interchange (the fluorescent stream reflected upon Bowman’s face and helmet upon which we see the corridor as a projection).
Ambiguous space: a shortcut from animation to live action cinema
With the introduction of computers to filmmaking, the spectrum of tools allowing for processing of imagery grew considerably, facilitating chirurgical incisions, letting cuts proliferate in a more in-depth manner, while the stitched-together patient would emerge with no visible scars. Animated films, especially experimental shorts (such as the first computer films by John Whitney Sr., beginning with Catalog (1961), which introduced the idea of morphing and sequences of transformative algorithms applied to on-screen objects), have evolved into the backbone of any big-budget action film circa 2018. Thereby, special effects entered mainstream live-action cinema and thoroughly reshaped the production pipeline, emerging soon after as their own separate category. Conversely, space in animation has always been an artificial construct, along with the characters themselves (bodies, contour lines etc), the convention of background images and their own laws of physics, which come into being only when acted out.
Along with digital special effects, new stages of film production quickly caught on, such as previsualisations of more complicated sequences (fight scenes, explosions, stunts etc), and animatics – an animated version of the storyboard. As a consequence, although still regarded as a waste product, a test ‘movie’ comes into being alongside the main feature. That was the case for Gravity, which was created not unlike a typical Pixar production. The final cut of the film was decided upon in the pre-production stage. While shooting (mainly in a bluescreen environment), a ‘virtual camera’ was programmed to perform smoother movements than a physical one could. Post-production is nowadays the lengthiest gestation period in a film’s production process, and involves colour correction, lighting adjustments and the addition of special effects to accompany traditional animatronics and digital compositing. In this way, a typical film begins its life as an animated storybook, with concept art and digital previsualizations, etc, and ends up as an animated film ‘in disguise’, harnessing live action with the dexterity of a professional puppeteer. By flowing into mainstream (mainly ‘action’) cinema, it doesn’t just introduce the issue, but reinforces the problem of imagery’s ambiguous status.
Michele Pierson rehearsed this possibility in 2002, speculating that digital special effects would effectively begin to ‘disappear’ as a visual category, as CGI became a more persistent and wide-ranging presence onscreen, and as the impulse towards photorealism in digital imaging eradicated the ‘bracketing of’ and stylistic foregrounding of special effects that Pierson had identified in earlier phases of the digital effects tradition.[13]
Two examples of animated films are analysed below, examining their visual strategies (which made their way, further on, into CGI-imbued live action cinema) – extracted from two anime classics, covering distinct sequences that are explicitly pure visual transitions conjoining separate settings. One created on the brink of the digital era, the other in its midst: Mamoru Oshii’s Ghost in the Shell (1995), and Satoshi Kon’s Paprika (2006). The latter is a film with computer-generated dream-reality transitions. The former features a main character in a sea-diving sequence, in which the director/animator plays around with the illusion of water reflections. As depicted, they are indistinguishable from the character and the environmental design. Meanwhile, Paprika constantly transitions its narrative between reality and dream, unnoticeably in the course of the plot, whereas the horror of waking up is ‘smoothed out’ with the use of digital special effects: blurring, twisting, morphing of the hand-drawn imagery, and all as abruptly as the morning bugle.
Ghost in the Shell: Re-surfacing
Halfway into the story, we find Ghost in the Shell’s central character, Major Kusanagi, taking some time off in a slow-paced ocean diving sequence. As she floats towards the water’s surface, we see her perfectly reflected in the upper right corner of the screen. Cut to a frame divided diagonally by the water line – the character in the lower left half of the screen rises floats gently upwards towards her double in the upper right. Despite our knowing the scene is set underwater, there seem to be no other indications, such as a watery blue hue, wavy shapes in the drawing, or a lack of focus. Which one of the two characters is Kusanagi, and which her reflection? Of course, both are images, as there was no real actor there to begin with. Mamoru Oshii frequently plays with pictorial conventions, creating equivocal 2D settings, depicting them at a fixed angle to reinforce an optical illusion that would have been shattered if presented stereoscopically. The water’s undisturbed surface, as painted, appears indistinguishable from a mirror, or polished chrome. Thus Oshii strains the limits of representation, demonstrating how images can imply, instead of merely depicting. Apparently, in their slavish attitude to the animated forefather, the creators of the live-action remake of Ghost in the Shell (2017, Rupert Sanders), tried to achieve a similar effect using not merely a CGI’d reflection, but a genuine double for the actress[14] descending from above – an image perfectly sharp and easily mistakable for the actress ascending from below.
Paprika: jumping fences
In Paprika, the ambiguity at play concerns the gradual intrusion of the dream world into concrete reality. Director Satoshi Kon’s team uses digital effects, such as morphing, to mark the transition from a dreaming life to a waking one. At some point in the story, Chiba, the main female character, is scrutinizing the apartment of her colleague (a former programmer). Descending to the basement, she suddenly realizes she’s wandered into a huge amusement park. She notices a doll bearing a significant resemblance to her colleague and approaches it, jumping over a fence which suddenly dissolves like a reflection in water. The barrier vanishes and Chiba finds herself in mid-air, having just jumped over the railing on the apartment’s balcony several floors above the ground. Digital embroidery makes the drawing undulate, morphing the safe space of the apartment into a vertiginous drop. Further in, Kon nullifies differences between the images shown, as the protagonist’s alter ego, Paprika, is able to use the spaces of television screens, billboards and picture frames as gateways to the ‘realities’ they depict.
By fusing characters and backgrounds, or simply treating them as items of equally artificial and abstract origin, animated film doesn’t withhold from following gradual and abrupt metamorphoses within the course (and space) of a long take. “One-shot animated films are formulated from the potential changes of the scenery, running without cutting interruptions. To overcome this narrative restriction, the singularity of the shot is mitigated by the fluency of transitions and transformations at a blank stage, and the division of the frame to support simultaneous storytelling.”[15] In both Ghost in the Shell and Paprika, the environment’s status is ambivalent, far from acting out its solid and static nature. In animation, a particular artist’s style often serves as a masking tool, preying on our habit of separating moving characters from static backgrounds, as if they were actors in actual spaces. Instead, Kon and Oshii prompt the viewer to discredit this ‘reality principle’, regarding it as nothing more than a construct, regardless of how convincing it looks and how engaging the plot. “What is notable is the extent to which the photorealist principle is adhered to even in the depiction of the most fantastical subject matter, and even within sequences which function as explicit ‘showcases’ for – and thus explicit acknowledgments of – computer-generated imagery (CGI).”[16]
In film, meaning is generally derived from the collision of two images, making montage the main rule of composition for ‘moving pictures’. In the Hollywood system this serves the principle of editing for continuity, in which “…actors’ movements are matched across cuts, and as the scene develops the shots get closer to the performers, carrying us to the heart of the drama […]”[17]. Furthermore, “…nearly all scenes in nearly all contemporary mass-market movies (and in most ‘independent’ films) are staged, shot, and cut according to principles which crystallized in the 1910s and 1920s.”[18] Post-classical style strives, as Bordwell remarks, for a sensation of intensified continuity, a “…traditional continuity amped up, raised to a higher pitch of emphasis.”[19] Cutting heightens awareness, as it requires the viewer to mentally reconstruct relationships between perceived images, especially as images tend to weaken in resemblance.[20] Thus, fragments of architectural spaces and shots of a scale model can consequently be turned into a virtual building in the audience’s imagination. Daniel Dayan summarised these rules in his ‘suture theory’ of the mental stitching together of visual information (frames, shots) into a scene, sequence, virtual environment, event… depending on the discursive approach we choose.
The evasion this account identifies is deep and pervasive: the reverse shot of the gazer […] sutures over that profound wound in our being […][;] suture, in other words, provides film spectators with the illusion of an origin for what they see. Film’s construction of seeing needs to be naturalized. More importantly, the construction of seeing needs to be naturalized.[21]
Another level – or ‘sequel’ to Dayan’s concept – comes with CGI. Seamless transitions linking contradicting environments and creating long takes with the aid of digital imagery aim at something different – involvement. “Digital imaging’s original incarnation was as a ‘special effect’, the ‘digital effects’ it first showcased in specific shots and later specific sequences of particular films […] At another level this was a way for early digital effects movies to trial visual effects artists’ capacity to integrate the digital with the pro-filmic in a convincing way, and to test out the spectator’s tolerance of the digital elements and the composited image within safe limits….”[22] We are easily fooled by impossible, illogical spatial constructions when they emerge as walked-through corridors traversed uninterruptedly, staircases and rooms that lose us in their maze, either of Escher’s or Industrial Light & Magic’s provenance. Having this principle of continuity when discussing the CGI-cast space of representation in mind, we can inspect the (in)famous incessant ‘take’ of (rather than in) Birdman as representative of this trend in transiting long takes from modernist cinema into action films, from Nostalgia (1983, Andrei Tarkovsky) to Gravity.
Birdman or (The Expected Case to Study)
It’s not an exaggeration to say that Alejandro González Iñárritu’s Birdman or (The Unexpected Virtue of Ignorance) is nothing but Hitchcock’s Rope (1948) cut together digitally. Prior to its triumphal procession at the 87th Academy Awards ceremony, the film’s antecedents included Alexander Sokurov’s Russian Ark (2002), Gaspar Noé’s Enter the Void (2009), and Robert Altman’s multiple prism narratives, such as Shortcuts (1993). All of these films attempted a sense of simultaneity, continuity and recreation of a life’s stream of events, and tried to bypass the shattered, montage-driven form of the world traditionally put on screen. Apart from the obvious importance of staging – the actors’ blocking, camera placement, rehearsals and the other preparatory activities that bring cinema closer to theatre – computer postproduction played a crucial part in Birdman, precisely because it made the illusion possible. Typically,
[a] shot is a single uninterrupted camera take with no perceptually detectable temporal or spatial discontinuities. Cinematic sequences are composed of a range of shots that present different vantage points on an action, event, or state of affairs for the purpose of narrating a fiction, depicting an environment, communicating a point of view […]. Shots and sequences can therefore be defined as recognitional prompts that present diagnostic information that enables viewers to perceptually recognize their content in much the same way they recognize everyday objects, actions, and events in ordinary contexts.[23]
Birdman’s famously long take – as is more frequently the case with cinematic displays of digitally-enhanced prowess – was in fact a composite of various shorter takes edited together into a single seamless transition. This further enhances the nervous, syncopated rhythm of the film as the viewer follows its central character, Riggan Thomson (Michael Keaton), nervously pacing the narrow corridors of a Broadway theatre and dealing with the various people and obstacles along the way. Three of the post-production tricks that made this seamless-looking feat possible are examined below, which include use of computer software to create an uninterrupted narrative flow.
- Matchmoving…
…is a common means of conjoining digital imagery with filmed footage. In the film, during Riggan’s nervous pacing, the theatre’s dimly lit interiors were a perfect opportunity for making digital seams, placed so as to preserve the lighting and colour consistency of the images. Unlike the fades to black practiced by Hitchcock, the seams here are invisible, conjoined by the graphic artists at Rodeo FX. They employed a variety of techniques, including use of three time-lapse sequences and the aforementioned matchmoving – the matching of camera angles, motion, lighting etc. between two separate shots, in order to insert CGI material into the scene. Even a CG camera was used to seamlessly make a move that would tie all the unrelated elements together. All of this effort resulted in about 100 digital ‘stitches’ altogether, including transitions to fully digital backgrounds.
Current computer technology has made it easier to incorporate motion into composited shots, even when using handheld cameras. […] In post-production, a computer can use the references to compute the camera’s position and thus render an image that matches the perspective and movement of the foreground perfectly. Modern advances in software and computational power have eliminated the need for accurate placement of the markers – the software figures out their position in space. A perceived disadvantage of this is that it requires a large camera movement, possibly encouraging modern film techniques where the camera is always in motion….[24]
In Birdman, the idea was to give the impression of uninterruptedness by combining scenes that normally wouldn’t be subjected to much post-processing (it’s usually action sequences that involve bluescreen environments, supporting wires and stuntmen), such as dialogue sequences that might require colour correction, but do not involve heavy use of visual effects. In the Birdman sequence examined here, the character engages in frequent actor-to-actor interplays, only to be suddenly pulled up out of them by a crane, into an SFX display. And so, the illusion of continuity resultant from cutting together scenes shot inside a greenscreen environment, with scenes shot on location or on soundstages (such as the opening shot of Riggan levitating in his dressing room and the endless corridor walk he goes on just a few minutes later), create a consistent plane of magical-realism in the film’s diegetic space. And this magical-realist take has often been seen in the emphasis on continuity and immersion in modernist cinema’s spatiotemporal durée, in which subsequent actions are as much corporeally justified as they are metaphorically. It moulds together historically disparate periods (as seen in the works of Carlos Saura, Theo Angelopoulos, Miklós Jancsó), immersing audiences in the real-time duration of the scene (Béla Tarr, Michelangelo Antonioni, Andrei Tarkovsky), with the intention of attaining the stasis of a fleeting moment. However, CGI long-takes usually want none of that, let alone those in Birdman. Here, smooth transitions are set up between spaces and moments so as to point towards the distorted mindset of the protagonist.
- Photogrammetry…
…is the technique of extracting information from two images from different POVs, setting compatible points and creating a 2.5D representation of the image. It enables the determination of the position of a camera from two (or more) separately-taken shots, or photographs, and on the basis of data gathered on location, reconstruction of a 3D model of the scene. This pre-dates traditional matte painting, and gives an impression of three-dimensionality. It’s also indispensable in shots with mirrors. In Birdman, crew reflections were digitally erased. Parts of the set had to be either obscured with a bluescreen or taken out in post-production using rotoscoping, for example, in the dressing room scenes in which the large mirrors would normally reflect the film crew. Instead, the filmed reflections were replaced with CG reflections of the actors only, as well as of objects lying on a table visible in the shot.
- Digital compositing…
…is what allows for the illusion to play out loud. In Birdman, it ‘erupts’ sporadically, most prominently in a brief episode emphasizing an outburst of anger from Riggan, as – when walking down the street – he suddenly transforms into his audacious alter-ego, the titular ‘Birdman’ superhero character. Simultaneously, audiences are shell-shocked by the fantastical (though still convincing) images of an aerial attack on the city. Parked cars exploding, debris falling from destroyed buildings, wreckage and fire from every corner of the until-recently peaceful urban scenery. On the other hand, digital intrusions are applied to small details too. The stuntman dressed in Birdman’s costume wasn’t blessed with Michael Keaton’s chin – he received that in post-production.
This coherence in the design of each frame (as well as their flow), can also be observed on the ‘molecular’ level, as the software used privileges the manipulation of curved lines directly on screen; it favours continuous surfaces and smooth forms, let alone the fluidity of camerawork, complex shots and transitions. It can also effortlessly recreate nearly infinite zoom, and has no problem with a scarcity of interrupting cuts. This appears as “…a return to what we might describe […] as cinema’s graphic anima: Image manipulation, retouching, color timing, editing and post-production operations, all now digital, have encouraged a heightened “picturization” of films, for example, by broadening the color palette and the ways it can be manipulated.”[25] What had once been achieved with great difficulty, is now made to appear smooth. Like Birdman’s dynamics, with the film’s amplitude rhythmically changing each scene’s ‘time signature’ (an intensified continuity), speeding up then slowing down, but never grinding to a halt. These concealed incongruities mask the fact that the conjunction of heterogeneous spaces result in new viewing habits and different tasks for the viewer. And not passive reception of the information projected, but active negotiation of instances in a stream of attractions. Immersion here means surrender to the apparatus of cinematic projection.
The emergent option of eliminating all montage edits whatsoever is much less constrained than it used to be, for example in Alfred Hitchcock’s 1948 Rope. Moreover, the promise of seamlessness acquired by any formerly disjointed sequence of images grants the filmmaker the ability to not only sustain the illusion of a long take, but erase any barriers that would have normally been posed by material objects – be it props, set decorations or even other actors, as in the case of Gravity.
The aesthetics of the film try to replicate the protagonist’s fearful and fascinated exploration of a horizonless world through 3-D cinematography and very long takes, which together induce in the spectator an equally ambivalent sense of disorientation and weightlessness. The reduced narration and the poetic exploration of zero-gravity turns the film into a laboratory of the senses, which brings the spectator close to the bodily experience of floating, drifting, and being suspended in space.[26]
Even actors’ bodies no longer pose any obstacles to the camera’s penetrative look, as they can be substituted by CG counterparts; the formerly impenetrable borders between interior and outside space (and in metaphoric terms, between dream and waking life, inner psyche and outer reality), can now be traversed without resistance. This relegates material objects on set to the status of artistic creations (in Birdman, the creators often replaced props such as the cosmetics on a dressing table, or a framed poster, with digital doubles), generated on an ‘animator’s desk’ ex nihilo.
Images of the real world can now blend with fiction’s images of possible worlds because they are constructed and perceived in the same way. And this point surely touches on the ethics of our faith in images, particularly with respect to the documentary contract and journalistic coverage of the world’s factual events.[27]
In animated films, the attributes of material objects and physical laws have to be implied, acted out; they need to give off an illusion of corporality through texture, or usually weight, through light play and the way characters interact with the object. Dematerializing them in live-action cinema, as with actors in a bluescreen environment (or the more frequent practice of bright green Christo-like wrapping of their body parts, indicating areas of later intervention for CG artists ), pares them down to the status borne by any other object. Threads are composited into a film’s fabric (virtual camera movements, CG puppets replacing actors), and regarded as a coherent whole. “With motion capture something like the opposite occurs: here, a photo-realist image achieves the flexibility of an animated image.”[28] Altogether, this has given rise to the category of animage, which stresses the actual ‘fabric’ of the filmic spectacle, constituted not out of a montage of shots representative of real-life environments, figures, and events, but – as is frequently the case – an animated narrative encrusted with photographic ‘skins’ only in the post-production stage. “This, then, is animage: an animated image that is already no longer an image (it is no longer an impression of the world precisely), something conveyed by the privative prefix ‘a’. But animage is also – and now more than ever – an image that moves to the beat of animation.”[29]
There is an ontological shift in the represented space we perceive, which – out of a continuous flux – forms the underlying principle of most digital interventions. “In digital cinema […] there is no such thing as a still image, no punctual moment. There is only a consistent process of becoming (and unbecoming), based on the binary sequencing of zeros and ones, which creates a constant relay of appearing and vanishing, of presence and absence.”[30] This becomes evident when we compare a simple travelling scene from Birdman with a similar idea executed nearly 40 years earlier, in Antonioni’s The Passenger (1975). In both, the beholding eye – the camera – appears as a disembodied entity, traversing walls and material obstacles; in the Antonioni film it passes through the bars on a window separating the hotel room in which David Locke dies, from the courtyard outside. In Iñárritu, the obstacle covers the entrance to Riggan’s dressing room, as the crane climbs up to his balcony following a lengthy time-lapse sequence. Collaging, or making a photomontage out of disparate spaces, gives rise to the illusion of the camera’s all-penetrating gaze – the disembodied floating of the spectator’s eye is given much more than a mere ‘backstage pass’ into Riggan’s floor show, turning it into an absolute beholder. Such swift hovering about a virtual set implies a bit more than a delusion of grandeur. In fact, it reintroduces filmic space as a 3D model, in which territory we are to manoeuvre, vastly removed from the notions of classical construction, the ‘tutor code’ of cinema that dresses up and stitches together the projected show from fragments, presented to the camera’s restrained immobility.
Conclusive remarks
The Eisensteinian concept of the dominant, indicating aspects of the film frame or scene, is brought to the fore as it denotes both aural and visual layers of the spectacle. In the age of CGI, the same factors can easily be emphasised through colour correction, or elaborate camera movements (amongst other methods), meant to channel and direct the audience’s attention towards the particular element of the representation considered by its creators as the most pregnant with significance. Eisenstein writes: “[o]rthodox montage is montage on the dominant, i.e. the combination of shots according to their dominating indications. Montage according to tempo. Montage according to the chief tendency within the frame. Montage according to the length (continuance) of the shots, and so on. This is montage according to the foreground.”[31] A logical extension of this comes with a composting strategy to guide the audience’s attention. In other words, providing them with visual cues. The Soviet film director regarded lighting effects, framing, camera movements, composition of the cadre, sound, texture and other aspects manipulated by the filmmaker as a means of evoking a certain engagement on the viewer’s part, focusing their attention on specific elements of the screen’s tapestry; elements that embody the general meaning of the scene. With contemporary productions, this strategy is repurposed by means of colour correction, digitally-added lens flare, vibrant luminescence, or manipulated brightness levels.
What is the consequence of this kind of multi-aspect use of digital processing, compositing of a homogenous environment in which the look, mediated by the camera, is invisibly paired with CG additions? As in a Eisenstein’s own Alexander Nevsky (1938), the space of representation becomes coupled with vision – an aspect that, when experienced in 3D, redesigns whole shots in a way that aims at the viewer’s cone of vision.
[I]n light of the revival of 3-D images, the screen is no longer only a visual container framing the image, but has expanded into the auditorium space, further blurring the boundaries between inside and outside, in-here and out-there. It now opens up a virtual space that extends in depth, alternately thrusting itself menacingly out towards the spectators and pulling them into an enveloping embrace.[32]
Objects are thrown at us, and we intuitively dodge them just before realising we didn’t have to. That ubiquitous strategy of creating an immersive spectacle will probably soon fall into decline, both as antecedents and nemeses of Birdman quickly grow in numbers. Awareness of these strategies involving a pliant ‘interface’ raises questions about what is real and what is simulated. The next logical step for any self-conscious film made in the digital age would be to engage a thematic exploration of interruptions, blemishes, and borderline cases, in which digital intrusions into (supposedly) material reality cause an involution of the latter.
[I]t is our contention that in the era of digital cinema, the body and the senses are if anything even more central for a theoretical understanding of the film experience, whether it is the feeling of bodily presence created through digital sound, the sensory overload and profusion of detail achieved by high-definition digital images when projected in an IMAX theatre, or the ‘freedom’ to have ‘movies to go’ on portable devices and to control their sequence and flow with our hands.[33]
In this regard, Ari Folman’s The Congress (2013) and Leos Carax’s Holy Motors (2012) both raise the subject of digital alteration of what we see on screen more directly, revealing the technique, demonstrating glitches, and philosophising about the future condition of filmmaking. Such films engage – even on their margins – a discussion of digital paraphernalia, pointing to the ‘engines at work’ underneath the representations we see on screen. An example of this might be temporal masking, which results from compression, making use of “[t]he human visual system [as it] takes a while to adapt to abrupt scene changes. During this period it is less sensitive to details, and images may be represented in a coarser way.”[34] Emphasizing, at the same time, the ‘lossy’ aspects in coding visual imagery, precisely by a display of digital artefacts, that “[a]t low qualities […] become very visible and take the shape of abrupt changes in luminance and color between neighboring blocks, due to the JPEG processing that is performed independently for each block. This is why compression artifacts are often called blocks, or blocking artifacts.”[35] But the digital paradigm shift is rarely taken into consideration when talking about contemporary film. Not just because of Hollywood’s timidity in discussing face transplants for their major productions, but also due to a reluctance, maybe even inability, to pay attention to the invisible world of code behind the glossy, lossless surface. Soon, cinema may well be without any material reality outside the machine, as long as it remains armed with vast libraries of data from the physical world; a hermetic hermitage of digitized props, ready to be used and reused in any future spectacle.
Finally, as an afterthought, let us revise Dayan’s view that stitching (suture) was an automatic activity on the part of the viewer, who was critically aware of – even if accustomed to – the constructed reality they perceive, although symbolically desirous of writing themselves into the filmic space as witnessed in cinemas (in CinemaScope, preferably!). “To see the film is not to perceive the frame, the camera angle and distance, etc. The space between planes or objects on the screen is perceived as real, hence the viewer may perceive himself (in relation to this space) as fluidity, expansion, elasticity.”[36] Just like any other entrant in the Academy Award for Best Visual Effects category, Birdman and Gravity, the Ghost in the Shell live-action remake (2017, Rupert Sanders), and Ready Player One (2018, Steven Spielberg) all inevitably do just that; throwing their audiences (or should we say, their audience, as it’s unlikely that we’re speaking of a revolving cast of characters), into the midst of a DVFX hailstorm. Sooner than expected, we could find ourselves confronted with a seamless cloth of digitally composited and enhanced reality. Then, it will be our turn to rip the stitches apart.
References
Ackland-Snow Terry, Laybourn Wendy, The Art of Illusion: Production Design for Film and Television, (Marlborough: Crowood Press) (2017).
Barsacq Leon, Caligari’s Cabinet and Other Grand Illusions, (New York: New York Graphic Society) (1977).
Bertalmío Marcelo, Image Processing for Cinema, (Boca Raton, London and New York: CRC Press) (2014).
Bordwell David, ”Intensified Continuity: Visual Style in Contemporary American Film”, Film Quarterly 55:3 (Spring 2002), pp. 16-28.
Bryant Antony and Pollock Griselda (ed.), Digital and Other Virtualities: Renegotiating the Image, (London and New York: I.B. Tauris, 2010).
Butte George, “Suture and the Narration of Subjectivity in Film”, Poetics Today 29:2 (Summer 2008), pp. 277-308.
Dayan Daniel, “The Tutor-Code of Classical Cinema”, Film Quarterly 28:1 (Autumn 1974), pp. 22-31.
Dobbert Tom, Matchmoving: The Invisible Art of Camera Tracking (San Francisco and London: Sybex) (2005).
Eisenstein Sergei, “The Filmic Fourth Dimension” (1929), in Film Form: Essays in Film Theory, Jay Leyda (ed. and transl.) (New York and London: Harcourt) (1949).
Elsaesser Thomas, Hagener Malte, Film Theory: An Introduction through the Senses, (New York and London: Routledge) (2015).
Gaudreault André, Marion Philippe, The End of Cinema? A Medium in Crisis in the Digital Age, John Belton (ed.), Timothy Barnard (transl.) (New York: Columbia University Press) (2015).
Hernández María Lorenzo, “The Double Sense of Animated Images: A View on the Paradoxes of Animation as a Visual Language”, Animation Studies 2 (2007), https://journal.animationstudies.org/maria-lorenzo-hernandez-the-double-sense-of-animated-images/, date accessed 9 July 2018.
Pethő Ágnes (ed.), The Cinema of Sensations (Newcastle upon Tyne: Cambridge Scholars Publishing) (2015), pp. 36-44.
Prince Stephen, Digital Visual Effects in Cinema: The Seduction of Reality (New Brunswick, New Jersey and London: Rutgers University Press) (2012).
Purse Lisa, Digital Imaging in Popular Cinema, (Edinburgh: Edinburgh University Press) (2013).
Ramírez Juan Antonio, Architecture for the Screen: A Critical Study of Set Design in Hollywood’s Golden Age, (Jefferson and London: McFarland & Company) (2012).
Shimamura Arthur P. (ed), Psychocinematics: Exploring Cognition at the Movies, (Oxford and New York: Oxford University Press) (2013).
Tawa Michael, Agencies of the Frame: Tectonic Strategies in Cinema and Architecture, (Newcastle upon Tyne: Cambridge Scholars Publishing) (2011).
Whitlock Cathy, Designs on Film: A Century of Hollywood At Direction (Sydney, London, New York: HarperCollins) (2010).
Notes
[1] Terry Ackland-Snow, Wendy Laybourn, The Art of Illusion: Production Design for Film and Television, (Marlborough: Crowood Press) (2017), p. 42.
[2] Daniel Dayan, ”The
Tutor-Code of Classical Cinema”, Film Quarterly 28:1 (Autumn 1974), p. 30.
[3] Not ‘sequence’, as it is crucial to analyse this part of the film as taking place within the same interior, as well as noticing its constructed perspective which brings the illusion into being.
[4] Daniel Dayan, p. 28.
[5] Juan Antonio Ramírez, Architecture for the Screen: A Critical Study of Set Design in Hollywood’s Golden Age, (Jefferson and London: McFarland & Company) (2012), p. 63.
[6] Lisa Purse, Digital Imaging in Popular Cinema, (Edinburgh: Edinburgh University Press) (2013), p. 2.
[7] Daniel Dayan, pp. 28-29.
[8] This has been historically motivated by the intention to optimize “… viewing positions by decomposing events into different shots, each of them showing the event part preferably from an appropriate position and viewpoint, [which] does not come for free but instead implies a reduced spatial coherence across shots that goes along with increased cost of cognitive processing.” [Stephen Schwan, “The Art of Simplifying Events”, in Psychocinematics. Exploring Cognition at the Movies, ed. Arthur P. Shimamura (Oxford and New York: Oxford University Press) (2013), p. 222.].
[9] André Gaudreault and Philippe Marion, The End of Cinema? A Medium in Crisis in the Digital Age, ed. John Belton, transl. Timothy Barnard (New York: Columbia University Press) (2015), p. 65.
[10] Leon Barsacq, Caligari’s Cabinet and Other Grand Illusions (New York: New York Graphic Society) (1977), pp. 44-45.
[11] Michael Tawa, Agencies of the Frame: Tectonic Strategies in Cinema and Architecture, (Newcastle Upon Tyne: Cambridge Scholars Publishing) (2011), p. 30.
[12] Juan Antonio Ramírez, p. 83.
[13] Idem, p. 24.
[14] See: Lisa Purse, p. 27. A similar discussion is conducted there on a sequence from Live Free or Die Hard (2007, Len Wiseman), in which John McClane looks past his perfectly clear reflection in a glass pane, enhanced this way to direct the viewers’ attention to the symbolic act of confronting oneself.
[15] María Lorenzo Hernández, “The Double Sense of Animated Images: A View on the Paradoxes of Animation as a Visual Language”, Animation Studies 2 (2007), https://journal.animationstudies.org/maria-lorenzo-hernandez-the-double-sense-of-animated-images/, date accessed 9 July 2018, p. 40.
[16] Idem, p. 6.
[17] David Bordwell, ”Intensified Continuity: Visual Style in Contemporary American Film”, Film Quarterly 55:3 (Spring 2002), p. 16.
[18] David Bordwell, p. 24.
[19] Idem, p. 16.
[20] The case of abstract and experimental cinema, but also the premise on which Eisenstein’s intellectual montage was founded.
[21] George Butte, ”Suture and the Narration of Subjectivity in Film“, Poetics Today 29:2 (Summer 2008), p. 283.
[22] Lisa Purse, p. 18.
[23] Noël Carroll and William P. Seeley, ”Cognitivism, Psychology, and Neuroscience: Movies as Attentional Engines”, in Psychocinematics: Exploring Cognition at the Movies, ed. Arthur P. Shimamura (Oxford and New York: Oxford University Press) (2013), p. 62.
[24] André Gaudreault and Philippe Marion, p. 161.
[25] Idem, p. 162.
[26] Thomas Elsaesser and Malte Hagener, Film Theory: An Introduction through the Senses, (New York and London: Routledge) (2015), p. 124.
[27] André Gaudreault and Philippe Marion, p. 69.
[28] Idem, p. 165.
[29] Idem, p. 175.
[30] Antony Bryant and Griselda Pollock, “Editors’ Introduction”, in Digital and Other Virtualities: Renegotiating the Image, ed. Antony Bryant and Griselda Pollock, (London and New York: I.B. Tauris, 2010), p. 8.
[31] Sergei Eisenstein, “The Filmic Fourth Dimension” (1929), in Film Form: Essays in Film Theory, ed. and transl. Jay Leyda (New York and London: Harcourt) (1949), p. 64.
[32] Idem, p. 43.
[33] Idem, p. 195.
[34] Marcelo Bertalmío, Image Processing for Cinema, (Boca Raton, London and New York: CRC Press) (2014), p. 103.
[35] Idem, p. 108.
[36] Daniel Dayan, p. 29.