Here's a nice article that postPerspective did about the process. I'll cut and past as well as link!
VFX bring wheatpaste poster to life in ‘Paper Heart’ music video
Leave a reply
Each January, The Silver Sound Showdown music video festival and battle of the bands takes place at Brooklyn Bowl. It pairs the winning director and winning band together for a chance to make a music video with Silver Sound Studios in New York City. It was here that Paper Heart, the music video directed by Nick Snyder, produced by Silver Sound and featuring the bandBlood and Glass, was born.
“Paper Heart, one of the most ambitious Showdown collaborations to date,” according to festival director and producer Cory Choy, features Blood and Glass’s lead singer Lisa Moore as a wheatpaste poster on walls across Brooklyn. It was shot on a Red Scarlet camera, features effects created in Adobe’s After Effects, Photoshop and was edited on Adobe Premiere Pro.
Why the wheatpaste-poster look? LA-based Snyder (@nickwsnyder) works in the arts district of downtown, where he sees inspiration in everything. He also liked the idea that the nature and lifespan of the wheatpaste poster seemed to play nicely into the “themes of isolation and fragility found in the song.”
Director Nick Snyder, right.
Snyder’s Silver Showdown-winning video Lost Boy Found also combined the techniques of live performance, compositing and animation — silhouettes of actors were composited into a fantasy shadow puppet world — so this was a realm he was comfortable in.
The Production After several months of prep, Snyder and the band made their way to New York City for the two-day shoot. The first day was dedicated to shooting plates. Locations around Brooklyn had been scouted by Silver Sound, Google street-viewed by Snyder prior to arrival, and then scouted in person. So by the time production began, specific moments had been planned to take place in a handful of selected locations. The remaining moments were narrowed down to areas where the filmmakers anticipated chance discoveries. Snyder, DP Andres Cardona and a skeleton crew set out onto the streets of New York to shoot with their Red Scarlet.
Going Green The second day was shot at Parlay Studios in Jersey City and dedicated exclusively to greenscreen shots. During a brief break in between days, Snyder analyzed the plates. While he shot listed and storyboarded, he also left room for improvisation and collaboration.
To aid lead singer Lisa Moore in her characterization, extra attention was given to wardrobe, makeup and props. “For example, it was decided beforehand that her prop cane would become a matchstick and that after using it, the matchstick would shrivel and blacken,” explains Snyder. “The art director constructed a practical burnt matchstick prop, but rather than swapping it out during Lisa’s performance, the prop was shot suspended in front of the greenscreen. Then, using an LED light on Lisa’s unburnt cane, I tracked the movement of the matchstick in After Effects. I then replaced it with the burnt matchstick seen at the end of the video.”
The same technique was used for the origami birds that interact with Moore throughout the video. Practical birds were made, shot against the greenscreen and keyed out in post. The intention was that they could be keyframed in After Effects, but their natural movement would allow for a slightly more organic feel. It’s also a good time saver. “Green apple boxes, chroma key gloves and even crew members wrapped in green blankets were used to achieve the effect of tactile contact within the video,” explains Snyder. “The performance moments were shot from start to finish in various sizes, and shooting in 4K allowed for any Lisa/plate size relationship miscalculations.”
The Post The next step was assembly. This involved mapping Moore to the building surface plates. Premiere Pro was used to assemble performance shots in raw R3D and narrow down her best takes. For performance takes, a six-panel export was made to quickly compare her gestures from the narrowed down shots. From there, a preliminary pass was made on pairing Moore with the plates by adding the chroma key effect in Premiere. “This simplified version of After Effects’ Keylight allowed us to see what was working without having to check all the shots in the much more sluggish After Effects video playback,” says Snyder. “Additionally, once the assembled shots were ready for AE, the greenscreen clips with this chroma key effect would stay in the metadata of the shot.” Another time saver, he says, was that once the Moore/plate relationships were locked and a cut was close to locked, the compositing could begin.
To save space and make for faster save times, Snyder chose to create separate After Effects files for each shot. The first step was to finalize the look for “Wheatpaste Lisa.” After some trial and error, a look was established and a master file was created that could be imported into each After Effects file, but the process for creating the look wasn’t as easy as copying and pasting a LUT. In some cases, upwards of 20 pre-comps were used.
According to Snyder, the basic process went like this. “The greenscreen shot was keyed out using Keylight, adjusting for spill and greenscreen inconsistencies. Luckily, the DP did an excellent job at lighting Lisa, so this was a breeze. If there was an issue, a simple matte choker was used. Then, this was precomped and a minimal texture was brought in to dirty it up a bit. The overlay blending mode was often used as well as an image mask. It was precomped again; an off-white stroke was added using Layer Style Stroke. This effect was used to create the white-edge poster look. The stroke size and precomp level varied from shot to shot, depending on the size of shot Lisa was in and also the texture of the plate onto which she was to be composited. At this point the look starts to emerge a bit, but a few steps remained in order to completely bring Wheatpaste Lisa to life.”
For Paper Heart, a combination of Adobe CC’s Glass and Texturize were used to give Moore a convincing paper texture as well as authentic surface imperfections, explains Snyder. Most often, two bump maps were used — one for generic surface texture and lighting, plus a second to pick up the surface of the wall behind her. For the second, a high contrast grayscale image was created in Photoshop to bring out the important parts. Using Dynamic Linking, Snyder was able to paint over parts of the bump map that were less important, save and view the results in After Effects.
Lastly, two layers needed to be created to mimic ink on paper and human error. This would also come into play later in the video as the iterations of Wheatpaste Lisa start to erode away. “For this effect, the comp had to be duplicated. Unfortunately, After Effects comp duplication only duplicates the top comp,” explains Snyder. “So in order to duplicate all of the nested comps, a purchased Script called True Comp Duplicator had to be used. The newly duplicated comp was then brought into the original comp and placed below. Using the Fill effect, this comp was colored off-white. Then, to add the finishing touches, some final grungifying had to be done to the top later. Using Photoshop, 5K resolution brush strokes and alpha channel grunge effects were created on multiple layers. Once imported into AE, these could be used in the top Lisa comp. Using the Silhouette Alpha blending mode, the grungy paintbrush strokes subtracting bits of Wheatpaste Lisa, creating imperfections and rough edges that exposed the off-white layer beneath it.
“Finally, back in the master comp with the two Lisa layers, those were precomped once more. At this point, the look was more or less complete,” he continues. “But from shot-to-shot, additional work was sometimes required to successfully composite Lisa onto the plates.” Some additional tools used were Roughen Edges, another Matte Choker and occasionally another round of Silhouette Alpha grungy paint brush strokes.
For lighting, Snyder used either the 4-Color Gradient or Gradient ramp on an Adjustment Layer or on a Solid set to the Hard Light Blending Mode. Opacity was usually in the 10-20 percent range.
During the process, Snyder and Silver Sound discovered that Wheatpaste Lisa’s movement looked best at 12fps. “We wanted to underscore the fact that Wheatpaste Lisa was an actual wheatpaste entity existing in her own little universe, not just a video projection,” explains Silver Sound’s Choy. “So the choppier feel of 12fps was used to make Lisa’s motions a little less fluid, a little more animation-y and other worldly feeling.” For this effect, the Posterize Time effect was used.
Throughout the compositing process, Snyder created H.264 proxy files from the transcoded R3D footage. This was especially helpful with the origami birds. To save space, the birds were rendered out on their own at a much smaller file size and then re-imported.
The Death of Wheatpaste Lisa Finally, Wheatpaste Lisa had to die. To achieve the effect of wheatpaste poster weathering, both layers of Wheatpaste Lisa had to erode. “Back inside the top layer — the double layer Lisa comp — individual brush strokes and grunge effects were animated with Silhouette Alpha as their blending mode,” describes Snyder. “Once the weathering looked satisfactory, these animated layers were copied, pasted into the bottom layer Lisa comp and adjusted in movement and timing. This allowed for the top layer to erode just before the bottom layer, pushing the compositing one step closer toward realism. Occasionally, one final matte choker and/or an animated mask was used on the final precomp to eliminate any stray particles or to insure that she dissolved away completely.
Once complete, the shots were rendered at 4K ProRes 4444. The final shots were delivered to Silver Sound colorist Vlad Kucherov with Moore separated from the building surface plates. Using DaVinci Resolve, Kucherov worked with Snyder to achieve satisfactory look that worked well for the video concept while also helping sell the compositing realism. Having the layers separated gave Silver Sound more control during this process by being able to adjust the levels independently. The goal was to find a look that plays to the feel of the song, but also gives the video a confident personalized look of its own.
“In the end, Paper Heart is the result of careful planning, post-production experimentation, lots of hair pulling and creating a concept that exists within a strict set of limitations,” concludes Snyder.