Post-production Sync Help

[jpsound]

Hey everyone, 

 

I work for a university production studio where we specialize in multitrack audio recording and offer multi camera video capture. I say it in this way because, unlike film, we lock an edit in the audio realm before its sent to the video editing folks. With our current workflow the video editors have to spend a good deal of time manually realigning takes of video to the audio mix before the creative editing process can take place. 

 

Currently we record, edit, and mix our audio in Protools 11 and edit video in FinalCut Pro X. Each of the video cameras captures onto a separate SxS card and timecode and genlock are not distributed throughout the studio. For some reason we haven't run into video sync issues for a few years now, so I consider ourselves lucky that we have’t had more problems.

 

The audio edits come from several non-contiguous takes over the course of a production. The video editors then have to go back through, reference the Protools session, and scrub through to find the corresponding sections of video in FCP X. Each take is slated manually, but often the audio edits come from relatively small sections within the take. 

 

We’re trying to find a better solution: 1. To automatically reassemble the video to match the audio edits, or 2. That would allow us to edit the video and audio simultaneously. Keep in mind that when we edit audio we have up to 48 channels and need similar editing power as Protools. 

 

If you're unfamiliar with Protools, the reason this won’t work immediately with timecode between PT and the cameras is because the timeline treats timecode as relative for each clip. For example, if a clip is recorded at 00:02:18:14 and moved in the timeline during editing, the clip’s timecode stamp when exported will be its new position in the timeline. So we cant use the clip’s timecode to line it up with the original video because it will be incorrect. 

 

I know this is a long question with a lot of conditions, but our workflow is somewhere between broadcast and film, and due to the nature of the material we record, audio takes precedence over video. It’s a unique situation.

 

I’m looking for any comments or suggestions regarding how we can improve our workflow, including the addition of equipment / timecode / sync / software / etc… and wondering if there are any other studios out there with a similar production goal.

[Jay Rose]

1.  To automatically reassemble the video to match the audio edits, or 2. That would allow us to edit the video and audio simultaneously

#1 sounds like an intractable video problem. This is so backwards from the normal flow that I doubt anybody has dedicated software. But since you know the start timestamp of each audio clip (or at least you know what it was from the shoot, before PT changes it), before placing sound in the timeline could you

1) plant a loud spike or beep in the prodn audio clip itself, very close to the start and before the part you'll use. You'll probably have to do this in a different editor, like Peak or SoundForge, so it stays on the same file and isn't broken into what PT considers separate clips.

2) figure the timecode for that beep (depending on the editing software, possibly read it directly ... or compute it manually from a start time you find in a broadcast wave reader)

3)  rename the clip so it's now called the timecode for the beep

4)  edit in PT

6) export with handles that include the beep.

7) pix editor lines up his source video so its timecode per the audio filename lines up with the audio beep

After pix is edited, you can ignore all the versions with handles and do your actual mix to pix using the PT layout from step 4.

 

#2 is a better way: Sync the wild recordings to pix using slates, then edit the sound along with the pix. You'll have to do it in an NLE like Final Cut, and your edits will be constrained to framelines, but you can smooth them out in PT later. If you ingest the audio files and don't do any level or eq changes in FCP, they'll stay pristine. Then you can do a final mix from and OMF or AAF. That's the usual workflow. 

If there are more iso tracks than the pix editor wants to deal with, ingest them anyway. Cut them locked to the main mix track and matching pix. Just keep them muted until you generate the OMF.

[Wandering Ear]

You could try using Ltc on an audio track and feeding that timecode to the cameras. . Keep that Ltc track locked to the audio as you edit,  then export that track to the video editor. Use fcpauxtc or similar program to read the edited Ltc audio clips to generate a list of tc values that correspond to each edit,  then assemble the video from that edl. I have not tried this specifically,  so some testing and tweaking would be required, but I don't see why it wouldn't work.  

 

[Philip Perkins]

Stop using PT for production recording--it is part of your sync problem.  We use a recording system that records the correct system TC timestamp to ALL files (from TC and clock that is also avail to the cameras if the producers are sensible enough to use cameras that will input such signals--not at all a given anymore), that is looking at a very stable clock.  Even if the cameras cannot (or aren't allowed to) take ext TC, you can still get the onboard generators close to the audio TC and start them.  As was said a slate @ some point will help you determine the TC offset for that clip.  Having the audio (MT and live mix) have the correct TC stamp REALLY simplifies audio post.   They can do their cut from a multiclip in which the picture is synced to the TC of the live audio mix, then the edit EDL will reflect source TCs that reference the MT master.  FCX is kind of TC unfriendly but it is possible.  You can export an AAF from FCX via X2Pro, and conform the MT audio to the picture that way, also.

philp

Another tip--DO NOT cut cameras unless you absolutely have to--like dead batts of out of media.  Longer runs=fewer sync points=happier editors.

[CraigF]

I hope you don't tell the students that they will get work in the industry with that ass backwards work flow.

[jpsound]

Alright! We're off to a good start here with the comments we've gotten so far. I think the most promising might be AuxTC Reader, so we'll put that on our list of things to test. Thanks everyone for your help.

A few more comments:

I'm just interested in what others are using - Phillip, what recording system are you using? Something like Boom Recorder or Metacorder with a TC input? Or something else?

Also,

ass backwards work flow

Yeah, we know it's weird. The world class musicians we're recording care more about how they sound than how they look. It may not be legitimate to you, but our clients beg to differ.

 

[Philip Perkins]

Those computer based systems have worked very well for me, yes.  These days I use a pair of JoeCo BBRs for this, another great choice might be a Sound Devices 970.

phil p

[CraigF]

Alright! We're off to a good start here with the comments we've gotten so far. I think the most promising might be AuxTC Reader, so we'll put that on our list of things to test. Thanks everyone for your help.

A few more comments:

I'm just interested in what others are using - Phillip, what recording system are you using? Something like Boom Recorder or Metacorder with a TC input? Or something else?

Also,

Yeah, we know it's weird. The world class musicians we're recording care more about how they sound than how they look. It may not be legitimate to you, but our clients beg to differ.

 

well given how you opened, you left out a ton of detail

I work for a university production studio where we specialize in multitrack audio recording and offer multi camera video capture. I say it in this way because, unlike film, we lock an edit in the audio realm before its sent to the video editing folks. With our current workflow the video editors have to spend a good deal of time manually realigning takes of video to the audio mix before the creative editing process can take place. 

OK the proper way to do this assuming you have a HD rig with Sync HD and a master TC generator as well at your Sync Generator to feed all devices  

In the Pro Tools session you set up Audio tracks as normal for multi-tracking and mute them (or send them all to a dead patch bus) then also set up a set of Aux tracks with inputs matching the Audio tracks bussed to a mono or stereo Audio track  to do a live rough mix for the editors to cut to (exporting the mix with the time stamp). Then you can mix the raw tracks at your leisure and everything matches via TC (but include head & tail slates)

Usually audio mix is not done until there is at least rough cut to mix to

[Marc Wielage]

I hope you don't tell the students that they will get work in the industry with that ass backwards work flow.

I'm often in shock by how little students are being taught in modern colleges as far as the right ways to approach timecode, workflow, and good editing practices.

The first thing they have to do is to stop shooting on toy cameras without timecode. There are inexpensive cameras out there that you can at least attach timecode boxes to, and once you do that, the soundtrack and picture will automatically lock up in almost all editing programs. But even then, you still should be using a timecode slate and a clap just as reference, for takes where the timecode fails, or takes where the camera is deliberately shooting off-speed. 

I've dealt with many neophytes who are absolutely terrified of sound sync and timecode, and they rarely seem to grasp that all these problems were solved more than 25 years ago. All they need to do is use the right equipment, ask the right questions, and always test the workflow prior to production.

Pro Tools is not ideal for this kind of recording, but it can work provided you jump through a few hoops.

Edited July 16, 2015 by Marc Wielage

[Jay Rose]

The world class musicians we're recording care more about how they sound than how they look.

Then don't worry. Seriously.

Cut the music so you and they are happy. Let a good pix editor, who knows music, piece the visuals together using the ref track from the camera (fed from the board) as a guide. I've seen editors do amazing jobs with this, and with long-shot dialog that had nothing to do with the spoken words. 

If there's a sync issue,

1) cover with b-roll

2) refer to above quote.

 

 

[Ze Frias]

You could try using Ltc on an audio track and feeding that timecode to the cameras. . Keep that Ltc track locked to the audio as you edit,  then export that track to the video editor. Use fcpauxtc or similar program to read the edited Ltc audio clips to generate a list of tc values that correspond to each edit,  then assemble the video from that edl. I have not tried this specifically,  so some testing and tweaking would be required, but I don't see why it wouldn't work.  

 

I think this is a very sensible solution, if everyone is comfortable with the workflow. Make sure you test the workflow thoroughly before implementing and using.

[jpsound]

In response to Craig:

Yeah, that's the way to do it only if audio and video is one long take. For us, it's not. Also, we're dealing with a broadcast mix in a lot less clunky way than using Auxes in PT.

In response to Marc:

Someone at some point made the assumption that we're shooting on cameras with no TC. That's not true, so getting past that part isn't a problem. There are some people at my workplace that are in fact scared of TC, but I'm not. Instead, I'm trying to streamline our editing process, which even though is backwards from the industry (audio before video), is necessary for our productions.

And finally, for Jay:

We have an editor that has gotten good at linking the video back to the audio, and our method currently works pretty well for him. It just takes a while, especially because we sometimes are editing audio down to a specific note. We want to make the process faster and more accessible for future editors who come in without knowing how we do things.

 

Long story short, WORKFLOW TEST!

Again, thanks for the help and interesting discussion.

 

[CraigF]

you know if you laid it all out instead of generalities we might be able to help more

[nickreich]

I actually see this 'backwards' workflow quite regularly - my main business is recording live shows for use in TVCs, EPKs up to full cinema releases of filmed live theatre shows. With Musicals, there is often a fair bit of back-and-forth between the Composer / Musical Supervisor and the Picture Editor / Director as to which takes to use, if one is visually better and the other Musically better. In the end, I suspect the Composer wins out as far as the soundtrack is concerned, but the Picture Editor will then do his/her best to cheat preferred vision on top as long as it looks in sync - popping in cutaways to nudge sync as required. We usually just Export Session as Text in ProTools (with Show User Timestamps selected - which should have your original timeline position, matching TOD timecode if ProTools or you original recorder was chasing the same TC as cameras) to produce a 'paper EDL' for the Pic Editor to see which takes we used where. What the Assistant Editor does with that, I'm not sure.

EdiLoad from Sounds In Sync can create EDLs from a ProTools Session - which may offer the start of a automated solution. You would record with PT chasing the same TOD TC as cameras (using Lockit boxes, for example), so the clips have the correct original timestamp to match picture. Do your audio edit back to a place earlier in the timeline. Drop the session file into EdiLoad and choose one track for it to reference, that has contiguous clips throughout the PT session (I'd use a guide track that doesn't appear in the mix, so it doesn't need fades - just mute the clip in PT but edit it with the rest of the show - it has contain a real clip, not a Clip Group). I don't have a copy of EdiLoad at home so I can't test it, but in theory you can export a standard EDL which may be imported into your video editor as a guide to re-assembling picture. Worth getting a Demo of EdiLoad to try it out.

[Christian Spaeth]

As unusual as this seems for most of us who know audio only as "slave" to video, I also see formats like this more and more often. Studio live sessions with video, concert recordings with video etc., all productions where the video edit follows the audio edit, as opposed to the jobs we're on. I see this as a positive process. Maybe we shouldn't criticize them for doing something "new" or different. That said, I believe the OP will do good testing some of the ideas that have been mentioned on this thread, as there doesn't seem to be standard procedure on this kind of production yet.

[CraigF]

my problem is not with the workflow, my problem is the OP's lack of detail on the project & goals from the start

[Wandering Ear]

I thought the OP was pretty clear with what they they were trying to accomplish.  Just because it's not the most common workflow, doesn't mean it shouldn't be done.  Just might take a little more testing to do it without trouble.

[Marc Wielage]

If the problem is caused by the cameras stopping and restarting during a live performance, then you have three options:

1) use timecode camcorders jammed to the same TC source as the multitrack audio. That way, when the camera stops, you can instantly sync-up the new start point to the existing soundtrack. 

2) have the cameras continuously roll without stopping.

3) bite the bullet and just manually re-sync the sound every single time the camera stops and restarts. 

I have done the latter on several very large-scale projects (one of which used 22 simultaneous cameras), and it can be done. All you have to do is have the time and skill to get it done. It helps to go through the sound and create a paper index of everything that happens in the concert and reference it to something visual, like a light change, some kind of on-stage action, or some other kind of event. 

At the worst, maybe you can have them roll a lower-res (but still timecoded) camera as a wideshot that absolutely never stops. Use that as a reference for the entire show. That way, you'd have both a visual and an aural reference for the other cameras as they move for close ups, stop and reposition, and so on. I have used this trick before on productions that ignored timecode slates, and just having one solid reference saved the whole show.

Edited July 22, 2015 by Marc Wielage

[jpsound]

Again, thanks everyone for some good suggestions. Nick, I'll take a look at EdiLoad and see if that can help us out. I'm happy to hear that Christian says that this workflow is appearing elsewhere... perhaps down the road there will be a single editing solution or workflow that solves all the problems we're having.

Although I have what I need, I'll try to clarify once more for Craig and Marc solely for their understanding.

Let's first consider how things are done in a recording studio, so let's say Marc is our singer and Craig is our engineer.

Marc is going to lay down a few base takes of his hit new single. These takes are continuous from the start of his song to the end. Then, because maybe his throat got dry in measure 25, he wants to re-record that measure and comp it in. So he does. And maybe he comps a few more measures, too. In the end, Craig bounces out a mix that is not just composed of the base take but also a bunch of comps - but together they add up to one full song.

In the audio-only world this is pretty easy. When Marc wants to go back and re-sing measure 25, Craig recalls the transport to measure 24 or so, hits record, and then crossfades the in and the out. Simple.

But how do you do this when you're also rolling video? You can't, because you just don't capture video into a 'comp' like you would audio. Instead, PT and cameras are rolling quasi-continuously through the entire session, and a producer sits there with notes to say "Let's go back to measure 25". Marc goes back and sings measure 25, which is not captured over the original base take but instead somewhere farther along in the audio and video files.

During all this, Marc's job doesn't get harder; he just sings when the producer or Craig tells him to. Craig's job actually gets easier, because he doesn't have to move the audio transport around as much. 

My job, as editor, gets much harder.

After the session, Craig follows the producers notes and splices all the comps into the base take. In our example, he would take the audio at "Measure 25 Take 1" and move it to overlap measure 25 in the base take. That comp, which originally was in time with the video (even assuming timecode and tri-level sync for everything), has now been shifted somewhere else in the PT timeline.

So without a better solution, I sit there in FCPx scrubbing through until I find the video that was recorded during "Measure 25 Take 1" and then dragging it so it aligns with where the audio for that take is mixed.

That's the issue we're having. I'm not sure if I can be more clear than that.

Edited July 30, 2015 by jpsound

[Jay Rose]

Just curious: have you asked in a video editors' group?

 

[jpsound]

Just curious: have you asked in a video editors' group?

 

No, not yet. Any good suggestions?

Thanks!

[Jim Feeley]

Perhaps the Avid-L. Lots of good editors there, including folks who've produced and cut multicam live music performance. The list has been around for a long time...many of the editors also cut in other systems, are PT aware, etc... 

https://groups.yahoo.com/neo/groups/Avid-L2/info

You could maybe try Creative Cow. Some good people there, but also a lot of new-user stuff; the SN isn't as good at Avid-L.

https://www.creativecow.net

I follow Avid-L. I don't follow the Cow (though some good editor friends do).

[Marc Wielage]

But how do you do this when you're also rolling video? You can't, because you just don't capture video into a 'comp' like you would audio. Instead, PT and cameras are rolling quasi-continuously through the entire session, and a producer sits there with notes to say "Let's go back to measure 25". Marc goes back and sings measure 25, which is not captured over the original base take but instead somewhere farther along in the audio and video files.

Well, to me, the answer is you need two editors: a picture editor and a sound editor. The two have to be edited separately. Cut the picture to tell the story, use a (very rough) mix that works for purposes of the documentary, and then have a music editor come in and finesse the sound edit to use the exact phrases and other moments that are needed from the multitrack session. 

I think trying to make one editor responsible for all of it is just too much. I guess it's technically doable if you're just doing one song, but if this is hours and hours of footage, it could be extremely problematic, especially in FCPX. I'm also not a fan of doing any sound editing in a video editing program like Avid, FCP, Premiere, or anything else. To me, none of them are precise enough or have the features of a system like Pro Tools, which is what you really need on the sound side.

There are editors that can bounce back and forth between FCP and Pro Tools, or Avid and Pro Tools. Walter Murch is one of them, and has actually one Oscars for both.