Saner postproduction process for multi-take music videos?

[Larry Sanbourne]

Hi,

 

I self-record classical chamber music videos with 2 cameras with time-of-day timecode, RME gear (prob switching to Sonosax/Merging) feeding into Reaper. My post workflow takes me forever and makes me hate my life. Can anyone help me improve? What I do is:

 

1. In Reaper, divide audio into takes, renaming clips T1, T2, .... Mix, including very detailed fader automation.

2. In Resolve, make a long timeline with all video takes. Sync each audio take with video using timecode + waveform sync.

3. In Resolve, divide that long timeline into single-take multicam sequences (set in/out points, create compound clip, convert to multicam sequence).

4. In Reaper, edit the audio based on the viewing of video takes.

 

Then comes the craziest step of all:

 

5. In Reaper, export timeline to EDL format and run a custom Python script to convert this to OpenTimelineIO, which Resolve can import as a timeline.

 

There *must* be a better way, right? Hope someone can help!  Thank you!!

[The Documentary Sound Guy]

I can't quite follow exactly what kind of edits you are turning out (are you editing highlight videos that don't include the whole piece?  Do you pick one take and cut to that take?).  It would help to know more about what the end product is supposed to look like.

 

From what I can see though, you seem to be logging everything twice, in two different programs, and doing way more mixing than you need.  You'd probably benefit from following a more traditional video workflow, i.e.:  picture edit, picture lock, and then worry about mixing once picture is done.  That would presumably mean you no longer have to mix every take, and you only mix what actually ends up on screen.  It would also mean you spend less time on conform processes jumping between programs.  Essentially, do everything you need to do in Resolve first (picture), export to Reaper, then do everything you need to do in Reaper (sound), and finish in reaper.  Or, worst case, you export a single bounce from Reaper and recombine just the finished track with the finished video in Resolve.  No need to mess around with scripts and EDLs.  (On the other, hand, if your EDL script process works as is, it sounds like it wouldn't take a lot of time.  But you are complaining about it, so I'm guessing it doesn't work very smoothly).

 

That lets you skip most of step 1, and the necessary parts (divide into takes) can be done in Resolve.  You don't need to edit picture with anything except the raw audio.  Sync should be your first step so you only have to do your logging once, since both video and audio are logged together.  Since you seem to be are rolling one long take for audio and video, you can also build your multiclips earlier as well:  Do it just after sync, before you cut anything up, so you only have to create one multiclip, which can then be cut up for logging.  Do your video edit, then flatten out all your multiclips so all the audio tracks are explicitly visible, then export the final video edit as a file, and export all the audio as OMF or AAF back into Reaper for your mix.  This is the most complex step, and you'll need to test it out to get the setting right, but you'll want to export with handles (so you have access to audio beyond the hard video edit points), and you'll have to negotiate the tempermental intricacies of getting OMF / AAF into Reaper.  Do your mix in Reaper, export the result, and recombine in either Resolve or another finishing tool ... it's not that complicated to combine a single video and audio track as long as the start time doesn't change.

[Larry Sanbourne]

Thanks so much for taking the time to reply.

 

I'm editing a whole piece. E.g. in one aria there might be an edit every 10-15 seconds (ideally, where a camera angle change would make sense, or else a little audio patch that doesn't affect overall timing). I've been doing an overall "rough" audio mix so I can pick takes with clear audio (closer mics than just the main pair). Then I've been editing the audio in Reaper and trying to create a video timeline with those same edits in Resolve (the process outlined above). The custom script actually works, but it requires a lot of hideously boring prep work (renaming media clips to T1, T2, T3; gluing media so timestamps in the edited audio are relative to take start times).

 

Re sync, since I'm sometimes recording myself and cameras might be far away, I stop rolling everything only when we stop for more than like 5 mins. (I wish I could start/stop everything with one button press…) It would indeed be vastly vaster to sync A/V before chopping into takes, so thanks very much for that.

 

I had no idea one could export video + audio with the handles! In my old audio-first workflow, every audio edit requires finessing the edit point by 1-5 ms at a time and varying the crossfade length. Due to timing differences in how musicians played each take, sometimes moving an edit point by 10ms also requires moving the current clip (and thus all subsequent ones) a couple ms earlier in the timeline. In your recommended video-first workflow, would that require a roundtrip, and is that possible?

 

> flatten out all your multiclips so all the audio tracks are explicitly visible

 

Before sync, are you recommending I put all the audio tracks into Reaper? I think this is the only way to be able to do finer mixing (fader riding) in Reaper after the fact, but I'm just checking that I understood. So for an orchestral video I would have to figure out how to get 10+ audio files, for each time we started/stopped, into Resolve and stacked up. (I see why video-focused audio recorders output multiwavs…)

 

THANK YOU so much for all these tips. If your video-first workflow could indeed work, this will save hours of tedium!

[The Documentary Sound Guy]

My approach to live edits has always been to pick a single take and stick to it, and have enough camera angles to make that work.  Also, with live performance videos, one take is all you get...  Gluing together a single performance from multiple audio takes isn't something I've taken on, so I see the complexity.

 

If cutting between multiple performances is a necessary part of the workflow, I'd suggest completely separating the audio edit and the mix.  Do the audio edit in Resolve as part of picture edit, and the mix in Reaper at the end.  Resolve's Edit page *can* do sub-frame audio edits and make sub-frame sync changes relative the picture, but it's not immediately obvious how.  I forget the exact process, but you should be able to google it.  If all else fails, you can also make sub-frame tweaks in the Fairlight page (which works off the same timeline as the edit page, and you can switch back and forth at will).  I don't recommend Fairlight for any other form of audio editing; I've just learned the hard way that it's not stable with plugins or automation, but it does work fine for simple edits.  That would save you needing to roundtrip into Reaper to fix every cut.

 

39 minutes ago, Larry Sanbourne said:

Before sync, are you recommending I put all the audio tracks into Reaper?

No, I'm trying to avoid that at all costs.  Flattening the multiclips is a pre-export step at the end of the process after you have picture lock.  Don't touch Reaper until you are done with the picture edit.  Resolve's handling of multiple audio channels is ... odd ... but if you take the time to figure it out, it does work.  I've just done this exact workflow getting all my multiclipped interviews into ProTools with all the original audio tracks.

[Larry Sanbourne]

I will look into the subframe audio edits. The issue is just that unless I doing crossfades (which are much harder to control in Resolve than Reaper, unless I missed a way when I last tested), I can't always tell exactly where the edit point should go. I.e. often I'll do a crossfade at the given edit point, find it sounds bad, move it over by a tiny bit, and then have to change where the edit boundary is. But I'll reread the Resolve manual re crossfades in case that could help.

 

If I don't put the (non-camera) audio tracks into Resolve, what audio am I actually editing in Resolve? Just an AB pair? Was hoping to be able to edit in Resolve by listening to roughly mixed audio, but then I don't know how I'd export the Resolve edit for more detailed mixing.

[The Documentary Sound Guy]

If you can get the timing right in Resolve, I'd save the dissolves until you finesse in Reaper.  But maybe you need the dissolve in order to know the timing works?  Could be you may need to use more of the Fairlight page than you might want for that ... Fairlight definitely has a bit more control, and can definitely move the cut into a sub-frame position if that's what you need.  I'd also add that there's a few different ways to create fades in Resolve ... in addition to dissolve "effects", you can drag the corners of the clips down and overlap across tracks.  Not saying it's as good as Reaper, but if you are going to adjust your workflow anyway, there's probably some more you can refine in your use of Resolve.

 

As far as what you edit, I'd bring the entire poly-wave file into Resolve and use "Clip Attributes" to just enable the AB pair and hide the rest of the tracks.  Then, the point of flattening out the multiclip is to expand all the channels before you export to Reaper so you get all the channels on individual tracks when it gets to Reaper.

 

I don't know that I can help that much more ... you need to bring it into Resolve and experiment a bit I think.

[Larry Sanbourne]

Hello friends! I finally stopped shooting videos long enough to postproduce some videos again.

 

I tried an edit that was entirely done in Resolve. This worked super well for a video that had just a single stereo audio track. I didn't sync it with "Auto Sync Audio" but rather just put the media on the timeline, one take after another. This was clunky for comparing sections of similar material but I did end up with an edit in a few hours.

 

Then I tried an edit, this time with 6-channel audio, that was entirely done in Resolve with "Auto Sync Audio": replacing the original clip audio with multi-track, creating multicam sequences from each take, putting those onto their own timelines for evaluation (including using Resolve's take evaluation UI…kind of clunky but worked). This was helpful for being able to reject audio takes due to camera issues and ultimately I decided to use a single take with some audio patches - which I just did in Reaper since that's much faster for me than Fairlight.

 

However, I ran into some questions - surely due to my own stupidity:

 

Auto Sync Audio

- What determines the order of channels? It is coming out different each time.
- After sync, how can I see source filenames of the added audio channels without adding the clip to a timeline? It's hard to tell what has gotten added.
- Sometimes artists record multiple takes in the same video clip (didn't start/stop cameras). Subclips have to be divided up for each take for each camera. Is there a way to do this more efficiently?

Create Multicam Clip
- What determines the order of the angles in the timeline (V1, V2, etc)? Again, this came out differently sometimes.
- How can I manipulate audio in the many-channels-per-angle multicam clip? It shows each angle as “4 Stereo”
- Sometimes audio continued after the end of video and I would like to “cheat” by using some of this audio for a different video take. How do I recover this since it gets cut off at the end of the video?

 

As you can see, I ran into some issues. Perhaps the tenor (no pun intended - these are opera videos) of my questions will reveal a different approach that works better. Or maybe I should go back to Premiere… Or figure out how to get ZVE-10 II + Reaper on a Mac to start/stop simultaneously via clicking one remote control…

[The Documentary Sound Guy]

For multiclips, a lot of your questions can be answered by using the "Open multiclip in timeline" option from the context menu.  That will let you look 'inside' the multiclip so you can rearrange video tracks, extend audio tracks within the multiclip etc.  I've found audio within multiclips to be somewhat clunky, as the entire multiclip can only have one channel format, and it doesn't adapt if you switch 'angles' to a different audio track, and it doesn't let you combine or mix audio tracks.  I believe Resolve re-did how this worked in the latest version, so I may not be up-to-date.  You might want to experiment with using multiclips only for video tracks, and then auto-sync the your audio takes to the multiclip (as if it was a regular video clip).  This will give you a multiclip synced and linked with all your audio takes, and will let you manipulate all the audio tracks without being limited to having just one active audio "angle".

 

For subclipping multiple takes, can you not just cut up the single multiclip (with linked audio tracks) after you've made your multiclips and synced everything?  I don't understand your workflow well enough to know where your inefficiency is, but that is how I would do it.

 

A really important (and unintuitive) feature is the audio tab of "Clip Attributes", which is more or less how audio routing is controlled on a 'clip' level.  This lets you determine how Resolve sees the raw audio tracks embedded in a single video clip or polywave file, and helps identify stereo pairs vs. mono tracks etc.  You can also disable / hide unused or unneeded tracks to prevent them from cluttering things up in Fairlight.  Also important is the track format for each of the audio tracks on the timeline.  This should probably be set to whatever format you are working in (stereo?) for all tracks.  Resolve uses a not-entirely clear method for mapping channels between clip format and track format, but in general, as long as the clips identify your mono channels as mono and your stereo channels as stereo, everything should end up more or less where you expect it to.

 

I'm not super familiar with Auto Sync Audio, but hopefully the "Clip Attributes" mentioned above help sort out some of the issues.  For seeing audio filenames, you need to have the metadata tab open and the audio clip selected, and the filename is listed near the top of the metadata tab.  It's also listed in the bin itself next to the clip name (you may have to scroll over, as there's a LOT of data about each clip, and you may have to enable "File Path" by right-clicking on the data headers at the top of the bin).  Finally, if your audio is hidden in a multiclip, you will need to be selecting the audio clip *inside* the multiclip using the "Open in timeline" option I mentioned above".  This is probably another reason you may want to set up audio linked *to* the multiclip rather than embedded *inside* it.

[Larry Sanbourne]

Thanks for your reply! I'm upgrading Resolve from 19 to 20 and perhaps that will help some of this. 

 

I'd love to use the multicam clips for video tracks and sync the audio takes to the multicam clip. But I tried selecting a WAV and a multicam clip and I don't have the Auto Sync Audio option anymore - it only shows up with plain video clip + WAV. Am I missing something there?  It would be amazing to be able to manipulate all the audio tracks without being limited to audio "angles"…

 

For the subclipping, this works if I have a big video file with corresponding audio. Unfortunately sometimes we stopped the audio recording so artists could listen back, but we didn't stop all cameras since some were mounted high in inaccessible places. Ideally I'd have a way to remote start/stop all cameras and audio at the same time but I don't know how to do that (even with ZVE-10 II, which is supposed to be able to handle this but in practice the bluetooth connection is unreliable.)

 

Thanks again!

[The Documentary Sound Guy]

53 minutes ago, Larry Sanbourne said:

I'd love to use the multicam clips for video tracks and sync the audio takes to the multicam clip. But I tried selecting a WAV and a multicam clip and I don't have the Auto Sync Audio option anymore - it only shows up with plain video clip + WAV. Am I missing something there?  It would be amazing to be able to manipulate all the audio tracks without being limited to audio "angles"…

 

I may not have used Auto Sync Audio when I did this.  A quick way to do it without that would be to duplicate the multiclip so you have copies in sync stacked on top of each other.  You want one copy for the video, and one for each audio source.  Make sure the appropriate audio 'angle' is selected for each copy.  Flatten all of the 'audio' multiclips.  That gets you all your audio sources in sync with the video multicliip.  Unlink everything, delete the redundant video tracks and the audio track from the one remaining multiclip.  Re-link everything that's left.

You could probably also do this by only syncing one audio track into the multiclip, flattening only that multiclip, and then auto-syncing all the remaining audio tracks once you've got the first track in sync.  There's some room for experimentation to see what works quickest for you.

 

53 minutes ago, Larry Sanbourne said:

For the subclipping, this works if I have a big video file with corresponding audio. Unfortunately sometimes we stopped the audio recording so artists could listen back, but we didn't stop all cameras since some were mounted high in inaccessible places. Ideally I'd have a way to remote start/stop all cameras and audio at the same time but I don't know how to do that (even with ZVE-10 II, which is supposed to be able to handle this but in practice the bluetooth connection is unreliable.)

 

I would probably just create a giant multiclip for the whole day, and then cut it up accordingly.  You'll get blank angles where there wasn't any recording happening, but if you did have at least one camera recording the whole time, everything should end up in order.  If you have the option in the future, you might consider a timecode-based workflow, as you are pushing the utility of waveform-sync to the limits I think.