Conformed original BWAVs are out of sync by up to a frame compared to the AAF

[Erve]

Lmao at this thread! But yes it should be corrected. A conform should be easy and precise with current technology. I think they could both be fixed without either company doing any major coding overhaul to their existing architecture.

It seems to me that MC is not adding data -unless at the end of the file where it really has no ill effect. It is instead sliding the sub frame file start point to the previous (or latter) frame start - thus the offset. It could simply notice sub frames coming in and add silent data (fill in) before the subframe to the preceding frame edge keeping its architecture within the application to frame edges only.

Also, couldn't zaxcom do the same without rewriting major code in the OS? Zaxcom recorders write to MARF files which are then converted to wav files using zaxconvert. Just tell zaxconvert to add non data (fill in) from the preceding frame edge to the first sample wherever it may lay in the subframe.

I think everyone should fix it.. MC should be able to understand sub frames and zaxcom should also start their files on frame edges... Because.. Why not? It would be more logical for zaxcom and more accommodating for ProTools.

Then again maybe I am missing something...

BTW - J.B. you are hilarious. Senator too (on this one!)

[pindrop]

Wouldn't non data fill in cause noticeable dropouts or are they so short one doesn't notice them?

[sonmi451]

Wouldn't non data fill in cause noticeable dropouts?

 

Yes. When the machine does a seamless changeover when the 2GB or 4GB maximum file size is reached (depending on your settings) in extra-long recordings. Another reason BTW why the other machines record at 00-frames. When you create a new cue on-the-fly it will seamlessly patch together with the previous one in the NLE.

 

Something that happens very rarely on features but can be helpful on documentary shoots where it's helpful to create new cues on-the-fly without stopping.

 

Since the deva doesn't create BWAVs audio before actually mirroring I'm sure there are options to treat the created BWAVs in all sorts of ways.

 

Alternatively the AVID could force-truncate the files on import but again: creating files on 00-frames in the first place would have other advantages as well (cross-jamming scenarios, changing iXML fps flags after the files have been created without changing I/O times etc etc.)

[chriswoolf]

My head is beginning to ache with the multiple confusions that keep recurring in this thread.

 

The first point is surely not to confuse "sync" and "timecode" - they may be connected but they are different elements. If sync between two machines (doesn't matter if they are picture or sound) is drifting, or "phasing" is occurring, then we can be sure they are not running at the same speed. The variation may be absolute speed errors or short term "velocity" errors, but it is still "speed". That has to get solved before anything else.

 

NLEs, by default, play everything in sample sync so we can guess that the errors that occur are most commonly down to location recording - and the usual balls-up is failing to have the camera and audio recorder clocks locked to each other. For sample-accurate stuff (yes, that's perfectly feasible) that demands cables and central locking to a house sync.

 

Plesiochronous operation - locking all recorders to super-accurate crystals with similar thermal drift patterns (Ambient Lockit and Deneke) - and plesiochronous-plus operation - the same system but with the addition of RF frame sync phasing (Blue Lockits or TC Buddy) - provide low drift ~speed~ errors but not sample accuracy.

 

This sync business has to be got right ~before~ timecode numbering is applied. In a great many cases it isn't. Free-running a Red camera and hoping that a Zaxcom Deva (for instances) will track without any assistance is a foolish act of faith.  They won't.

 

With plesiochronous systems sync will predictably jump a frame every now and again because there is no conversation between record machines, and the coincidence of frame boundaries cannot be guaranteed. This isn't drift - this is random single-frame offset that moves forward or back depending on the exact frame boundary point.

 

Timecoding as a frame address system works wonderfully if the sync issue is properly addressed as a priority. In practice the very haphazard (and usually ill-informed) method of attaching TC numbers on location - forcing unframed TC into a camera, using TC generators that have no relation to video or wordclock sync, failure to rejam recorders after power-down or varispeeding, etc - causes most of the sync problems we commonly meet. These have nothing to do with BWAV structures.

 

We do have some entirely functional standards for the positioning of a TC number in relation to video frames, and precise relationship of audio sample numbers to TC frames too. We also have some useful defacto ones such as starting and stopping on 00 numbers, which are not ~essential~ but massively reduce problems where TC rates, such as cross-jamming.

 

And of course, being European, I smirk at the fact that the US got frame rates in such a mess that they have to resort to drop-frame etc, but since I'm not aware that anyone uses non-integer codes on location I'm not sure they are quite the problem that people suggest.

 

But let's underline that synchronisation is a problem that starts at acquisition and ends at broadcast or release print. It isn't something that occurs in isolation at post, and it isn't a localised failure of the BWAV specification.

 

Chris Woolf

[pindrop]

My head is beginning to ache with the multiple confusions that keep recurring in this thread.

 

The first point is surely not to confuse "sync" and "timecode" - they may be connected but they are different elements. If sync between two machines (doesn't matter if they are picture or sound) is drifting, or "phasing" is occurring, then we can be sure they are not running at the same speed. The variation may be absolute speed errors or short term "velocity" errors, but it is still "speed". That has to get solved before anything else.

 

NLEs, by default, play everything in sample sync so we can guess that the errors that occur are most commonly down to location recording - and the usual balls-up is failing to have the camera and audio recorder clocks locked to each other. For sample-accurate stuff (yes, that's perfectly feasible) that demands cables and central locking to a house sync.

 

Plesiochronous operation - locking all recorders to super-accurate crystals with similar thermal drift patterns (Ambient Lockit and Deneke) - and plesiochronous-plus operation - the same system but with the addition of RF frame sync phasing (Blue Lockits or TC Buddy) - provide low drift ~speed~ errors but not sample accuracy.

 

This sync business has to be got right ~before~ timecode numbering is applied. In a great many cases it isn't. Free-running a Red camera and hoping that a Zaxcom Deva (for instances) will track without any assistance is a foolish act of faith.  They won't.

 

With plesiochronous systems sync will predictably jump a frame every now and again because there is no conversation between record machines, and the coincidence of frame boundaries cannot be guaranteed. This isn't drift - this is random single-frame offset that moves forward or back depending on the exact frame boundary point.

 

Timecoding as a frame address system works wonderfully if the sync issue is properly addressed as a priority. In practice the very haphazard (and usually ill-informed) method of attaching TC numbers on location - forcing unframed TC into a camera, using TC generators that have no relation to video or wordclock sync, failure to rejam recorders after power-down or varispeeding, etc - causes most of the sync problems we commonly meet. These have nothing to do with BWAV structures.

 

We do have some entirely functional standards for the positioning of a TC number in relation to video frames, and precise relationship of audio sample numbers to TC frames too. We also have some useful defacto ones such as starting and stopping on 00 numbers, which are not ~essential~ but massively reduce problems where TC rates, such as cross-jamming.

 

And of course, being European, I smirk at the fact that the US got frame rates in such a mess that they have to resort to drop-frame etc, but since I'm not aware that anyone uses non-integer codes on location I'm not sure they are quite the problem that people suggest.

 

But let's underline that synchronisation is a problem that starts at acquisition and ends at broadcast or release print. It isn't something that occurs in isolation at post, and it isn't a localised failure of the BWAV specification.

 

Chris Woolf

Thank you for your useful observations very informative - couple of things though - I've worked on PBS programs in Europe where 29.976 DF is specified as the working rate on location.

 

We also have some useful defacto ones such as starting and stopping on 00 numbers, which are not ~essential~ but massively reduce problems where TC rates, such as cross-jamming.

Think there might be something missing from this sentence so I can't understand it?

 

NLEs, by default, play everything in sample sync

I thought we'd established that Avid MC plays everything by whole picture frame sync which is the origin of what we've been discussing?

 

We do have some entirely functional standards for the positioning of a TC number in relation to video frames, and precise relationship of audio sample numbers to TC frames too.

As I understood it both Takev and Courtney said we didn't? But I could be wrong and perhaps they'll comment more knowledgeably than I.

 

The more this thread runs the more it seems to me that Zaxcom should just get on and address the issue by starting and stopping Zaxcom files on 00 of frames like SD and Aaton, which if nothing else, is a very functional workaround.

If it was easy I think Zaxcom would have already have done it. It has AFAIK been looked in to and it is possible but would involve a lot of work.

[Erve]

Chriswoolf - getting accurate TC on location is a separate issue (and important - yes) but I think less so, as sub frame accuracy is not probably that important (unnoticeable). Surely though, when the avid exports OMF AAF based on location files and then conformed in PT with same location files, it should be an exact match. While I agree that in years past it was always just part of post to deal with sync, it is kind of ridiculous as a process when it really doesn't need to be now. Imagine going through every single shot and adjusting every single region by unknown amounts - that is often thousands of files! All unnecessary... And less accurate.

By adding fill in data to the previous frame boundary would yes cause a momentary dropout for less than 1 frame on continuous takes, but the previous clip has that missing sound data so nothing would be lost. Just requires extending the last region to its end (less than 1 frame). Granted this is not perfect but far easier than resyncing every shot by unknown amounts on micro scale, and only ever required when conforming sound that lies on the first or last frame of the sound files.

I would not truncate files to frame boundaries because then there would be potential for lost data, but fill in data would accrue no losses.

So 99.9% of the time there will be no issue at all and problem is solved 100%. In the rare case that first or last frame of a sound file is used, then PT editor just drags out from the previous region. Seems like a far better situation that would save countless hours, no risk, but yes still a small and rare caveat that must be understood by post.. I'm sure they wouldn't mind considering how easy it is to extend a region boundary to its end and the fact they don't have to resyncing EVERYTHING anymore.. I have done this resyncing before and it does suck! It takes a long time, has potential for error, and doesn't need to be done (from a technological standpoint). I don't want to throw any work at us location sound mixers and these solutions would not - it would be unnoticeable to us and 99% better for post.

This actually covers us.. No more post questions; "why are your deva files out of sync" and no lost data. It is the best short term solution that is doable and has no real negative.

Of course if there was a complete recoding overhaul it could be solved in a better way but recoding to that degree is time consuming and expensive! I'd rather zaxcom be working on new gear and just fill in to previous frame boundary with zaxconvert and be done with it... Also MC do this same on audio file import. If I chose 1 it would be for MC to do this on import, then in PT there would not even be the caveat mentioned above.

So MC should definitely do this so their video and audio platforms can match - duh. Zaxconvert should have the option "Fill in silence to previous frame boundary" option incase MC or a different platform being used does not do this.

BUT

The more I know, the more I know that I know nothing.

(And Senator - the 99.9% statements above are not real figures - just expressions) haha

[pindrop]

Courtney has made the most succinct explanation of what SD and Aaton do - no dropouts, no backfilling with silence, no extending regions involved,

 

to quote -

"To avoid a lot of these errors and calculation problems Sound Devices used their pre-record buffer to backfill the start to the nearest second of the TC clock.  They also continue recording until the next second transition after you hit the stop button so each file starts and stops on an even second transition which should always make it better to verify you got the math correct when applying the frame rate calculation  for the time stamps."

 

Problem solved.

Is it really so difficult for Zaxcom to do the same?

[macrecorder]

That is the $64,000 question, Glenn?

[Erve]

Yes this could be done but seems to require a deeper level of recoding - thus time and money. Backfilling is not the only solution but potentially the most practical? And makes this whole "issue" benign.

There is ideal thinking and practical problem solving. You are thinking ideal - good - but don't hold your breathe...

[studiomprd]

" and did start the recordings on a :00 frame, "

and at '0' crossing

[Constantin]

I am so glad that more people have joined this thread who are technically much better versed in this matter than I am.

" and did start the recordings on a :00 frame, " and at '0' crossing

Really, zero crossing, too? Does that affect sync as well? Didn't know that.

[glenn]

I had no idea this thread was even here until I got a call asking me to look at it.

 

We are now looking into it to see if a change is possible. I am told it is not simple due to buffer sizes and disk cluster size. Also playback of audio between segments without gaps may be an issue.

 

I will contact AVID this week to see if I can convince them to do the right thing and fix it on their side. After all, with 30 years of TC referenced recordings from linear tape to DAT to Deva it is wrong to me to now say "we as an industry need to redefine the TC system as only being valid if a recording starts on a Zero frame and ends the last frame before zero frame." (Yes I quoted myself)

 

As pointed out here even if this is fixed by AVID or Zaxcom picture and sound will never be in perfect sync as both systems are 

usually asyncronus. No two camera types have the same picture to TC sync relationship.  Since genlocked cameras seem to be going away in favor of DSLR types I only think this will get worse not better.

 

In the defense of the SMPTE TC standard. I do not think they ever had a concept of file based recording in the 1960s. They did invent color TV and decided it would be good to run it at 29.97. Now that was a very bad choice.

 

After 17 years of Deva file based recording I think the concept of this AVID time code problem being a deciding factor in what recorder is used on a production is not worthy of consideration.

 

Glenn

[macrecorder]

Ok, but what about the proposed workaround which would be to do it, as an option, at the mirroring/zaxconvert stage? Is that easier to implement?

[studiomprd]

" They did invent color TV and decided it would be good to run it at 29.97. "

just to clarify,  it was NTSC that gave us the .1% solution for compatible (as required by the FCC) color television

[glenn]

" They did invent color TV and decided it would be good to run it at 29.97. "

just to clarify,  it was NTSC that gave us the .1% solution for compatible (as required by the FCC) color television

 

Fair enough. Still a bad idea.

 

Glenn

[studiomprd]

"Still a bad idea. "

very, but it seemed like the thing to do, at the time...

BTW, it was SMPTE that gave us "Drop Frame" timecode counting... bad name!

[Marc Wielage]

This sync business has to be got right ~before~ timecode numbering is applied. In a great many cases it isn't. Free-running a Red camera and hoping that a Zaxcom Deva (for instances) will track without any assistance is a foolish act of faith.  They won't.

 

I wouldn't say it's Zaxcom's fault. If you put two Red cameras side by side, jam one to the other, then hit record and stop all day long, I guarantee you, they will not be in frame-accurate sync with each other. Not without a genlock reference. 

[Erve]

Yes you are right Marc and Glenn and many others who have pointed out that PERFECT sync between cameras and recorders has many issues to be solved before it is realized, BUT I see this topic to be about something different - really it's about sync between MC and PT (both avid products) not being in PERFECT sync with each other when they exchange files - Which IS possible a few ways:

1) Location recorders deliver sound starting on frame starts (whether this be at time of recording or during mirroring in zaxcoms case)

2) MC to overhaul their software to allow audio at subframes

3) MC to backfill audio to previous frame boundary on import (BEST OPTION IMO)

X) Surely there are other ways I'm not thinking of...

But if avid refuses to act, it wouldn't hurt if backfilling auido was a selectable mirroring option for zaxcom recorders... Right? This wouldn't affect "disk clusters or buffers" and could save A LOT of work for posties. And even though its AVIDs compatibility problem, the problem does only exist when using zaxcom recorders so it may not be a bad idea for this OPTION (means selectable for certain circumstances where it makes sense) to be available.

I agree avid should fix things since its their products incompatibility, but someone should step up and solve it to make the sound industries workflow more efficient and more accurate. Think of the poor postie spending countless hours nudging waveforms into alignment by subframes!!

My 2 cents..

I think this horse might be dead? I'll check back to see if its still squirming.

[Constantin]

the problem does only exist when using zaxcom recorders.

That's not true, there are others, such as Boom Recorder.

[Constantin]

Yes you are right Marc and Glenn and many others who have pointed out that PERFECT sync between cameras and recorders has many issues to be solved before it is realized, BUT I see this topic to be about something different - really it's about sync between MC and PT (both avid products) not being in PERFECT sync with each other when they exchange files

Yes, that's right.

And many here have said that there are many reasons why cameras and audio recorders are never in perfect sync and this is just one problem of many. And that is true, but let me point out what Frank already said in post #124. this issue recurs in audio post, again and again, when camera-audio sync has long been fixed.

[Erve]

Yes, that is what I was saying...

[Constantin]

Yes, that is what I was saying...

I know, I was agreeing with you. Hence my first sentence under your post: "Yes, that's right"

[Erve]

Yeah, that's what I said!

Just messing with you - haha

[chriswoolf]

... No two camera types have the same picture to TC sync relationship.  Since genlocked cameras seem to be going away in favor of DSLR types I only think this will get worse not better...

 

Glenn

 

Firstly, if TC is practised correctly picture frame and TC ~do~ have a defined relationship - sync word starts at the frame boundary. The only reason this doesn't always happen is when recordists try to force timecode numbers without paying attention to genlock.

 

Secondly, there are an awful lot of professional level cameras out there with proper genlock facilities, not least because of the need to lock them for 3D or or other synchronous picture purposes, What's more, Ambient, Deneke and Timecode Buddy provide systems that can achieve properly phased timecode without a problem - it's just that recordists/camera departments with limited engineering knowledge are scared stiff of using these facilities.

 

At a recent Timecode Training day we had some superb samples of a day's files played on an Avid and showing the classic frame sync hopping of non-genlocked camera / separate sound system. The drift of the plesiochronous system was nominally quite low but increased as the devices warmed up through the day.

 

 

I wouldn't say it's Zaxcom's fault. If you put two Red cameras side by side, jam one to the other, then hit record and stop all day long, I guarantee you, they will not be in frame-accurate sync with each other. Not without a genlock reference. 

 

I wasn't aiming to assign blame anywhere - my comment was "instances". No two crystal controlled devices will ever run in lock. The best plesiochronous systems might achieve <0.5ppm errors but they will never be sample locked. That can only ever be achieved using a single reference. The best that can be achieved without cables currently are the "plesiochronous plus" systems that use close-tolerance crystals backed up by a wireless network to give frame phasing. The current precision is ~10microseconds - good, but not sample accurate.

 

I do understand the thoughts about Avid sample shifting, and Zaxcom 00 framing, but unless (near) synchronism is achieved at the initial stage everything in post must inevitably be a case of fiddling something visually/aurally acceptable - there is no "correct" reference to work from. You can reduce (but not remove) drift by using better crystal oscillators (and some machinery is much better than others) but you can't prevent frame hopping unless you use genlock.

 

Chris Woolf

[Constantin]

unless (near) synchronism is achieved at the initial stage everything in post must inevitably be a case of fiddling something visually/aurally acceptable - there is no "correct" reference to work from. You can reduce (but not remove) drift by using better crystal oscillators (and some machinery is much better than others) but you can't prevent frame hopping unless you use genlock.

I do have a feeling like we went over this already, but no one is talking about absolute perfect sync. No one says we need this now. And if they were, they woudn't be after reading this thread.

The point is to avoid having to fix the same problem twice, once in telecine, or at the assistant editor's station or wherever, and the twice in sound post after re-conform. And maybe even once more after the foley session, and then again after... It goes on. Frank really said it all in his posts.

I don't know why we have to keep going on and on about the fact that no two devices will easily ever run in sync and that there are many other potential sync problems. ... unless I am totally mistaken, which is a possibility