Jump to content
M Sisco

Delivering files from multiple recorders

Recommended Posts

9 hours ago, Bouke said:

Guys,

I just made something that can do exactly this. (It works under lab circumstances here.)

Input mono or poly files, output one big poly.

It does not matter if the files start at the same time or not.

Track names / other metadata is retained, in both the BEXT and iXML chunk.

@pindrop, if you can elaborate on what you wolud like to see, I can probably build it in.

You can reply here (so others might add,) or drop me a line directly.

I hope to have a beta version on my site tomorrow.

 

Bouke

I would suggest having the option to add silence at the heads and tails of files shorter than the longest. I can see how on certain instances it would be better to heir on the side of caution and not "remove" any data for the sake of a common start/stop point. 

 

That said, it would also be nice to have the option to select a "master" track and have your app trim all other tracks to match it. This would be great for those using bodypack recorders in conjunction with a main bag or cart recorder, that would normally be recording continuously and independent from each other. Example of this would be Zaxcom TRXs recording on transmitter boot up.

Share this post


Link to post
Share on other sites
On 8/31/2017 at 5:05 PM, vale said:

 

ALE export?

Vale,

That is not difficult, but I fail to see the point.

Could you elaborate on why you want this?

Avid imports a lot of metadata already. What does it not do then so you would need the ALE route?

 

Meanwhile, I have a working version for both Win and Mac.

https://www.videotoolshed.com/product/bwfmerge/

 

 

 

Bouke

@all,

It took some time, as I wanted it to be good, and sample accurate. (And i've managed to do so!)

I have a working version.

It accepts all kind of input files (mono, poly, even sr / bd can differ), and lets you merge them into poly files.

Metadata is retained for all inputs (well, channel name. Comment is taken from the first / most important one.)

You can also set the output patching if you want to re-order the tracks.

 

You can choose if you want one recorder to be 'master', and have other recordings trimmed to that in/out,

or have (a gazillion) different files, where as long as there is overlap between any files all will become one large one.

Multiple stop/starts of a rogue recorder is supported as well!

 

Get the demo, toy with it, feed me with anything you like to see!

(It's working, but still I like to add a lot more, but I need input!)

 

Last but not least, should I make a new thread about this?

 

Bouke

 

 

 

Share this post


Link to post
Share on other sites
On 09 settembre 2017 at 4:45 PM, Bouke said:

Vale,

That is not difficult, but I fail to see the point.

Could you elaborate on why you want this?

Avid imports a lot of metadata already. What does it not do then so you would need the ALE route?

 

I really don't know why as well, but it happened that we were asked to provide ALE reports (i.e. for a couple of Netflix project last month). We told them that all the metadata were written on the audiofiles and on our digital sound reports but they still wanted the ALE report. So, since you're starting from scratch, I was asking.

 

Thanks anyway, I will try the demo.

 

V.

 

Share this post


Link to post
Share on other sites

ALE , PDF and CSV have all been  offered as options on Cantar sound reports since the first Cantar X1 sound report function was introduced. They must have had their reasons. A sound editor I know has always asked for ALEs - they are helpful in some way.

 

 

Share this post


Link to post
Share on other sites

Hi guys,

I'm highly interested in the why of ALE use. (Maybe in the old days it served a purpose, but nowadays...)

One big problem I've encountered with Avid is the UMID memory.

It is hard to test multiple outputs if the UMID does not change, as Avid (thus ProTools as well I assume) will take metadata from it's database rather than the file.

That's not bad for normal production, but a PITA when testing.

(If you output with Master selected, import that and do the same test again without, you may NOT see what's actually there.)

So, next step is to generate a new UMID. If someone has specs on what kind of UMID is used for BWF, please tell me!

 

Meanwhile, version 0.9.1 is out that at least upnumbers the UMID by 1 point from the source UMID

(And there are a few other improvements.)

Get it here:

https://www.videotoolshed.com/product/bwfmerge/

And there are a few screenshots explaining what the 'master' function does.

(in short, trimming / padding  all extra input to one recorders clip in/out.)

 

Bouke

 

Share this post


Link to post
Share on other sites
On 31. August 2017 at 7:05 AM, RPSharman said:

So, in short, if we want to deliver one CF card with 9+ tracks in one Wav Poly file per take, then we just need a machine with more tracks.

 

In breathless anticipation of the new 716 or 816 (funny) from Sound Devices.

 

Or you can now download Bouke's new software. I have worked with it for a while now and it's working really well. I had this production where I needed more tracks than one 788T could offer, so I C-Linked my second 788T into one larger recorder. I actually physically linked them, too, which worked really well. See the picture below.

This means they were in perfect sync regarding Timecode and sample clock. Both recorders had an external hard drive connected to and at wrap time, all I needed to do was take those two hdd's and connect them to my Laptop, open Bouke's software, choose both source folders, define a target folder and hit "make poly". Depending on file sizes (obviously), the program needs a few minutes, meanwhile I could collect my lavs and other things. Once the merge is done, the program even creates a sound report for the new large files and drops that into the target folder. Then I'd just hand the drive to the DIT who would copy just the one folder like he always would. For post, these are just large polywavs, so nothing noticeable for them. 

For those with little time at wrap, the merging process could be moved to either the DIT or the assistant editor. In any case, it's only a matter of minutes. 

 

For me, this has breathed new life into my 788t's. Now I can calmly wait for the next recorder to suit my needs, while I can still comfortably record up to 16 inputs, almost as easily as if it were one recorder. 

 

Bouke said that this will also work with two different recorders, which are linked by TC only, so for the OP, this is the solution he is looking for. 

 

IMG_7008.jpg

Share this post


Link to post
Share on other sites

That is cool!! I now wish I still had my other 788T - although I could probably re-buy at less than I sold it for.

 

Does it merge the metadata, like track names and such?

Share this post


Link to post
Share on other sites
1 hour ago, RPSharman said:

That is cool!! I now wish I still had my other 788T - although I could probably re-buy at less than I sold it for.

 

Does it merge the metadata, like track names and such?

 

Yes, all metadata is retained from both files, especially track names. I only input notes on one recorder, as that is enough of course. So after the merge you have maybe 10 tracks from recorder 1 and then tracks 11 - 15 from recorder 2, all with their own track names. 

You could also do a mixdown on each recorder. In my case I didn't do that, though. I had a mix on recorder 2, but didn't arm the mix tracks. I routed the mixdown into tracks 7&8 of recorder 1, but also didn't arm those tracks, instead I included these in the mixdown on recorder 1, and thus had a regular mixdown and Iso's of all inputs. 

 

Here is the link to the program. The website is well worth a visit in any case

 

https://www.videotoolshed.com/product/bwfmerge/

 

 

Share this post


Link to post
Share on other sites

Hi Bouke,

 

I'm curious how BWFmerge would behave if the 'Master' folder contained several short takes, and the other folder contained TC Sync'd longer takes that overlap several of the 'master' takes. This is typically what would happen when one is using one of the new body-pack sized recorders (such as a Lectro PDR, the little Tascams or similar) on some talent who you may be having trouble covering with a wireless mic, while recording other talent with a boom or wireless into your main recorder - such as a Sound Devices 788. . You normally jam sync the PDR recorder before fitting it to the actor, after which you may do several takes and setups of a scene before getting the PDR back from them and stopping it (other readers, please refrain from mentioning the Zaxcom solution to this - it's not relevant to my situation). Would BWFmerge be able to slice chunks out of the PDR Recorder's single long file to merge with the master Poly BWF individual takes from the main recorder? If not, it might be worth seeing if you can add this function. It would be a big deal in making the use of these little recorders something that requires less work in Post, as while it's pretty easy for a proper audio post person to handle, it's a big task for a picture edit assistant, and they may need that performer's track for the edit long before a Sound Professional is involved.

Share this post


Link to post
Share on other sites

Regarding the ALE thing. Avid uses the BEXT chunk to derive track names. Only the first 8 track names will be in the BEXT chunk on account of size limitation. So - if you have poly files with more than 8 tracks, Avid will only load the first 8 track names. If you import an ALE file into Avid, it can find the rest of the track names. Yves-Marie Omnes and I have been 'working on' Avid to try and get them to use iXML, and they would then be able to import all the track names.

 

Bourke - if I sent you some mono tracks from an Audio Ltd 1010 digitak radio mic transmitter - might you have a look at them please? They dont have proper start times, but I believe the TC value is written into the filename - would you be able to incorporate these into a poly merge with other files?

 

Many thanks, Simon B

Share this post


Link to post
Share on other sites

@Nick,

Sure, If I understand you right, it can do this. Just get the demo and toy around.

@Simon,

Afaik, Avid uses iXML when you AMA instead of import.

Not sure about the 8 channel limitation, but there is an option to disable stuff from the BEXT chunk to use all 256 chars for track names as much as possible.

I'll have some other stuff to do now, will look again in a couple of hours.

You can send me testfiles, but why not toy yourself with the demo? (And, if you have files without proper timecode, what do you expect that will happen :-) ? However, I could read TC in filenames, but that will be a different project.)

 

@all, I'm still developing. Current version over here can go over the 4 gig limit by creating RF64 files, and there will be a proper manual. Hope to have a newer version online at the end of the day.

2 hours ago, nickreich said:

Would BWFmerge be able to slice chunks out of the PDR Recorder's single long file to merge with the master Poly BWF individual takes from the main recorder?

 

To make it clear, you can sync the whole shebang if you don't pick a 'master' folder in BWF merge, or pick a the master as master folder.

If you pick the short individual takes as a master folder, (they can live inside different folders if you must), the short clips will become 'master', and yes, the long file will be plucked where needed.

The timeline function will show you!

 

Share this post


Link to post
Share on other sites
On 10/13/2017 at 7:04 AM, Constantin said:

 

Yes, all metadata is retained from both files, especially track names. I only input notes on one recorder, as that is enough of course. So after the merge you have maybe 10 tracks from recorder 1 and then tracks 11 - 15 from recorder 2, all with their own track names. 

You could also do a mixdown on each recorder. In my case I didn't do that, though. I had a mix on recorder 2, but didn't arm the mix tracks. I routed the mixdown into tracks 7&8 of recorder 1, but also didn't arm those tracks, instead I included these in the mixdown on recorder 1, and thus had a regular mixdown and Iso's of all inputs. 

 

Here is the link to the program. The website is well worth a visit in any case

 

https://www.videotoolshed.com/product/bwfmerge/

 

 

Beauty - thank you sir - this is very useful

 

Share this post


Link to post
Share on other sites

Ok,

Version 1.0.3 is now on my site.

RF64 support, proper manual.

ToDo is ALE.

 

4 hours ago, Bash said:

Only the first 8 track names will be in the BEXT chunk on account of size limitation.

 

This is not true. (At least not on my Avid.)

On Import, BEXT is used, and if XX track names are inside the first 256 bytes of the BEXT chunk, they will import.

On AMA link, iXML is used for metadata. But, consolidating a Poly Wav takes way more time than a regular import, hence I will add ALE support.

(so you can import the ALE, then batch import the files and the ALE metadata will be used.)

 

Any more wishes?

 

Share this post


Link to post
Share on other sites

Warning, 1.0.3 has a bug where the channels get shifted!

So, if you currently have 1.0.3, trash it and get 1.0.4 from my site!

https://www.videotoolshed.com/handcrafted-timecode-tools/bwfmerge/

(When adding RF64 support, I needed 24 bytes more in the header, those got trimmed off the sound data. If your output was a multiple of 8 channels, you would have missed one sample, but in all other cases the channels got shifted where the track names stayed intact.)

 

 

Share this post


Link to post
Share on other sites

1.0.5 is on my site now, with ALE export next to the sound report.

(And some info about it in the manual.)

 

I would like some feedback to make this thing a proper replacement for WaveAgent though.

 

Share this post


Link to post
Share on other sites

Hello,

 

If I have two CF cards from two different 788T recorders, not all files will exist on both recorders. The common files will have same file name and start and end points (exact) as the machines will be linked. I will only roll two machines if there are more than 7 ISO characters and a mix, however. I don't have the setup currently to try out this demo. What exactly is the procedure to merge files from one CF to the other, or must you pick a third destination to send the files to?

Share this post


Link to post
Share on other sites
6 hours ago, RPSharman said:

Hello,

 

If I have two CF cards from two different 788T recorders, not all files will exist on both recorders. The common files will have same file name and start and end points (exact) as the machines will be linked. I will only roll two machines if there are more than 7 ISO characters and a mix, however. I don't have the setup currently to try out this demo. What exactly is the procedure to merge files from one CF to the other, or must you pick a third destination to send the files to?

 

Yes, you can pick the destination folder which can be wherever you want it to be. 

If there are some files which don't have a TC overlap with other files (within the chosen folders), they won't get touched and you'd need to make sure they still end up in the destination folder. 

Also, they won't make it into the sound report I think, but I'm not sure, I haven't tried it. Bouke?

Share this post


Link to post
Share on other sites
23 minutes ago, Constantin said:

Also, they won't make it into the sound report I think, but I'm not sure, I haven't tried it. Bouke?

Not yet.

We've talked about it to include a 'also copy no-match files' options but I haven't done it yet.\
(It's not that hard though...)

Share this post


Link to post
Share on other sites

I have 2x USB ports on my laptop. If I take a CF card from each recorder, plug them into a reader in each port, can I pick the destination folder to be on one of those two cards? Of will I need to create a destination folder on a third piece of media or on the laptop hard drive?

 

Realistically, I would have the data transfer person on set load the program and just do this all for me, but only on long projects. On short jobs, I'd want to do it on my own.

Share this post


Link to post
Share on other sites

Yes, just try it.

But, is an SD card faster / more convenient than a harddisk?

Even if you don't have two recorders, just record a clip twice with the same TC (or about, as long as there is overlap it will work.)

Or duplicate a clip to a second card.

This  (and the manual) will answer all your questions!

 

Share this post


Link to post
Share on other sites

Yes, it shouldn't really matter if it's hdd's or cf's. I used HDD's, because they held 1TB and I could have everything on one HDD: both recorders's files, and the merged files. And I could take them home for a quick backup. 

I bought Lacie Rugged drives, as they have one option with both a Firewire 800 port and a USB3. So I could hook it up to the 788 which had a free firewire port and to my Laptop which only has USB ports. 

 

Bouke, I think the auto-copy feature of non-merge files would be quite useful. 

On the one hand, just so that those files won't be forgotten and on the other hand to have them included in the sound report

Share this post


Link to post
Share on other sites

Hi Constantin (and others)

Building in copy of non-merged files now.

But, having a bit of trouble on how it should behave.

 

If you use a master folder,

If there is no patching, and the 'rogue' files live in the master folder, it's no problem.

(the sound report / ALE will report the track names starting from 1 and up number)

If there is patching, and the 'rogue' files live in the master folder, there is also no problem as the files will be patched.

 

BUT, if the files live in a non-master folder, what to do? Add silent tracks? Re-assign channel index?

(That shows the track name as correct, but Avid does not set the patching by itself, so I doubt that will be useful)

Adding silence tracks seems the way to go, but makes for unneeded bigger files.

 

So, what to do?

(And the patching becomes really unpredictable if the setup changes inside a batch....)

Share this post


Link to post
Share on other sites

I had my recorders arranged in a master/slave setup. The tracks recorded are sorted in a descending order of importance. So the Mix tracks are tracks 1&2, boom on 3, lead actor on 4 and so on. I think most of us arrange their tracks in a similar fashion. That means I will always record to recorder 1, but not always to recorder 2, so 1 is in every sense the master, and will also become the master folder in your program. So I would say there shouldn't be files in the folder for recorder 2 if there aren't any in the folder for recorder 1. 

 

I can't think of a scenario where this might be different. 

 

I had my recorders arranged in a master/slave setup. The tracks recorded are sorted in a descending order of importance. So the Mix tracks are tracks 1&2, boom on 3, lead actor on 4 and so on. I think most of us arrange their tracks in a similar fashion. That means I will always record to recorder 1, but not always to recorder 2, so 1 is in every sense the master, and will also become the master folder in your program. So I would say there shouldn't be files in the folder for recorder 2 if there aren't any in the folder for recorder 1. 

 

I can't think of a scenario where this might be different. 

 

Share this post


Link to post
Share on other sites
3 hours ago, Constantin said:

 

I can't think of a scenario where this might be different. 

 

No, you don't but I can imagine that Nick does something different.

(Rolling long takes with his master setup, where some body packs (not master) roll short takes, and might NOT overlap the master setup.)

 

Nevertheless, it now copies / patches all the files, no matter the overlap, and it will set patching correct, and will put everything in the ALE / Sound Report.

I've also added warning colors for custom patching / double channel patching.

So, new version on my site.

There will be more to do. I've got testclips from what I think is a steam powered 844, where chunks are un-even sized in the chunk size, but padded with a zero. (so current version refuses to load them...)

That will be fixed tomorrow...

 

Another thing, how do you add comments? Do you have a keyboard attached?

I could make something that someone else creates a file during the shoot that can be merged with files so you don't have to worry about that.

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×