Jump to content
joshneal

Recording 360 sound for Oculus Rift in a narrative context

Recommended Posts

  I recently received a call from a production company I've worked for in the past that is starting a new project to record sound for the Oculus Rift virtual reality headset.  Typically the Oculus Rift is used for 3D/360 gaming, but this project intends to record 360 video either doc style or narrative.  The example of the first scenario they want to capture is inside of a locker room with a coach talking to players.
  Being that this is fairly uncharted territory for video production, these producers have asked me if I have any thoughts on how to best record audio for this type of shooting situation.  I've looked into binaural recording, however this would only work for one perspective - if the viewer turns their head 90 degrees the binaural microphones would also have to shift with the viewers movements to match the perspective - this isn't possible to record in the field as far as I can tell.  You would almost have to have a binaural pair of mics for every 15 degrees or so to capture binaural audio in 360 similar to the way the cameras are working in a 360 array.  But the problem is I don't think the software exists to stitch it all together - at least not from what I can find so far, please tell me if anyone knows otherwise.
  The way that individual sounds are modeled to give them a 3D perspective is that each sound is recorded clean normally and then placed into a 3D modeling software like Unity 3D and based on its x,y,z position the dynamics of the sound are manipulated by the software so that it sounds in perspective to its position relative to the viewer.  This works well for individual sounds - sound effects and whatnot.  But what I'm trying to figure out is how, using some sort of configuration of mics/equipment, to record 360 room ambience that can also match perspective based on which direction the viewer's head is turned.
  Forgive me if this is confusing or if I'm asking for a solution to a situation that barely exists yet, I'm just throwing this out there to see if any of you might have any ideas or be able to point me in the right direction.

Share this post


Link to post
Share on other sites

Soundfield mics. I think they've been around since the 80's or earlier. (Total guess)

http://www.tslproducts.com/soundfield-type/soundfield-microphones/

The kit to record periphonic surround sound definitely does exist. Plus they sound great and are good to use. Integrating it with programming/movement information from a VR headset is a different matter. Love to try the end result.

Stuart

Share this post


Link to post
Share on other sites

If you are only interested in lateral head turns, an IRT Cross (aka IRT Square) mic configuration (4 Cardioid mics with their capsules 90deg to each other on the corners of a 20cm square, from memory - check the Schoeps website) may work as well as a Soundfield or similar tetrahedral mic for capture. It's the subsequent processing for head tracking that they'll need to sort out. You may be able to feed the 4 signals into the existing object-based panner you described above as four fixed locations which it will rotate in response to whatever head tracking system it uses. Fascinating project.

Share this post


Link to post
Share on other sites

Sorry if this appears twice or thrice in various forms - my 'puter seems to be playing up.

 

Not able to answer your question - the question being what subsequent processing might be used on whatever multichannel recording system is used, but since Ambisonics and IRT are brought up (and IRT is just one cardioid configuration of several that Williams describes in his multichannel angle/distance ratios) - you might want to have a look at the TRINNOV system.

 

8 track omni in a set configuration with subsequent processing to output multichannel surround. Based in Paris I think, but English info and tech stuff available on their web site. Interesting.

 

Jez Adamson

Share this post


Link to post
Share on other sites

I think recording this is even the easy part. It's the playback with the head tracking that's tricky. As mentioned, Beyerdynamic's system is probably your best bet.

Share this post


Link to post
Share on other sites

Just to add that I've had a Soundfield mic since 1989 and have been involved with Ambisonics since 1979 and these days there's a wealth of versatile software available, quite a lot of it free, and a wealth of experience from academic institutions around the world. Low cost mics are available from Len Moskowitz at http://www.core-sound.com and from Brahma at http://www.embracecinemagear.com/brahma-ambisonic-microphone.html although their web-site's a bit temperamental.

 

The original Soundfield microphone company was bought out by TSL at http://www.tslproducts.com/ but because this is a large organisation, the personal service that one used to get from Ken, Pieter and Richard at Wakefield is now rather lacking. A colleague was quoted twelve weeks for a replacement cable for her SPS200 recently, which is not exactly user-friendly.

 

Currently, I use an ST450 and a Core Sound TetraMic: both have their uses. The ST450 outputs at line level, has a full Rycote set-up and is used for all my outdoor recordings, which covers about 85% of the recording work I do. The Core-Sound is tiny, easily hidden in a live music recording situation, for example, but has a higher noise level and needs good mic pre-amps to fulfill its full potential. It's also rather more difficult to isolate from wind and vibration, although Rycote will make you a modified Lyre suspension for it and it will fit inside an extended BBG.

 

There's a reasonably up-to-date list of software available here: http://en.wikipedia.org/wiki/List_of_Ambisonic_Software.

 

Regards,

 

John

Share this post


Link to post
Share on other sites

I have a SPS200 and I couldn't be more happier with this microphone, sounds great, is small (Can fit in any regular zep) and when is combined by Harpex B software have unlimited possibilities. 

Share this post


Link to post
Share on other sites

If you have the time, you might want to send your SPS200 to Len at Core Sound who will produce a set of calibration files for your microphone that will improve its performance. The only downside is that you'll need to use an extra plug-in before Harpex, but those who have had it done are very pleased with the results. He can do it for Soundfield Mk IVs as well.

 

Regards,

 

John

 

I have a SPS200 and I couldn't be more happier with this microphone, sounds great, is small (Can fit in any regular zep) and when is combined by Harpex B software have unlimited possibilities. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×