future of foley?

[Shastapete]

 

[Philip Perkins]

OK.  So this gives us a more or less appropriate sound for the picture.  I wonder if these scientists understand that Foley (among other sound-add tasks in audio post) is an interpretive art, expected to deepen and reinforce the director's vision of the story?  Like it's not "paint-by-numbers"?  I hope they do.

[Constantin]

Also, I've seen only a few movies in which someone walks around with a drumstick beating things.

[Mobilemike]

Foley is so much more than just "sound goes here". It can enhance a soundtrack in amazing ways by not being literally accurate  to what's in the picture. That's one of the major skills of a great Foley artist - interpreting the sounds to enhance the emotion of a film much like a musical soundtrack does. You can make things sound larger then life, or scarier, or older, or more modern, or wetter, or grosser than they appear in the picture just by varying the sound effect you hear for the visual. That's the kind of thing a computer will never be able to reproduce - the creativity of an artist's craft. If you want your film to sound bland and cookie cutter though then I'm sure this is an excellent tool

-Mike

 

[Shastapete]

I agree with all of you guys on the art of Foley, it is a difficult craft to do well.

When watching the clip I had a "video game sound" vibe, where sounds synced temporally but didn't sync emotionally. The flip side is that I'm amazed at the creations by computer neural nets. We are now seeing some of the first creative works done by computers. 

I hope you've all watched the surreal "Sunspring"

It is interesting where the future of creative works is going

[Philip Perkins]

Like a bad dream.....

 

[Mike Robertson]

I worked on that - the table read went more like a comedy. I was surprised  they went dramatic with it. The stage direction and dialog got pretty hard to follow but I thought everyone did a good job going along with it. The song at the end was also written by AI, for lyrics - I think it did great since they are more subjective anyway.

 

Back to the original post - I think having a algorithm automatically place footsteps in sync would be very helpful. 

[Jim Feeley]

That "Visually Indicated Sounds" video at the top is from MIT’s big Computer Science and Artificial Intelligence Lab. I've been there; they do some hip stuff and while they have a finely-attuned sense of the value of hype, they're not as gee-whizzy as the Media Lab. 

The slightly breathless MIT news release on the work:

http://news.mit.edu/2016/artificial-intelligence-produces-realistic-sounds-0613

The homepage, which includes the video Pete posted as well as a PDF of their paper (which I skimmed...neat, but some tricky mathy bits), and a couple other  things:

http://vis.csail.mit.edu

A caveat: :-)

"Our data was collected us- ing two wooden (hickory) drumsticks, and an SLR camera with a 29.97 Hz framerate. We used a ZOOM H1 external audio recorder, and a Rode VideoMic Pro microphone." 

 

BTW- one of the professors involved in this project also worked on (or oversaw) this cool work:

The Visual Microphone: Passive Recovery of Sound from Video

 

[IronFilm]

I blogged recently on AI in film:
http://ironfilm.co.nz/scifi-short-film-written-by-ai/

At the moment the skill of AI is like a toddler, not great!
But AI will quickly grow up (for instance just look at AI player chess, or more recently Go), and before we know it as good as a 9yo kid, and then as good as a young teenager, then as good as a (non-skilled) adult! After that is only a matter of time before it takes on professionals.