Jump to content

Archiving Digital Media


Jeff Wexler

Recommended Posts

I found this article originally published in 2007. I do not necessarily agree with everything the author is saying but it is definitely food for thought.

New York Times: Media & Advertising

The Afterlife Is Expensive for Digital Movies

By MICHAEL CIEPLY Published: December 23, 2007

(LOS ANGELES) Time was, a movie studio could pack up a picture and all of its assorted bloopers, alternate takes and other odds and ends as soon as the production staff was done with them, and ship them off to the salt mine. Literally.

Having figured out that really big money comes from reselling old films — on broadcast television, then cable, videocassettes, DVDs, and so on — companies like Warner Bros. and Paramount Pictures for decades have been tucking their 35-millimeter film masters and associated source material into archives, some of which are housed in a Kansas salt mine, or in limestone mines in Kansas and Pennsylvania.

A picture could sit for many, many years, cool and comfortable, until some enterprising executive decided that the time was ripe for, say, a Wallace Beery special collection timed to a 25th- anniversary 3-D rerelease of “Barton Fink,” with a hitherto unseen, behind-the-scenes peek at the Coen brothers trying to explain a Hollywood in-joke to John Turturro.

It was a file-and-forget system that didn’t cost much, and made up for the self-destructive sins of an industry that discarded its earliest works or allowed films on old flammable stock to degrade. (Indeed, only half of the feature films shot before 1950 survive.)

But then came digital. And suddenly the film industry is wrestling again with the possibility that its most precious assets, the pictures, aren’t as durable as they used to be.

The problem became public, but just barely, last month, when the science and technology council of the Academy of Motion Picture Arts and Sciences released the results of a yearlong study of digital archiving in the movie business. Titled “The Digital Dilemma,” the council’s report surfaced just as Hollywood’s writers began their walkout. Busy walking, or dodging, the picket lines, industry types largely missed the report’s startling bottom line: To store a digital master record of a movie costs about $12,514 a year, versus the $1,059 it costs to keep a conventional film master.

Much worse, to keep the enormous swarm of data produced when a picture is “born digital” — that is, produced using all-electronic processes, rather than relying wholly or partially on film — pushes the cost of preservation to $208,569 a year, vastly higher than the $486 it costs to toss the equivalent camera negatives, audio recordings, on-set photographs and annotated scripts of an all-film production into the cold-storage vault.

All of this may seem counterintuitive. After all, digital magic is supposed to make information of all kinds more available, not less. But ubiquity, it turns out, is not the same as permanence. In a telephone interview earlier this month, Milton Shefter, a longtime film preservationist who helped prepare the academy’s report, said the problems associated with digital movie storage, if not addressed, could point the industry “back to the early days, when they showed a picture for a week or two, and it was thrown away.”

Mr. Shefter and his associates do not contend that films are actually on the verge of becoming quite that ephemeral. But they do see difficulties and trends that could point many movies or the source material associated with them toward “digital extinction” over a relatively short span of years, unless something changes.

At present, a copy of virtually all studio movies — even those like “Click” or “Miami Vice” that are shot using digital processes — is being stored in film format, protecting the finished product for 100 years or more. For film aficionados, the current practice is already less than perfect. Regardless of how they are shot, most pictures are edited digitally, and then a digital master is transferred to film, which can result in an image of lower quality than a pure film process — and this is what becomes stored for the ages.

But over the next couple of decades, archivists reason, the conversion of theaters to digital projection will sharply reduce the overall demand for film, eventually making it a sunset market for the main manufacturers, Kodak, Fujifilm and Agfa. At that point, pure digital storage will become the norm, bringing with it a whole set of problems that never troubled film.

To begin with, the hardware and storage media — magnetic tapes, disks, whatever — on which a film is encoded are much less enduring than good old film. If not operated occasionally, a hard drive will freeze up in as little as two years. Similarly, DVDs tend to degrade: according to the report, only half of a collection of disks can be expected to last for 15 years, not a reassuring prospect to those who think about centuries. Digital audiotape, it was discovered, tends to hit a“brick wall” when it degrades. While conventional tape becomes scratchy, the digital variety becomes unreadable.

Difficulties of that sort are compounded by constant change in technology. As one generation of digital magic replaces the next, archived materials must be repeatedly “migrated” to the new format, or risk becoming unreadable. Thus, NASA scientists found in 1999 that they were unable to read digital data saved from a Viking space probe in 1975; the format had long been obsolete. All of that makes digital archiving a dynamic rather than static process, and one that costs far more than studios have been accustomed to paying in the past — no small matter, given that movie companies rely on their libraries for about one-third of their $36 billion in annual revenue, according to a recent assessment by the research service Global Media Intelligence.

“It’s been in the air since we started talking about doing things digitally,” Chris Cookson,

president of Warner’s technical operations and chief technology officer, said of the archiving quandary.

One of the most perplexing realities of a digital production like “Superman Returns” is that it sometimes generates more storable material than conventional film, creating new questions about what to save. Such pile-ups can occur, for instance, when a director or cinematographer who no longer has to husband film stock simply allows cameras to remain running for long stretches while working out scenes.

Much of the resulting data may be no more worth saving that the misspellings and awkward phrases deleted from a newspaper reporter’s word-processing screen. Then again, a telling exchange between star and filmmaker might yield gold as a “special feature” on some future home-viewing format — so who wants to be responsible for tossing it into the digital dustbin? For now, studios are saving as much of this digital ephemera as possible, storing it on tapes or drives in vaults not unlike those that house traditional film. But how much of that material will be migrated when technology shifts in 7 or 10 years is anyone’s guess. (And archiving practices in the independent film world run the gamut, from studied preservation to complete inattention, noted Andrew Maltz, director of the academy’s science and technology council.)

According to Mr. Shefter, a universal standard for storage technology would go far toward reducing a problem that would otherwise grow every time the geniuses who create digital hardware come up with something a little better than their last bit of wizardry.

As the report put it, “If we allow technological obsolescence to repeat itself, we are tied either to continuously increasing costs — or worse — the failure to save important assets.”

In other words, we could be watching Wallace Beery long after more contemporary images are gone.

Link to comment
Share on other sites

Interesting and, in a way, scary.

I've always been a firm believer in film. We have a saying here in Sweden that goes "the old one is the oldest" which means, what has worked will work, what's right is right.

Senator has that great saying, "tripping over dollars to pick up pennies", please correct me if I'm wrong.

I'm sure someone will figure something out. I mean, it would be horrible if producers would have to actually PAY for the archiving of a film! Preposterous thought that.

Jokes aside, It's scary how much money talks. Film has a huge cultural value and impact as well. Something producers, and obviously some writers, fail to realize. Cus you can't write it into a budget... Sigh

Link to comment
Share on other sites

Let's put this in perspective:

Digital shooting has been around barely a decade. Heck, dense digital processing of _any_ type is only a few decades old. In archiving terms, it's all brand-new... and we haven't worked out all the problems yet.

Movie film has been around more than century. But how many of that first decade's films are still around? How many of the first quarter century, in fact? Nitrate self-destructed, formats died, early color processes lost both individual colors _and_ registration over time...

Or how much film sound from the first years of that medium have survived? Anybody regularly playing, or even finding, original optical production tracks?

Yeah, we've still got problems to solve. But I suspect solving the problems of digital archiving will be a lot faster than solving those of film (or paper books, before it). There's a lot of pressure to save all kinds of digital data - business, legal, academic, you name it - and our industry can reap the benefit.

---

We're all still early adopters, because the whole technology is still in its early stages.

(Anybody want to buy an almost new DA98? How about some timecode DAT decks? Or even my BetaSP deck?)

Link to comment
Share on other sites

I've seen a film format that records what looks like frame of video snow--digital info--it allows the old archive practice of doing BW color separations of the neg to be combined in a single strip of 35mm film (still the most known-stable archive format for movies).

In recording there are still arguments about what an archived session is, including flattened stems/files, lay off to 2 inch analog etc..

phil p

Link to comment
Share on other sites

I've seen a film format that records what looks like frame of video snow--digital info--it allows the old archive practice of doing BW color separations of the neg to be combined in a single strip of 35mm film (still the most known-stable archive format for movies).

In recording there are still arguments about what an archived session is, including flattened stems/files, lay off to 2 inch analog etc..

phil p

I was reading through the article thinking why something like this hasn't been implemented. In projection, the Dolby Digital or SDDS information is read off the film print - the data for it is read from next to or between the sprocket holes. We know that digital images take up a lot more data storage space than audio, but they have the whole 35mm frame to play with. Print each digital frame to some film, put it in old school storage. Might even keep the digital arm of Kodak, Fuji and Agfa alive for a few more years. Bonus!

200px-35mm_film_audio_macro.jpg

Link to comment
Share on other sites

  • 3 weeks later...

A local production company I know backs up their movies on LTO tape. Digital archiving is definitely a market with room for growth and will take time to mature.

LTOs can be cranky to use. It takes more than 4 hours to back up just 1.5TB of data to an LTO-5. Some current films (like Girl with the Dragon Tattoo) comprise about 55TB of data just for the basic film and 5.1 soundtracks. And you really have to go to multiple copies and store them in different places.

I also know of two major, major projects (one of which is an older film that made many hundreds of millions of dollars) where half of the LTO tapes went bad, and a lot of the visual work had to be redone from scratch. This is a very scary business, fraught with problems.

I can almost guarantee you that a brand-new piece of 35mm negative will play 50 years from now. An LTO tape... not so much.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...