A cloud post-production workflow: edit proxies, Hot and Archive storage, and conform pulls

Most of the shows we work with used to keep every card on a RAID for the life of the show. That works, but it means paying for fast storage on terabytes of footage nobody touches between the edit and the online, and it ties the show to wherever the RAID physically is.

On CloudStudio a show moves through four stages instead. Each stage reads different media, and each one uses the storage tier that fits it. This post walks through the whole workflow. The linked posts go deeper on each step.

The workflow at a glance

1. Ingest: camera cards upload to Hot storage, and edit proxies are made during the upload.
2. Offline: editors cut on the proxies from Hot, on CloudStudio workstations.
3. Archive: once editorial is working from proxies, the camera originals move to the Archive tier.
4. Online and color: when the cut locks, a conform pull restores only the originals the timeline uses, back to Hot.

After delivery, media the show has to keep can move to the Deep tier.

1. Ingest: camera cards to Hot, with edit proxies

Camera media uploads to Hot storage through the transfer agent. In the same transfer, you can turn on edit proxies and choose DNxHR LB for Avid or ProRes Proxy for Premiere Pro and DaVinci Resolve. When the upload finishes, our media workers make the proxies and write them to a Hot storage pool, so nobody has to transcode cards overnight on a spare machine.

A proxy hour is about 16 to 20 GB. A 4K ProRes 4444 hour is about 450 GB. Read how edit proxies work.

2. Offline on proxies

Editors mount the proxy pool on their CloudStudio workstations and cut as they would from a local drive. Because the proxies are small, many editors can play several streams each from the same storage, and the full proxy library stays online for the whole schedule.

3. Camera originals move to Archive

Once editorial is working from proxies, the camera originals don’t need to be on Hot. They move to the Archive tier, which costs much less to keep media on. Every file is checked against its Hot copy before the move counts as done, and in our own moves we have measured 446 to 686 MB/s. Nothing changes for the editors, because they never opened the originals.

Read how Hot, Archive and Deep storage are split.

4. Online and color: the conform pull

When the cut locks, the post team uploads the XML from the timeline. The conform pull matches each proxy in the cut back to its camera original, including the extra segments RED splits long clips into, and restores only those files to Hot for the online editor and the colorist. Nothing is transcoded or rewrapped, so the files keep their original names.

On one docuseries, an episode’s online was 505 files and 3.88 TB out of a 115.6 TB show archive. It was back on Hot in 2 hours 34 minutes, with every file checked against its archive copy. When the v2 cut came in two days later, it needed 4 more files. Read the full job numbers.

After delivery: Deep

Shows that must keep camera originals and masters for years after delivery can move them to the Deep tier, which is tape-backed and the cheapest place to keep media. Restores from Deep take longer and are arranged on request.

What changes compared with a RAID per show

The show stores all of its media for the whole schedule, but only the working set sits on fast storage.
Proxies are made once, where the footage lands, instead of on each editor’s machine.
Editors and colorists on CloudStudio workstations mount the same storage, so there are no drives to ship.
The online brings back only what the cut uses. When a new version of the cut arrives, only files that weren’t already restored need to come back.