High Density Encoding for ARRIRAW Image files
Date & Time
Wednesday, November 11, 2020, 8:00 PM - 8:30 PM
Session Type
Technical session
Brian Gaffney

What is High Density Encoding? Codex High Density Encoding (HDE) is an easy to use tool for large format ARRIRAW workflows to control the ever-increasing data footprint of RAW data. Studios are demanding 4K and RAW image data to archive and future-proof the negative. Cinematic productions are in the process of transitioning from 2K to 4K (uncompressed in size) with camera manufacturers such as ARRI, offering up to 6.5K uncompressed images. High Frame Rate (HFR) and High Dynamic Range (HDR) add to this large format data footprint, leading to significant cost increases as well as storage and transmission challenges. Uncompressed RAW data costs more to transport and store than compressed ProRes files. An uncompressed Bayer pattern image can be reduced to a HDE file size not much larger than a corresponding ProRes 4444 XQ mezzanine file. Except that with Codex HDE, you can access the original pixel values that were encoded. Codex High Density Encoding (HDE) can be used to compress raw Bayer pattern data (e.g. from a digital image sensor) in such a way that decoded data is identical to the raw data. It has been widely tested and implemented with ARRI cameras and the ARRIRAW format with encoded files typically around 60% of their original size. HDE is designed to provide high encode and decode throughput for use with systems with high data rates (e.g. high resolution or high frame rate cameras), real-time playback, faster than real-time data transfers and low-latency decoding. A wide variety of third-party applications can be used to decode files containing HDE bitstreams for data management, transcoding, color grading and visual effects purposes making HDE an ideal tool for workflow Management optimization. The presentation will describes the structure of data encoded with Codex High Density Encoding (HDE), along with algorithms for decoding the stored data. It is the intent of the presentation to describe the structure and encoding of all fields in the HDE bitstream, detailing how 3rd party vendors and their applications can decode HDE bitstreams and correctly identify its structure. Recent details on SMPTE RDD51 for HDE will also be presented. Encoding results in ARRIRAW images that are typically 60% of original size. HDE works on Bayer pattern images of any size. Data reduction is comparable to best-in-class lossless codecs. Provides fast encoding and decoding speeds. Files, once decoded, are a bit-for-bit perfect match to the original file

Technical Depth of Presentation
The presentation will detail in a basic and simple way and describe the very complex math that is quite advanced behind this algorithm. This presentation compliments a recent SMPTE RDD submission, RDD 51 on the data encoding specification for HDE.
What Attendees will Benefit Most from this Presentation
Studio Management. Producers. Post Production and VFX Supervisors. Facility management and engineering teams who deal with RAW data during the course of workflow management on a digital; production. Cloud vendors and storage partners.
Take-Aways from this Presentation
1. Preserve RAW data while reducing storage demands. 2. Bit Exact Loss Encoding of ARRIRAW data. 3. SMPTE RDD 51 description on HDE encoding process.