The HDR pipeline

One master

The engine has several HDR policies, and each one takes its numbers from the caller. Tone mapping a PQ file down to SDR needs an operator, a mode, a source peak and a target white; placing an SDR file into PQ needs an operator and a peak. None of them has a default. For hosts that want the engine’s own answer, there is one built-in path,ColorPolicy::Master, and every number it uses is fixed in the engine and stated back in the report’sMasterReport.

The master is linear light in PixlRGB, the engine’s working space, scaled so that1.0 is 203 cd/m². That is the graphics white of ITU-R BT.2408, the level a reference HDR display shows diffuse white at, so SDR content placed in the master sits where an HDR display expects it. Everything above 1.0 is headroom, carried unclipped through every stage: the grade, framing, the resize and overlays. Nothing is clipped until the output decides what it can hold.

Every source in

Every source enters the master the same way, and each way in is a standard’s, not a guess:

• SDR (JPEG, PNG, TIFF, WebP, AVIF, JPEG XL): through its own ICC profile or CICP code points into linear light, its white at 1.0.
• PQ: through the SMPTE ST 2084 EOTF to absolute luminance, then divided by 203.
• HLG: through the inverse OETF and the OOTF for a 1000 cd/m² display, as BT.2100 specifies.
• Gain-map files: ISO 21496-1 (a HEIF tmap item or a JPEG’s APP2), Adobe’shdrgm XMP, and Apple’s legacy maps in HEIC and JPEG. The map is applied at the file’s own alternate headroom, so the base’s white is 1.0. A file whose map states no headroom is refused, not given one.
• A RAW developed in Scene: the engine’s own develop delivers linear PixlRGB in 32-bit floats, unclamped, highlights above sensor white included. It enters the master as it stands.

Grey stays one channel throughout. The order of the stages is fixed: decode, gain map, orientation, lens correction, retouching, the grade, framing, resize, output sharpening and overlays, then the egress. Every stage sees headroom whatever the output will be.

The peak and the ceiling

HDR metadata in the wild rarely says what the pixels hold: most PQ files state no peak at all. So the master measures its own. At the egress, after framing, the resize and overlays, the peak is the 99.9th percentile of luminance, by nearest rank. The histogram has 1/1024-stop bins from 2⁻⁸ of white to the top, with integer counts, so the peak is the same at any thread count. Fully transparent pixels are left out. A peak in the first bin above white counts as white, so a white that arrives a rounding error high rolls nothing off.

A host can state the peak instead. A preview passes its measured figure back, so a region of the picture renders exactly as the whole does. A region asked to measure its own peak is refused, because its peak would differ from the picture’s.

With headroom on, the master is held to a ceiling: the measured peak, or any level from 203 to 10 000 cd/m². A peak above the ceiling rolls off with BT.2390’s Hermite curve, using the largest knee for which the curve stays monotone; a peak below it is clipped at it. The report states the peak, the limit and how many samples the limit changed.

The display tone map

Without headroom, everything above white has to fit under it. The first version, v0, was BT.2390 applied to each channel separately. Rolling each channel off on its own turns hue: a saturated highlight’s strongest channel flattens first, so orange drifts toward yellow and blue toward cyan. Version 1 rolled off luminance alone; that kept hue, but near white a colour of white’s luminance can only be white, so it lost chroma.

Version 2, the one the master runs, does two things:

1. The roll-off runs on n = Y + 0.5 × (max(c, Y) − Y), half-way between luminance and the largest channel, and every channel is scaled by the same factor. A saturated highlight keeps its colour by ending a little below white; a grey one rolls off exactly as on luminance.
2. The path to white. Near the ceiling, bright saturated colours are drawn gently toward the boundary of what the output can show, along their own hue line in ICtCp at their own intensity. A straight line to white in linear light turns hue as the eye sees it; ICtCp keeps it. The boundary is found by the Illinois method in about six evaluations a colour.

A pixel below the knee with every channel inside keeps its exact bits, and with nothing above white no tone map runs at all. An unedited photograph through the master is its conversion to Display P3, sample for sample. On the photograph lifted a stop, the tone map leaves under a tenth of a plain clip’s pixels at pure white, keeps 7.5 times as many distinct colours where the clip flattens a highlight, and leaves every pixel below a quarter of white exactly as the clip does. The path to white costs about 1.3 s on a 24 MP frame at 8 threads, about 0.15 s at a 2048-pixel preview.

Colours past the primaries

An edit can push colours outside the output’s gamut: Display P3 without headroom, Rec.2020 for PQ. Version 2 of the gamut compression moves only those colours. Along each pixel’s ICtCp hue line, it finds where the line leaves the target’s primaries. The 99.9th percentile of how far past the boundary the picture reaches sets a soft knee, and colours above the knee are eased onto the boundary with hue kept. A picture that already fits moves no pixel and keeps its bytes. One step per output, never chained.

Measured on the P3 photograph at a quarter size with saturation doubled (reach 1.47, knee 0.82): over the 71 000 pixels past P3, the hue shift is 0.00° median and 0.01° at the 95th percentile, against a plain clip’s 0.23° and 0.70°. Bright colours inside P3’s sides keep the clip’s bits. A region rendered with the preview’s stated peak and reach is the full render, cut.

One flag, every sink

The master has one switch, headroom. What each sink receives follows from what it can hold:

With headroom on, a gain-map file’s base is the headroom-off rendition. Flipping the flag changes only the headroom; an SDR viewer sees the same picture either way. Every HDR output measures MaxCLL and MaxFALL on the pixels it produced and writes them where the format has a place: JPEG XL’s intensity_target, PNG’scLLI, HEIF’s clli. A source’s own light-level metadata never survives a transform: it described other pixels.

• 4 msan unedited P3 photo, proven exact at plan time and copied
• 0.88 slifted a stop, into 16-bit PQ JPEG XL
• 2.98 slifted a stop, into JPEG with its gain map

The colour stage on the 24 MP JPEG, 8 threads, i7-14700F.

Gain maps, full size

A gain map is computed where both renditions exist: the master after its ceiling, and the SDR base after it has been rounded. Gains are taken per channel in the base’s primaries (one channel for grey), with gamma 1 and offsets of 1/64. Each channel’s range is what its pixels need over the colours the base can hold.

The map is written at full size. The base is tone mapped pixel by pixel, so the gain changes as sharply as the picture’s edges. At half size, a reader’s rebuild of even a lossless file missed the held master by up to 1.3 stops at fine detail. At full size the rebuild is within 0.04 stops, and files are 23–39% larger.

The rebuild never passes the ceiling. Lossy coding moves both base and map: without a cap, a white base’s map code pushed to the top of a widened range rebuilt 0.49 stops past the ceiling. A safety margin of that size would have dimmed every highlight by 30%. Instead, each channel’s top gain is capped at the gain that takes a white base to the ceiling, so no code a lossy codec produces can rebuild above it.

Read back against the PQ export of the same master, in 10-bit PQ codes (mean / p99): a lossless AVIF is within 0.13 / 0.38, a JPEG at quality 100, 4:4:4, within 0.93 / 5.07, and MaxCLL within 0.3%. The JPEG is an UltraHDR file (ISO 21496-1 with hdrgm XMP and an MPF index); the AVIF carries an ISO tmap item.

iPhone HDR, read as Apple reads it

An iPhone 17 writes an ISO 21496-1 map beside Apple’s older one; an iPhone 13 Pro writes only the older one. When both are present the engine reads the ISO map. The older map is applied the way Core Image applies it. The headroom comes from MakerNote tags 33 and 48 by Apple’s published formula, and the gain is1 + (min(H, h) − 1) · map^2.2 — a pure 2.2 gamma on the map, even where it is tagged linear, rather than the Rec.709 curve Apple’s document gives. The map’s headroom tag is a ratio, not stops; the file’s three statements of its headroom agree only read that way.

The test corpus is eleven HEICs from two iPhones: iOS 17.5 to 27.0, 12 and 48 MP, ultra wide, tele, the front camera (rotated and mirrored), a Live Photo and an edited crop. Against Apple’s own renders on macOS 27.0, within the bounds set before the comparison:

HDR is compared over the pixels where Apple’s render has no negative channel. Core Image keeps a float YCbCr conversion’s out-of-range values; the engine clips at the integer decode. On files carrying both maps, the ISO and legacy readings agree to a median of 0.032 stops at most. The base matches partly because the engine also upsamples HEIC chroma as Core Image does (see Codecs).

Effects above white

Grain, clarity, the colour wheels and a vignette’s highlight protection all weigh their effect by tone. In SDR the tone is clamped at white, so a highlight lifted into headroom would lose its grain the moment it passed 1.0. In an HDR pipeline with headroom the tone is read on PQ’s perceptual scale, up to 1000 cd/m² in the master (BT.2408’s reference display), where white sits at 0.77. The weight is a function of the pixel alone, so a region is still the full render cut, and the headroom flag never changes it.

The master cache

A host renders an edited photo many times, and everything before the grade — the decode, a gain map applied, orientation, enhancement, the lens correction — is the same every time. On a RAW or a 48 MP HEIC it is also the expensive part. The master cache keeps the master at that point, the seam, as a lossless float JPEG XL with apixl box, and a render from it is the render from the original, byte for byte.

The box records the engine version, the source’s SHA-256, and a digest of every request field that acts before the seam: the RAW develop, enhancement, the lens, orientation, the channel count. A render whose fields differ, or a cache made by another engine version, is refused with StaleCache naming the field, and the host makes the cache again. A plain JPEG XL reader sees the master’s floats as linear PixlRGB, alpha included.

Effort 1, 8 threads. A 2048-pixel PQ preview of the road photograph takes 0.49 s from the cache against 0.68 s from the original; a RAW or a 48 MP HEIC also saves its decode. Tested byte for byte, both ways, on an RGBA PNG through a lens, the PQ HEIC, an UltraHDR JPEG, an iPhone HEIC with Apple’s map, and a CR2 in Scene, each rendered as a graded, straightened, cropped, resized PQ PNG, the master’s floats, a region, an SDR JPEG and a gain-map JPEG.

Verification

The claims on this page are held by tests in the engine’s repository, and by other readers:

• Identity where nothing happens. An unedited photograph through the master is its conversion to Display P3, sample for sample, and the 24 MP JPEG at quality 90 through the command line, byte for byte.
• Headroom into PQ equals a plain linear expansion at 203 cd/m² on over 99% of pixels (all below the ceiling in every channel); MaxCLL is the ceiling within 0.5 cd/m², and a ceiling of 400 is never passed.
• Every source kind — JPEG, PNG, JPEG XL, HEIC, TIFF, WebP, PQ, HLG, gain-map files and a Scene CR2 — comes out as PQ JPEG XL and as a base with a three-channel map in AVIF and JPEG.
• Threads. 1 thread and 7 give the same bytes on every HDR path.
• SDR previews. An SDR rendition of an HDR master equals a separate tone map of its PQ export to one code at most, mean ≤ 0.005.
• Other readers. libavif tone maps the engine’s gain maps as the engine does (MaxCLL 795 against 797 cd/m² at 2 stops). On a Mac, Preview shows the PQ and HLG files in HDR, and Core Image renders them as the engine reads them (0.017 stops apart for PQ, 0.055 for HLG) and applies the ISO gain maps as the engine does (0.017 stops).

Sources and method

Times are on an Intel i7-14700F under Linux at 8 threads unless stated. The Apple comparisons ran once on the owner’s MacBook Pro (M2 Pro, macOS 27.0); the renders are pinned by SHA-256 and the tests that read them run by hand.

• The master: tests/master.rs — an_unedited_photo_is_its_conversion_to_display_p3, a_rebuilt_lossy_file_never_passes_the_ceiling, with_headroom_a_float_sink_gets_the_master_in_linear_pixlrgb, grain_carries_on_above_white_in_the_master.
• Hue and chroma: tests/tone_map_hue.rs (ICtCp; the photograph held, the corpus by hand); the_boundary_is_where_a_bisection_puts_it.
• Why hue is measured in ICtCp: in psychophysical hue matching on HDR stimuli across nine colour spaces, “the hue linearity of ICTCP color space was the best” (Wang, Wei and Qu, Optics Express 30(25):44896, 2022, 10.1364/OE.475433). That per-channel tone curves turn hue is also shown by Burke, Smith and Zink, “Color Volume and Hue-preservation in HDR Tone Mapping”, SMPTE Motion Imaging Journal 129(4), 2020, 10.5594/JMI.2020.2984046.
• Gamut compression: tests/gamut.rs.
• HDR paths and threads: tests/hdr.rs — every_hdr_path_is_byte_identical_whatever_the_thread_count; RAW into headroom: tests/raw_hdr.rs, a_scene_raw_graded_above_white_reaches_headroom_in_pq.
• Gain maps: tests/gainmap.rs; the iPhone corpus and Apple’s renders: tests/heic.rs, tests/heic/pixl-heic-reference.swift.
• The cache: tests/master_cache.rs; timings from the_cache_codec_measured and preview_times_at_2048_measured (by hand).
• Interoperability: ci/avif-interop/ (libavif 1.3.0, libheif 1.23.5), and the device check on macOS 27.