Thank you Harry and all other contributors for your work on this. Responses
inline -
On Mon, 23 Oct 2023, Pekka Paalanen wrote:
> On Fri, 20 Oct 2023 11:23:28 -0400
> Harry Wentland wrote:
>
> > On 2023-10-20 10:57, Pekka Paalanen wrote:
> > > On Fri, 20 Oct 2023 16:22:56 +0200
> > > Sebastian Wick wrote:
> > >
> > >> Thanks for continuing to work on this!
> > >>
> > >> On Thu, Oct 19, 2023 at 05:21:22PM -0400, Harry Wentland wrote:
> > >>> v2:
> > >>> - Update colorop visualizations to match reality (Sebastian, Alex Hung)
> > >>> - Updated wording (Pekka)
> > >>> - Change BYPASS wording to make it non-mandatory (Sebastian)
> > >>> - Drop cover-letter-like paragraph from COLOR_PIPELINE Plane Property
> > >>>section (Pekka)
> > >>> - Use PQ EOTF instead of its inverse in Pipeline Programming example
> > >>> (Melissa)
> > >>> - Add "Driver Implementer's Guide" section (Pekka)
> > >>> - Add "Driver Forward/Backward Compatibility" section (Sebastian,
> > >>> Pekka)
> > >
> > > ...
> > >
> > >>> +An example of a drm_colorop object might look like one of these::
> > >>> +
> > >>> +/* 1D enumerated curve */
> > >>> +Color operation 42
> > >>> +├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
> > >>> matrix, 3x4 matrix, 3D LUT, etc.} = 1D enumerated curve
> > >>> +├─ "BYPASS": bool {true, false}
> > >>> +├─ "CURVE_1D_TYPE": enum {sRGB EOTF, sRGB inverse EOTF, PQ EOTF,
> > >>> PQ inverse EOTF, …}
> > >>> +└─ "NEXT": immutable color operation ID = 43
I know these are just examples, but I would also like to suggest the possibility
of an "identity" CURVE_1D_TYPE. BYPASS = true might get different results
compared to setting an identity in some cases depending on the hardware. See
below for more on this, RE: implicit format conversions.
Although NVIDIA hardware doesn't use a ROM for enumerated curves, it came up in
offline discussions that it would nonetheless be helpful to expose enumerated
curves in order to hide the vendor-specific complexities of programming
segmented LUTs from clients. In that case, we would simply refer to the
enumerated curve when calculating/choosing segmented LUT entries.
Another thing that came up in offline discussions is that we could use multiple
color operations to program a single operation in hardware. As I understand it,
AMD has a ROM-defined LUT, followed by a custom 4K entry LUT, followed by an
"HDR Multiplier". On NVIDIA we don't have these as separate hardware stages, but
we could combine them into a singular LUT in software, such that you can combine
e.g. segmented PQ EOTF with night light. One caveat is that you will lose
precision from the custom LUT where it overlaps with the linear section of the
enumerated curve, but that is unavoidable and shouldn't be an issue in most
use-cases.
Actually, the current examples in the proposal don't include a multiplier color
op, which might be useful. For AMD as above, but also for NVIDIA as the
following issue arises:
As discussed further below, the NVIDIA "degamma" LUT performs an implicit fixed
point to FP16 conversion. In that conversion, what fixed point 0x maps
to in floating point varies depending on the source content. If it's SDR
content, we want the max value in FP16 to be 1.0 (80 nits), subject to a
potential boost multiplier if we want SDR content to be brighter. If it's HDR PQ
content, we want the max value in FP16 to be 125.0 (10,000 nits). My assumption
is that this is also what AMD's "HDR Multiplier" stage is used for, is that
correct?
>From the given enumerated curves, it's not clear how they would map to the
above. Should sRGB EOTF have a max FP16 value of 1.0, and the PQ EOTF a max FP16
value of 125.0? That may work, but it tends towards the "descriptive" notion of
assuming the source content, which may not be accurate in all cases. This is
also an issue for the custom 1D LUT, as the blob will need to be converted to
FP16 in order to populate our "degamma" LUT. What should the resulting max FP16
value be, given that we no longer have any hint as to the source content?
I think a multiplier color op solves all of these issues. Named curves and
custom 1D LUTs would by default assume a max FP16 value of 1.0, which would then
be adjusted by the multiplier. For 80 nit SDR content, set it to 1, for 400
nit SDR content, set it to 5, for HDR PQ content, set it to 125, etc.
> > >>> +
> > >>> +/* custom 4k entry 1D LUT */
> > >>> +Color operation 52
> > >>> +├─ "TYPE": immutable enum {1D enumerated curve, 1D LUT, 3x3
> > >>> matrix, 3x4 matrix, 3D LUT, etc.} = 1D LUT
> > >>> +├─ "BYPASS": bool {true, false}
> > >>> +├─ "LUT_1D_SIZE": immutable range = 4096
> > >>> +├─ "LUT_1D": blob
> > >>> +└─ "NEXT": immutable color operation ID = 0
> > >
> > > ...
> > >
> > >>> +Driver Forward/Backward Compatibility
> > >>> +=
> > >>> +
> > >>> +As this is uAPI drivers can't regress color pipelines that
