On Mon, Apr 26, 2021 at 01:38:49PM -0400, Harry Wentland wrote: > > ## Introduction > > We are looking to enable HDR support for a couple of single-plane and > multi-plane scenarios. To do this effectively we recommend new > interfaces to drm_plane. Below I'll give a bit of background on HDR and > why we propose these interfaces.
I think this is on of the topics that would tremendously benefit from the uapi rfc process, with lots of compositor people involved. https://dri.freedesktop.org/docs/drm/gpu/rfc/ Also for this I think we really do need a pretty solid understanding of the involve compositor protocols, otherwise the kernel uapi is going to be for naught. -Daniel > > > ## Defining a pixel's luminance > > Currently the luminance space of pixels in a framebuffer/plane presented to > the display is not well defined. It's usually assumed to be in a 2.2 or 2.4 > gamma space and has no mapping to an absolute luminance value but is > interpreted in relative terms. > > Luminance can be measured and described in absolute terms as candela per > meter squared, or cd/m2, or nits. Even though a pixel value can be mapped to > luminance in a linear fashion to do so without losing a lot of detail > requires 16-bpc color depth. The reason for this is that human perception can > distinguish roughly between a 0.5-1% luminance delta. A linear representation > is suboptimal, wasting precision in the highlights and losing precision in > the shadows. > > A gamma curve is a decent approximation to a human's perception of luminance, > but the PQ (perceptual quantizer) function [1] improves on it. It also > defines the luminance values in absolute terms, with the highest value being > 10,000 nits and the lowest 0.0005 nits. > > Using a content that's defined in PQ space we can approximate the real world > in a much better way. > > Here are some examples of real-life objects and their approximate luminance > values: > > | Object | Luminance in nits | > | ----------------- | ----------------- | > | Sun | 1.6 million | > | Fluorescent light | 10,000 | > | Highlights | 1,000 - sunlight | > | White Objects | 250 - 1,000 | > | Typical objects | 1 - 250 | > | Shadows | 0.01 - 1 | > | Ultra Blacks | 0 - 0.0005 | > > > ## Describing the luminance space > > **We propose a new drm_plane property to describe the Eletro-Optical Transfer > Function (EOTF) with which its framebuffer was composed.** Examples of EOTF > are: > > | EOTF | Description > | > | --------- > |:------------------------------------------------------------------------- | > | Gamma 2.2 | a simple 2.2 gamma > | > | sRGB | 2.4 gamma with small initial linear section > | > | PQ 2084 | SMPTE ST 2084; used for HDR video and allows for up to 10,000 > nit support | > | Linear | Linear relationship between pixel value and luminance value > | > > > ## Mastering Luminances > > Now we are able to use the PQ 2084 EOTF to define the luminance of pixels in > absolute terms. Unfortunately we're again presented with physical limitations > of the display technologies on the market today. Here are a few examples of > luminance ranges of displays. > > | Display | Luminance range in nits | > | ------------------------ | ----------------------- | > | Typical PC display | 0.3 - 200 | > | Excellent LCD HDTV | 0.3 - 400 | > | HDR LCD w/ local dimming | 0.05 - 1,500 | > > Since no display can currently show the full 0.0005 to 10,000 nits luminance > range the display will need to tonemap the HDR content, i.e to fit the > content within a display's capabilities. To assist with tonemapping HDR > content is usually accompanied with a metadata that describes (among other > things) the minimum and maximum mastering luminance, i.e. the maximum and > minimum luminance of the display that was used to master the HDR content. > > The HDR metadata is currently defined on the drm_connector via the > hdr_output_metadata blob property. > > It might be useful to define per-plane hdr metadata, as different planes > might have been mastered differently. > > > ## SDR Luminance > > Since SDR covers a smaller luminance range than HDR, an SDR plane might look > dark when blended with HDR content. Since the max HDR luminance can be quite > variable (200-1,500 nits on actual displays) it is best to make the SDR > maximum luminance value configurable. > > **We propose a drm_plane property to specfy the desired maximum luminance of > the SDR plane in nits.** This allows us to map the SDR content predictably > into HDR's absolute luminance space. > > > ## Let There Be Color > > So far we've only talked about luminance, ignoring colors altogether. Just > like in the luminance space, traditionally the color space of display outputs > has not been well defined. Similar to how an EOTF defines a mapping of pixel > data to an absolute luminance value, the color space maps color information > for each pixel onto the CIE 1931 chromaticity space. This can be thought of > as a mapping to an absolute, real-life, color value. > > A color space is defined by its primaries and white point. The primaries and > white point are expressed as coordinates in the CIE 1931 color space. Think > of the red primary as the reddest red that can be displayed within the color > space. Same for green and blue. > > Examples of color spaces are: > > | Color Space | Description | > | ----------- | ------------------------------------------ | > | BT 601 | similar to BT 709 | > | BT 709 | used by sRGB content; ~53% of BT 2020 | > | DCI-P3 | used by most HDR displays; ~72% of BT 2020 | > | BT 2020 | standard for most HDR content | > > The color space is defined in DRM for YCbCr planes via the color_encoding > property of the drm_plane. > > **We propose to add definitions for the RGB variants of the BT color spaces.** > > > ## Color Primaries and White Point > > Just like displays can currently not represent the entire 0.0005 - 10,000 > nits HDR range of the PQ 2084 EOTF, they are currently not capable of > representing the entire BT.2020 color Gamut. For this reason video content > will often specify the color primaries and white point used to master the > video, in order to allow displays to be able to map the image as best as > possible onto the display's gamut. > > > ## Displays and Tonemapping > > External displays are able to do their own tone and color mapping, based on > the mastering luminance, color primaries, and white space defined in the HDR > metadata. > > Internal panels (which are currently few and far between) usually don't > include the complex HW to do tone and color mapping on their own and will > require the display driver to perform appropriate mapping. > > > ## Pixel Formats > > The pixel formats, such as ARGB8888, ARGB2101010, P010, or FP16 are unrelated > to color space and EOTF definitions. HDR pixels can be formatted in different > ways but in order to not lose precision HDR content requires at least 10 bpc > precision. For this reason ARGB2101010, P010, and FP16 are the obvious > candidates for HDR. ARGB2101010 and P010 have the advantage of requiring only > half the bandwidth as FP16, while FP16 has the advantage of enough precision > to operate in a linear space, i.e. without EOTF. > > > ## Proposed use-cases > > Although the userspace side of this work is still in the early stages it is > clear that we will want to support the following two use-cases: > > **One XRGB2101010 HDR Plane:** A single, composited plane of HDR content. The > use-case is a video player on a desktop with the compositor owning the > composition of SDR and HDR content. The content shall be PQ BT.2020 > formatted. The drm_connector's hdr_output_metadata shall be set. > > **One ARGB8888 SDR Plane + One P010 HDR Plane:** A normal 8bpc desktop plane, > with a P010 HDR video plane underlayed. The HDR plane shall be PQ BT.2020 > formatted. The desktop plane shall specify an SDR boost value. The > drm_connector's hdr_output_metadata shall be set. > > **One XRGB8888 SDR Plane - HDR output:** In order to support a smooth > transition we recommend an OS that supports HDR output to provide the > hdr_output_metadata on the drm_connector to configure the output for HDR, > even when the content is only SDR. This will allow for a smooth transition > between SDR-only and HDR content. In this use-case the SDR max luminance > value should be provided on the drm_plane. > > In DCN we will de-PQ or de-Gamma all input in order to blend in linear space. > For SDR content we will also apply any desired boost before blending. After > blending we will then re-apply the PQ EOTF and do RGB to YCbCr conversion if > needed. > > > ## Summary of proposed interface changes > > per drm_plane: > - new RGB color space definitions, mirroring the existing YUV color space > definitions > - new transfer function property > - new SDR maximum white level property > > > ## References > > [1] > https://en.wikipedia.org/wiki/High-dynamic-range_video#Perceptual_Quantizer > > > ## Further Reading > > https://gitlab.freedesktop.org/swick/wayland-protocols/-/blob/color/unstable/color-management/color.rst > http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP309.pdf > https://app.spectracal.com/Documents/White%20Papers/HDR_Demystified.pdf > > > Bhawanpreet Lakha (3): > drm/color: Add RGB Color encodings > drm/color: Add Color transfer functions for HDR/SDR > drm/color: Add sdr boost property > > .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 4 +- > .../gpu/drm/arm/display/komeda/komeda_plane.c | 4 +- > drivers/gpu/drm/arm/malidp_planes.c | 4 +- > drivers/gpu/drm/armada/armada_overlay.c | 4 +- > drivers/gpu/drm/drm_atomic_uapi.c | 8 ++ > drivers/gpu/drm/drm_color_mgmt.c | 84 +++++++++++++++++-- > drivers/gpu/drm/i915/display/intel_sprite.c | 4 +- > .../drm/i915/display/skl_universal_plane.c | 4 +- > drivers/gpu/drm/nouveau/dispnv04/overlay.c | 4 +- > drivers/gpu/drm/omapdrm/omap_plane.c | 4 +- > drivers/gpu/drm/sun4i/sun8i_vi_layer.c | 4 +- > drivers/gpu/drm/tidss/tidss_plane.c | 6 +- > include/drm/drm_color_mgmt.h | 25 +++++- > include/drm/drm_plane.h | 30 +++++++ > 14 files changed, 173 insertions(+), 16 deletions(-) > > -- > 2.31.0 > > _______________________________________________ > dri-devel mailing list > dri-de...@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx
