If the block diagram is not aligned due to mail client, please refer to
the attached .txt file. Hope thats slightly better :).
Regards
Shashank
On 1/10/2019 8:32 PM, Sharma, Shashank wrote:
Hello All,
This mail is to propose a design for enabling HDR support in
Wayland/Weston stack, using display engine capabilities, and get more
feedback and input from community.
Here are few points (you might already know these), about HDR
framebuffers, videos and displays:
- HDR content/buffers are composed in REC2020 colorspace, with bit
depth 10/12/16 BPC. Some of the popular formats are P010,P012,P016.
- HDR content come with their own Metadata to be applied to get the
right luminance at the display device.
- The metadata can be of two type 1. static 2. dynamic . For
simplicity, this solution is focusing on static HDR only (HDR10 standard)
- HDR content also provide its supported EOTF (electro optical
transfer function) information, which is a curve (like SRGB gamma
curve). One popular EOTF is PQ(ST2084).
- HDR capable displays mention their EOTF and HDR metadata support
information in EDID CEA-861-G blocks.
- Normal SRGB buffers are composed in SRGB color space following
REC709 specifications.
- For accurate blending in display engines, we need to make sure
following:
- All the buffers are in same colorspace (Rec 709 or Rec 2020)
- All the buffers are liner (gamma/EOTF removed)
- All the buffers are tone mapped in same zone (HDR or SDR)
Please refer to the block diagram below, which presents a simple case
of a HDR P010 movie playback, with HDR buffers as video buffers, and
SDR buffers as subtitles. The subsystem looks and works like this:
- A client decodes the buffer (using FFMpeg for example) and gets the
two buffers, one with video (HDR) and one subtitles (SDR)
- Client passes following information to the compositor:
- The actual buffers
- Their colorspace infromation, BT2020 for HDR buffer, REC709 for
SDR buffer (planning to add a new protocol extension for this)
- The HDR metadata of the content (planning to add new protocol
for this)
- Compositors actions:
- Reads the End display's HDR capabilities from display EDID.
Assume its an HDR HDMI monitor.
- Compositor tone maps every view's framebuffer to match tone of
end display, applying a libVA filter. In this example:
- The SDR subtitles frame will go through SDR to HDR tone
mapping (called S2H)
- The HDR video frame will go through HDR to HDR tone mapping
(called H2H) if the HDR capabilities of monitor and content are different.
- Now both the buffers and the monitor are in the same tone
mapped range.
- As the end display is HDR capable, and one of the content frame
is HDR, the compositor will prepare all other planes for color space
conversion (CSC) from REC709->REC2020 using plane CSC property.
- As the CSC and blending should be done in liner space,
compositor will also use plane level degamma to make the buffers linear.
- These actions will make sure that, during blending:
- All the buffers are in same colorspace (REC2020)
- All the buffers are linear
- All the buffers are tone mapped (HDR)
- The plane level color properties patch, for DRM can be found
here: https://patchwork.freedesktop.org/series/30875/
- Now, in order to re-apply the HDR curve, compositor will apply
CRTC level gamma, so that the output buffer is non-linear again.
- To pass the output HDR information to kernel, so that it can
create and send AVI-info-frames to HDMI, compositor will set Connector
HDR metadata property.
- Code for the same can be found here:
https://patchwork.freedesktop.org/series/25091/
- And they will ever live happily after :).
Please provide inputs, feedbacks and suggestions for this design and
plan, so that we can improve out half cooked solution, and start
sending the patches.
+------------------+ +-------------------+
| SDR Buffer subtitles | HDR Buffer video
| (REC 709 colorsp) | (REC 2020 colorsp |
| | | |
+-------+----------+ +-------+-----------+
| |
| |
| |
+------v---------------------------v------------+ +--------------+
| Compositor: v | |
LibVA |
| - assigns views to overlays
+---------> Tone mapping |
| - prepare plane/CRTC color properties
<---------+ SDR to HDR |
| for linear blending in display
| | HDR to SDR |
+------+-----------------------------+----------+ +--------------+
| |
| Tone mapped | Tone mapped
| non-linear-Rec709 |
non-linear Rec2020
+------v------+ +-------v--------+
SRGB Degamma | |EOTF as degamma |
|(Plane) | |(Plane) |
| | | |
+------+------+ +-------+--------+
Tone mapped linear Rec 709 | |
+------v------+ | Tone mapped
| CSC/CTM | |
non-linear Rec2020
| REC709->2020| |
| | |
+------+------+ |
| Tone mapped linear |
| Rec 2020 |
+------v-----------------------------v---------+
| Blender |
| |
+--------------------+-------------------------+
| Tone mapped linear Rec2020
+--------------------v-------------------------+ Tone mapped
| OETF(CRTC Gamma, post blending)
| non-linear Rec2020 +------------------+
| +----------------> | HDMI monitor |
+----------------------------------------------+ +------------------+
+------------------+ +-------------------+
| SDR Buffer subtitles | HDR Buffer video
| (REC 709 colorsp) | (REC 2020 colorsp |
| | | |
+-------+----------+ +-------+-----------+
| |
| |
| |
+------v---------------------------v------------+
+--------------+
| Compositor: v |
| LibVA |
| - assigns views to overlays
+---------> Tone mapping |
| - prepare plane/CRTC color properties
<---------+ SDR to HDR |
| for linear blending in display |
| HDR to SDR |
+------+-----------------------------+----------+
+--------------+
| |
| Tone mapped | Tone mapped
| non-linear-Rec709 | non-linear Rec2020
+------v------+ +-------v--------+
SRGB Degamma | |EOTF as degamma |
|(Plane) | |(Plane) |
| | | |
+------+------+ +-------+--------+
Tone mapped linear Rec 709 | |
+------v------+ | Tone mapped
| CSC/CTM | | non-linear Rec2020
| REC709->2020| |
| | |
+------+------+ |
| Tone mapped linear |
| Rec 2020 |
+------v-----------------------------v---------+
| Blender |
| |
+--------------------+-------------------------+
| Tone mapped linear Rec2020
+--------------------v-------------------------+ Tone
mapped
| OETF(CRTC Gamma, post blending) |
non-linear Rec2020 +------------------+
|
+----------------> | HDMI monitor |
+----------------------------------------------+
+------------------+
_______________________________________________
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/wayland-devel
