RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> -Original Message- > From: Christopher Braga > Sent: Friday, September 8, 2023 1:39 AM > To: Shankar, Uma ; intel-...@lists.freedesktop.org; > dri- > de...@lists.freedesktop.org > Cc: Borah, Chaitanya Kumar ; wayland- > de...@lists.freedesktop.org > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane > Color Pipeline > > > > On 8/29/2023 12:03 PM, Uma Shankar wrote: > > Add the documentation for the new proposed Plane Color Pipeline. > > > > Co-developed-by: Chaitanya Kumar Borah > > > > Signed-off-by: Chaitanya Kumar Borah > > Signed-off-by: Uma Shankar > > --- > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++ > > 1 file changed, 394 insertions(+) > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst > > b/Documentation/gpu/rfc/plane_color_pipeline.rst > > new file mode 100644 > > index ..60ce515b6ea7 > > --- /dev/null > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst > > @@ -0,0 +1,394 @@ > > +=== > > + Plane Color Pipeline: A UAPI proposal > > +=== > > + > > +To build the proposal on, lets take the premise of a color pipeline > > +as shown below. > > + > Hi Uma, > Thanks for posting this. A few comments, with some echoing the sentiments of > other commenters on the patch set. Hi Christopher, Thanks for your inputs on the series and my apologies for a late reply. > > + +---+ > > + |RAM| > > + | +--++-++-+ | > > + | | FB 1 || FB 2 || FB N| | > > + | +--++-++-+ | > > + +---+ > > + | Plane Color Hardware Block | > > + ++ > > + | +---v-+ +---v---+ +---v--+ | > > + | | Plane A | | Plane B | | Plane N | | > > + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | > > + | +---+-+ +---+---+ +---+--+ | > > + | | | || > > + | +---v-+ +---v---+ +---v--+ | > > + | |Plane A | | Plane B | | Plane N | | > > + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > > + | +---+-+ ++--+ ++-+ | > > + | | | | | > > + | +---v-+ +v--+ +v-+ | > > + | | Plane A | | Plane B | | Plane N | | > > + | |Post-CSC | | Post-CSC | | Post-CSC | | > > + | +---+-+ ++--+ ++-+ | > > + | | | | | > > + ++ > > ++--v--v---v---| > > +|| || > > +|| Pipe Blender|| > > ++++ > > +||| > > +|+---v--+ | > > +|| Pipe Pre-CSC| | > > +|| | | > > +|+---+--+ | > > +||Pipe Color | > > +|+---v--+ Hardware| > > +|| Pipe CSC/CTM| | > > +|| | | > > +|+---+--+ | > > +||| > > +|+---v--+ | > > +|| Pipe Post-CSC | | > > +|| | | > > +|+---+--+ | > > +||| > > ++-+ > > + | > > + v > > +Pipe Output > > + > > +Each plane consists of the following color blocks > > + * Pre-CSC : This block can used to linearize the input frame buffer data. > > + The linear data then can be further acted on by the following > > + color hardware blocks in the display hardware pipeline > > + > > + * CSC/CTM: Used to program color transformation matrix, this block is used > > +to perform color space conversions like BT2020 to BT709 or > &g
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
On Thu, 7 Sep 2023 12:31:47 + "Shankar, Uma" wrote: > > -Original Message- > > From: Pekka Paalanen > > Sent: Tuesday, September 5, 2023 5:03 PM > > To: Shankar, Uma > > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar > > ; dri-devel@lists.freedesktop.org; wayland- > > de...@lists.freedesktop.org; Harry Wentland ; > > Sebastian Wick ; ville.syrj...@linux.intel.com; > > Jonas Adahl > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane > > Color Pipeline > > > > On Mon, 4 Sep 2023 13:44:49 + > > "Shankar, Uma" wrote: > > > > > > -Original Message- > > > > From: dri-devel On Behalf > > > > Of Pekka Paalanen > > > > Sent: Wednesday, August 30, 2023 5:59 PM > > > > To: Shankar, Uma > > > > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar > > > > ; dri-devel@lists.freedesktop.org; > > > > wayland- de...@lists.freedesktop.org > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed > > > > Plane Color Pipeline > > > > > > > > On Wed, 30 Aug 2023 08:59:36 + > > > > "Shankar, Uma" wrote: > > > > > > > > > > -Original Message- > > > > > > From: Harry Wentland > > > > > > Sent: Wednesday, August 30, 2023 1:10 AM > > > > > > To: Shankar, Uma ; > > > > > > intel-...@lists.freedesktop.org; dri- > > > > > > de...@lists.freedesktop.org > > > > > > Cc: Borah, Chaitanya Kumar ; > > > > > > wayland- de...@lists.freedesktop.org > > > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for > > > > > > proposed Plane Color Pipeline > > > > > > > > > > > > > > > > > > > > > > > > On 2023-08-29 12:03, Uma Shankar wrote: > > > > > > > Add the documentation for the new proposed Plane Color Pipeline. > > > > > > > > > > > > > > Co-developed-by: Chaitanya Kumar Borah > > > > > > > > > > > > > > Signed-off-by: Chaitanya Kumar Borah > > > > > > > > > > > > > > Signed-off-by: Uma Shankar > > > > > > > --- > > > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 > > > > > > > ++ > > > > > > > 1 file changed, 394 insertions(+) > > > > > > > create mode 100644 > > > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > > > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > new file mode 100644 > > > > > > > index ..60ce515b6ea7 > > > > > > > --- /dev/null > > > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > > ... > > > > > > > > Hi Uma! > > > > > > Thanks Pekka for the feedback and useful inputs. > > > > Hi Uma, > > > > sorry to say, but the overall feeling I get from this proposal is that it > > is just the > > current color related KMS properties wrapped in a pipeline blob. That is > > not the > > re-design I believe we are looking for, and the feeling is based on several > > details > > that are just copied from the current property design. Also the "private" > > stuff has > > to go. > > Hi Pekka, > Ok, got the concerns in general. We will try to evaluate in deeper detail the > property based design and come back if there are some issues or inputs. > > At Intel we don't need private as of now, but we thought of having an option > to > enable any custom hardware or vendor. But we can drop the same for now if > community doesn't feel the need for it. > > > All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT > > tap > > distribution, and parametrized curves are good development, but right now we > > are looking at things one step higher level: the overall color pipeline > > design and > > how to represent any operation. Most of this series is considering details > > below > > the current attention level,
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
On 8/29/2023 12:03 PM, Uma Shankar wrote: Add the documentation for the new proposed Plane Color Pipeline. Co-developed-by: Chaitanya Kumar Borah Signed-off-by: Chaitanya Kumar Borah Signed-off-by: Uma Shankar --- .../gpu/rfc/plane_color_pipeline.rst | 394 ++ 1 file changed, 394 insertions(+) create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst new file mode 100644 index ..60ce515b6ea7 --- /dev/null +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst @@ -0,0 +1,394 @@ +=== + Plane Color Pipeline: A UAPI proposal +=== + +To build the proposal on, lets take the premise of a color pipeline as shown +below. + Hi Uma, Thanks for posting this. A few comments, with some echoing the sentiments of other commenters on the patch set. + +---+ + |RAM| + | +--++-++-+ | + | | FB 1 || FB 2 || FB N| | + | +--++-++-+ | + +---+ + | Plane Color Hardware Block | + ++ + | +---v-+ +---v---+ +---v--+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-+ +---+---+ +---+--+ | + | | | || + | +---v-+ +---v---+ +---v--+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-+ ++--+ ++-+ | + | | | | | + | +---v-+ +v--+ +v-+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-+ ++--+ ++-+ | + | | | | | + ++ ++--v--v---v---| +|| || +|| Pipe Blender|| ++++ +||| +|+---v--+ | +|| Pipe Pre-CSC| | +|| | | +|+---+--+ | +||Pipe Color | +|+---v--+ Hardware| +|| Pipe CSC/CTM| | +|| | | +|+---+--+ | +||| +|+---v--+ | +|| Pipe Post-CSC | | +|| | | +|+---+--+ | +||| ++-+ + | + v +Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline + + * CSC/CTM: Used to program color transformation matrix, this block is used +to perform color space conversions like BT2020 to BT709 or BT601 +etc. This block acts on the linearized data coming from the +Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will get blended. +There is a similar set of HW blocks available at pipe/crtc level which acts on +this blended data. + +Below is a sample usecase fo video playback with sub-titles and playback +controls + +┌┐┌─┐ ┌─┐┌─┐ +│FB1 ││PRE-CSC │ │ CTM Matrix ││ POST-CSC│ +│├───►│Linearize├►│ BT709 to├───►│ SDR to HDR │ +│BT709 SDR ││ │ │ BT2020 ││ Tone Mapping├─┐ +└┘└─┘ └─┘└─┘ │ +(subtitles) │ + │ +┌┐┌─┐ ┌─┐┌─┐ │ +│FB2 ││PRE-CSC │ │ CTM Matrix ││ POST-CSC│ │ +│├───►│Linearize├
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> -Original Message- > From: Pekka Paalanen > Sent: Tuesday, September 5, 2023 5:03 PM > To: Shankar, Uma > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar > ; dri-devel@lists.freedesktop.org; wayland- > de...@lists.freedesktop.org; Harry Wentland ; > Sebastian Wick ; ville.syrj...@linux.intel.com; > Jonas Adahl > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane > Color Pipeline > > On Mon, 4 Sep 2023 13:44:49 + > "Shankar, Uma" wrote: > > > > -Original Message- > > > From: dri-devel On Behalf > > > Of Pekka Paalanen > > > Sent: Wednesday, August 30, 2023 5:59 PM > > > To: Shankar, Uma > > > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar > > > ; dri-devel@lists.freedesktop.org; > > > wayland- de...@lists.freedesktop.org > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed > > > Plane Color Pipeline > > > > > > On Wed, 30 Aug 2023 08:59:36 + > > > "Shankar, Uma" wrote: > > > > > > > > -Original Message- > > > > > From: Harry Wentland > > > > > Sent: Wednesday, August 30, 2023 1:10 AM > > > > > To: Shankar, Uma ; > > > > > intel-...@lists.freedesktop.org; dri- > > > > > de...@lists.freedesktop.org > > > > > Cc: Borah, Chaitanya Kumar ; > > > > > wayland- de...@lists.freedesktop.org > > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for > > > > > proposed Plane Color Pipeline > > > > > > > > > > > > > > > > > > > > On 2023-08-29 12:03, Uma Shankar wrote: > > > > > > Add the documentation for the new proposed Plane Color Pipeline. > > > > > > > > > > > > Co-developed-by: Chaitanya Kumar Borah > > > > > > > > > > > > Signed-off-by: Chaitanya Kumar Borah > > > > > > > > > > > > Signed-off-by: Uma Shankar > > > > > > --- > > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 > > > > > > ++ > > > > > > 1 file changed, 394 insertions(+) > > > > > > create mode 100644 > > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > new file mode 100644 > > > > > > index ..60ce515b6ea7 > > > > > > --- /dev/null > > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > ... > > > > > > Hi Uma! > > > > Thanks Pekka for the feedback and useful inputs. > > Hi Uma, > > sorry to say, but the overall feeling I get from this proposal is that it is > just the > current color related KMS properties wrapped in a pipeline blob. That is not > the > re-design I believe we are looking for, and the feeling is based on several > details > that are just copied from the current property design. Also the "private" > stuff has > to go. Hi Pekka, Ok, got the concerns in general. We will try to evaluate in deeper detail the property based design and come back if there are some issues or inputs. At Intel we don't need private as of now, but we thought of having an option to enable any custom hardware or vendor. But we can drop the same for now if community doesn't feel the need for it. > All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT > tap > distribution, and parametrized curves are good development, but right now we > are looking at things one step higher level: the overall color pipeline > design and > how to represent any operation. Most of this series is considering details > below > the current attention level, hence I'm paying attention only to the first few > patches. We will need to precisely describe the hardware in userspace. Number of luts, precision, segments etc.., we can't just pass EOTF's as enum from userspace and let driver put hardcoded values to LUT. This will be nothing but an extension of descriptive behaviour. This will be needed as there are multiple colorspaces possible and even LUTS can be used to perform tone mapping. So, we need userspace to be able to program luts directly. This is something we must expose to userspace. We will check i
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
On Mon, 4 Sep 2023 13:44:49 + "Shankar, Uma" wrote: > > -Original Message- > > From: dri-devel On Behalf Of Pekka > > Paalanen > > Sent: Wednesday, August 30, 2023 5:59 PM > > To: Shankar, Uma > > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar > > ; dri-devel@lists.freedesktop.org; wayland- > > de...@lists.freedesktop.org > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane > > Color Pipeline > > > > On Wed, 30 Aug 2023 08:59:36 + > > "Shankar, Uma" wrote: > > > > > > -Original Message- > > > > From: Harry Wentland > > > > Sent: Wednesday, August 30, 2023 1:10 AM > > > > To: Shankar, Uma ; > > > > intel-...@lists.freedesktop.org; dri- de...@lists.freedesktop.org > > > > Cc: Borah, Chaitanya Kumar ; > > > > wayland- de...@lists.freedesktop.org > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed > > > > Plane Color Pipeline > > > > > > > > > > > > > > > > On 2023-08-29 12:03, Uma Shankar wrote: > > > > > Add the documentation for the new proposed Plane Color Pipeline. > > > > > > > > > > Co-developed-by: Chaitanya Kumar Borah > > > > > > > > > > Signed-off-by: Chaitanya Kumar Borah > > > > > > > > > > Signed-off-by: Uma Shankar > > > > > --- > > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 > > > > > ++ > > > > > 1 file changed, 394 insertions(+) > > > > > create mode 100644 > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > new file mode 100644 > > > > > index ..60ce515b6ea7 > > > > > --- /dev/null > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > ... > > > > Hi Uma! > > Thanks Pekka for the feedback and useful inputs. Hi Uma, sorry to say, but the overall feeling I get from this proposal is that it is just the current color related KMS properties wrapped in a pipeline blob. That is not the re-design I believe we are looking for, and the feeling is based on several details that are just copied from the current property design. Also the "private" stuff has to go. All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT tap distribution, and parametrized curves are good development, but right now we are looking at things one step higher level: the overall color pipeline design and how to represent any operation. Most of this series is considering details below the current attention level, hence I'm paying attention only to the first few patches. More below. > > > > > +This color pipeline is then packaged within a blob for the user > > > > > +space to retrieve it. Details can be found in the next section > > > > > + > > > > > > > > Not sure I like blobs that contain other blob ids. > > > > > > It provides flexibility and helps with just one interface to > > > userspace. Its easy to handle and manage once we get the hang of it 😊. > > > > > > We can clearly define the steps of parsing and data structures to be > > > used while interpreting and parsing the blobs. > > > > Don't forget extendability. Possibly every single struct will need some > > kind of > > versioning, and then it's not simple to parse anymore. Add to that new/old > > kernel > > vs. old/new userspace, and it seems a bit nightmarish to design. > > Structure to be used to interpret the blob should be defined as UAPI only and > is not > expected to change once agreed upon. It should be interpreted like a standard > property. > So structure to be used, say for 3dLut or 1dlut or CTM operations should be > standardized > and fixed. No versioning of structure should be done and same > rules/restrictions as of UAPI > property should be applied. That sounds like a UAPI that cannot be extended, either. So in a few years we'd be looking at replacing it. Or maybe you are just re-inventing the KMS object property system as blobs? Replacing a single KMS property is much easier than replacing the foundations of the whole color pipeline design. > ... > > I have a feeling that introspect
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> -Original Message- > From: dri-devel On Behalf Of Pekka > Paalanen > Sent: Wednesday, August 30, 2023 5:59 PM > To: Shankar, Uma > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar > ; dri-devel@lists.freedesktop.org; wayland- > de...@lists.freedesktop.org > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane > Color Pipeline > > On Wed, 30 Aug 2023 08:59:36 + > "Shankar, Uma" wrote: > > > > -Original Message- > > > From: Harry Wentland > > > Sent: Wednesday, August 30, 2023 1:10 AM > > > To: Shankar, Uma ; > > > intel-...@lists.freedesktop.org; dri- de...@lists.freedesktop.org > > > Cc: Borah, Chaitanya Kumar ; > > > wayland- de...@lists.freedesktop.org > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed > > > Plane Color Pipeline > > > > > > > > > > > > On 2023-08-29 12:03, Uma Shankar wrote: > > > > Add the documentation for the new proposed Plane Color Pipeline. > > > > > > > > Co-developed-by: Chaitanya Kumar Borah > > > > > > > > Signed-off-by: Chaitanya Kumar Borah > > > > > > > > Signed-off-by: Uma Shankar > > > > --- > > > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++ > > > > 1 file changed, 394 insertions(+) > > > > create mode 100644 > > > > Documentation/gpu/rfc/plane_color_pipeline.rst > > > > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst > > > > new file mode 100644 > > > > index ..60ce515b6ea7 > > > > --- /dev/null > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst > > ... > > Hi Uma! Thanks Pekka for the feedback and useful inputs. > > > > +This color pipeline is then packaged within a blob for the user > > > > +space to retrieve it. Details can be found in the next section > > > > + > > > > > > Not sure I like blobs that contain other blob ids. > > > > It provides flexibility and helps with just one interface to > > userspace. Its easy to handle and manage once we get the hang of it 😊. > > > > We can clearly define the steps of parsing and data structures to be > > used while interpreting and parsing the blobs. > > Don't forget extendability. Possibly every single struct will need some kind > of > versioning, and then it's not simple to parse anymore. Add to that new/old > kernel > vs. old/new userspace, and it seems a bit nightmarish to design. Structure to be used to interpret the blob should be defined as UAPI only and is not expected to change once agreed upon. It should be interpreted like a standard property. So structure to be used, say for 3dLut or 1dlut or CTM operations should be standardized and fixed. No versioning of structure should be done and same rules/restrictions as of UAPI property should be applied. New vs old userspace problem exists even today as you rightly highlighted in mail below, however we are planning to propose that we clean the hardware state once the userspace client switches or same client switches the pipeline. > Also since it's records inside a single blob, it's like a new file > format: every record needs a standard header that allows skipping it > appropriately if userspace does not understand it, or you need a standard > index > telling where everything is. Making all records the same size would waste > space, > and extendability requires variable size. The design currently implements 1 hardware block by a struct drm_color_op data structure. Multiple such blocks make the pipeline. So userspace just needs to get the pipeline and then parse blocks 1 by 1. For blocks which it doesn't understand it can just skip and move to the next one. Each block is differentiated by a unique "name" standardized by an enum which will be part of the UAPI. Thus we will have scope for variable size blob to represent the particular hardware pipeline, userspace can parse and implement whichever blocks it understands. Only rule defined by UAPI is the way the respective block is to be parsed and programmed. > I also would not assume that we can declare a standard set of blocks and that > nothing else will be needed. The existing hardware is too diverse for that > from > what I have understood. I assume that some hardware have blocks unique to > them, and they want to at least expose that functionality through a UAPI that > allows at least generic
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
On Wed, 30 Aug 2023 08:59:36 +
"Shankar, Uma" wrote:
> > -Original Message-
> > From: Harry Wentland
> > Sent: Wednesday, August 30, 2023 1:10 AM
> > To: Shankar, Uma ; intel-...@lists.freedesktop.org;
> > dri-
> > de...@lists.freedesktop.org
> > Cc: Borah, Chaitanya Kumar ; wayland-
> > de...@lists.freedesktop.org
> > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> > Color
> > Pipeline
> >
> >
> >
> > On 2023-08-29 12:03, Uma Shankar wrote:
> > > Add the documentation for the new proposed Plane Color Pipeline.
> > >
> > > Co-developed-by: Chaitanya Kumar Borah
> > >
> > > Signed-off-by: Chaitanya Kumar Borah
> > > Signed-off-by: Uma Shankar
> > > ---
> > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++
> > > 1 file changed, 394 insertions(+)
> > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
> > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > new file mode 100644
> > > index ..60ce515b6ea7
> > > --- /dev/null
> > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
...
Hi Uma!
> > > +This color pipeline is then packaged within a blob for the user space
> > > +to retrieve it. Details can be found in the next section
> > > +
> >
> > Not sure I like blobs that contain other blob ids.
>
> It provides flexibility and helps with just one interface to userspace. Its
> easy to handle and
> manage once we get the hang of it 😊.
>
> We can clearly define the steps of parsing and data structures to be used
> while interpreting
> and parsing the blobs.
Don't forget extendability. Possibly every single struct will need some
kind of versioning, and then it's not simple to parse anymore. Add to
that new/old kernel vs. old/new userspace, and it seems a bit
nightmarish to design.
Also since it's records inside a single blob, it's like a new file
format: every record needs a standard header that allows skipping it
appropriately if userspace does not understand it, or you need a
standard index telling where everything is. Making all records the same
size would waste space, and extendability requires variable size.
I also would not assume that we can declare a standard set of blocks
and that nothing else will be needed. The existing hardware is too
diverse for that from what I have understood. I assume that some
hardware have blocks unique to them, and they want to at least expose
that functionality through a UAPI that allows at least generic
enumeration of functionality, even if it needs specialized userspace
code to actually make use of.
If we go with
+struct drm_color_op {
+ enum color_op_block name;
+ enum color_op_type type;
+ u32 blob_id;
+ u32 private_flags;
+};
as in your proposal, I believe it can work (sorry, looking further
down, I have assumed too much of 'type'), but the enumerations will
become long, and the details blob_id is still specific to 'type'. This
is unavoidable, but we can still choose the form between blobs and
properties, integers and strings.
I have a feeling that introspection will be valuable here, to help
people understand what their hardware could do if they had the code to
use it. 'name' and 'type' being integers require a translation table to
strings before they are readable, and it would be best if the kernel
itself provided that translation.
I don't understand how 'private_flags' could be useful. There must not
be any "hidden" features. Everything a block can be programmed to do
via this UAPI must be clearly documented, there cannot be anything
private. If two hardware versions of a block differ in a meaningful or
significant way, they need to be exposed as different types of blocks.
OTOH, if one goes with a (new) DRM object with string named properties
model, all that struct versioning and file format hassle has mostly a
clear and well-understood solution. We only need to define the rules of
how userspace needs to deal with properties or values it does not
understand, so that the kernel can keep adding more.
Therefore, I'm not yet convinced with the "all blobs" design.
> > > +Exposing a color pipeline to user space
> > > +===
> > > +
> > > +To advertise the available color pipelines, an immutable ENUM
> > > +property "GET_COLOR_PIPELINE" is introduced.
> > > +This enum property has blob id's as valu
RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
> -Original Message- > From: Harry Wentland > Sent: Wednesday, August 30, 2023 1:10 AM > To: Shankar, Uma ; intel-...@lists.freedesktop.org; > dri- > de...@lists.freedesktop.org > Cc: Borah, Chaitanya Kumar ; wayland- > de...@lists.freedesktop.org > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane > Color > Pipeline > > > > On 2023-08-29 12:03, Uma Shankar wrote: > > Add the documentation for the new proposed Plane Color Pipeline. > > > > Co-developed-by: Chaitanya Kumar Borah > > > > Signed-off-by: Chaitanya Kumar Borah > > Signed-off-by: Uma Shankar > > --- > > .../gpu/rfc/plane_color_pipeline.rst | 394 ++ > > 1 file changed, 394 insertions(+) > > create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst > > b/Documentation/gpu/rfc/plane_color_pipeline.rst > > new file mode 100644 > > index ..60ce515b6ea7 > > --- /dev/null > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst > > @@ -0,0 +1,394 @@ > > +=== > > + Plane Color Pipeline: A UAPI proposal > > +=== > > + > > +To build the proposal on, lets take the premise of a color pipeline > > +as shown below. > > + > > + +---+ > > + |RAM| > > + | +--++-++-+ | > > + | | FB 1 || FB 2 || FB N| | > > + | +--++-++-+ | > > + +---+ > > + | Plane Color Hardware Block | > > + ++ > > + | +---v-+ +---v---+ +---v--+ | > > + | | Plane A | | Plane B | | Plane N | | > > + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | > > + | +---+-+ +---+---+ +---+--+ | > > + | | | || > > + | +---v-+ +---v---+ +---v--+ | > > + | |Plane A | | Plane B | | Plane N | | > > + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | > > + | +---+-+ ++--+ ++-+ | > > + | | | | | > > + | +---v-+ +v--+ +v-+ | > > + | | Plane A | | Plane B | | Plane N | | > > + | |Post-CSC | | Post-CSC | | Post-CSC | | > > + | +---+-+ ++--+ ++-+ | > > + | | | | | > > + ++ > > ++--v--v---v---| > > +|| || > > +|| Pipe Blender|| > > ++++ > > +||| > > +|+---v--+ | > > +|| Pipe Pre-CSC| | > > +|| | | > > +|+---+--+ | > > +||Pipe Color | > > +|+---v--+ Hardware| > > +|| Pipe CSC/CTM| | > > +|| | | > > +|+---+--+ | > > +||| > > +|+---v--+ | > > +|| Pipe Post-CSC | | > > +|| | | > > +|+---+--+ | > > +||| > > ++-+ > > + | > > + v > > +Pipe Output > > + > > +Each plane consists of the following color blocks > > + * Pre-CSC : This block can used to linearize the input frame buffer data. > > + The linear data then can be further acted on by the following > > + color hardware blocks in the display hardware pipeline > > + > > + * CSC/CTM: Used to program color transformation matrix, this block is used > > +to perform color space conversions like BT2020 to BT709 or > > BT601 > > +etc. This block acts on the linearized data coming from the > > +Pre-CSC HW block. > > + > > + * Post-CSC: This HW block can be used to non-linea
Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline
On 2023-08-29 12:03, Uma Shankar wrote: Add the documentation for the new proposed Plane Color Pipeline. Co-developed-by: Chaitanya Kumar Borah Signed-off-by: Chaitanya Kumar Borah Signed-off-by: Uma Shankar --- .../gpu/rfc/plane_color_pipeline.rst | 394 ++ 1 file changed, 394 insertions(+) create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst b/Documentation/gpu/rfc/plane_color_pipeline.rst new file mode 100644 index ..60ce515b6ea7 --- /dev/null +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst @@ -0,0 +1,394 @@ +=== + Plane Color Pipeline: A UAPI proposal +=== + +To build the proposal on, lets take the premise of a color pipeline as shown +below. + + +---+ + |RAM| + | +--++-++-+ | + | | FB 1 || FB 2 || FB N| | + | +--++-++-+ | + +---+ + | Plane Color Hardware Block | + ++ + | +---v-+ +---v---+ +---v--+ | + | | Plane A | | Plane B | | Plane N | | + | | Pre-CSC | | Pre-CSC | | Pre-CSC | | + | +---+-+ +---+---+ +---+--+ | + | | | || + | +---v-+ +---v---+ +---v--+ | + | |Plane A | | Plane B | | Plane N | | + | |CSC/CTM | | CSC/CTM | | CSC/CTM | | + | +---+-+ ++--+ ++-+ | + | | | | | + | +---v-+ +v--+ +v-+ | + | | Plane A | | Plane B | | Plane N | | + | |Post-CSC | | Post-CSC | | Post-CSC | | + | +---+-+ ++--+ ++-+ | + | | | | | + ++ ++--v--v---v---| +|| || +|| Pipe Blender|| ++++ +||| +|+---v--+ | +|| Pipe Pre-CSC| | +|| | | +|+---+--+ | +||Pipe Color | +|+---v--+ Hardware| +|| Pipe CSC/CTM| | +|| | | +|+---+--+ | +||| +|+---v--+ | +|| Pipe Post-CSC | | +|| | | +|+---+--+ | +||| ++-+ + | + v +Pipe Output + +Each plane consists of the following color blocks + * Pre-CSC : This block can used to linearize the input frame buffer data. + The linear data then can be further acted on by the following + color hardware blocks in the display hardware pipeline + + * CSC/CTM: Used to program color transformation matrix, this block is used +to perform color space conversions like BT2020 to BT709 or BT601 +etc. This block acts on the linearized data coming from the +Pre-CSC HW block. + + * Post-CSC: This HW block can be used to non-linearize frame buffer data to + match the sink. Another use case of it could be to perform Tone + mapping for HDR use-cases. + +Data from multiple planes will then be fed to pipe/crtc where it will get blended. +There is a similar set of HW blocks available at pipe/crtc level which acts on +this blended data. + +Below is a sample usecase fo video playback with sub-titles and playback +controls + +┌┐┌─┐ ┌─┐┌─┐ +│FB1 ││PRE-CSC │ │ CTM Matrix ││ POST-CSC│ +│├───►│Linearize├►│ BT709 to├───►│ SDR to HDR │ +│BT709 SDR ││ │ │ BT2020 ││ Tone Mapping├─┐ +└┘└─┘ └─┘└─┘ │ +(subtitles) │ + │ +┌┐┌─┐ ┌─┐┌─┐ │ +│FB2 ││PRE-CSC │ │ CTM Matrix ││ POST-CSC│ │ +│├───►│Linearize├►│ BT601 to├───►│ SDR to HDR ├───┐ │ +│BT601 SDR ││ │ │ BT2020 ││ Tone Mapping│ │ │ +
