Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
On Fri, 2018-10-05 at 13:49 -0300, Mauro Carvalho Chehab wrote:
> Em Fri, 05 Oct 2018 12:37:43 -0300
> Ezequiel Garcia escreveu:
>
> > On Fri, 2018-10-05 at 09:10 -0300, Mauro Carvalho Chehab wrote:
> > > Em Thu, 04 Oct 2018 20:39:31 -0300
> > > Ezequiel Garcia escreveu:
> > >
> > > > On Mon, 2018-10-01 at 14:54 -0300, Ezequiel Garcia wrote:
> > > > > On Fri, 2018-09-28 at 14:33 +0200, Hans Verkuil wrote:
> > > > > > On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> > > > > > > This series adds support for JPEG encoding via the VPU block
> > > > > > > present in Rockchip platforms. Currently, support for RK3288
> > > > > > > and RK3399 is included.
> > > > > > >
> > > > > > > Please, see the previous versions of this cover letter for
> > > > > > > more information.
> > > > > > >
> > > > > > > Compared to v5, the only change is in the
> > > > > > > V4L2_CID_JPEG_QUANTIZATION
> > > > > > > control. We've decided to support only baseline profile,
> > > > > > > and only add 8-bit luma and chroma tables.
> > > > > > >
> > > > > > > struct v4l2_ctrl_jpeg_quantization {
> > > > > > >__u8luma_quantization_matrix[64];
> > > > > > >__u8chroma_quantization_matrix[64];
> > > > > > > };
> > > > > > >
> > > > > > > By doing this, it's clear that we don't currently support anything
> > > > > > > but baseline.
> > > > > > >
> > > > > > > This series should apply cleanly on top of
> > > > > > >
> > > > > > > git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
> > > > > > >
> > > > > > > If everyone is happy with this series, I'd like to route the
> > > > > > > devicetree
> > > > > > > changes through the rockchip tree, and the rest via the media
> > > > > > > subsystem.
> > > > > >
> > > > > > OK, so I have what is no doubt an annoying question: do we really
> > > > > > need
> > > > > > a JPEG_RAW format?
> > > > > >
> > > > >
> > > > > Not annoying, as it helps clarify a few things :-)
> > > > > I think we do need the JPEG_RAW format. The way I see it, using
> > > > > JPEG opens a can of worms...
> > > > >
> > > > > > The JPEG header is really easy to parse for a decoder and really
> > > > > > easy to
> > > > > > prepend to the compressed image for the encoder.
> > > > > >
> > > > > > The only reason I can see for a JPEG_RAW is if the image must start
> > > > > > at
> > > > > > some specific address alignment. Although even then you can just
> > > > > > pad the
> > > > > > JPEG header that you will add according to the alignment
> > > > > > requirements.
> > > > > >
> > > > > > I know I am very late with this question, but I never looked all
> > > > > > that
> > > > > > closely at what a JPEG header looks like. But this helped:
> > > > > >
> > > > > > https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
> > > > > >
> > > > > > and it doesn't seem difficult at all to parse or create the header.
> > > > > >
> > > > > >
> > > > >
> > > > > ... I think that having JPEG_RAW as the compressed format
> > > > > is much more clear for userspace, as it explicitly specifies
> > > > > what is expected.
> > > > >
> > > > > This way, for a stateless encoder, applications are required
> > > > > to set quantization and/or entropy tables, and are then in
> > > > > charge of using the compressed JPEG_RAW payload in whatever form
> > > > > they want. Stupid simple.
> > > > >
> > > > > On the other side, if the stateless encoder driver supports
> > > > > JPEG (creating headers in-kernel), it means that:
> > > > >
> > > > > *) applications should pass a quality control, if supported,
> > > > > and the driver will use hardcoded tables or...
> > > > >
> > > > > *) applications pass quantization control and, if supported,
> > > > > entropy control. The kernel uses them to create the JPEG frame.
> > > > > But also, some drivers (e.g. Rockchip), use default entropy
> > > > > tables, which should now be in the kernel.
> > > > >
> > > > > So the application would have to query controls to find out
> > > > > what to do. Not exactly hard, but I think having the JPEG_RAW
> > > > > is much simpler and more clear.
> > > > >
> > > > > Now, for stateless decoders, supporting JPEG_RAW means
> > > > > the application has to pass quantization and entropy
> > > > > controls, probably using the Request API.
> > > > > Given the application has parsed the JPEG,
> > > > > it knows the width and height and can request
> > > > > buffers accordingly.
> > > > >
> > > > > The hardware is stateless, and so is the driver.
> > > > >
> > > > > On the other hand, supporting JPEG would mean that
> > > > > drivers will have to parse the image, extracting
> > > > > the quantization and entropy tables.
> > > > >
> > > > > Format negotiation is now more complex,
> > > > > either we follow the stateful spec, introducing a little
> > > > > state machine in the driver... or we use the Request API,
> > > > > but that means parsing on both sides kernel and application.
> > > > >
Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
Em Fri, 05 Oct 2018 12:37:43 -0300
Ezequiel Garcia escreveu:
> On Fri, 2018-10-05 at 09:10 -0300, Mauro Carvalho Chehab wrote:
> > Em Thu, 04 Oct 2018 20:39:31 -0300
> > Ezequiel Garcia escreveu:
> >
> > > On Mon, 2018-10-01 at 14:54 -0300, Ezequiel Garcia wrote:
> > > > On Fri, 2018-09-28 at 14:33 +0200, Hans Verkuil wrote:
> > > > > On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> > > > > > This series adds support for JPEG encoding via the VPU block
> > > > > > present in Rockchip platforms. Currently, support for RK3288
> > > > > > and RK3399 is included.
> > > > > >
> > > > > > Please, see the previous versions of this cover letter for
> > > > > > more information.
> > > > > >
> > > > > > Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
> > > > > > control. We've decided to support only baseline profile,
> > > > > > and only add 8-bit luma and chroma tables.
> > > > > >
> > > > > > struct v4l2_ctrl_jpeg_quantization {
> > > > > >__u8luma_quantization_matrix[64];
> > > > > >__u8chroma_quantization_matrix[64];
> > > > > > };
> > > > > >
> > > > > > By doing this, it's clear that we don't currently support anything
> > > > > > but baseline.
> > > > > >
> > > > > > This series should apply cleanly on top of
> > > > > >
> > > > > > git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
> > > > > >
> > > > > > If everyone is happy with this series, I'd like to route the
> > > > > > devicetree
> > > > > > changes through the rockchip tree, and the rest via the media
> > > > > > subsystem.
> > > > >
> > > > > OK, so I have what is no doubt an annoying question: do we really need
> > > > > a JPEG_RAW format?
> > > > >
> > > >
> > > > Not annoying, as it helps clarify a few things :-)
> > > > I think we do need the JPEG_RAW format. The way I see it, using
> > > > JPEG opens a can of worms...
> > > >
> > > > > The JPEG header is really easy to parse for a decoder and really easy
> > > > > to
> > > > > prepend to the compressed image for the encoder.
> > > > >
> > > > > The only reason I can see for a JPEG_RAW is if the image must start at
> > > > > some specific address alignment. Although even then you can just pad
> > > > > the
> > > > > JPEG header that you will add according to the alignment requirements.
> > > > >
> > > > > I know I am very late with this question, but I never looked all that
> > > > > closely at what a JPEG header looks like. But this helped:
> > > > >
> > > > > https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
> > > > >
> > > > > and it doesn't seem difficult at all to parse or create the header.
> > > > >
> > > > >
> > > >
> > > > ... I think that having JPEG_RAW as the compressed format
> > > > is much more clear for userspace, as it explicitly specifies
> > > > what is expected.
> > > >
> > > > This way, for a stateless encoder, applications are required
> > > > to set quantization and/or entropy tables, and are then in
> > > > charge of using the compressed JPEG_RAW payload in whatever form
> > > > they want. Stupid simple.
> > > >
> > > > On the other side, if the stateless encoder driver supports
> > > > JPEG (creating headers in-kernel), it means that:
> > > >
> > > > *) applications should pass a quality control, if supported,
> > > > and the driver will use hardcoded tables or...
> > > >
> > > > *) applications pass quantization control and, if supported,
> > > > entropy control. The kernel uses them to create the JPEG frame.
> > > > But also, some drivers (e.g. Rockchip), use default entropy
> > > > tables, which should now be in the kernel.
> > > >
> > > > So the application would have to query controls to find out
> > > > what to do. Not exactly hard, but I think having the JPEG_RAW
> > > > is much simpler and more clear.
> > > >
> > > > Now, for stateless decoders, supporting JPEG_RAW means
> > > > the application has to pass quantization and entropy
> > > > controls, probably using the Request API.
> > > > Given the application has parsed the JPEG,
> > > > it knows the width and height and can request
> > > > buffers accordingly.
> > > >
> > > > The hardware is stateless, and so is the driver.
> > > >
> > > > On the other hand, supporting JPEG would mean that
> > > > drivers will have to parse the image, extracting
> > > > the quantization and entropy tables.
> > > >
> > > > Format negotiation is now more complex,
> > > > either we follow the stateful spec, introducing a little
> > > > state machine in the driver... or we use the Request API,
> > > > but that means parsing on both sides kernel and application.
> > > >
> > > > Either way, using JPEG_RAW is just waaay simpler and puts
> > > > things where they belong.
> > > >
> > >
> > > As discussed on IRC, I'm sending a v7 for this series,
> > > fixing only the checkpatch issues and the extra line in the
> > > binding document.
> > >
> > > We've agreed to move forward with JPEG_RAW, for the
Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
On Fri, 2018-10-05 at 09:10 -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 04 Oct 2018 20:39:31 -0300
> Ezequiel Garcia escreveu:
>
> > On Mon, 2018-10-01 at 14:54 -0300, Ezequiel Garcia wrote:
> > > On Fri, 2018-09-28 at 14:33 +0200, Hans Verkuil wrote:
> > > > On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> > > > > This series adds support for JPEG encoding via the VPU block
> > > > > present in Rockchip platforms. Currently, support for RK3288
> > > > > and RK3399 is included.
> > > > >
> > > > > Please, see the previous versions of this cover letter for
> > > > > more information.
> > > > >
> > > > > Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
> > > > > control. We've decided to support only baseline profile,
> > > > > and only add 8-bit luma and chroma tables.
> > > > >
> > > > > struct v4l2_ctrl_jpeg_quantization {
> > > > >__u8luma_quantization_matrix[64];
> > > > >__u8chroma_quantization_matrix[64];
> > > > > };
> > > > >
> > > > > By doing this, it's clear that we don't currently support anything
> > > > > but baseline.
> > > > >
> > > > > This series should apply cleanly on top of
> > > > >
> > > > > git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
> > > > >
> > > > > If everyone is happy with this series, I'd like to route the
> > > > > devicetree
> > > > > changes through the rockchip tree, and the rest via the media
> > > > > subsystem.
> > > >
> > > > OK, so I have what is no doubt an annoying question: do we really need
> > > > a JPEG_RAW format?
> > > >
> > >
> > > Not annoying, as it helps clarify a few things :-)
> > > I think we do need the JPEG_RAW format. The way I see it, using
> > > JPEG opens a can of worms...
> > >
> > > > The JPEG header is really easy to parse for a decoder and really easy to
> > > > prepend to the compressed image for the encoder.
> > > >
> > > > The only reason I can see for a JPEG_RAW is if the image must start at
> > > > some specific address alignment. Although even then you can just pad the
> > > > JPEG header that you will add according to the alignment requirements.
> > > >
> > > > I know I am very late with this question, but I never looked all that
> > > > closely at what a JPEG header looks like. But this helped:
> > > >
> > > > https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
> > > >
> > > > and it doesn't seem difficult at all to parse or create the header.
> > > >
> > > >
> > >
> > > ... I think that having JPEG_RAW as the compressed format
> > > is much more clear for userspace, as it explicitly specifies
> > > what is expected.
> > >
> > > This way, for a stateless encoder, applications are required
> > > to set quantization and/or entropy tables, and are then in
> > > charge of using the compressed JPEG_RAW payload in whatever form
> > > they want. Stupid simple.
> > >
> > > On the other side, if the stateless encoder driver supports
> > > JPEG (creating headers in-kernel), it means that:
> > >
> > > *) applications should pass a quality control, if supported,
> > > and the driver will use hardcoded tables or...
> > >
> > > *) applications pass quantization control and, if supported,
> > > entropy control. The kernel uses them to create the JPEG frame.
> > > But also, some drivers (e.g. Rockchip), use default entropy
> > > tables, which should now be in the kernel.
> > >
> > > So the application would have to query controls to find out
> > > what to do. Not exactly hard, but I think having the JPEG_RAW
> > > is much simpler and more clear.
> > >
> > > Now, for stateless decoders, supporting JPEG_RAW means
> > > the application has to pass quantization and entropy
> > > controls, probably using the Request API.
> > > Given the application has parsed the JPEG,
> > > it knows the width and height and can request
> > > buffers accordingly.
> > >
> > > The hardware is stateless, and so is the driver.
> > >
> > > On the other hand, supporting JPEG would mean that
> > > drivers will have to parse the image, extracting
> > > the quantization and entropy tables.
> > >
> > > Format negotiation is now more complex,
> > > either we follow the stateful spec, introducing a little
> > > state machine in the driver... or we use the Request API,
> > > but that means parsing on both sides kernel and application.
> > >
> > > Either way, using JPEG_RAW is just waaay simpler and puts
> > > things where they belong.
> > >
> >
> > As discussed on IRC, I'm sending a v7 for this series,
> > fixing only the checkpatch issues and the extra line in the
> > binding document.
> >
> > We've agreed to move forward with JPEG_RAW, for the reasons
> > stated above.
> >
> > Plus, we've agreed to keep this in staging until the userspace
> > support for JPEG_RAW format is clear.
>
> The problem is that a new format is actually a V4L2 core change, and
> not something for staging.
>
> IMHO, it is prudent to wait for an userspace code before hushing
> merging this
Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
Em Thu, 04 Oct 2018 20:39:31 -0300
Ezequiel Garcia escreveu:
> On Mon, 2018-10-01 at 14:54 -0300, Ezequiel Garcia wrote:
> > On Fri, 2018-09-28 at 14:33 +0200, Hans Verkuil wrote:
> > > On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> > > > This series adds support for JPEG encoding via the VPU block
> > > > present in Rockchip platforms. Currently, support for RK3288
> > > > and RK3399 is included.
> > > >
> > > > Please, see the previous versions of this cover letter for
> > > > more information.
> > > >
> > > > Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
> > > > control. We've decided to support only baseline profile,
> > > > and only add 8-bit luma and chroma tables.
> > > >
> > > > struct v4l2_ctrl_jpeg_quantization {
> > > >__u8luma_quantization_matrix[64];
> > > >__u8chroma_quantization_matrix[64];
> > > > };
> > > >
> > > > By doing this, it's clear that we don't currently support anything
> > > > but baseline.
> > > >
> > > > This series should apply cleanly on top of
> > > >
> > > > git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
> > > >
> > > > If everyone is happy with this series, I'd like to route the devicetree
> > > > changes through the rockchip tree, and the rest via the media
> > > > subsystem.
> > >
> > > OK, so I have what is no doubt an annoying question: do we really need
> > > a JPEG_RAW format?
> > >
> >
> > Not annoying, as it helps clarify a few things :-)
> > I think we do need the JPEG_RAW format. The way I see it, using
> > JPEG opens a can of worms...
> >
> > > The JPEG header is really easy to parse for a decoder and really easy to
> > > prepend to the compressed image for the encoder.
> > >
> > > The only reason I can see for a JPEG_RAW is if the image must start at
> > > some specific address alignment. Although even then you can just pad the
> > > JPEG header that you will add according to the alignment requirements.
> > >
> > > I know I am very late with this question, but I never looked all that
> > > closely at what a JPEG header looks like. But this helped:
> > >
> > > https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
> > >
> > > and it doesn't seem difficult at all to parse or create the header.
> > >
> > >
> >
> > ... I think that having JPEG_RAW as the compressed format
> > is much more clear for userspace, as it explicitly specifies
> > what is expected.
> >
> > This way, for a stateless encoder, applications are required
> > to set quantization and/or entropy tables, and are then in
> > charge of using the compressed JPEG_RAW payload in whatever form
> > they want. Stupid simple.
> >
> > On the other side, if the stateless encoder driver supports
> > JPEG (creating headers in-kernel), it means that:
> >
> > *) applications should pass a quality control, if supported,
> > and the driver will use hardcoded tables or...
> >
> > *) applications pass quantization control and, if supported,
> > entropy control. The kernel uses them to create the JPEG frame.
> > But also, some drivers (e.g. Rockchip), use default entropy
> > tables, which should now be in the kernel.
> >
> > So the application would have to query controls to find out
> > what to do. Not exactly hard, but I think having the JPEG_RAW
> > is much simpler and more clear.
> >
> > Now, for stateless decoders, supporting JPEG_RAW means
> > the application has to pass quantization and entropy
> > controls, probably using the Request API.
> > Given the application has parsed the JPEG,
> > it knows the width and height and can request
> > buffers accordingly.
> >
> > The hardware is stateless, and so is the driver.
> >
> > On the other hand, supporting JPEG would mean that
> > drivers will have to parse the image, extracting
> > the quantization and entropy tables.
> >
> > Format negotiation is now more complex,
> > either we follow the stateful spec, introducing a little
> > state machine in the driver... or we use the Request API,
> > but that means parsing on both sides kernel and application.
> >
> > Either way, using JPEG_RAW is just waaay simpler and puts
> > things where they belong.
> >
>
> As discussed on IRC, I'm sending a v7 for this series,
> fixing only the checkpatch issues and the extra line in the
> binding document.
>
> We've agreed to move forward with JPEG_RAW, for the reasons
> stated above.
>
> Plus, we've agreed to keep this in staging until the userspace
> support for JPEG_RAW format is clear.
The problem is that a new format is actually a V4L2 core change, and
not something for staging.
IMHO, it is prudent to wait for an userspace code before hushing
merging this upstream.
-
I'm concerned about adding a JPEG_RAW format. We had this discussion
in the past. On that time, it was opted to add a code inside the
drivers that will be adding the jpeg headers, on the cases where
the hardware was providing headerless frames.
Among the reasons, different drivers may be
Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
On Mon, 2018-10-01 at 14:54 -0300, Ezequiel Garcia wrote:
> On Fri, 2018-09-28 at 14:33 +0200, Hans Verkuil wrote:
> > On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> > > This series adds support for JPEG encoding via the VPU block
> > > present in Rockchip platforms. Currently, support for RK3288
> > > and RK3399 is included.
> > >
> > > Please, see the previous versions of this cover letter for
> > > more information.
> > >
> > > Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
> > > control. We've decided to support only baseline profile,
> > > and only add 8-bit luma and chroma tables.
> > >
> > > struct v4l2_ctrl_jpeg_quantization {
> > >__u8luma_quantization_matrix[64];
> > >__u8chroma_quantization_matrix[64];
> > > };
> > >
> > > By doing this, it's clear that we don't currently support anything
> > > but baseline.
> > >
> > > This series should apply cleanly on top of
> > >
> > > git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
> > >
> > > If everyone is happy with this series, I'd like to route the devicetree
> > > changes through the rockchip tree, and the rest via the media subsystem.
> >
> > OK, so I have what is no doubt an annoying question: do we really need
> > a JPEG_RAW format?
> >
>
> Not annoying, as it helps clarify a few things :-)
> I think we do need the JPEG_RAW format. The way I see it, using
> JPEG opens a can of worms...
>
> > The JPEG header is really easy to parse for a decoder and really easy to
> > prepend to the compressed image for the encoder.
> >
> > The only reason I can see for a JPEG_RAW is if the image must start at
> > some specific address alignment. Although even then you can just pad the
> > JPEG header that you will add according to the alignment requirements.
> >
> > I know I am very late with this question, but I never looked all that
> > closely at what a JPEG header looks like. But this helped:
> >
> > https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
> >
> > and it doesn't seem difficult at all to parse or create the header.
> >
> >
>
> ... I think that having JPEG_RAW as the compressed format
> is much more clear for userspace, as it explicitly specifies
> what is expected.
>
> This way, for a stateless encoder, applications are required
> to set quantization and/or entropy tables, and are then in
> charge of using the compressed JPEG_RAW payload in whatever form
> they want. Stupid simple.
>
> On the other side, if the stateless encoder driver supports
> JPEG (creating headers in-kernel), it means that:
>
> *) applications should pass a quality control, if supported,
> and the driver will use hardcoded tables or...
>
> *) applications pass quantization control and, if supported,
> entropy control. The kernel uses them to create the JPEG frame.
> But also, some drivers (e.g. Rockchip), use default entropy
> tables, which should now be in the kernel.
>
> So the application would have to query controls to find out
> what to do. Not exactly hard, but I think having the JPEG_RAW
> is much simpler and more clear.
>
> Now, for stateless decoders, supporting JPEG_RAW means
> the application has to pass quantization and entropy
> controls, probably using the Request API.
> Given the application has parsed the JPEG,
> it knows the width and height and can request
> buffers accordingly.
>
> The hardware is stateless, and so is the driver.
>
> On the other hand, supporting JPEG would mean that
> drivers will have to parse the image, extracting
> the quantization and entropy tables.
>
> Format negotiation is now more complex,
> either we follow the stateful spec, introducing a little
> state machine in the driver... or we use the Request API,
> but that means parsing on both sides kernel and application.
>
> Either way, using JPEG_RAW is just waaay simpler and puts
> things where they belong.
>
As discussed on IRC, I'm sending a v7 for this series,
fixing only the checkpatch issues and the extra line in the
binding document.
We've agreed to move forward with JPEG_RAW, for the reasons
stated above.
Plus, we've agreed to keep this in staging until the userspace
support for JPEG_RAW format is clear.
Regards,
Ezequiel
Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
On Fri, 2018-09-28 at 14:33 +0200, Hans Verkuil wrote:
> On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> > This series adds support for JPEG encoding via the VPU block
> > present in Rockchip platforms. Currently, support for RK3288
> > and RK3399 is included.
> >
> > Please, see the previous versions of this cover letter for
> > more information.
> >
> > Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
> > control. We've decided to support only baseline profile,
> > and only add 8-bit luma and chroma tables.
> >
> > struct v4l2_ctrl_jpeg_quantization {
> >__u8luma_quantization_matrix[64];
> >__u8chroma_quantization_matrix[64];
> > };
> >
> > By doing this, it's clear that we don't currently support anything
> > but baseline.
> >
> > This series should apply cleanly on top of
> >
> > git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
> >
> > If everyone is happy with this series, I'd like to route the devicetree
> > changes through the rockchip tree, and the rest via the media subsystem.
>
> OK, so I have what is no doubt an annoying question: do we really need
> a JPEG_RAW format?
>
Not annoying, as it helps clarify a few things :-)
I think we do need the JPEG_RAW format. The way I see it, using
JPEG opens a can of worms...
> The JPEG header is really easy to parse for a decoder and really easy to
> prepend to the compressed image for the encoder.
>
> The only reason I can see for a JPEG_RAW is if the image must start at
> some specific address alignment. Although even then you can just pad the
> JPEG header that you will add according to the alignment requirements.
>
> I know I am very late with this question, but I never looked all that
> closely at what a JPEG header looks like. But this helped:
>
> https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
>
> and it doesn't seem difficult at all to parse or create the header.
>
>
... I think that having JPEG_RAW as the compressed format
is much more clear for userspace, as it explicitly specifies
what is expected.
This way, for a stateless encoder, applications are required
to set quantization and/or entropy tables, and are then in
charge of using the compressed JPEG_RAW payload in whatever form
they want. Stupid simple.
On the other side, if the stateless encoder driver supports
JPEG (creating headers in-kernel), it means that:
*) applications should pass a quality control, if supported,
and the driver will use hardcoded tables or...
*) applications pass quantization control and, if supported,
entropy control. The kernel uses them to create the JPEG frame.
But also, some drivers (e.g. Rockchip), use default entropy
tables, which should now be in the kernel.
So the application would have to query controls to find out
what to do. Not exactly hard, but I think having the JPEG_RAW
is much simpler and more clear.
Now, for stateless decoders, supporting JPEG_RAW means
the application has to pass quantization and entropy
controls, probably using the Request API.
Given the application has parsed the JPEG,
it knows the width and height and can request
buffers accordingly.
The hardware is stateless, and so is the driver.
On the other hand, supporting JPEG would mean that
drivers will have to parse the image, extracting
the quantization and entropy tables.
Format negotiation is now more complex,
either we follow the stateful spec, introducing a little
state machine in the driver... or we use the Request API,
but that means parsing on both sides kernel and application.
Either way, using JPEG_RAW is just waaay simpler and puts
things where they belong.
> I also think there are more drivers (solo6x10) that
> manipulate the JPEG header.
Well, I've always thought this was kind of suboptimal.
Thanks,
Ezequiel
Re: [PATCH v6 0/6] Add Rockchip VPU JPEG encoder
On 09/17/2018 07:30 PM, Ezequiel Garcia wrote:
> This series adds support for JPEG encoding via the VPU block
> present in Rockchip platforms. Currently, support for RK3288
> and RK3399 is included.
>
> Please, see the previous versions of this cover letter for
> more information.
>
> Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
> control. We've decided to support only baseline profile,
> and only add 8-bit luma and chroma tables.
>
> struct v4l2_ctrl_jpeg_quantization {
>__u8luma_quantization_matrix[64];
>__u8chroma_quantization_matrix[64];
> };
>
> By doing this, it's clear that we don't currently support anything
> but baseline.
>
> This series should apply cleanly on top of
>
> git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
>
> If everyone is happy with this series, I'd like to route the devicetree
> changes through the rockchip tree, and the rest via the media subsystem.
OK, so I have what is no doubt an annoying question: do we really need
a JPEG_RAW format?
The JPEG header is really easy to parse for a decoder and really easy to
prepend to the compressed image for the encoder.
The only reason I can see for a JPEG_RAW is if the image must start at
some specific address alignment. Although even then you can just pad the
JPEG header that you will add according to the alignment requirements.
I know I am very late with this question, but I never looked all that
closely at what a JPEG header looks like. But this helped:
https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
and it doesn't seem difficult at all to parse or create the header.
I also think there are more drivers (solo6x10) that manipulate the JPEG
header.
Regards,
Hans
>
> Compliance
> ==
>
> (Same results as v3)
>
> v4l2-compliance SHA: d0f4ea7ddab6d0244c4fe1e960bb2aaeefb911b9, 64 bits
>
> Compliance test for device /dev/video0:
>
> Driver Info:
> Driver name : rockchip-vpu
> Card type: rockchip,rk3399-vpu-enc
> Bus info : platform: rockchip-vpu
> Driver version : 4.18.0
> Capabilities : 0x84204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04204000
> Video Memory-to-Memory Multiplanar
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : rockchip-vpu
> Model: rockchip-vpu
> Serial :
> Bus info :
> Media version: 4.18.0
> Hardware revision: 0x (0)
> Driver version : 4.18.0
> Interface Info:
> ID : 0x030c
> Type : V4L Video
> Entity Info:
> ID : 0x0001 (1)
> Name : rockchip,rk3399-vpu-enc-source
> Function : V4L2 I/O
> Pad 0x0102 : Source
> Link 0x0208: to remote pad 0x105 of entity
> 'rockchip,rk3399-vpu-enc-proc': Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 2 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not
[PATCH v6 0/6] Add Rockchip VPU JPEG encoder
This series adds support for JPEG encoding via the VPU block
present in Rockchip platforms. Currently, support for RK3288
and RK3399 is included.
Please, see the previous versions of this cover letter for
more information.
Compared to v5, the only change is in the V4L2_CID_JPEG_QUANTIZATION
control. We've decided to support only baseline profile,
and only add 8-bit luma and chroma tables.
struct v4l2_ctrl_jpeg_quantization {
__u8luma_quantization_matrix[64];
__u8chroma_quantization_matrix[64];
};
By doing this, it's clear that we don't currently support anything
but baseline.
This series should apply cleanly on top of
git://linuxtv.org/hverkuil/media_tree.git br-cedrus tag.
If everyone is happy with this series, I'd like to route the devicetree
changes through the rockchip tree, and the rest via the media subsystem.
Compliance
==
(Same results as v3)
v4l2-compliance SHA: d0f4ea7ddab6d0244c4fe1e960bb2aaeefb911b9, 64 bits
Compliance test for device /dev/video0:
Driver Info:
Driver name : rockchip-vpu
Card type: rockchip,rk3399-vpu-enc
Bus info : platform: rockchip-vpu
Driver version : 4.18.0
Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : rockchip-vpu
Model: rockchip-vpu
Serial :
Bus info :
Media version: 4.18.0
Hardware revision: 0x (0)
Driver version : 4.18.0
Interface Info:
ID : 0x030c
Type : V4L Video
Entity Info:
ID : 0x0001 (1)
Name : rockchip,rk3399-vpu-enc-source
Function : V4L2 I/O
Pad 0x0102 : Source
Link 0x0208: to remote pad 0x105 of entity
'rockchip,rk3399-vpu-enc-proc': Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 2 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
test blocking wait: OK
test MMAP: OK
test USERPTR: OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total: 48, Succeeded: 48, Failed: 0, Warnings: 0
v6:
- As agreed with Hans change the
