[staging:staging-testing] BUILD SUCCESS WITH WARNING 0e3136771b58dda2bec0a0bfedab119e77a88c9b
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git staging-testing branch HEAD: 0e3136771b58dda2bec0a0bfedab119e77a88c9b staging: mt7621-dts: properly organize pcie node Warning in current branch: drivers/comedi/drivers/jr3_pci.c:507:22: warning: variable 'min_full_scale' set but not used [-Wunused-but-set-variable] Warning ids grouped by kconfigs: gcc_recent_errors |-- alpha-allmodconfig | `-- drivers-comedi-drivers-jr3_pci.c:warning:variable-min_full_scale-set-but-not-used |-- ia64-allmodconfig | `-- drivers-comedi-drivers-jr3_pci.c:warning:variable-min_full_scale-set-but-not-used `-- sparc-allyesconfig `-- drivers-comedi-drivers-jr3_pci.c:warning:variable-min_full_scale-set-but-not-used elapsed time: 724m configs tested: 117 configs skipped: 2 gcc tested configs: arm defconfig arm64allyesconfig arm64 defconfig arm allyesconfig arm allmodconfig x86_64 allyesconfig riscvallmodconfig i386 allyesconfig riscvallyesconfig sh se7705_defconfig sh sh03_defconfig sh ecovec24_defconfig mipsar7_defconfig powerpc asp8347_defconfig sparc64 alldefconfig mipse55_defconfig mipsvocore2_defconfig powerpcicon_defconfig powerpcsocrates_defconfig powerpc tqm5200_defconfig sh sdk7780_defconfig m68k atari_defconfig m68kmvme147_defconfig arm integrator_defconfig ia64 tiger_defconfig m68k bvme6000_defconfig powerpc tqm8548_defconfig m68km5272c3_defconfig mips ip22_defconfig powerpc makalu_defconfig m68kq40_defconfig powerpc sbc8548_defconfig arm s5pv210_defconfig arm lpc32xx_defconfig armzeus_defconfig s390 allyesconfig arm pcm027_defconfig powerpc mpc837x_mds_defconfig powerpc ppa8548_defconfig arm imx_v4_v5_defconfig arm footbridge_defconfig riscv allnoconfig powerpc mpc83xx_defconfig arm pxa910_defconfig powerpc mpc836x_mds_defconfig armvt8500_v6_v7_defconfig powerpc mpc836x_rdk_defconfig archsdk_defconfig mips rbtx49xx_defconfig shsh7785lcr_defconfig ia64 allmodconfig ia64defconfig ia64 allyesconfig m68k allmodconfig m68kdefconfig m68k allyesconfig nios2 defconfig arc allyesconfig nds32 allnoconfig nds32 defconfig nios2allyesconfig cskydefconfig alpha defconfig alphaallyesconfig xtensa allyesconfig h8300allyesconfig arc defconfig sh allmodconfig parisc defconfig s390 allmodconfig parisc allyesconfig s390defconfig sparcallyesconfig sparc defconfig i386defconfig mips allyesconfig mips allmodconfig powerpc allyesconfig powerpc allmodconfig powerpc allnoconfig x86_64 randconfig-a001-20210505 x86_64 randconfig-a003-20210505 x86_64 randconfig-a005-20210505 x86_64 randconfig-a002-20210505 x86_64 randconfig-a006-20210505 x86_64 randconfig-a004-20210505 i386 randconfig-a003-20210505 i386 randconfig-a006-20210505 i386 randconfig-a001-20210505 i386 randconfig-a005-20210505 i386
Re: [PATCH v10 6/9] media: uapi: Add a control for HANTRO driver
On Wed, 2021-05-05 at 16:55 +0200, Hans Verkuil wrote: > On 20/04/2021 14:10, Benjamin Gaignard wrote: > > The HEVC HANTRO driver needs to know the number of bits to skip at > > the beginning of the slice header. > > That is a hardware specific requirement so create a dedicated control > > for this purpose. > > > > Signed-off-by: Benjamin Gaignard > > --- > > .../userspace-api/media/drivers/hantro.rst | 19 +++ > > .../userspace-api/media/drivers/index.rst | 1 + > > include/media/hevc-ctrls.h | 13 + > > 3 files changed, 33 insertions(+) > > create mode 100644 Documentation/userspace-api/media/drivers/hantro.rst > > > > diff --git a/Documentation/userspace-api/media/drivers/hantro.rst > > b/Documentation/userspace-api/media/drivers/hantro.rst > > new file mode 100644 > > index ..cd9754b4e005 > > --- /dev/null > > +++ b/Documentation/userspace-api/media/drivers/hantro.rst > > @@ -0,0 +1,19 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > + > > +Hantro video decoder driver > > +=== > > + > > +The Hantro video decoder driver implements the following driver-specific > > controls: > > + > > +``V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (integer)`` > > + Specifies to Hantro HEVC video decoder driver the number of data (in > > bits) to > > + skip in the slice segment header. > > + If non-IDR, the bits to be skipped go from syntax element > > "pic_output_flag" > > + to before syntax element "slice_temporal_mvp_enabled_flag". > > + If IDR, the skipped bits are just "pic_output_flag" > > + (separate_colour_plane_flag is not supported). > > I'm not very keen on this. Without this information the video data cannot be > decoded, or will it just be suboptimal? > > The problem is that a generic decoder would have to know that the HW is a > hantro, Applications can just query which controls are exposed by a video device, and if this control is found, then it means it needs to be set. > and then call this control. If they don't (and are testing on non-hantro HW), > then > it won't work, thus defeating the purpose of the HW independent decoder API. > > Since hantro is widely used, and if there is no other way to do this beside > explitely > setting this control, then perhaps this should be part of the standard HEVC > API. > Non-hantro drivers that do not need this can just skip it. > The decision to move it out of the HEVC API is not really to avoid setting it. In the end, most/all applications will end up required to set this ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [RFC RESEND 2/3] media: uapi: Add VP9 stateless decoder controls
Hi Hans, just a partial reply, I'll let Andrzej extend. Le jeudi 29 avril 2021 à 12:20 +0200, Hans Verkuil a écrit : > > + - ``frame_width_minus_1`` > > + - Add 1 to get the frame width expressed in pixels. > > + * - __u16 > > + - ``frame_height_minus_1`` > > + - Add 1 to get the frame height expressed in pixels. > > These two fields are weird. Isn't this defined by setting the output format? > And why the 'minus_1'? > > > + * - __u16 > > + - ``render_width_minus_1`` > > + - Add 1 to get the expected render width expressed in pixels. This is > > + not used during the decoding process but might be used by HW > > scalers to > > + prepare a frame that's ready for scanout. > > + * - __u16 > > + - render_height_minus_1 > > + - Add 1 to get the expected render height expressed in pixels. This > > is > > + not used during the decoding process but might be used by HW > > scalers to > > + prepare a frame that's ready for scanout. > > No idea what these fields are about. I suspect this can be defined by setting > the capture format, but I'm not sure. We have the same for other codecs. Each codec bitstream include the coded and the display size in some form. For H264/H265 that was passed as as number of macroblock and a crop rectangle. For VP9 value - 1 is as defined in the spec. As 0 is not valid, the boolean coder can save some bits when storing the value. Though, for parameters, we usually start with the way it's encoded first, and change it only if we think it make sense. We'll take note and consider this whenever we have a second driver to look at. Now, why do we include both here. Well in fact, the HW may have some extra constraints. Which means there will be up to 3 frame sizes that matters. We can't also determine if the display/render or coded size will be used for minim CAPTURE format, as this will in fact depends if a post processor will be used or not. So to recap, we limit the CAPTURE format base on this information, and not the opposite. The width/height on OUTPUT FMT has been define as meaningless in the spec (to align with stateful I suppose ?). This way, the driver got all the information aligned with how it works inside the codec, without having to do a translation dance, and then properly implement CAPTURE TRY_FMT base on that. To make an analogy with stateful codec, this replaces the queuing of a frame that contains codec headers. We skip the SRC_CH events, since this is no longer asynchronous. Nicolas ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v10 6/9] media: uapi: Add a control for HANTRO driver
Le mercredi 05 mai 2021 à 16:18 +0100, John Cox a écrit :
> > The HEVC HANTRO driver needs to know the number of bits to skip at
> > the beginning of the slice header.
> > That is a hardware specific requirement so create a dedicated control
> > for this purpose.
> >
> > Signed-off-by: Benjamin Gaignard
> > ---
> > .../userspace-api/media/drivers/hantro.rst| 19 +++
> > .../userspace-api/media/drivers/index.rst | 1 +
> > include/media/hevc-ctrls.h| 13 +
> > 3 files changed, 33 insertions(+)
> > create mode 100644 Documentation/userspace-api/media/drivers/hantro.rst
> >
> > diff --git a/Documentation/userspace-api/media/drivers/hantro.rst
> > b/Documentation/userspace-api/media/drivers/hantro.rst
> > new file mode 100644
> > index ..cd9754b4e005
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/drivers/hantro.rst
> > @@ -0,0 +1,19 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +Hantro video decoder driver
> > +===
> > +
> > +The Hantro video decoder driver implements the following driver-specific
> > controls:
> > +
> > +``V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (integer)``
> > +Specifies to Hantro HEVC video decoder driver the number of data (in
> > bits) to
> > +skip in the slice segment header.
> > +If non-IDR, the bits to be skipped go from syntax element
> > "pic_output_flag"
> > +to before syntax element "slice_temporal_mvp_enabled_flag".
> > +If IDR, the skipped bits are just "pic_output_flag"
> > +(separate_colour_plane_flag is not supported).
>
> What happens if it is a dependant_slice_segement or
> output_flag_present_flag? Those flags are all dependant on
> dependant_slice_segement being false. I'm guessing 0 but it maybe
> should be documented.
Zero indeed.
> Likewise if output_flag_present_flag is false pic_output_flag will not
> be coded, so maybe express it as "after slice_type" rather than "before
> pic_output_flag"?
Should work too.
>
> Regards
>
> John Cox
>
> > +.. note::
> > +
> > +This control is not yet part of the public kernel API and
> > +it is expected to change.
> > diff --git a/Documentation/userspace-api/media/drivers/index.rst
> > b/Documentation/userspace-api/media/drivers/index.rst
> > index 1a9038f5f9fa..12e3c512d718 100644
> > --- a/Documentation/userspace-api/media/drivers/index.rst
> > +++ b/Documentation/userspace-api/media/drivers/index.rst
> > @@ -33,6 +33,7 @@ For more details see the file COPYING in the source
> > distribution of Linux.
> >
> > ccs
> > cx2341x-uapi
> > +hantro
> > imx-uapi
> > max2175
> > meye-uapi
> > diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> > index 8e0109eea454..b713eeed1915 100644
> > --- a/include/media/hevc-ctrls.h
> > +++ b/include/media/hevc-ctrls.h
> > @@ -224,4 +224,17 @@ struct v4l2_ctrl_hevc_decode_params {
> > __u64 flags;
> > };
> >
> > +/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2
> > */
> > +#define
> > V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC
> > | 0x1200)
> > +/*
> > + * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
> > + * the number of data (in bits) to skip in the
> > + * slice segment header.
> > + * If non-IDR, the bits to be skipped go from syntax element
> > "pic_output_flag"
> > + * to before syntax element "slice_temporal_mvp_enabled_flag".
> > + * If IDR, the skipped bits are just "pic_output_flag"
> > + * (separate_colour_plane_flag is not supported).
> > + */
> > +#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP
> > (V4L2_CID_CODEC_HANTRO_BASE
> > + 0)
> > +
> > #endif
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v10 6/9] media: uapi: Add a control for HANTRO driver
Le 05/05/2021 à 16:55, Hans Verkuil a écrit :
On 20/04/2021 14:10, Benjamin Gaignard wrote:
The HEVC HANTRO driver needs to know the number of bits to skip at
the beginning of the slice header.
That is a hardware specific requirement so create a dedicated control
for this purpose.
Signed-off-by: Benjamin Gaignard
---
.../userspace-api/media/drivers/hantro.rst| 19 +++
.../userspace-api/media/drivers/index.rst | 1 +
include/media/hevc-ctrls.h| 13 +
3 files changed, 33 insertions(+)
create mode 100644 Documentation/userspace-api/media/drivers/hantro.rst
diff --git a/Documentation/userspace-api/media/drivers/hantro.rst
b/Documentation/userspace-api/media/drivers/hantro.rst
new file mode 100644
index ..cd9754b4e005
--- /dev/null
+++ b/Documentation/userspace-api/media/drivers/hantro.rst
@@ -0,0 +1,19 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Hantro video decoder driver
+===
+
+The Hantro video decoder driver implements the following driver-specific
controls:
+
+``V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (integer)``
+Specifies to Hantro HEVC video decoder driver the number of data (in bits)
to
+skip in the slice segment header.
+If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
+to before syntax element "slice_temporal_mvp_enabled_flag".
+If IDR, the skipped bits are just "pic_output_flag"
+(separate_colour_plane_flag is not supported).
I'm not very keen on this. Without this information the video data cannot be
decoded, or will it just be suboptimal?
Without that information the video can't be decoded.
The problem is that a generic decoder would have to know that the HW is a
hantro,
and then call this control. If they don't (and are testing on non-hantro HW),
then
it won't work, thus defeating the purpose of the HW independent decoder API.
Since hantro is widely used, and if there is no other way to do this beside
explitely
setting this control, then perhaps this should be part of the standard HEVC API.
Non-hantro drivers that do not need this can just skip it.
Even if I put this parameter in decode_params structure that would means that a
generic
userland decoder will have to know how the compute this value for hantro HW
since it
isn't something that could be done on kernel side.
Regards,
Benjamin
Regards,
Hans
+
+.. note::
+
+This control is not yet part of the public kernel API and
+it is expected to change.
diff --git a/Documentation/userspace-api/media/drivers/index.rst
b/Documentation/userspace-api/media/drivers/index.rst
index 1a9038f5f9fa..12e3c512d718 100644
--- a/Documentation/userspace-api/media/drivers/index.rst
+++ b/Documentation/userspace-api/media/drivers/index.rst
@@ -33,6 +33,7 @@ For more details see the file COPYING in the source
distribution of Linux.
ccs
cx2341x-uapi
+hantro
imx-uapi
max2175
meye-uapi
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index 8e0109eea454..b713eeed1915 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -224,4 +224,17 @@ struct v4l2_ctrl_hevc_decode_params {
__u64 flags;
};
+/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
+#define V4L2_CID_CODEC_HANTRO_BASE
(V4L2_CTRL_CLASS_CODEC | 0x1200)
+/*
+ * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
+ * the number of data (in bits) to skip in the
+ * slice segment header.
+ * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
+ * to before syntax element "slice_temporal_mvp_enabled_flag".
+ * If IDR, the skipped bits are just "pic_output_flag"
+ * (separate_colour_plane_flag is not supported).
+ */
+#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_CODEC_HANTRO_BASE + 0)
+
#endif
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v10 6/9] media: uapi: Add a control for HANTRO driver
>The HEVC HANTRO driver needs to know the number of bits to skip at
>the beginning of the slice header.
>That is a hardware specific requirement so create a dedicated control
>for this purpose.
>
>Signed-off-by: Benjamin Gaignard
>---
> .../userspace-api/media/drivers/hantro.rst| 19 +++
> .../userspace-api/media/drivers/index.rst | 1 +
> include/media/hevc-ctrls.h| 13 +
> 3 files changed, 33 insertions(+)
> create mode 100644 Documentation/userspace-api/media/drivers/hantro.rst
>
>diff --git a/Documentation/userspace-api/media/drivers/hantro.rst
>b/Documentation/userspace-api/media/drivers/hantro.rst
>new file mode 100644
>index ..cd9754b4e005
>--- /dev/null
>+++ b/Documentation/userspace-api/media/drivers/hantro.rst
>@@ -0,0 +1,19 @@
>+.. SPDX-License-Identifier: GPL-2.0
>+
>+Hantro video decoder driver
>+===
>+
>+The Hantro video decoder driver implements the following driver-specific
>controls:
>+
>+``V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (integer)``
>+Specifies to Hantro HEVC video decoder driver the number of data (in
>bits) to
>+skip in the slice segment header.
>+If non-IDR, the bits to be skipped go from syntax element
>"pic_output_flag"
>+to before syntax element "slice_temporal_mvp_enabled_flag".
>+If IDR, the skipped bits are just "pic_output_flag"
>+(separate_colour_plane_flag is not supported).
What happens if it is a dependant_slice_segement or
output_flag_present_flag? Those flags are all dependant on
dependant_slice_segement being false. I'm guessing 0 but it maybe
should be documented.
Likewise if output_flag_present_flag is false pic_output_flag will not
be coded, so maybe express it as "after slice_type" rather than "before
pic_output_flag"?
Regards
John Cox
>+.. note::
>+
>+This control is not yet part of the public kernel API and
>+it is expected to change.
>diff --git a/Documentation/userspace-api/media/drivers/index.rst
>b/Documentation/userspace-api/media/drivers/index.rst
>index 1a9038f5f9fa..12e3c512d718 100644
>--- a/Documentation/userspace-api/media/drivers/index.rst
>+++ b/Documentation/userspace-api/media/drivers/index.rst
>@@ -33,6 +33,7 @@ For more details see the file COPYING in the source
>distribution of Linux.
>
> ccs
> cx2341x-uapi
>+hantro
> imx-uapi
> max2175
> meye-uapi
>diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
>index 8e0109eea454..b713eeed1915 100644
>--- a/include/media/hevc-ctrls.h
>+++ b/include/media/hevc-ctrls.h
>@@ -224,4 +224,17 @@ struct v4l2_ctrl_hevc_decode_params {
> __u64 flags;
> };
>
>+/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
>+#define V4L2_CID_CODEC_HANTRO_BASE
>(V4L2_CTRL_CLASS_CODEC | 0x1200)
>+/*
>+ * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
>+ * the number of data (in bits) to skip in the
>+ * slice segment header.
>+ * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
>+ * to before syntax element "slice_temporal_mvp_enabled_flag".
>+ * If IDR, the skipped bits are just "pic_output_flag"
>+ * (separate_colour_plane_flag is not supported).
>+ */
>+#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP
>(V4L2_CID_CODEC_HANTRO_BASE + 0)
>+
> #endif
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v10 6/9] media: uapi: Add a control for HANTRO driver
On 20/04/2021 14:10, Benjamin Gaignard wrote:
> The HEVC HANTRO driver needs to know the number of bits to skip at
> the beginning of the slice header.
> That is a hardware specific requirement so create a dedicated control
> for this purpose.
>
> Signed-off-by: Benjamin Gaignard
> ---
> .../userspace-api/media/drivers/hantro.rst| 19 +++
> .../userspace-api/media/drivers/index.rst | 1 +
> include/media/hevc-ctrls.h| 13 +
> 3 files changed, 33 insertions(+)
> create mode 100644 Documentation/userspace-api/media/drivers/hantro.rst
>
> diff --git a/Documentation/userspace-api/media/drivers/hantro.rst
> b/Documentation/userspace-api/media/drivers/hantro.rst
> new file mode 100644
> index ..cd9754b4e005
> --- /dev/null
> +++ b/Documentation/userspace-api/media/drivers/hantro.rst
> @@ -0,0 +1,19 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +Hantro video decoder driver
> +===
> +
> +The Hantro video decoder driver implements the following driver-specific
> controls:
> +
> +``V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (integer)``
> +Specifies to Hantro HEVC video decoder driver the number of data (in
> bits) to
> +skip in the slice segment header.
> +If non-IDR, the bits to be skipped go from syntax element
> "pic_output_flag"
> +to before syntax element "slice_temporal_mvp_enabled_flag".
> +If IDR, the skipped bits are just "pic_output_flag"
> +(separate_colour_plane_flag is not supported).
I'm not very keen on this. Without this information the video data cannot be
decoded, or will it just be suboptimal?
The problem is that a generic decoder would have to know that the HW is a
hantro,
and then call this control. If they don't (and are testing on non-hantro HW),
then
it won't work, thus defeating the purpose of the HW independent decoder API.
Since hantro is widely used, and if there is no other way to do this beside
explitely
setting this control, then perhaps this should be part of the standard HEVC API.
Non-hantro drivers that do not need this can just skip it.
Regards,
Hans
> +
> +.. note::
> +
> +This control is not yet part of the public kernel API and
> +it is expected to change.
> diff --git a/Documentation/userspace-api/media/drivers/index.rst
> b/Documentation/userspace-api/media/drivers/index.rst
> index 1a9038f5f9fa..12e3c512d718 100644
> --- a/Documentation/userspace-api/media/drivers/index.rst
> +++ b/Documentation/userspace-api/media/drivers/index.rst
> @@ -33,6 +33,7 @@ For more details see the file COPYING in the source
> distribution of Linux.
>
> ccs
> cx2341x-uapi
> +hantro
> imx-uapi
> max2175
> meye-uapi
> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> index 8e0109eea454..b713eeed1915 100644
> --- a/include/media/hevc-ctrls.h
> +++ b/include/media/hevc-ctrls.h
> @@ -224,4 +224,17 @@ struct v4l2_ctrl_hevc_decode_params {
> __u64 flags;
> };
>
> +/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2
> */
> +#define V4L2_CID_CODEC_HANTRO_BASE
> (V4L2_CTRL_CLASS_CODEC | 0x1200)
> +/*
> + * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
> + * the number of data (in bits) to skip in the
> + * slice segment header.
> + * If non-IDR, the bits to be skipped go from syntax element
> "pic_output_flag"
> + * to before syntax element "slice_temporal_mvp_enabled_flag".
> + * If IDR, the skipped bits are just "pic_output_flag"
> + * (separate_colour_plane_flag is not supported).
> + */
> +#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP
> (V4L2_CID_CODEC_HANTRO_BASE + 0)
> +
> #endif
>
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v10 8/9] media: hantro: Introduce G2/HEVC decoder
Hi Benjamin,
Some comments:
On 20/04/2021 14:10, Benjamin Gaignard wrote:
> Implement all the logic to get G2 hardware decoding HEVC frames.
> It supports up level 5.1 HEVC stream.
> It doesn't support yet 10 bits formats or the scaling feature.
>
> Add HANTRO HEVC dedicated control to skip some bits at the beginning
> of the slice header. That is very specific to this hardware so can't
> go into uapi structures. Computing the needed value is complex and requires
> information from the stream that only the userland knows so let it
> provide the correct value to the driver.
>
> Signed-off-by: Benjamin Gaignard
> Co-developed-by: Adrian Ratiu
> Signed-off-by: Adrian Ratiu
> Co-developed-by: Ezequiel Garcia
> Signed-off-by: Ezequiel Garcia
> ---
> drivers/staging/media/hantro/Makefile | 2 +
> drivers/staging/media/hantro/hantro.h | 2 +
> drivers/staging/media/hantro/hantro_drv.c | 36 ++
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++
> drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++
> drivers/staging/media/hantro/hantro_hevc.c| 327 ++
> drivers/staging/media/hantro/hantro_hw.h | 49 ++
> 7 files changed, 1201 insertions(+)
> create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
>
> diff --git a/drivers/staging/media/hantro/Makefile
> b/drivers/staging/media/hantro/Makefile
> index 743ce08eb184..0357f1772267 100644
> --- a/drivers/staging/media/hantro/Makefile
> +++ b/drivers/staging/media/hantro/Makefile
> @@ -9,12 +9,14 @@ hantro-vpu-y += \
> hantro_h1_jpeg_enc.o \
> hantro_g1_h264_dec.o \
> hantro_g1_mpeg2_dec.o \
> + hantro_g2_hevc_dec.o \
> hantro_g1_vp8_dec.o \
> rk3399_vpu_hw_jpeg_enc.o \
> rk3399_vpu_hw_mpeg2_dec.o \
> rk3399_vpu_hw_vp8_dec.o \
> hantro_jpeg.o \
> hantro_h264.o \
> + hantro_hevc.o \
> hantro_mpeg2.o \
> hantro_vp8.o
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index e50d39b51902..a70c386de6f1 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -221,6 +221,7 @@ struct hantro_dev {
> * @jpeg_enc:JPEG-encoding context.
> * @mpeg2_dec: MPEG-2-decoding context.
> * @vp8_dec: VP8-decoding context.
> + * @hevc_dec:HEVC-decoding context.
> */
> struct hantro_ctx {
> struct hantro_dev *dev;
> @@ -247,6 +248,7 @@ struct hantro_ctx {
> struct hantro_jpeg_enc_hw_ctx jpeg_enc;
> struct hantro_mpeg2_dec_hw_ctx mpeg2_dec;
> struct hantro_vp8_dec_hw_ctx vp8_dec;
> + struct hantro_hevc_dec_hw_ctx hevc_dec;
> };
> };
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index d9a3a5ef9330..905a69758b37 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -281,6 +281,26 @@ static int hantro_jpeg_s_ctrl(struct v4l2_ctrl *ctrl)
> return 0;
> }
>
> +static int hantro_hevc_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct hantro_ctx *ctx;
> +
> + ctx = container_of(ctrl->handler,
> +struct hantro_ctx, ctrl_handler);
> +
> + vpu_debug(1, "s_ctrl: id = %d, val = %d\n", ctrl->id, ctrl->val);
> +
> + switch (ctrl->id) {
> + case V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP:
> + ctx->hevc_dec.ctrls.hevc_hdr_skip_length = ctrl->val;
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> static const struct v4l2_ctrl_ops hantro_ctrl_ops = {
> .try_ctrl = hantro_try_ctrl,
> };
> @@ -289,6 +309,10 @@ static const struct v4l2_ctrl_ops hantro_jpeg_ctrl_ops =
> {
> .s_ctrl = hantro_jpeg_s_ctrl,
> };
>
> +static const struct v4l2_ctrl_ops hantro_hevc_ctrl_ops = {
> + .s_ctrl = hantro_hevc_s_ctrl,
> +};
> +
> static const struct hantro_ctrl controls[] = {
> {
> .codec = HANTRO_JPEG_ENCODER,
> @@ -409,6 +433,18 @@ static const struct hantro_ctrl controls[] = {
> .cfg = {
> .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP,
> + .name = "Hantro HEVC slice header skip bytes",
> + .type = V4L2_CTRL_TYPE_INTEGER,
> + .min = 0,
> + .def = 0,
> + .max = 0x100,
> + .step = 1,
> +
Re: [PATCH v10 0/9] Add HANTRO G2/HEVC decoder support for IMX8MQ
Hi Benjamin, On 20/04/2021 14:10, Benjamin Gaignard wrote: > The IMX8MQ got two VPUs but until now only G1 has been enabled. > This series aim to add the second VPU (aka G2) and provide basic > HEVC decoding support. > > To be able to decode HEVC it is needed to add/update some of the > structures in the uapi. In addition of them one HANTRO dedicated > control is required to inform the driver of the number of bits to skip > at the beginning of the slice header. > The hardware require to allocate few auxiliary buffers to store the > references frame or tile size data. This series clashes with this patch: https://patchwork.linuxtv.org/project/linux-media/patch/20210427071554.625-1-jernej.skra...@siol.net/ and this patch series: https://patchwork.linuxtv.org/project/linux-media/cover/20210401144336.2495479-1-emil.l.veli...@gmail.com/ For both PRs are pending. It's probably better to wait until this is merged before rebasing this series. And if drivers are going to be moved out of staging, leaving only HEVC support in staging, then I'd wait until that is done as well. Regards, Hans > > The driver has been tested with fluster test suite stream. > For example with this command: ./fluster.py run -ts JCT-VC-HEVC_V1 -d > GStreamer-H.265-V4L2SL-Gst1.0 > > version 10: > - Shorter version of the previous series without ctrl block patches >and no DT modifications. >The scope of this series is limited to HEVC support. > > version 9: > - Corrections in commits messages. > - Define the dedicated control in hevc-controls.h > - Add note in documentation. > - Change max value of the dedicated control. > - Rebased on media_tree/master branch. > > version 8: > - Add reviewed-by and ack-by tags > - Fix the warnings reported by kernel test robot > - Only patch 9 (adding dedicated control), patch 11 (HEVC support) and >patch 13 (DT changes) are still missing of review/ack tag. > > version 7: > - Remove 'q' from syscon phandle name to make usable for iMX8MM too. >Update the bindings documentation. > - Add review/ack tags. > - Rebase on top of media_tree/master > - Be more accurate when computing the size of the memory needed motion >vectors. > - Explain why the all clocks need to set in the both DT node. > > version 6: > - fix the errors reported by kernel test robot > > version 5: > - use syscon instead of VPU reset driver. > - Do not break kernel/DT backward compatibility. > - Add documentation for dedicated Hantro control. > - Fix the remarks done by Ezequeil (typo, comments, unused function) > - Run v4l2-compliance without errors (see below). > - Do not add field to distinguish version, check postproc reg instead > > version 4: > - Split the changes in hevc controls in 2 commits to make them easier to > review. > - Change hantro_codec_ops run() prototype to return errors > - Hantro v4l2 dedicated control is now only an integer > - rebase on top of VPU reset changes posted here: > https://www.spinics.net/lists/arm-kernel/msg878440.html > - Various fix from previous remarks > - Limit the modifications in API to what the driver needs > > version 3: > - Fix typo in Hantro v4l2 dedicated control > - Add documentation for the new structures and fields > - Rebased on top of media_tree for-linus-5.12-rc1 tag > > version 2: > - remove all change related to scaling > - squash commits to a coherent split > - be more verbose about the added fields > - fix the comments done by Ezequiel about dma_alloc_coherent usage > - fix Dan's comments about control copy, reverse the test logic > in tile_buffer_reallocate, rework some goto and return cases. > - be more verbose about why I change the bindings > - remove all sign-off expect mime since it is confusing > - remove useless clocks in VPUs nodes > > Benjamin Gaignard (9): > media: hevc: Add fields and flags for hevc PPS > media: hevc: Add decode params control > media: hantro: change hantro_codec_ops run prototype to return errors > media: hantro: Define HEVC codec profiles and supported features > media: hantro: Only use postproc when post processed formats are > defined > media: uapi: Add a control for HANTRO driver > media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control > media: hantro: Introduce G2/HEVC decoder > media: hantro: IMX8M: add variant for G2/HEVC codec > > .../userspace-api/media/drivers/hantro.rst| 19 + > .../userspace-api/media/drivers/index.rst | 1 + > .../media/v4l/ext-ctrls-codec.rst | 108 +++- > .../media/v4l/vidioc-queryctrl.rst| 6 + > drivers/media/v4l2-core/v4l2-ctrls.c | 28 +- > drivers/staging/media/hantro/Makefile | 2 + > drivers/staging/media/hantro/hantro.h | 13 +- > drivers/staging/media/hantro/hantro_drv.c | 99 ++- > .../staging/media/hantro/hantro_g1_h264_dec.c | 10 +- > .../media/hantro/hantro_g1_mpeg2_dec.c| 4 +- > .../staging/media/hantro/hantro_g1_vp8_dec.c
[RFCv2 2/3] media: uapi: Add VP9 v4l2 library
Provide code common to vp9 drivers in one central location.
Signed-off-by: Andrzej Pietrasiewicz
---
drivers/media/v4l2-core/Kconfig|4 +
drivers/media/v4l2-core/Makefile |1 +
drivers/media/v4l2-core/v4l2-vp9.c | 1831
include/media/v4l2-vp9.h | 168 +++
4 files changed, 2004 insertions(+)
create mode 100644 drivers/media/v4l2-core/v4l2-vp9.c
create mode 100644 include/media/v4l2-vp9.h
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index bf49f83cb86f..69d1a0bc06cc 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -52,6 +52,10 @@ config V4L2_JPEG_HELPER
config V4L2_H264
tristate
+# Used by drivers that need v4l2-vp9.ko
+config V4L2_VP9
+ tristate
+
# Used by drivers that need v4l2-mem2mem.ko
config V4L2_MEM2MEM_DEV
tristate
diff --git a/drivers/media/v4l2-core/Makefile b/drivers/media/v4l2-core/Makefile
index e4cd589b99a5..438c0adaf144 100644
--- a/drivers/media/v4l2-core/Makefile
+++ b/drivers/media/v4l2-core/Makefile
@@ -22,6 +22,7 @@ obj-$(CONFIG_VIDEO_TUNER) += tuner.o
obj-$(CONFIG_V4L2_MEM2MEM_DEV) += v4l2-mem2mem.o
obj-$(CONFIG_V4L2_H264) += v4l2-h264.o
+obj-$(CONFIG_V4L2_VP9) += v4l2-vp9.o
obj-$(CONFIG_V4L2_FLASH_LED_CLASS) += v4l2-flash-led-class.o
diff --git a/drivers/media/v4l2-core/v4l2-vp9.c
b/drivers/media/v4l2-core/v4l2-vp9.c
new file mode 100644
index ..8609cc46b646
--- /dev/null
+++ b/drivers/media/v4l2-core/v4l2-vp9.c
@@ -0,0 +1,1831 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * V4L2 VP9 helpers.
+ *
+ * Copyright (C) 2021 Collabora, Ltd.
+ *
+ * Author: Andrzej Pietrasiewicz
+ */
+
+#include
+
+#include
+
+const u8 v4l2_vp9_kf_y_mode_prob[10][10][9] = {
+ {
+ /* above = dc */
+ { 137, 30, 42, 148, 151, 207, 70, 52, 91 }, /*left = dc */
+ { 92, 45, 102, 136, 116, 180, 74, 90, 100 }, /*left = v */
+ { 73, 32, 19, 187, 222, 215, 46, 34, 100 }, /*left = h */
+ { 91, 30, 32, 116, 121, 186, 93, 86, 94 }, /*left = d45 */
+ { 72, 35, 36, 149, 68, 206, 68, 63, 105 }, /*left = d135*/
+ { 73, 31, 28, 138, 57, 124, 55, 122, 151 }, /*left = d117*/
+ { 67, 23, 21, 140, 126, 197, 40, 37, 171 }, /*left = d153*/
+ { 86, 27, 28, 128, 154, 212, 45, 43, 53 }, /*left = d207*/
+ { 74, 32, 27, 107, 86, 160, 63, 134, 102 }, /*left = d63 */
+ { 59, 67, 44, 140, 161, 202, 78, 67, 119 }, /*left = tm */
+ }, { /* above = v */
+ { 63, 36, 126, 146, 123, 158, 60, 90, 96 }, /*left = dc */
+ { 43, 46, 168, 134, 107, 128, 69, 142, 92 }, /*left = v */
+ { 44, 29, 68, 159, 201, 177, 50, 57, 77 }, /*left = h */
+ { 58, 38, 76, 114, 97, 172, 78, 133, 92 }, /*left = d45 */
+ { 46, 41, 76, 140, 63, 184, 69, 112, 57 }, /*left = d135*/
+ { 38, 32, 85, 140, 46, 112, 54, 151, 133 }, /*left = d117*/
+ { 39, 27, 61, 131, 110, 175, 44, 75, 136 }, /*left = d153*/
+ { 52, 30, 74, 113, 130, 175, 51, 64, 58 }, /*left = d207*/
+ { 47, 35, 80, 100, 74, 143, 64, 163, 74 }, /*left = d63 */
+ { 36, 61, 116, 114, 128, 162, 80, 125, 82 }, /*left = tm */
+ }, { /* above = h */
+ { 82, 26, 26, 171, 208, 204, 44, 32, 105 }, /*left = dc */
+ { 55, 44, 68, 166, 179, 192, 57, 57, 108 }, /*left = v */
+ { 42, 26, 11, 199, 241, 228, 23, 15, 85 }, /*left = h */
+ { 68, 42, 19, 131, 160, 199, 55, 52, 83 }, /*left = d45 */
+ { 58, 50, 25, 139, 115, 232, 39, 52, 118 }, /*left = d135*/
+ { 50, 35, 33, 153, 104, 162, 64, 59, 131 }, /*left = d117*/
+ { 44, 24, 16, 150, 177, 202, 33, 19, 156 }, /*left = d153*/
+ { 55, 27, 12, 153, 203, 218, 26, 27, 49 }, /*left = d207*/
+ { 53, 49, 21, 110, 116, 168, 59, 80, 76 }, /*left = d63 */
+ { 38, 72, 19, 168, 203, 212, 50, 50, 107 }, /*left = tm */
+ }, { /* above = d45 */
+ { 103, 26, 36, 129, 132, 201, 83, 80, 93 }, /*left = dc */
+ { 59, 38, 83, 112, 103, 162, 98, 136, 90 }, /*left = v */
+ { 62, 30, 23, 158, 200, 207, 59, 57, 50 }, /*left = h */
+ { 67, 30, 29, 84, 86, 191, 102, 91, 59 }, /*left = d45 */
+ { 60, 32, 33, 112, 71, 220, 64, 89, 104 }, /*left = d135*/
+ { 53, 26, 34, 130, 56, 149, 84, 120, 103 }, /*left = d117*/
+ { 53, 21, 23, 133, 109, 210, 56, 77, 172 }, /*left = d153*/
+ { 77, 19, 29, 112, 142, 228, 55, 66, 36 }, /*left = d207*/
+
[RFCv2 3/3] media: rkvdec: Add the VP9 backend
From: Boris Brezillon
The Rockchip VDEC supports VP9 profile 0 up to 4096x2304@30fps. Add
a backend for this new format.
Signed-off-by: Boris Brezillon
Signed-off-by: Ezequiel Garcia
Signed-off-by: Adrian Ratiu
Signed-off-by: Andrzej Pietrasiewicz
---
drivers/staging/media/rkvdec/Kconfig |1 +
drivers/staging/media/rkvdec/Makefile |2 +-
drivers/staging/media/rkvdec/rkvdec-vp9.c | 1084 +
drivers/staging/media/rkvdec/rkvdec.c | 52 +-
drivers/staging/media/rkvdec/rkvdec.h |6 +
5 files changed, 1140 insertions(+), 5 deletions(-)
create mode 100644 drivers/staging/media/rkvdec/rkvdec-vp9.c
diff --git a/drivers/staging/media/rkvdec/Kconfig
b/drivers/staging/media/rkvdec/Kconfig
index c02199b5e0fd..dc7292f346fa 100644
--- a/drivers/staging/media/rkvdec/Kconfig
+++ b/drivers/staging/media/rkvdec/Kconfig
@@ -9,6 +9,7 @@ config VIDEO_ROCKCHIP_VDEC
select VIDEOBUF2_VMALLOC
select V4L2_MEM2MEM_DEV
select V4L2_H264
+ select V4L2_VP9
help
Support for the Rockchip Video Decoder IP present on Rockchip SoCs,
which accelerates video decoding.
diff --git a/drivers/staging/media/rkvdec/Makefile
b/drivers/staging/media/rkvdec/Makefile
index c08fed0a39f9..cb86b429cfaa 100644
--- a/drivers/staging/media/rkvdec/Makefile
+++ b/drivers/staging/media/rkvdec/Makefile
@@ -1,3 +1,3 @@
obj-$(CONFIG_VIDEO_ROCKCHIP_VDEC) += rockchip-vdec.o
-rockchip-vdec-y += rkvdec.o rkvdec-h264.o
+rockchip-vdec-y += rkvdec.o rkvdec-h264.o rkvdec-vp9.o
diff --git a/drivers/staging/media/rkvdec/rkvdec-vp9.c
b/drivers/staging/media/rkvdec/rkvdec-vp9.c
new file mode 100644
index ..d40e4ff7e3f6
--- /dev/null
+++ b/drivers/staging/media/rkvdec/rkvdec-vp9.c
@@ -0,0 +1,1084 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip Video Decoder VP9 backend
+ *
+ * Copyright (C) 2019 Collabora, Ltd.
+ * Boris Brezillon
+ * Copyright (C) 2021 Collabora, Ltd.
+ * Andrzej Pietrasiewicz
+ *
+ * Copyright (C) 2016 Rockchip Electronics Co., Ltd.
+ * Alpha Lin
+ */
+
+/*
+ * For following the vp9 spec please start reading this driver
+ * code from rkvdec_vp9_run() followed by rkvdec_vp9_done().
+ */
+
+#include
+#include
+#include
+#include
+
+#include "rkvdec.h"
+#include "rkvdec-regs.h"
+
+#define RKVDEC_VP9_PROBE_SIZE 4864
+#define RKVDEC_VP9_COUNT_SIZE 13232
+#define RKVDEC_VP9_MAX_SEGMAP_SIZE 73728
+
+struct rkvdec_vp9_intra_mode_probs {
+ u8 y_mode[105];
+ u8 uv_mode[23];
+};
+
+struct rkvdec_vp9_intra_only_frame_probs {
+ u8 coef_intra[4][2][128];
+ struct rkvdec_vp9_intra_mode_probs intra_mode[10];
+};
+
+struct rkvdec_vp9_inter_frame_probs {
+ u8 y_mode[4][9];
+ u8 comp_mode[5];
+ u8 comp_ref[5];
+ u8 single_ref[5][2];
+ u8 inter_mode[7][3];
+ u8 interp_filter[4][2];
+ u8 padding0[11];
+ u8 coef[2][4][2][128];
+ u8 uv_mode_0_2[3][9];
+ u8 padding1[5];
+ u8 uv_mode_3_5[3][9];
+ u8 padding2[5];
+ u8 uv_mode_6_8[3][9];
+ u8 padding3[5];
+ u8 uv_mode_9[9];
+ u8 padding4[7];
+ u8 padding5[16];
+ struct {
+ u8 joint[3];
+ u8 sign[2];
+ u8 class[2][10];
+ u8 class0_bit[2];
+ u8 bits[2][10];
+ u8 class0_fr[2][2][3];
+ u8 fr[2][3];
+ u8 class0_hp[2];
+ u8 hp[2];
+ } mv;
+};
+
+struct rkvdec_vp9_probs {
+ u8 partition[16][3];
+ u8 pred[3];
+ u8 tree[7];
+ u8 skip[3];
+ u8 tx32[2][3];
+ u8 tx16[2][2];
+ u8 tx8[2][1];
+ u8 is_inter[4];
+ /* 128 bit alignment */
+ u8 padding0[3];
+ union {
+ struct rkvdec_vp9_inter_frame_probs inter;
+ struct rkvdec_vp9_intra_only_frame_probs intra_only;
+ };
+};
+
+/* Data structure describing auxiliary buffer format. */
+struct rkvdec_vp9_priv_tbl {
+ struct rkvdec_vp9_probs probs;
+ u8 segmap[2][RKVDEC_VP9_MAX_SEGMAP_SIZE];
+};
+
+struct rkvdec_vp9_refs_counts {
+ u32 eob[2];
+ u32 coeff[3];
+};
+
+struct rkvdec_vp9_inter_frame_symbol_counts {
+ u32 partition[16][4];
+ u32 skip[3][2];
+ u32 inter[4][2];
+ u32 tx32p[2][4];
+ u32 tx16p[2][4];
+ u32 tx8p[2][2];
+ u32 y_mode[4][10];
+ u32 uv_mode[10][10];
+ u32 comp[5][2];
+ u32 comp_ref[5][2];
+ u32 single_ref[5][2][2];
+ u32 mv_mode[7][4];
+ u32 filter[4][3];
+ u32 mv_joint[4];
+ u32 sign[2][2];
+ /* add 1 element for align */
+ u32 classes[2][11 + 1];
+ u32 class0[2][2];
+ u32 bits[2][10][2];
+ u32 class0_fp[2][2][4];
+ u32 fp[2][4];
+ u32 class0_hp[2][2];
+ u32 hp[2][2];
+ struct rkvdec_vp9_refs_counts ref_cnt[2][4][2][6][6];
+};
+
+struct rkvdec_vp9_intra_frame_symbol_counts {
+
[RFCv2 1/3] media: uapi: Add VP9 stateless decoder controls
Add the VP9 stateless decoder controls plus the documentation that goes
with it.
Signed-off-by: Boris Brezillon
Signed-off-by: Ezequiel Garcia
Signed-off-by: Adrian Ratiu
Signed-off-by: Andrzej Pietrasiewicz
---
.../userspace-api/media/v4l/biblio.rst| 10 +
.../media/v4l/ext-ctrls-codec-stateless.rst | 547 ++
.../media/v4l/pixfmt-compressed.rst | 15 +
.../media/v4l/vidioc-g-ext-ctrls.rst | 8 +
.../media/v4l/vidioc-queryctrl.rst| 12 +
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/v4l2-ctrls.c | 229
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h| 4 +
include/uapi/linux/v4l2-controls.h| 425 ++
include/uapi/linux/videodev2.h| 6 +
11 files changed, 1259 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/biblio.rst
b/Documentation/userspace-api/media/v4l/biblio.rst
index 64d241daf63c..051982896375 100644
--- a/Documentation/userspace-api/media/v4l/biblio.rst
+++ b/Documentation/userspace-api/media/v4l/biblio.rst
@@ -417,3 +417,13 @@ VP8
:title: RFC 6386: "VP8 Data Format and Decoding Guide"
:author:J. Bankoski et al.
+
+.. _vp9:
+
+VP9
+===
+
+
+:title: VP9 Bitstream & Decoding Process Specification
+
+:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan
Hunt (Argon Design)
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 3fc04daa9ffb..2d15d2d419a6 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1244,3 +1244,550 @@ FWHT Flags
* - __u8
- ``padding[3]``
- Applications and drivers must set this to zero.
+
+.. _v4l2-codec-stateless-vp9:
+
+``V4L2_CID_STATELESS_VP9_COMPRESSED_HDR_PROBS (struct)``
+Stores VP9 probabilities updates as parsed from the current compressed
frame
+header. A value of zero in an array element means no update of the relevant
+probability. Motion vector-related updates contain a new value or zero. All
+other updates contain values translated with inv_map_table[] (see 6.3.5 in
+:ref:`vp9`).
+
+.. c:type:: v4l2_ctrl_vp9_compressed_hdr_probs
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
+
+.. flat-table:: struct v4l2_ctrl_vp9_compressed_hdr_probs
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u8
+ - ``tx8[2][1]``
+ - TX 8x8 probabilities delta.
+* - __u8
+ - ``tx16[2][2]``
+ - TX 16x16 probabilities delta.
+* - __u8
+ - ``tx32[2][3]``
+ - TX 32x32 probabilities delta.
+* - __u8
+ - ``coef[4][2][2][6][6][3]``
+ - Coefficient probabilities delta.
+* - __u8
+ - ``skip[3]``
+ - Skip probabilities delta.
+* - __u8
+ - ``inter_mode[7][3]``
+ - Inter prediction mode probabilities delta.
+* - __u8
+ - ``interp_filter[4][2]``
+ - Interpolation filter probabilities delta.
+* - __u8
+ - ``is_inter[4]``
+ - Is inter-block probabilities delta.
+* - __u8
+ - ``comp_mode[5]``
+ - Compound prediction mode probabilities delta.
+* - __u8
+ - ``single_ref[5][2]``
+ - Single reference probabilities delta.
+* - __u8
+ - ``comp_mode[5]``
+ - Compound reference probabilities delta.
+* - __u8
+ - ``y_mode[4][9]``
+ - Y prediction mode probabilities delta.
+* - __u8
+ - ``uv_mode[10][9]``
+ - UV prediction mode probabilities delta.
+* - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+* - __u8
+ - ``partition[16][3]``
+ - Partition probabilities delta.
+* - __u8
+ - ``mv.joint[3]``
+ - Motion vector joint probabilities delta.
+* - __u8
+ - ``mv.sign[2]``
+ - Motion vector sign probabilities delta.
+* - __u8
+ - ``mv.class[2][10]``
+ - Motion vector class probabilities delta.
+* - __u8
+ - ``mv.class0_bit[2]``
+ - Motion vector class0 bit probabilities delta.
+* - __u8
+ - ``mv.bits[2][10]``
+ - Motion vector bits probabilities delta.
+* - __u8
+ - ``mv.class0_fr[2][2][3]``
+ - Motion vector class0 fractional bit probabilities delta.
+* - __u8
+ - ``mv.fr[2][3]``
+ - Motion vector fractional bit probabilities delta.
+* - __u8
+ - ``mv.class0_hp[2]``
+ - Motion vector class0 high precision fractional bit probabilities delta.
+* - __u8
+ - ``mv.hp[2]``
+ - Motion vector high precision fractional bit probabilities delta.
+
+``V4L2_CID_STATELESS_VP9_FRAME (struct)``
+Specifies the frame parameters for the associated VP9 frame decode request.
+This includes the necessary
[RFCv2 0/3] vp9 v4l2 stateless uapi
Dear All, This is a follow-up work for https://patchwork.linuxtv.org/project/linux-media/list/?series=5268 I addressed Hans's comments. Changes: v1..v2: - improve documentation - imrpove coding style - factor out of common vp9 code into v4l2-vp9.c - rename V4L2_CID_STATELESS_VP9_FRAME_DECODE_PARAMS into V4L2_CID_STATELESS_VP9_FRAME This is still sent as an RFC because the works for adding the second driver (g2@imx8) are ongoing. The v1 was an RFC on stateless uapi for vp9 decoding with v4l2, which was based on https://lkml.org/lkml/2020/11/2/1043, but had been substantially reworked. The important change was that the v4l2 control used to pass boolean decoder probabilities had been made unidirectional, and was renamed V4L2_CID_STATELESS_VP9_COMPRESSED_HDR_PROBS. In the original proposal from Boris, to queue a frame the userspace must fully dequeue the previous one, which effectively results in a forced lockstep behavior and defeats vb2's capability to enqueue multiple buffers. Such a design was a consequence of backward probability updates being performed by the kernel driver (which has direct access to appropriate counter values) but forward probability updates being coupled with compressed header parsing performed by the userspace. In vp9 the boolean decoder used to decode the bitstream needs certain parameters to work. Those are probabilities, which change with each frame. After each frame is decoded it is known how many times a given symbol occured in the frame, so the probabilities can be adapted. This process is known as backward probabilities update. A next frame header can also contain information which modifies probabilities resulting from backward update. The said modification is called forward probabilities update. The data for backward update is generated by the decoder hardware, while the data for forward update is prepared by reading the compressed frame header. The natural place to parse something is userspace, while the natural place to access hardware-provided counters is the kernel. Such responsibilties assignment was used in the original work. To overcome the lockstep, we moved forward probability updates to the kernel, while leaving parsing them in userspace. This way the v4l2 control which is used to pass the probs becomes unidirectional (user->kernel) and the userspace can keep parsing and enqueueing succeeding frames. If a particular driver parses the compressed header and does backward probability updates on its own then V4L2_CID_STATELESS_VP9_COMPRESSED_HDR_PROBS does not need to be used. This series adds vp9 uapi in proper locations, which means it is a proper, "official" uapi, as opposed to staging uapi which was proposed in the above mentioned lkml thread. The series adds vp9 support to rkvdec driver. Rebased onto media_tree, requires this patch: https://patchwork.linuxtv.org/project/linux-media/patch/20210505122347.7576-2-andrze...@collabora.com/ You can test rkvdec implementation with gstreamer, please clone gstreamer and then use these branches for -base and -bad: https://gitlab.freedesktop.org/dwlsalmeida/gst-plugins-base/-/tree/vp9-upstream-padding https://gitlab.freedesktop.org/dwlsalmeida/gst-plugins-bad/-/tree/vp9-upstream Example invocation: without format conversion: gst-launch-1.0 filesrc location=Big_Buck_Bunny_1080_10s_1MB.webm ! parsebin ! v4l2slvp9dec ! filesink location=out.yuv with format conversion to match vpxdec output: gst-launch-1.0 filesrc location=Big_Buck_Bunny_1080_10s_1MB.webm ! parsebin ! v4l2slvp9dec ! videoconvert ! video/x-raw,format=I420 ! filesink location=out.yuv I kindly ask for your comments. Andrzej Pietrasiewicz (2): media: uapi: Add VP9 stateless decoder controls media: uapi: Add VP9 v4l2 library Boris Brezillon (1): media: rkvdec: Add the VP9 backend .../userspace-api/media/v4l/biblio.rst| 10 + .../media/v4l/ext-ctrls-codec-stateless.rst | 547 + .../media/v4l/pixfmt-compressed.rst | 15 + .../media/v4l/vidioc-g-ext-ctrls.rst |8 + .../media/v4l/vidioc-queryctrl.rst| 12 + .../media/videodev2.h.rst.exceptions |2 + drivers/media/v4l2-core/Kconfig |4 + drivers/media/v4l2-core/Makefile |1 + drivers/media/v4l2-core/v4l2-ctrls.c | 229 +++ drivers/media/v4l2-core/v4l2-ioctl.c |1 + drivers/media/v4l2-core/v4l2-vp9.c| 1831 + drivers/staging/media/rkvdec/Kconfig |1 + drivers/staging/media/rkvdec/Makefile |2 +- drivers/staging/media/rkvdec/rkvdec-vp9.c | 1084 ++ drivers/staging/media/rkvdec/rkvdec.c | 52 +- drivers/staging/media/rkvdec/rkvdec.h |6 + include/media/v4l2-ctrls.h|4 + include/media/v4l2-vp9.h | 168 ++ include/uapi/linux/v4l2-controls.h| 425 include/uapi/linux/videodev2.h|
[PATCHv2 3/3] media: cedrus: Fix .buf_prepare
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type. If an OUTPUT buffer has a zero bytesused set by userspace then v4l2-core will set it to buffer length. If we overwrite bytesused for OUTPUT buffers, too, then vb2_get_plane_payload() will return incorrect value which might be then written to hw registers by the driver in cedrus_h264.c or cedrus_vp8.c. Signed-off-by: Andrzej Pietrasiewicz --- drivers/staging/media/sunxi/cedrus/cedrus_video.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index b62eb8e84057..bf731caf2ed5 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c @@ -457,7 +457,13 @@ static int cedrus_buf_prepare(struct vb2_buffer *vb) if (vb2_plane_size(vb, 0) < pix_fmt->sizeimage) return -EINVAL; - vb2_set_plane_payload(vb, 0, pix_fmt->sizeimage); + /* +* Buffer's bytesused must be written by driver for CAPTURE buffers. +* (for OUTPUT buffers, if userspace passes 0 bytesused, v4l2-core sets +* it to buffer length). +*/ + if (V4L2_TYPE_IS_CAPTURE(vq->type)) + vb2_set_plane_payload(vb, 0, pix_fmt->sizeimage); return 0; } -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCHv2 2/3] media: hantro: Fix .buf_prepare
The driver should only set the payload on .buf_prepare if the buffer is CAPTURE type. If an OUTPUT buffer has a zero bytesused set by userspace then v4l2-core will set it to buffer length. If we overwrite bytesused for OUTPUT buffers, too, then vb2_get_plane_payload() will return incorrect value which might be then written to hw registers by the driver in hantro_g1_h264_dec.c. Signed-off-by: Andrzej Pietrasiewicz --- drivers/staging/media/hantro/hantro_v4l2.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 1bc118e375a1..7ccc6405036a 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -639,7 +639,14 @@ static int hantro_buf_prepare(struct vb2_buffer *vb) ret = hantro_buf_plane_check(vb, pix_fmt); if (ret) return ret; - vb2_set_plane_payload(vb, 0, pix_fmt->plane_fmt[0].sizeimage); + /* +* Buffer's bytesused must be written by driver for CAPTURE buffers. +* (for OUTPUT buffers, if userspace passes 0 bytesused, v4l2-core sets +* it to buffer length). +*/ + if (V4L2_TYPE_IS_CAPTURE(vq->type)) + vb2_set_plane_payload(vb, 0, pix_fmt->plane_fmt[0].sizeimage); + return 0; } -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCHv2 1/3] media: rkvdec: Fix .buf_prepare
From: Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the
buffer is CAPTURE type. If an OUTPUT buffer has a zero bytesused
set by userspace then v4l2-core will set it to buffer length.
If we overwrite bytesused for OUTPUT buffers, too, then
vb2_get_plane_payload() will return incorrect value which might be then
written to hw registers by the driver in rkvdec-h264.c.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Ezequiel Garcia
Signed-off-by: Adrian Ratiu
[Changed the comment and used V4L2_TYPE_IS_CAPTURE macro]
Signed-off-by: Andrzej Pietrasiewicz
---
drivers/staging/media/rkvdec/rkvdec.c | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c
b/drivers/staging/media/rkvdec/rkvdec.c
index d821661d30f3..cd65ad2af8d4 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -481,7 +481,15 @@ static int rkvdec_buf_prepare(struct vb2_buffer *vb)
if (vb2_plane_size(vb, i) < sizeimage)
return -EINVAL;
}
- vb2_set_plane_payload(vb, 0, f->fmt.pix_mp.plane_fmt[0].sizeimage);
+
+ /*
+* Buffer's bytesused must be written by driver for CAPTURE buffers.
+* (for OUTPUT buffers, if userspace passes 0 bytesused, v4l2-core sets
+* it to buffer length).
+*/
+ if (V4L2_TYPE_IS_CAPTURE(vq->type))
+ vb2_set_plane_payload(vb, 0,
f->fmt.pix_mp.plane_fmt[0].sizeimage);
+
return 0;
}
--
2.17.1
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCHv2 0/3] Fix .buf_prepare
Drivers should only set the payload on .buf_prepare if the buffer is CAPTURE type. If an OUTPUT buffer has a zero bytesused set by userspace then v4l2-core will set it to buffer length. If we overwrite bytesused for OUTPUT buffers, too, then vb2_get_plane_payload() will return incorrect value which might be then written to hw registers by the driver. Andrzej Pietrasiewicz (2): media: hantro: Fix .buf_prepare media: cedrus: Fix .buf_prepare Ezequiel Garcia (1): media: rkvdec: Fix .buf_prepare drivers/staging/media/hantro/hantro_v4l2.c| 9 - drivers/staging/media/rkvdec/rkvdec.c | 10 +- drivers/staging/media/sunxi/cedrus/cedrus_video.c | 8 +++- 3 files changed, 24 insertions(+), 3 deletions(-) base-commit: 0b276e470a4d43e1365d3eb53c608a3d208cabd4 -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] media: rkvdec: Fix .buf_prepare
Hi Ezequiel,
W dniu 04.05.2021 o 13:56, Ezequiel Garcia pisze:
Hi Andrzej,
Thanks a lot for picking this up.
On Tue, 2021-05-04 at 13:37 +0200, Andrzej Pietrasiewicz wrote:
From: Ezequiel Garcia
The driver should only set the payload on .buf_prepare if the
buffer is CAPTURE type. If an OUTPUT buffer has a zero bytesused
set by userspace then v4l2-core will set it to buffer length.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Ezequiel Garcia
Signed-off-by: Adrian Ratiu
Signed-off-by: Andrzej Pietrasiewicz
---
@Hans: I haven't had anyone complain about the issue. The fix is needed for
the rkvdec vp9 work, so I think 5.14 is fine.
drivers/staging/media/rkvdec/rkvdec.c | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c
b/drivers/staging/media/rkvdec/rkvdec.c
index d821661d30f3..ef2166043127 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -481,7 +481,15 @@ static int rkvdec_buf_prepare(struct vb2_buffer *vb)
if (vb2_plane_size(vb, i) < sizeimage)
return -EINVAL;
}
- vb2_set_plane_payload(vb, 0, f->fmt.pix_mp.plane_fmt[0].sizeimage);
+
+ /*
+ * Buffer bytesused is written by driver for CAPTURE buffers.
+ * (if userspace passes 0 bytesused for OUTPUT buffers, v4l2-core sets
+ * it to buffer length).
+ */
+ if (!V4L2_TYPE_IS_OUTPUT(vq->type))
Please use V4L2_TYPE_IS_CAPTURE here.
Also, why is this change needed in rkvdec, but not in cedrus
or hantro?
As a matter of fact I think it is needed in all three, because later on,
whenever a driver uses vb2_get_plane_payload(), without such a patch it
will get an invalid number and write that to a hardware register, causing
incorrect behavior.
I will respond with a v2 series.
Regards,
Andrzej
___
devel mailing list
de...@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
