Re: [PATCH v2 3/7] gpio: separate gpio driver from pinctrl-rockchip driver
Hi Johan, On Sun, 11 Apr 2021 at 14:57, Heiko Stübner wrote: > > Hi Johan, > > Am Sonntag, 11. April 2021, 19:51:52 CEST schrieb Johan Jonker: > > Hi, > > > > When I check "rockchip,gpio-bank" with YAML it turns out that > > rk3288-veyron-XXX has 'gpio-line-names' as 'extra' property. > > It is not defined in the "rockchip,pinctrl.txt" document, but in > > ~/.local/lib/python3.5/site-packages/dtschema/schemas/gpio/gpio.yaml > > > > Where is that in use? > > In this driver or external? > > Can it be removed from mainline dts? > > gpio-line-names gets defined in devicetree/bindings/gpio/gpio.txt > which isn't converted to yaml yet: > On top of what Heiko says, feel free to convert Documentation/devicetree/bindings/gpio/gpio.txt to YAML :-) Thanks! Ezequiel
[PATCH] dt-bindings: pinctrl: rockchip: add RK3568 SoC support
Add RK3568/RK3566 SoC support to pinctrl. Signed-off-by: Ezequiel Garcia --- Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt b/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt index d3eae61a340d..91fab614c381 100644 --- a/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt +++ b/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt @@ -33,6 +33,7 @@ Required properties for iomux controller: "rockchip,rk3328-pinctrl": for Rockchip RK3328 "rockchip,rk3368-pinctrl": for Rockchip RK3368 "rockchip,rk3399-pinctrl": for Rockchip RK3399 + "rockchip,rk3568-pinctrl": for Rockchip RK3568 - rockchip,grf: phandle referencing a syscon providing the "general register files" -- 2.30.0
Re: [PATCH RESEND 0/7] gpio-rockchip driver
Hi Jianqun, I tried applying this on top of "pinctrl: rockchip: add support for rk3568", but I got some conflicts. If you could add some information about how patches should be applied to the cover letter, that'd be really helpful :-) Also, I've noticed several of these GPIOs patches, is this the most up to date? You can add some information about superseeding patches in the cover letter as well. Thanks, Ezequiel On Wed, 24 Mar 2021 at 03:46, Jianqun Xu wrote: > > Separate gpio driver from pinctrl driver, and support v2 controller. > > Jianqun Xu (7): > pinctrl/rockchip: separate struct rockchip_pin_bank to a head file > pinctrl/pinctrl-rockchip.h: add pinctrl device to gpio bank struct > gpio: separate gpio driver from pinctrl-rockchip driver > gpio/rockchip: use struct rockchip_gpio_regs for gpio controller > gpio/rockchip: support next version gpio controller > gpio/rockchip: always enable clock for gpio controller > gpio/rockchip: drop irq_gc_lock/irq_gc_unlock for irq set type > > drivers/gpio/Kconfig | 8 + > drivers/gpio/Makefile | 1 + > drivers/gpio/gpio-rockchip.c | 758 > drivers/pinctrl/pinctrl-rockchip.c | 909 + > drivers/pinctrl/pinctrl-rockchip.h | 286 + > 5 files changed, 1072 insertions(+), 890 deletions(-) > create mode 100644 drivers/gpio/gpio-rockchip.c > create mode 100644 drivers/pinctrl/pinctrl-rockchip.h > > -- > 2.25.1 > > >
Re: [Outreachy kernel][PATCH] staging: media: hantro: Rewrite macro function in lower case
On Fri, 2021-04-09 at 19:25 +0200, Julia Lawall wrote: > > > On Fri, 9 Apr 2021, ascordeiro wrote: > > > Em sex, 2021-04-09 às 13:44 -0300, Ezequiel Garcia escreveu: > > > Hi Aline, > > > > Hi Ezequiel, > > > > > > On Fri, 2021-04-09 at 10:54 -0300, Aline Santana Cordeiro wrote: > > > > Rewrite macros resembling functions #define HANTRO_PP_REG_WRITE > > > > and #define HANTRO_PP_RED_WRITE_S in lower case, according with > > > > code style. > > Maybe you can see if these macros can be converted to static inline > functions. Macros don't provide any type checking. > I'd like to leave these as they are for now, tbh ^_^ A more massive refactoring is needed in this driver, but it needs proper testing and it's a more complex plan. Thanks! Ezequiel
Re: [Outreachy kernel][PATCH] staging: media: hantro: Rewrite macro function in lower case
On Fri, 2021-04-09 at 14:00 -0300, ascordeiro wrote: > Em sex, 2021-04-09 às 13:44 -0300, Ezequiel Garcia escreveu: > > Hi Aline, > > Hi Ezequiel, > > > > On Fri, 2021-04-09 at 10:54 -0300, Aline Santana Cordeiro wrote: > > > Rewrite macros resembling functions #define HANTRO_PP_REG_WRITE > > > and #define HANTRO_PP_RED_WRITE_S in lower case, according with > > > code style. > > > > > > > Where is this written in the Coding Style? > > I found this in section 12, about Macros, Enums and RTL in both > references: > https://www.kernel.org/doc/html/latest/process/coding-style.html > https://elixir.bootlin.com/linux/latest/source/Documentation/process/coding-style.rst It seems this """ CAPITALIZED macro names are appreciated but macros resembling functions may be named in lower case. """ It's just a suggestion, not a big deal. In general, if checkpatch.pl is not complaining, it's not worth the trouble. So let's skip this fix. Thanks a lot! Ezequiel
Re: [Outreachy kernel][PATCH 1/2 v3] staging: media: hantro: Align line break to the open parenthesis in file hantro_hw.h
Hi Aline, On Fri, 2021-04-09 at 09:24 -0300, Aline Santana Cordeiro wrote: > Aligns line break with the remaining function arguments > to the open parenthesis. Issue found by checkpatch. > > Signed-off-by: Aline Santana Cordeiro > --- > Changes since v2: > - Rename the commit messages properly > > Changes since v1: > - Send patchset without the cover-letter Why did you remove the cover letter? What was wrong with it? Thanks! Ezequiel
Re: [Outreachy kernel][PATCH] staging: media: hantro: Rewrite macro function in lower case
Hi Aline,
On Fri, 2021-04-09 at 10:54 -0300, Aline Santana Cordeiro wrote:
> Rewrite macros resembling functions #define HANTRO_PP_REG_WRITE
> and #define HANTRO_PP_RED_WRITE_S in lower case, according with
> code style.
>
Where is this written in the Coding Style?
Thanks!
Ezequiel
> Signed-off-by: Aline Santana Cordeiro
> ---
> drivers/staging/media/hantro/hantro_postproc.c | 34
> +-
> 1 file changed, 17 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_postproc.c
> b/drivers/staging/media/hantro/hantro_postproc.c
> index 6d2a8f2a..06279c0 100644
> --- a/drivers/staging/media/hantro/hantro_postproc.c
> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> @@ -12,14 +12,14 @@
> #include "hantro_hw.h"
> #include "hantro_g1_regs.h"
>
> -#define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
> +#define hantro_pp_reg_write(vpu, reg_name, val) \
> { \
> hantro_reg_write(vpu, \
> &(vpu)->variant->postproc_regs->reg_name, \
> val); \
> }
>
> -#define HANTRO_PP_REG_WRITE_S(vpu, reg_name, val) \
> +#define hantro_pp_reg_write_s(vpu, reg_name, val) \
> { \
> hantro_reg_write_s(vpu, \
> &(vpu)->variant->postproc_regs->reg_name, \
> @@ -61,7 +61,7 @@ void hantro_postproc_enable(struct hantro_ctx *ctx)
> return;
>
> /* Turn on pipeline mode. Must be done first. */
> - HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x1);
> + hantro_pp_reg_write_s(vpu, pipeline_en, 0x1);
>
> src_pp_fmt = VPU_PP_IN_NV12;
>
> @@ -79,19 +79,19 @@ void hantro_postproc_enable(struct hantro_ctx *ctx)
> dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
> dst_dma = vb2_dma_contig_plane_dma_addr(_buf->vb2_buf, 0);
>
> - HANTRO_PP_REG_WRITE(vpu, clk_gate, 0x1);
> - HANTRO_PP_REG_WRITE(vpu, out_endian, 0x1);
> - HANTRO_PP_REG_WRITE(vpu, out_swap32, 0x1);
> - HANTRO_PP_REG_WRITE(vpu, max_burst, 16);
> - HANTRO_PP_REG_WRITE(vpu, out_luma_base, dst_dma);
> - HANTRO_PP_REG_WRITE(vpu, input_width, MB_WIDTH(ctx->dst_fmt.width));
> - HANTRO_PP_REG_WRITE(vpu, input_height,
> MB_HEIGHT(ctx->dst_fmt.height));
> - HANTRO_PP_REG_WRITE(vpu, input_fmt, src_pp_fmt);
> - HANTRO_PP_REG_WRITE(vpu, output_fmt, dst_pp_fmt);
> - HANTRO_PP_REG_WRITE(vpu, output_width, ctx->dst_fmt.width);
> - HANTRO_PP_REG_WRITE(vpu, output_height, ctx->dst_fmt.height);
> - HANTRO_PP_REG_WRITE(vpu, orig_width, MB_WIDTH(ctx->dst_fmt.width));
> - HANTRO_PP_REG_WRITE(vpu, display_width, ctx->dst_fmt.width);
> + hantro_pp_reg_write(vpu, clk_gate, 0x1);
> + hantro_pp_reg_write(vpu, out_endian, 0x1);
> + hantro_pp_reg_write(vpu, out_swap32, 0x1);
> + hantro_pp_reg_write(vpu, max_burst, 16);
> + hantro_pp_reg_write(vpu, out_luma_base, dst_dma);
> + hantro_pp_reg_write(vpu, input_width, MB_WIDTH(ctx->dst_fmt.width));
> + hantro_pp_reg_write(vpu, input_height,
> MB_HEIGHT(ctx->dst_fmt.height));
> + hantro_pp_reg_write(vpu, input_fmt, src_pp_fmt);
> + hantro_pp_reg_write(vpu, output_fmt, dst_pp_fmt);
> + hantro_pp_reg_write(vpu, output_width, ctx->dst_fmt.width);
> + hantro_pp_reg_write(vpu, output_height, ctx->dst_fmt.height);
> + hantro_pp_reg_write(vpu, orig_width, MB_WIDTH(ctx->dst_fmt.width));
> + hantro_pp_reg_write(vpu, display_width, ctx->dst_fmt.width);
> }
>
> void hantro_postproc_free(struct hantro_ctx *ctx)
> @@ -146,5 +146,5 @@ void hantro_postproc_disable(struct hantro_ctx *ctx)
> if (!vpu->variant->postproc_regs)
> return;
>
> - HANTRO_PP_REG_WRITE_S(vpu, pipeline_en, 0x0);
> + hantro_pp_reg_write_s(vpu, pipeline_en, 0x0);
> }
Re: [PATCH 1/2 v2] staging: media: hantro: Align line break to the open parenthesis in file hantro_hw.h
Ola Aline, Welcome to the kernel community. Hope you enjoy some of this Outreachy adventures. Normally, when you submit a v2, we want to know what changed between the first submission and v2. If you are subscribed to linux-media, you can read some of the series with a vN+1 and look how it's done. Examples: https://www.spinics.net/lists/linux-media/msg190043.html https://www.spinics.net/lists/linux-media/msg189923.html I'm sure your Outreachy mentors can tell you more. On Thu, 2021-04-08 at 11:07 -0300, Aline Santana Cordeiro wrote: > Aligns line break with the remaining function arguments > to the open parenthesis. Issue found by checkpatch. > > Signed-off-by: Aline Santana Cordeiro > --- > drivers/staging/media/hantro/hantro_hw.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/media/hantro/hantro_hw.h > b/drivers/staging/media/hantro/hantro_hw.h > index 34c9e46..a650b9c 100644 > --- a/drivers/staging/media/hantro/hantro_hw.h > +++ b/drivers/staging/media/hantro/hantro_hw.h > @@ -207,7 +207,7 @@ hantro_h264_mv_size(unsigned int width, unsigned int > height) > void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx); > void rk3399_vpu_mpeg2_dec_run(struct hantro_ctx *ctx); > void hantro_mpeg2_dec_copy_qtable(u8 *qtable, > - const struct v4l2_ctrl_mpeg2_quantization *ctrl); > + const struct v4l2_ctrl_mpeg2_quantization > *ctrl); > int hantro_mpeg2_dec_init(struct hantro_ctx *ctx); > void hantro_mpeg2_dec_exit(struct hantro_ctx *ctx); >
[PATCH v5 10/10] media: uapi: move MPEG-2 stateless controls out of staging
Until now, the MPEG-2 V4L2 API was not exported as a public API,
and only defined in a private media header (media/mpeg2-ctrls.h).
After reviewing the MPEG-2 specification in detail, and reworking
the controls so they match the MPEG-2 semantics properly,
we can consider it ready.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec-stateless.rst | 218 +
.../media/v4l/ext-ctrls-codec.rst | 219 --
.../media/v4l/pixfmt-compressed.rst | 10 +-
.../media/v4l/vidioc-g-ext-ctrls.rst | 12 +
drivers/media/v4l2-core/v4l2-ctrls.c | 12 +-
drivers/staging/media/hantro/hantro_drv.c | 6 +-
.../media/hantro/hantro_g1_mpeg2_dec.c| 6 +-
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 6 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 6 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 6 +-
include/media/mpeg2-ctrls.h | 128 --
include/media/v4l2-ctrls.h| 1 -
include/uapi/linux/v4l2-controls.h| 114 +
include/uapi/linux/videodev2.h| 3 +
14 files changed, 373 insertions(+), 374 deletions(-)
delete mode 100644 include/media/mpeg2-ctrls.h
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 3fc04daa9ffb..c38826024c58 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1244,3 +1244,221 @@ FWHT Flags
* - __u8
- ``padding[3]``
- Applications and drivers must set this to zero.
+
+.. _v4l2-codec-stateless-mpeg2:
+
+``V4L2_CID_STATELESS_MPEG2_SEQUENCE (struct)``
+Specifies the sequence parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the sequence header and sequence extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. c:type:: v4l2_ctrl_mpeg2_sequence
+
+.. raw:: latex
+
+\small
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{1.4cm}|p{6.5cm}|p{9.4cm}|
+
+.. flat-table:: struct v4l2_ctrl_mpeg2_sequence
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u16
+ - ``horizontal_size``
+ - The width of the displayable part of the frame's luminance component.
+* - __u16
+ - ``vertical_size``
+ - The height of the displayable part of the frame's luminance component.
+* - __u32
+ - ``vbv_buffer_size``
+ - Used to calculate the required size of the video buffering verifier,
+ defined (in bits) as: 16 * 1024 * vbv_buffer_size.
+* - __u16
+ - ``profile_and_level_indication``
+ - The current profile and level indication as extracted from the
+ bitstream.
+* - __u8
+ - ``chroma_format``
+ - The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4).
+* - __u8
+ - ``reserved``
+ - Applications and drivers must set this to zero.
+* - __u32
+ - ``flags``
+ - See :ref:`MPEG-2 Sequence Flags `.
+
+.. _mpeg2_sequence_flags:
+
+``MPEG-2 Sequence Flags``
+
+.. cssclass:: longtable
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - ``V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE``
+ - 0x0001
+ - Indication that all the frames for the sequence are progressive instead
+ of interlaced.
+
+.. raw:: latex
+
+\normalsize
+
+``V4L2_CID_STATELESS_MPEG2_PICTURE (struct)``
+Specifies the picture parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the picture header and picture coding extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. c:type:: v4l2_ctrl_mpeg2_picture
+
+.. raw:: latex
+
+\small
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{1.0cm}|p{5.6cm}|p{10.7cm}|
+
+.. flat-table:: struct v4l2_ctrl_mpeg2_picture
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u64
+ - ``backward_ref_ts``
+ - Timestamp of the V4L2 capture buffer to use as backward reference, used
+with B-coded and P-coded frames. The timestamp refers to the
+ ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
+ :c:func:`v4l2_timeval_to_ns()` function to convert the struct
+ :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``forward_ref_ts``
+ - Timestamp for the V4L2 capture buffer to use as forward reference, used
+with B-coded frames. The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer
[PATCH v5 09/10] media: uapi: Move the MPEG-2 stateless control type out of staging
Move the MPEG-2 stateless control types out of staging,
and re-number it to avoid any confusion.
Signed-off-by: Ezequiel Garcia
---
include/media/mpeg2-ctrls.h| 4
include/uapi/linux/videodev2.h | 4
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/media/mpeg2-ctrls.h b/include/media/mpeg2-ctrls.h
index 2b0084b2519b..0b0ac8fcd220 100644
--- a/include/media/mpeg2-ctrls.h
+++ b/include/media/mpeg2-ctrls.h
@@ -16,10 +16,6 @@
#define V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE
(V4L2_CID_CODEC_BASE+253)
/* enum v4l2_ctrl_type type values */
-#define V4L2_CTRL_TYPE_MPEG2_QUANTISATION 0x0131
-#define V4L2_CTRL_TYPE_MPEG2_SEQUENCE 0x0132
-#define V4L2_CTRL_TYPE_MPEG2_PICTURE 0x0133
-
#define V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE0x0001
/**
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8d15f6ccc4b4..2206f3cd28a4 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1804,6 +1804,10 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_FWHT_PARAMS = 0x0220,
V4L2_CTRL_TYPE_VP8_FRAME= 0x0240,
+
+ V4L2_CTRL_TYPE_MPEG2_QUANTISATION = 0x0250,
+ V4L2_CTRL_TYPE_MPEG2_SEQUENCE = 0x0251,
+ V4L2_CTRL_TYPE_MPEG2_PICTURE= 0x0252,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.30.0
[PATCH v5 08/10] media: controls: Log MPEG-2 stateless control in .std_log
Simply print the type of the control.
Signed-off-by: Ezequiel Garcia
---
drivers/media/v4l2-core/v4l2-ctrls.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 575553ee7e4a..722a208c647e 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1826,6 +1826,15 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_VP8_FRAME:
pr_cont("VP8_FRAME");
break;
+ case V4L2_CTRL_TYPE_MPEG2_QUANTISATION:
+ pr_cont("MPEG2_QUANTISATION");
+ break;
+ case V4L2_CTRL_TYPE_MPEG2_SEQUENCE:
+ pr_cont("MPEG2_SEQUENCE");
+ break;
+ case V4L2_CTRL_TYPE_MPEG2_PICTURE:
+ pr_cont("MPEG2_PICTURE");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
--
2.30.0
[PATCH v5 07/10] media: uapi: mpeg2: Remove V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS
The Hantro and Cedrus drivers work in frame-mode,
meaning they expect all the slices in a picture (either frame
or field structure) to be passed in each OUTPUT buffer.
These two are the only V4L2 MPEG-2 stateless decoders currently
supported. Given the VA-API drivers also work per-frame,
coalescing all the MPEG-2 slices in a buffer before the decoding
operation, it makes sense to not expect slice-mode drivers and
therefore remove V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS.
This is done to avoid carrying an unused interface. If needed,
this control can be added without breaking backwards compatibility.
Note that this would mean introducing a enumerator control to
specify the decoding mode (see V4L2_CID_STATELESS_H264_DECODE_MODE).
Signed-off-by: Ezequiel Garcia
Co-developed-by: Nicolas Dufresne
Signed-off-by: Nicolas Dufresne
---
.../media/v4l/ext-ctrls-codec.rst | 35 ---
.../media/v4l/pixfmt-compressed.rst | 6 ++--
.../media/v4l/vidioc-queryctrl.rst| 6
.../media/videodev2.h.rst.exceptions | 1 -
drivers/media/v4l2-core/v4l2-ctrls.c | 19 --
drivers/staging/media/hantro/hantro_drv.c | 5 ---
.../media/hantro/hantro_g1_mpeg2_dec.c| 9 ++---
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 8 ++---
drivers/staging/media/sunxi/cedrus/cedrus.c | 6
drivers/staging/media/sunxi/cedrus/cedrus.h | 1 -
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 --
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 2 --
include/media/mpeg2-ctrls.h | 16 -
include/media/v4l2-ctrls.h| 2 --
14 files changed, 7 insertions(+), 111 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index ebed3d8570f4..7d3bfdbed4ec 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1575,41 +1575,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. _v4l2-mpeg-mpeg2:
-``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS (struct)``
-Specifies the slice parameters (as extracted from the bitstream) for the
-associated MPEG-2 slice data. This includes the necessary parameters for
-configuring a stateless hardware decoding pipeline for MPEG-2.
-The bitstream parameters are defined according to :ref:`mpeg2part2`.
-
-.. note::
-
- This compound control is not yet part of the public kernel API and
- it is expected to change.
-
-.. c:type:: v4l2_ctrl_mpeg2_slice_params
-
-.. tabularcolumns:: |p{5.6cm}|p{4.6cm}|p{7.1cm}|
-
-.. cssclass:: longtable
-
-.. flat-table:: struct v4l2_ctrl_mpeg2_slice_params
-:header-rows: 0
-:stub-columns: 0
-:widths: 1 1 2
-
-* - __u32
- - ``bit_size``
- - Size (in bits) of the current slice data.
-* - __u32
- - ``data_bit_offset``
- - Offset (in bits) to the video data in the current slice data.
-* - __u32
- - ``quantiser_scale_code``
- - Code used to determine the quantization scale to use for the IDCT.
-* - __u8
- - ``reserved``
- - Applications and drivers must set this to zero.
-
``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE (struct)``
Specifies the sequence parameters (as extracted from the bitstream) for the
associated MPEG-2 slice data. This includes fields matching the syntax
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index bbbacbd65d6f..6c10a062adac 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -114,9 +114,9 @@ Compressed Formats
This format is adapted for stateless video decoders that implement a
MPEG-2 pipeline (using the :ref:`mem2mem` and :ref:`media-request-api`).
Metadata associated with the frame to decode is required to be passed
- through the ``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE``,
-``V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE``, and
``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS``
-controls. Quantisation matrices can optionally be specified through the
+ through the ``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE`` and
+``V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE`` controls.
+Quantisation matrices can optionally be specified through the
``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTISATION`` control.
See the :ref:`associated Codec Control IDs `.
Exactly one output and one capture buffer must be provided for use with
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index afc1505a3a7e..07e54029e1e9 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -417,12 +417,6
[PATCH v5 06/10] media: hantro/cedrus: Remove unneeded slice size and slice offset
The MPEG2_SLICE_PARAMS control is designed to refer to a single slice. However, the Hantro and Cedrus drivers operate in per-frame mode, and so does the current Ffmpeg and GStreamer implementations that are tested with these two drivers. In other words, the drivers are expecting all the slices in a picture (with either frame or field structure) to be contained in the OUTPUT buffer, which means the slice size and offset shouldn't be used. Signed-off-by: Ezequiel Garcia --- drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c | 4 ++-- drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c | 4 ++-- drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c | 7 +++ 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c index d35071e88116..c4040fba7c80 100644 --- a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c @@ -203,7 +203,7 @@ void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx) G1_REG_TOPFIELDFIRST_E(pic->flags & V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST); vdpu_write_relaxed(vpu, reg, G1_SWREG(4)); - reg = G1_REG_STRM_START_BIT(slice_params->data_bit_offset) | + reg = G1_REG_STRM_START_BIT(0) | G1_REG_QSCALE_TYPE(pic->flags & V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE) | G1_REG_CON_MV_E(pic->flags & V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV) | G1_REG_INTRA_DC_PREC(pic->intra_dc_precision) | @@ -212,7 +212,7 @@ void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx) vdpu_write_relaxed(vpu, reg, G1_SWREG(5)); reg = G1_REG_INIT_QP(1) | - G1_REG_STREAM_LEN(slice_params->bit_size >> 3); + G1_REG_STREAM_LEN(vb2_get_plane_payload(_buf->vb2_buf, 0)); vdpu_write_relaxed(vpu, reg, G1_SWREG(6)); reg = G1_REG_ALT_SCAN_FLAG_E(pic->flags & V4L2_MPEG2_PIC_FLAG_ALT_SCAN) | diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c index 18bd14704ebf..314269811244 100644 --- a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c +++ b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c @@ -177,7 +177,7 @@ void rk3399_vpu_mpeg2_dec_run(struct hantro_ctx *ctx) vdpu_write_relaxed(vpu, reg, VDPU_SWREG(50)); reg = VDPU_REG_INIT_QP(1) | - VDPU_REG_STREAM_LEN(slice_params->bit_size >> 3); + VDPU_REG_STREAM_LEN(vb2_get_plane_payload(_buf->vb2_buf, 0)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(51)); reg = VDPU_REG_APF_THRESHOLD(8) | @@ -220,7 +220,7 @@ void rk3399_vpu_mpeg2_dec_run(struct hantro_ctx *ctx) VDPU_REG_TOPFIELDFIRST_E(pic->flags & V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(120)); - reg = VDPU_REG_STRM_START_BIT(slice_params->data_bit_offset) | + reg = VDPU_REG_STRM_START_BIT(0) | VDPU_REG_QSCALE_TYPE(pic->flags & V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE) | VDPU_REG_CON_MV_E(pic->flags & V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV) | VDPU_REG_INTRA_DC_PREC(pic->intra_dc_precision) | diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c index 16e99792cf42..fd71cb175318 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c @@ -152,10 +152,9 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) /* Source offset and length in bits. */ - cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, -slice_params->data_bit_offset); + cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, 0); - reg = slice_params->bit_size - slice_params->data_bit_offset; + reg = vb2_get_plane_payload(>src->vb2_buf, 0) * 8; cedrus_write(dev, VE_DEC_MPEG_VLD_LEN, reg); /* Source beginning and end addresses. */ @@ -169,7 +168,7 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) cedrus_write(dev, VE_DEC_MPEG_VLD_ADDR, reg); - reg = src_buf_addr + DIV_ROUND_UP(slice_params->bit_size, 8); + reg = src_buf_addr + vb2_get_plane_payload(>src->vb2_buf, 0); cedrus_write(dev, VE_DEC_MPEG_VLD_END_ADDR, reg); /* Macroblock address: start at the beginning. */ -- 2.30.0
[PATCH v5 05/10] media: uapi: mpeg2: Move reference buffer fields
The forward and backwards references are specified per-picture
and not per-slice. Move it to V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec.rst | 28 +--
.../media/hantro/hantro_g1_mpeg2_dec.c| 6 ++--
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 6 ++--
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 4 +--
include/media/mpeg2-ctrls.h | 16 +--
5 files changed, 28 insertions(+), 32 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 8a0d6139db34..ebed3d8570f4 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1603,20 +1603,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u32
- ``data_bit_offset``
- Offset (in bits) to the video data in the current slice data.
-* - __u64
- - ``backward_ref_ts``
- - Timestamp of the V4L2 capture buffer to use as backward reference, used
-with B-coded and P-coded frames. The timestamp refers to the
- ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
- :c:func:`v4l2_timeval_to_ns()` function to convert the struct
- :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
-* - __u64
- - ``forward_ref_ts``
- - Timestamp for the V4L2 capture buffer to use as forward reference, used
-with B-coded frames. The timestamp refers to the ``timestamp`` field in
- struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
- function to convert the struct :c:type:`timeval` in struct
- :c:type:`v4l2_buffer` to a __u64.
* - __u32
- ``quantiser_scale_code``
- Code used to determine the quantization scale to use for the IDCT.
@@ -1712,6 +1698,20 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
+* - __u64
+ - ``backward_ref_ts``
+ - Timestamp of the V4L2 capture buffer to use as backward reference, used
+with B-coded and P-coded frames. The timestamp refers to the
+ ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
+ :c:func:`v4l2_timeval_to_ns()` function to convert the struct
+ :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``forward_ref_ts``
+ - Timestamp for the V4L2 capture buffer to use as forward reference, used
+with B-coded frames. The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
* - __u8
- ``picture_coding_type``
- Picture coding type for the frame covered by the current slice
diff --git a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
index f3494a70aa12..d35071e88116 100644
--- a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
@@ -101,12 +101,10 @@ hantro_g1_mpeg2_dec_set_buffers(struct hantro_dev *vpu,
struct hantro_ctx *ctx,
switch (pic->picture_coding_type) {
case V4L2_MPEG2_PIC_CODING_TYPE_B:
- backward_addr = hantro_get_ref(ctx,
- slice_params->backward_ref_ts);
+ backward_addr = hantro_get_ref(ctx, pic->backward_ref_ts);
fallthrough;
case V4L2_MPEG2_PIC_CODING_TYPE_P:
- forward_addr = hantro_get_ref(ctx,
- slice_params->forward_ref_ts);
+ forward_addr = hantro_get_ref(ctx, pic->forward_ref_ts);
}
/* Source bitstream buffer */
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
index 5b383906af59..18bd14704ebf 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
@@ -104,12 +104,10 @@ rk3399_vpu_mpeg2_dec_set_buffers(struct hantro_dev *vpu,
switch (pic->picture_coding_type) {
case V4L2_MPEG2_PIC_CODING_TYPE_B:
- backward_addr = hantro_get_ref(ctx,
- slice_params->backward_ref_ts);
+ backward_addr = hantro_get_ref(ctx, pic->backward_ref_ts);
fallthrough;
case V4L2_MPEG2_PIC_CODING_TYPE_P:
- forward_addr = hantro_get_ref(ctx,
- slice_params->forward_ref_ts);
+ forward_addr = hantro_get_ref(ctx, pic->forward_ref_ts);
}
/* Source bitstream buffer */
diff --git
[PATCH v5 04/10] media: uapi: mpeg2: Split sequence and picture parameters
Typically, bitstreams are composed of a sequence header,
followed by a number of picture header and picture coding extension
headers. Each picture can be composed by a number of slices.
Let's split the MPEG-2 uAPI to follow these semantics more closely,
allowing more usage flexibility. Having these controls splitted
allows applications to set a sequence control at the beginning
of a sequence, and then set a picture control for each frame.
While here add padding fields where needed, and document
the uAPI header thoroughly.
Note that the V4L2_CTRL_TYPE_{} defines had to be moved because
it clashes with existing ones. This is not really an issue
since they will be re-defined when the controls are moved
out of staging.
Signed-off-by: Ezequiel Garcia
Tested-by: Jonas Karlman
---
.../media/v4l/ext-ctrls-codec.rst | 47 ++---
.../media/v4l/pixfmt-compressed.rst | 5 +-
.../media/v4l/vidioc-queryctrl.rst| 12 +++
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/v4l2-ctrls.c | 57 ---
drivers/staging/media/hantro/hantro_drv.c | 10 ++
.../media/hantro/hantro_g1_mpeg2_dec.c| 14 +--
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 14 +--
drivers/staging/media/sunxi/cedrus/cedrus.c | 12 +++
drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +
.../staging/media/sunxi/cedrus/cedrus_dec.c | 4 +
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 8 +-
include/media/mpeg2-ctrls.h | 97 +++
include/media/v4l2-ctrls.h| 4 +
14 files changed, 228 insertions(+), 60 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 7d5ac7fb6579..8a0d6139db34 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1603,14 +1603,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u32
- ``data_bit_offset``
- Offset (in bits) to the video data in the current slice data.
-* - struct :c:type:`v4l2_mpeg2_sequence`
- - ``sequence``
- - Structure with MPEG-2 sequence metadata, merging relevant fields from
- the sequence header and sequence extension parts of the bitstream.
-* - struct :c:type:`v4l2_mpeg2_picture`
- - ``picture``
- - Structure with MPEG-2 picture metadata, merging relevant fields from
- the picture header and picture coding extension parts of the bitstream.
* - __u64
- ``backward_ref_ts``
- Timestamp of the V4L2 capture buffer to use as backward reference, used
@@ -1628,14 +1620,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u32
- ``quantiser_scale_code``
- Code used to determine the quantization scale to use for the IDCT.
+* - __u8
+ - ``reserved``
+ - Applications and drivers must set this to zero.
-.. c:type:: v4l2_mpeg2_sequence
+``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE (struct)``
+Specifies the sequence parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the sequence header and sequence extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. note::
+
+ This compound control is not yet part of the public kernel API and
+ it is expected to change.
+
+.. c:type:: v4l2_ctrl_mpeg2_sequence
.. cssclass:: longtable
.. tabularcolumns:: |p{1.4cm}|p{6.5cm}|p{9.4cm}|
-.. flat-table:: struct v4l2_mpeg2_sequence
+.. flat-table:: struct v4l2_ctrl_mpeg2_sequence
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
@@ -1657,6 +1663,9 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u8
- ``chroma_format``
- The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4).
+* - __u8
+ - ``reserved``
+ - Applications and drivers must set this to zero.
* - __u32
- ``flags``
- See :ref:`MPEG-2 Sequence Flags `.
@@ -1677,7 +1686,18 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- Indication that all the frames for the sequence are progressive instead
of interlaced.
-.. c:type:: v4l2_mpeg2_picture
+``V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE (struct)``
+Specifies the picture parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the picture header and picture coding extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. note::
+
+ This compound control is not yet part of the public kernel API and
+ it is expected to change.
+
+.. c:type:: v4l2_ctrl_mpeg2_picture
.. raw:: latex
@@ -1687,7 +1707,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. tabularcolumns:: |p{1.0cm}|p{5.6cm
[PATCH v5 03/10] media: uapi: mpeg2: Cleanup flags
Our current MPEG-2 uAPI uses 1-byte fields for MPEG-2 boolean syntax elements. Clean these by adding a 'flags' field and flag macro for each boolean syntax element. A follow-up change will refactor this uAPI so we don't need to add padding fields just yet. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- .../media/v4l/ext-ctrls-codec.rst | 77 +- drivers/media/v4l2-core/v4l2-async-core.c | 880 ++ drivers/media/v4l2-core/v4l2-ctrls.c | 14 +- .../media/hantro/hantro_g1_mpeg2_dec.c| 76 +- .../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 76 +- .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 38 +- include/media/mpeg2-ctrls.h | 36 +- 7 files changed, 1055 insertions(+), 142 deletions(-) create mode 100644 drivers/media/v4l2-core/v4l2-async-core.c diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index d9546f0aa2e8..7d5ac7fb6579 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1654,13 +1654,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``profile_and_level_indication`` - The current profile and level indication as extracted from the bitstream. -* - __u8 - - ``progressive_sequence`` - - Indication that all the frames for the sequence are progressive instead - of interlaced. * - __u8 - ``chroma_format`` - The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4). +* - __u32 + - ``flags`` + - See :ref:`MPEG-2 Sequence Flags `. + +.. _mpeg2_sequence_flags: + +``MPEG-2 Sequence Flags`` + +.. cssclass:: longtable + +.. flat-table:: +:header-rows: 0 +:stub-columns: 0 +:widths: 1 1 2 + +* - ``V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE`` + - 0x0001 + - Indication that all the frames for the sequence are progressive instead + of interlaced. .. c:type:: v4l2_mpeg2_picture @@ -1693,29 +1708,45 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``picture_structure`` - Picture structure (1: interlaced top field, 2: interlaced bottom field, 3: progressive frame). -* - __u8 - - ``top_field_first`` - - If set to 1 and interlaced stream, top field is output first. -* - __u8 - - ``frame_pred_frame_dct`` - - If set to 1, only frame-DCT and frame prediction are used. -* - __u8 - - ``concealment_motion_vectors`` - - If set to 1, motion vectors are coded for intra macroblocks. -* - __u8 - - ``q_scale_type`` +* - __u32 + - ``flags`` + - See :ref:`MPEG-2 Picture Flags `. + + +.. _mpeg2_picture_flags: + +``MPEG-2 Picture Flags`` + +.. cssclass:: longtable + +.. flat-table:: +:header-rows: 0 +:stub-columns: 0 +:widths: 1 1 2 + +* - ``V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST`` + - 0x0001 + - If set and it's an interlaced stream, top field is output first. +* - ``V4L2_MPEG2_PIC_FLAG_FRAME_PRED_DCT`` + - 0x0002 + - If set only frame-DCT and frame prediction are used. +* - ``V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV`` + - 0x0004 + - If set motion vectors are coded for intra macroblocks. +* - ``V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE`` + - 0x0008 - This flag affects the inverse quantization process. -* - __u8 - - ``intra_vlc_format`` +* - ``V4L2_MPEG2_PIC_FLAG_INTRA_VLC`` + - 0x0010 - This flag affects the decoding of transform coefficient data. -* - __u8 - - ``alternate_scan`` +* - ``V4L2_MPEG2_PIC_FLAG_ALT_SCAN`` + - 0x0020 - This flag affects the decoding of transform coefficient data. -* - __u8 - - ``repeat_first_field`` +* - ``V4L2_MPEG2_PIC_FLAG_REPEAT_FIRST`` + - 0x0040 - This flag affects the decoding process of progressive frames. -* - __u16 - - ``progressive_frame`` +* - ``V4L2_MPEG2_PIC_FLAG_PROGRESSIVE`` + - 0x0080 - Indicates whether the current frame is progressive. .. raw:: latex diff --git a/drivers/media/v4l2-core/v4l2-async-core.c b/drivers/media/v4l2-core/v4l2-async-core.c new file mode 100644 index ..cd9e78c63791 --- /dev/null +++ b/drivers/media/v4l2-core/v4l2-async-core.c @@ -0,0 +1,880 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * V4L2 asynchronous subdevice registration API + * + * Copyright (C) 2012-2013, Guennadi Liakhovetski + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +static int v4l2_async_notifier_call_bound(struct v4l2_async_notifier *n, + struct v4l2_subdev *subdev, + struct
[PATCH v5 02/10] media: uapi: mpeg2: rework quantisation matrices semantics
As stated in the MPEG-2 specification, section 6.3.7 "Quant matrix
extension":
Each quantisation matrix has a default set of values. When a
sequence_header_code is decoded all matrices shall be reset to
their default values. User defined matrices may be downloaded
and this can occur in a sequence_header() or in a
quant_matrix_extension().
The load_intra_quantiser_matrix syntax elements are transmitted
in the bistream headers, signalling that a quantisation matrix
needs to be loaded and used for pictures transmitted afterwards
(until the matrices are reset).
This "load" semantics are implemented in the V4L2 interface
without the need of any "load" flags: passing the control
is effectively a load.
Therefore, rework the V4L2_CID_MPEG_VIDEO_MPEG2_QUANTISATION
semantics to match the MPEG-2 semantics. Quantisation matrices
values are now initialized by the V4L2 control core to their
reset default value, and applications are expected to reset
their values as specified.
The quantisation control is therefore optional, and used to
load bitstream-defined values in the quantisation matrices.
Signed-off-by: Ezequiel Garcia
Co-developed-by: Nicolas Dufresne
Signed-off-by: Nicolas Dufresne
---
.../media/v4l/ext-ctrls-codec.rst | 17 -
drivers/media/v4l2-core/v4l2-ctrls.c | 21 ++
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 38 +--
include/media/mpeg2-ctrls.h | 5 ---
4 files changed, 23 insertions(+), 58 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 151d1c676b6e..d9546f0aa2e8 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1746,23 +1746,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
-* - __u8
- - ``load_intra_quantiser_matrix``
- - One bit to indicate whether to load the ``intra_quantiser_matrix``
data.
-* - __u8
- - ``load_non_intra_quantiser_matrix``
- - One bit to indicate whether to load the ``non_intra_quantiser_matrix``
- data.
-* - __u8
- - ``load_chroma_intra_quantiser_matrix``
- - One bit to indicate whether to load the
- ``chroma_intra_quantiser_matrix`` data, only relevant for non-4:2:0 YUV
- formats.
-* - __u8
- - ``load_chroma_non_intra_quantiser_matrix``
- - One bit to indicate whether to load the
- ``chroma_non_intra_quantiser_matrix`` data, only relevant for non-4:2:0
- YUV formats.
* - __u8
- ``intra_quantiser_matrix[64]``
- The quantisation matrix coefficients for intra-coded frames, in zigzag
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 74f5ca1a5f6c..5d92a2b33a6e 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -57,6 +57,18 @@ static bool is_new_manual(const struct v4l2_ctrl *master)
return master->is_auto && master->val == master->manual_mode_value;
}
+/* Default MPEG-2 quantisation coefficients, from the specification. */
+static const u8 mpeg2_intra_quant_matrix[64] = {
+ 8, 16, 16, 19, 16, 19, 22, 22,
+ 22, 22, 22, 22, 26, 24, 26, 27,
+ 27, 27, 26, 26, 26, 26, 27, 27,
+ 27, 29, 29, 29, 34, 34, 34, 29,
+ 29, 29, 27, 27, 29, 29, 32, 32,
+ 34, 34, 37, 38, 37, 35, 35, 34,
+ 35, 38, 38, 40, 40, 40, 48, 48,
+ 46, 46, 56, 56, 58, 69, 69, 83
+};
+
/* Returns NULL or a character pointer array containing the menu for
the given control ID. The pointer array ends with a NULL pointer.
An empty string signifies a menu entry that is invalid. This allows
@@ -1656,6 +1668,7 @@ static void std_init_compound(const struct v4l2_ctrl
*ctrl, u32 idx,
union v4l2_ctrl_ptr ptr)
{
struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params;
+ struct v4l2_ctrl_mpeg2_quantisation *p_mpeg2_quant;
struct v4l2_ctrl_vp8_frame *p_vp8_frame;
struct v4l2_ctrl_fwht_params *p_fwht_params;
void *p = ptr.p + idx * ctrl->elem_size;
@@ -1680,6 +1693,14 @@ static void std_init_compound(const struct v4l2_ctrl
*ctrl, u32 idx,
p_mpeg2_slice_params->picture.picture_coding_type =
V4L2_MPEG2_PICTURE_CODING_TYPE_I;
break;
+ case V4L2_CTRL_TYPE_MPEG2_QUANTISATION:
+ p_mpeg2_quant = p;
+
+ memcpy(p_mpeg2_quant->intra_quantiser_matrix,
+ mpeg2_intra_quant_matrix,
+ ARRAY_SIZE(mpeg2_intra_quant_matrix));
+ memset(p_mpeg2_quant->non_intra_quantiser_matrix, 16, 64);
+ break;
case V4L2_CTRL_TYPE_VP8_FRAME:
p_vp8_fram
[PATCH v5 01/10] media: uapi: mpeg2: Rename "quantization" to "quantisation"
The MPEG-2 specification refers to the quantisation matrices
using the word "quantisation". Make the V4L2 interface more
ergonomic by matching the MPEG-2 spec.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec.rst | 16 ++--
.../media/v4l/pixfmt-compressed.rst | 4 +--
.../media/v4l/vidioc-queryctrl.rst| 6 ++---
.../media/videodev2.h.rst.exceptions | 2 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 12 -
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../media/hantro/hantro_g1_mpeg2_dec.c| 17 +---
drivers/staging/media/hantro/hantro_hw.h | 2 +-
drivers/staging/media/hantro/hantro_mpeg2.c | 2 +-
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 14 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 4 +--
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 26 +--
include/media/mpeg2-ctrls.h | 6 ++---
include/media/v4l2-ctrls.h| 4 +--
16 files changed, 58 insertions(+), 63 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 188aef8e40d0..151d1c676b6e 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1722,8 +1722,8 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
\normalsize
-``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION (struct)``
-Specifies quantization matrices (as extracted from the bitstream) for the
+``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTISATION (struct)``
+Specifies quantisation matrices (as extracted from the bitstream) for the
associated MPEG-2 slice data.
.. note::
@@ -1731,7 +1731,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
This compound control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_ctrl_mpeg2_quantization
+.. c:type:: v4l2_ctrl_mpeg2_quantisation
.. tabularcolumns:: |p{0.8cm}|p{8.0cm}|p{8.5cm}|
@@ -1741,7 +1741,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
\small
-.. flat-table:: struct v4l2_ctrl_mpeg2_quantization
+.. flat-table:: struct v4l2_ctrl_mpeg2_quantisation
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
@@ -1765,24 +1765,24 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
YUV formats.
* - __u8
- ``intra_quantiser_matrix[64]``
- - The quantization matrix coefficients for intra-coded frames, in zigzag
+ - The quantisation matrix coefficients for intra-coded frames, in zigzag
scanning order. It is relevant for both luma and chroma components,
although it can be superseded by the chroma-specific matrix for
non-4:2:0 YUV formats.
* - __u8
- ``non_intra_quantiser_matrix[64]``
- - The quantization matrix coefficients for non-intra-coded frames, in
+ - The quantisation matrix coefficients for non-intra-coded frames, in
zigzag scanning order. It is relevant for both luma and chroma
components, although it can be superseded by the chroma-specific matrix
for non-4:2:0 YUV formats.
* - __u8
- ``chroma_intra_quantiser_matrix[64]``
- - The quantization matrix coefficients for the chominance component of
+ - The quantisation matrix coefficients for the chominance component of
intra-coded frames, in zigzag scanning order. Only relevant for
non-4:2:0 YUV formats.
* - __u8
- ``chroma_non_intra_quantiser_matrix[64]``
- - The quantization matrix coefficients for the chrominance component of
+ - The quantisation matrix coefficients for the chrominance component of
non-intra-coded frames, in zigzag scanning order. Only relevant for
non-4:2:0 YUV formats.
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 6dba70da822b..cba607f789f0 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -115,8 +115,8 @@ Compressed Formats
MPEG-2 pipeline (using the :ref:`mem2mem` and :ref:`media-request-api`).
Metadata associated with the frame to decode is required to be passed
through the ``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS`` control and
- quantization matrices can optionally be specified through the
- ``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION`` control.
+ quantisation matrices can optionally be specified through the
+ ``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTISATION`` control.
See the :ref:`associated Codec Control IDs `.
Exactly one output and one capture buffer must b
[PATCH v5 00/10] MPEG-2 stateless API cleanup and destaging
Hi everyone, Over the last few weeks Daniel Almeida and Nicolas Dufresne have been working on GStreamer v4l2codecs support for stateless MPEG-2 decoding. This allowed us to re-review the MPEG-2 specification and re-discuss the API with some additional insight. The API now looks quite cleaner, and hopefully ready for destaging. This series is mostly thanks to Daniel and Nicolas! Patches 1 to 7 cleanup the API, and patches 8 to 10 move it out of staging. This is tested on i.MX8M and RK3399 platforms, using GStreamer v4l2codecs, which will be upstream very soon. v5: * Rename "quantization" to "quantisation", so the terminology matches the MPEG-2 specification. This is the only change in v5, compared to v4. v4: * Rework and clarify quantization matrices control semantics. * Move reference buffer fields to the picture parameter control. * Remove slice parameters control. This can be added back in the future if needed, but for now it's not used. See patch 6/9 for details. * Destage the API. v3: * No API changes, just minor boilerplate fixes for the new controls to be properly exposed, initialized and validated. v2: * Fixed bad use of boolean negation in a flag, which was fortunately reported by 0day bot. Ezequiel Garcia (10): media: uapi: mpeg2: Rename "quantization" to "quantisation" media: uapi: mpeg2: rework quantisation matrices semantics media: uapi: mpeg2: Cleanup flags media: uapi: mpeg2: Split sequence and picture parameters media: uapi: mpeg2: Move reference buffer fields media: hantro/cedrus: Remove unneeded slice size and slice offset media: uapi: mpeg2: Remove V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS media: controls: Log MPEG-2 stateless control in .std_log media: uapi: Move the MPEG-2 stateless control type out of staging media: uapi: move MPEG-2 stateless controls out of staging .../media/v4l/ext-ctrls-codec-stateless.rst | 218 + .../media/v4l/ext-ctrls-codec.rst | 217 - .../media/v4l/pixfmt-compressed.rst | 11 +- .../media/v4l/vidioc-g-ext-ctrls.rst | 12 + .../media/v4l/vidioc-queryctrl.rst| 18 +- .../media/videodev2.h.rst.exceptions | 5 +- drivers/media/v4l2-core/v4l2-async-core.c | 880 ++ drivers/media/v4l2-core/v4l2-ctrls.c | 118 ++- drivers/staging/media/hantro/hantro_drv.c | 9 +- .../media/hantro/hantro_g1_mpeg2_dec.c| 110 +-- drivers/staging/media/hantro/hantro_hw.h | 2 +- drivers/staging/media/hantro/hantro_mpeg2.c | 2 +- .../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 106 +-- drivers/staging/media/sunxi/cedrus/cedrus.c | 10 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 5 +- .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +- .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 97 +- include/media/mpeg2-ctrls.h | 82 -- include/media/v4l2-ctrls.h| 11 +- include/uapi/linux/v4l2-controls.h| 114 +++ include/uapi/linux/videodev2.h| 7 + 21 files changed, 1492 insertions(+), 552 deletions(-) create mode 100644 drivers/media/v4l2-core/v4l2-async-core.c delete mode 100644 include/media/mpeg2-ctrls.h -- 2.30.0
Re: [PATCH v4 1/9] media: uapi: mpeg2: Rework quantization matrices semantics
Hi Nicolas, On Mon, 2021-03-29 at 16:53 -0400, Nicolas Dufresne wrote: > Le lundi 29 mars 2021 à 15:13 -0300, Ezequiel Garcia a écrit : > > As stated in the MPEG-2 specification, section 6.3.7 "Quant matrix > > extension": > > > > Each quantisation matrix has a default set of values. When a > > sequence_header_code is decoded all matrices shall be reset to > > their default values. User defined matrices may be downloaded > > and this can occur in a sequence_header() or in a > > quant_matrix_extension(). > > > > The load_intra_quantiser_matrix syntax elements are transmitted > > in the bistream headers, signalling that a quantization matrix > quantisation > > Not really a typo, just a suggestion to follow the specification spelling. I > would like to see concistant spelling the API. My rational is that you can > copy > and paste the strings when searching inside the specification PDF, and you > don't > mix both in the API like we do now. > Absolutely. Thanks for spotting this. > > needs to be loaded and used for pictures transmitted afterwards > > (until the matrices are reset). > > > > These "load" semantics are implemented in the V4L2 interface > > without the need of any "load" flags: passing the control > > is effectively a load. > > > > Therefore, rework the V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION > S > > > semantics to match the MPEG-2 semantics. Quantization matrices > s > > etc. > > > values are now initialized by the V4L2 control core to their > > reset default value, and applications are expected to reset > > their values as specified. > > > > The quantization control is therefore optional, and used to > > load bitstream-defined values in the quantization matrices. > > Perhaps: > > "The quantisation controls is therefore optional for decoding streams that > uses > the default matrices." > > A stack that would not handle the default, would have to read the control at > least once in order to avoid overriding valid values with 0s, not sure if that > is worth mentioning ? > Hm, not entirely sure. If application calls S_EXT_CTRL with 0s, then that's what will happen. Thanks, Ezequiel
Re: [PATCH v8 09/13] media: uapi: Add a control for HANTRO driver
Hi Benjamin,
Thanks for the patch.
On Thu, 2021-04-01 at 17:59 +0200, Benjamin Gaignard wrote:
> The HEVC HANTRO driver needs to know the number of bits to skip at
> the beginning of the slice header.
> That is a hardware specific requirement so create a dedicated control
> that this purpose.
>
> Signed-off-by: Benjamin Gaignard
> ---
> version 5:
> - Be even more verbose in control documentation.
> - Do not create class for the control.
> version 4:
> - The control is now an integer which is enough to provide the numbers
> of bits to skip.
> version 3:
> - Fix typo in field name
>
> .../userspace-api/media/drivers/hantro.rst | 14 ++
> .../userspace-api/media/drivers/index.rst | 1 +
> include/uapi/linux/v4l2-controls.h | 13 +
> 3 files changed, 28 insertions(+)
> create mode 100644 Documentation/userspace-api/media/drivers/hantro.rst
>
> diff --git a/Documentation/userspace-api/media/drivers/hantro.rst
> b/Documentation/userspace-api/media/drivers/hantro.rst
> new file mode 100644
> index ..78dcd2a44a03
> --- /dev/null
> +++ b/Documentation/userspace-api/media/drivers/hantro.rst
> @@ -0,0 +1,14 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +Hantro video decoder driver
> +===
> +
> +The Hantro video decoder driver implements the following driver-specific
> controls:
> +
> +``V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (integer)``
> + Specifies to Hantro HEVC video decoder driver the number of data (in
> bits) to
> + skip in the slice segment header.
> + If non-IDR, the bits to be skipped go from syntax element
> "pic_output_flag"
> + to before syntax element "slice_temporal_mvp_enabled_flag".
> + If IDR, the skipped bits are just "pic_output_flag"
> + (separate_colour_plane_flag is not supported).
> diff --git a/Documentation/userspace-api/media/drivers/index.rst
> b/Documentation/userspace-api/media/drivers/index.rst
> index 1a9038f5f9fa..12e3c512d718 100644
> --- a/Documentation/userspace-api/media/drivers/index.rst
> +++ b/Documentation/userspace-api/media/drivers/index.rst
> @@ -33,6 +33,7 @@ For more details see the file COPYING in the source
> distribution of Linux.
>
> ccs
> cx2341x-uapi
> + hantro
> imx-uapi
> max2175
> meye-uapi
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index f3376aafea65..1dfb874b6272 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -869,6 +869,19 @@ enum v4l2_mpeg_mfc51_video_force_frame_type {
> #define V4L2_CID_MPEG_MFC51_VIDEO_H264_ADAPTIVE_RC_STATIC
> (V4L2_CID_CODEC_MFC51_BASE+53)
> #define V4L2_CID_MPEG_MFC51_VIDEO_H264_NUM_REF_PIC_FOR_P
> (V4L2_CID_CODEC_MFC51_BASE+54)
>
> +/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2
> */
We are moving away from "MPEG" terminology for codecs.
> +#define V4L2_CID_CODEC_HANTRO_BASE
> (V4L2_CTRL_CLASS_CODEC | 0x1200)
Using V4L2_CTRL_CLASS_CODEC_STATELESS is IMO better,
since this belongs to a stateless decoder.
And also, since we are still a bit unsure about the
syntax of this parameter (given it's not documented):
how about keeping the V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP
definition in drivers/staging/media/hantro/hantro.h ?
This would be a hint for applications that this control
is a quirk.
Hans, Philipp, any thoughts on this?
Regards,
Ezequiel
Re: [PATCH v7 12/13] media: hantro: IMX8M: add variant for G2/HEVC codec
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote:
> Add variant to IMX8M to enable G2/HEVC codec.
> Define the capabilities for the hardware up to 3840x2160.
> G2 doesn't have postprocessor, use the same clocks and got it
> own interruption.
>
> Signed-off-by: Benjamin Gaignard
> Reviewed-by: Philipp Zabel
Reviewed-by: Ezequiel Garcia
> ---
> version 7:
> - Add Philipp Reviewed-by tag.
>
> version 5:
> - remove useless postproc fields for G2
>
> version 2:
> - remove useless clocks
>
> drivers/staging/media/hantro/hantro_drv.c | 1 +
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 76 -
> 3 files changed, 76 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index 33b8bd38eac1..ed380a8bef93 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -574,6 +574,7 @@ static const struct of_device_id of_hantro_match[] = {
> #endif
> #ifdef CONFIG_VIDEO_HANTRO_IMX8M
> { .compatible = "nxp,imx8mq-vpu", .data = _vpu_variant, },
> + { .compatible = "nxp,imx8mq-vpu-g2", .data = _vpu_g2_variant },
> #endif
> { /* sentinel */ }
> };
> diff --git a/drivers/staging/media/hantro/hantro_hw.h
> b/drivers/staging/media/hantro/hantro_hw.h
> index 5788188aae50..b4e7490bbe45 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -193,6 +193,7 @@ extern const struct hantro_variant rk3399_vpu_variant;
> extern const struct hantro_variant rk3328_vpu_variant;
> extern const struct hantro_variant rk3288_vpu_variant;
> extern const struct hantro_variant imx8mq_vpu_variant;
> +extern const struct hantro_variant imx8mq_vpu_g2_variant;
>
> extern const struct hantro_postproc_regs hantro_g1_postproc_regs;
>
> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> index 8d0c3425234b..6de43e0edc36 100644
> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> @@ -12,6 +12,7 @@
> #include "hantro.h"
> #include "hantro_jpeg.h"
> #include "hantro_g1_regs.h"
> +#include "hantro_g2_regs.h"
>
> #define CTRL_SOFT_RESET0x00
> #define RESET_G1 BIT(1)
> @@ -129,6 +130,26 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
> },
> };
>
> +static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
> + {
> + .fourcc = V4L2_PIX_FMT_NV12,
> + .codec_mode = HANTRO_MODE_NONE,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
> + .codec_mode = HANTRO_MODE_HEVC_DEC,
> + .max_depth = 2,
> + .frmsize = {
> + .min_width = 48,
> + .max_width = 3840,
> + .step_width = MB_DIM,
> + .min_height = 48,
> + .max_height = 2160,
> + .step_height = MB_DIM,
> + },
> + },
> +};
> +
> static irqreturn_t imx8m_vpu_g1_irq(int irq, void *dev_id)
> {
> struct hantro_dev *vpu = dev_id;
> @@ -147,6 +168,24 @@ static irqreturn_t imx8m_vpu_g1_irq(int irq, void
> *dev_id)
> return IRQ_HANDLED;
> }
>
> +static irqreturn_t imx8m_vpu_g2_irq(int irq, void *dev_id)
> +{
> + struct hantro_dev *vpu = dev_id;
> + enum vb2_buffer_state state;
> + u32 status;
> +
> + status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
> + state = (status & HEVC_REG_INTERRUPT_DEC_RDY_INT) ?
> + VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
> +
> + vdpu_write(vpu, 0, HEVC_REG_INTERRUPT);
> + vdpu_write(vpu, HEVC_REG_CONFIG_DEC_CLK_GATE_E, HEVC_REG_CONFIG);
> +
> + hantro_irq_done(vpu, state);
> +
> + return IRQ_HANDLED;
> +}
> +
> static int imx8mq_vpu_hw_init(struct hantro_dev *vpu)
> {
> struct device_node *np = vpu->dev->of_node;
> @@ -176,6 +215,13 @@ static void imx8m_vpu_g1_reset(struct hantro_ctx *ctx)
> imx8m_soft_reset(vpu, RESET_G1);
> }
>
> +static void imx8m_vpu_g2_reset(struct hantro_ctx *ctx)
> +{
> + struct hantro_dev *vpu = ctx->dev;
> +
> + imx8m_soft_reset(vpu, RESET_G2);
> +}
> +
> /*
> * Supported codec ops.
> */
> @@ -201,16 +247,28 @@ static const struct hantro_codec_ops
> imx8mq_vpu_co
Re: [PATCH v7 10/13] media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote: > Make sure that V4L2_PIX_FMT_HEVC_SLICE is correctly handle by v4l2 > of the driver. > > Signed-off-by: Benjamin Gaignard Reviewed-by: Ezequiel Garcia > --- > drivers/staging/media/hantro/hantro_v4l2.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/staging/media/hantro/hantro_v4l2.c > b/drivers/staging/media/hantro/hantro_v4l2.c > index 77d7fe62ce81..0655324fd0d4 100644 > --- a/drivers/staging/media/hantro/hantro_v4l2.c > +++ b/drivers/staging/media/hantro/hantro_v4l2.c > @@ -392,6 +392,7 @@ hantro_update_requires_request(struct hantro_ctx *ctx, > u32 fourcc) > case V4L2_PIX_FMT_MPEG2_SLICE: > case V4L2_PIX_FMT_VP8_FRAME: > case V4L2_PIX_FMT_H264_SLICE: > + case V4L2_PIX_FMT_HEVC_SLICE: > ctx->fh.m2m_ctx->out_q_ctx.q.requires_requests = true; > break; > default:
Re: [PATCH v7 08/13] media: hantro: Only use postproc when post processed formats are defined
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote:
> If the variant doesn't offert postprocessed formats make sure it will
> be ok.
>
> Signed-off-by: Benjamin Gaignard
Reviewed-by: Ezequiel Garcia
> ---
> drivers/staging/media/hantro/hantro.h | 8 ++--
> drivers/staging/media/hantro/hantro_postproc.c | 14 ++
> drivers/staging/media/hantro/hantro_v4l2.c | 4 +++-
> 3 files changed, 19 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index edb4561a6887..7a5ad93466c8 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -414,12 +414,8 @@ hantro_get_dst_buf(struct hantro_ctx *ctx)
> return v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
> }
>
> -static inline bool
> -hantro_needs_postproc(const struct hantro_ctx *ctx,
> - const struct hantro_fmt *fmt)
> -{
> - return !ctx->is_encoder && fmt->fourcc != V4L2_PIX_FMT_NV12;
> -}
> +bool hantro_needs_postproc(const struct hantro_ctx *ctx,
> + const struct hantro_fmt *fmt);
>
> static inline dma_addr_t
> hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> diff --git a/drivers/staging/media/hantro/hantro_postproc.c
> b/drivers/staging/media/hantro/hantro_postproc.c
> index 6d2a8f2a8f0b..ed8916c950a4 100644
> --- a/drivers/staging/media/hantro/hantro_postproc.c
> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> @@ -50,6 +50,20 @@ const struct hantro_postproc_regs hantro_g1_postproc_regs
> = {
> .display_width = {G1_REG_PP_DISPLAY_WIDTH, 0, 0xfff},
> };
>
> +bool hantro_needs_postproc(const struct hantro_ctx *ctx,
> + const struct hantro_fmt *fmt)
> +{
> + struct hantro_dev *vpu = ctx->dev;
> +
> + if (ctx->is_encoder)
> + return false;
> +
> + if (!vpu->variant->postproc_fmts)
> + return false;
> +
> + return fmt->fourcc != V4L2_PIX_FMT_NV12;
> +}
> +
> void hantro_postproc_enable(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
> diff --git a/drivers/staging/media/hantro/hantro_v4l2.c
> b/drivers/staging/media/hantro/hantro_v4l2.c
> index 1bc118e375a1..77d7fe62ce81 100644
> --- a/drivers/staging/media/hantro/hantro_v4l2.c
> +++ b/drivers/staging/media/hantro/hantro_v4l2.c
> @@ -55,7 +55,9 @@ static const struct hantro_fmt *
> hantro_get_postproc_formats(const struct hantro_ctx *ctx,
> unsigned int *num_fmts)
> {
> - if (ctx->is_encoder) {
> + struct hantro_dev *vpu = ctx->dev;
> +
> + if (ctx->is_encoder || !vpu->variant->postproc_fmts) {
> *num_fmts = 0;
> return NULL;
> }
Re: [PATCH v7 07/13] media: hantro: Define HEVC codec profiles and supported features
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote:
> Define which HEVC profiles (up to level 5.1) and features
> (no scaling, no 10 bits) are supported by the driver.
>
> Signed-off-by: Benjamin Gaignard
Reviewed-by: Ezequiel Garcia
> ---
> drivers/staging/media/hantro/hantro.h | 3 ++
> drivers/staging/media/hantro/hantro_drv.c | 58 +++
> 2 files changed, 61 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index 37b9ce04bd4e..edb4561a6887 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -35,6 +35,7 @@ struct hantro_codec_ops;
> #define HANTRO_MPEG2_DECODER BIT(16)
> #define HANTRO_VP8_DECODER BIT(17)
> #define HANTRO_H264_DECODERBIT(18)
> +#define HANTRO_HEVC_DECODERBIT(19)
> #define HANTRO_DECODERS0x
>
> /**
> @@ -100,6 +101,7 @@ struct hantro_variant {
> * @HANTRO_MODE_H264_DEC: H264 decoder.
> * @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
> * @HANTRO_MODE_VP8_DEC: VP8 decoder.
> + * @HANTRO_MODE_HEVC_DEC: HEVC decoder.
> */
> enum hantro_codec_mode {
> HANTRO_MODE_NONE = -1,
> @@ -107,6 +109,7 @@ enum hantro_codec_mode {
> HANTRO_MODE_H264_DEC,
> HANTRO_MODE_MPEG2_DEC,
> HANTRO_MODE_VP8_DEC,
> + HANTRO_MODE_HEVC_DEC,
> };
>
> /*
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index 02c5c2f1a88b..d9a3a5ef9330 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -245,6 +245,18 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
> + } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
> + const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
> +
> + if (sps->bit_depth_luma_minus8 !=
> sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
> + /* No scaling support */
> + return -EINVAL;
> }
> return 0;
> }
> @@ -351,6 +363,52 @@ static const struct hantro_ctrl controls[] = {
> .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> }
> }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
> + .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
> + .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
> + .min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> + .max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
> + .def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
> + .min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
> + .max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
> + .ops = _ctrl_ops,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> + },
> },
> };
>
Re: [PATCH v7 06/13] media: hantro: change hantro_codec_ops run prototype to return errors
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote:
> Change hantro_codec_ops run prototype from 'void' to 'int'.
> This allow to cancel the job if an error occur while configuring
> the hardware.
>
> Signed-off-by: Benjamin Gaignard
Reviewed-by: Ezequiel Garcia
> ---
> version 5:
> - forward hantro_h264_dec_prepare_run() return value in case
> of error
>
> drivers/staging/media/hantro/hantro_drv.c | 4 +++-
> .../staging/media/hantro/hantro_g1_h264_dec.c | 10 +++---
> .../media/hantro/hantro_g1_mpeg2_dec.c | 4 +++-
> .../staging/media/hantro/hantro_g1_vp8_dec.c | 6 --
> .../staging/media/hantro/hantro_h1_jpeg_enc.c | 4 +++-
> drivers/staging/media/hantro/hantro_hw.h | 19 ++-
> .../media/hantro/rk3399_vpu_hw_jpeg_enc.c | 4 +++-
> .../media/hantro/rk3399_vpu_hw_mpeg2_dec.c | 4 +++-
> .../media/hantro/rk3399_vpu_hw_vp8_dec.c | 6 --
> 9 files changed, 40 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index 595e82a82728..02c5c2f1a88b 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -161,7 +161,9 @@ static void device_run(void *priv)
>
> v4l2_m2m_buf_copy_metadata(src, dst, true);
>
> - ctx->codec_ops->run(ctx);
> + if (ctx->codec_ops->run(ctx))
> + goto err_cancel_job;
> +
> return;
>
> err_cancel_job:
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index 845bef73d218..5c792b7bcb79 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -273,13 +273,15 @@ static void set_buffers(struct hantro_ctx *ctx)
> vdpu_write_relaxed(vpu, ctx->h264_dec.priv.dma, G1_REG_ADDR_QTABLE);
> }
>
> -void hantro_g1_h264_dec_run(struct hantro_ctx *ctx)
> +int hantro_g1_h264_dec_run(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
> + int ret;
>
> /* Prepare the H264 decoder context. */
> - if (hantro_h264_dec_prepare_run(ctx))
> - return;
> + ret = hantro_h264_dec_prepare_run(ctx);
> + if (ret)
> + return ret;
>
> /* Configure hardware registers. */
> set_params(ctx);
> @@ -301,4 +303,6 @@ void hantro_g1_h264_dec_run(struct hantro_ctx *ctx)
> G1_REG_CONFIG_DEC_CLK_GATE_E,
> G1_REG_CONFIG);
> vdpu_write(vpu, G1_REG_INTERRUPT_DEC_E, G1_REG_INTERRUPT);
> +
> + return 0;
> }
> diff --git a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
> b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
> index 6386a3989bfe..5e8943d31dc5 100644
> --- a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
> @@ -155,7 +155,7 @@ hantro_g1_mpeg2_dec_set_buffers(struct hantro_dev *vpu,
> struct hantro_ctx *ctx,
> vdpu_write_relaxed(vpu, backward_addr, G1_REG_REFER3_BASE);
> }
>
> -void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx)
> +int hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
> struct vb2_v4l2_buffer *src_buf, *dst_buf;
> @@ -248,4 +248,6 @@ void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx)
>
> reg = G1_REG_DEC_E(1);
> vdpu_write(vpu, reg, G1_SWREG(1));
> +
> + return 0;
> }
> diff --git a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
> b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
> index 57002ba70176..96622a7f8279 100644
> --- a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
> @@ -425,7 +425,7 @@ static void cfg_buffers(struct hantro_ctx *ctx,
> vdpu_write_relaxed(vpu, dst_dma, G1_REG_ADDR_DST);
> }
>
> -void hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
> +int hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
> {
> const struct v4l2_ctrl_vp8_frame *hdr;
> struct hantro_dev *vpu = ctx->dev;
> @@ -438,7 +438,7 @@ void hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
>
> hdr = hantro_get_ctrl(ctx, V4L2_CID_STATELESS_VP8_FRAME);
> if (WARN_ON(!hdr))
> - return;
> + return -EINVAL;
>
> /* Reset segment_map buffer in keyframe */
> if (V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
> @@ -498,4 +498,6 @@ void hantro_g
Re: [PATCH v7 05/13] media: hevc: Add decode params control
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote: > Add decode params control and it associated structure to regroup > all the information that are needed to decode a reference frame as > it is describe in ITU-T Rec. H.265 section "8.3.2 Decoding process > for reference picture set". > > Adapt Cedrus driver to these changes. > > Signed-off-by: Benjamin Gaignard Reviewed-by: Ezequiel Garcia > --- > version 7: > - rebased on top of media_tree/master branch > > version 6: > - fix compilation errors > > .../media/v4l/ext-ctrls-codec.rst | 94 +++ > .../media/v4l/vidioc-queryctrl.rst | 6 ++ > drivers/media/v4l2-core/v4l2-ctrls.c | 26 +++-- > drivers/staging/media/sunxi/cedrus/cedrus.c | 6 ++ > drivers/staging/media/sunxi/cedrus/cedrus.h | 1 + > .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 + > .../staging/media/sunxi/cedrus/cedrus_h265.c | 12 ++- > include/media/hevc-ctrls.h | 29 -- > 8 files changed, 137 insertions(+), 39 deletions(-) > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > index 92314aec655a..7552869687f7 100644 > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > @@ -3181,9 +3181,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > * - __u8 > - ``pic_struct`` > - > - * - __u8 > - - ``num_active_dpb_entries`` > - - The number of entries in ``dpb``. > * - __u8 > - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]`` > - The list of L0 reference elements as indices in the DPB. > @@ -3191,22 +3188,8 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]`` > - The list of L1 reference elements as indices in the DPB. > * - __u8 > - - ``num_rps_poc_st_curr_before`` > - - The number of reference pictures in the short-term set that come > before > - the current frame. > - * - __u8 > - - ``num_rps_poc_st_curr_after`` > - - The number of reference pictures in the short-term set that come > after > - the current frame. > - * - __u8 > - - ``num_rps_poc_lt_curr`` > - - The number of reference pictures in the long-term set. > - * - __u8 > - - ``padding[7]`` > + - ``padding`` > - Applications and drivers must set this to zero. > - * - struct :c:type:`v4l2_hevc_dpb_entry` > - - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]`` > - - The decoded picture buffer, for meta-data about reference frames. > * - struct :c:type:`v4l2_hevc_pred_weight_table` > - ``pred_weight_table`` > - The prediction weight coefficients for inter-picture prediction. > @@ -3441,3 +3424,78 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > so this has to come from client. > This is applicable to H264 and valid Range is from 0 to 63. > Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1. > + > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)`` > + Specifies various decode parameters, especially the references picture > order > + count (POC) for all the lists (short, long, before, current, after) and > the > + number of entries for each of them. > + These parameters are defined according to :ref:`hevc`. > + They are described in section 8.3 "Slice decoding process" of the > + specification. > + > +.. c:type:: v4l2_ctrl_hevc_decode_params > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __s32 > + - ``pic_order_cnt_val`` > + - PicOrderCntVal as described in section 8.3.1 "Decoding process > + for picture order count" of the specification. > + * - __u8 > + - ``num_active_dpb_entries`` > + - The number of entries in ``dpb``. > + * - struct :c:type:`v4l2_hevc_dpb_entry` > + - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]`` > + - The decoded picture buffer, for meta-data about reference frames. > + * - __u8 > + - ``num_poc_st_curr_before`` > + - The number of reference pictures in the short-term set that come > before > + the current frame. > + * - __u8 > + - ``num_poc_st_curr_after`` > + - The number of reference pictures in the short-term set that come > after > + the current frame. > + * - __u8 > + - ``num_poc_lt_curr`` > + - The number of refe
Re: [PATCH v7 04/13] media: hevc: Add fields and flags for hevc PPS
On Mon, 2021-03-29 at 08:57 +0200, Benjamin Gaignard wrote:
> Add fields and flags as they are defined in
> 7.4.3.3.1 "General picture parameter set RBSP semantics of the
> H.265 ITU specification.
>
> Signed-off-by: Benjamin Gaignard
Reviewed-by: Ezequiel Garcia
> ---
> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 14 ++
> include/media/hevc-ctrls.h | 4
> 2 files changed, 18 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 188aef8e40d0..92314aec655a 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -2967,6 +2967,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> * - __u8
> - ``num_extra_slice_header_bits``
> -
> + * - __u8
> + - ``num_ref_idx_l0_default_active_minus1``
> + - Specifies the inferred value of num_ref_idx_l0_active_minus1
> + * - __u8
> + - ``num_ref_idx_l1_default_active_minus1``
> + - Specifies the inferred value of num_ref_idx_l1_active_minus1
> * - __s8
> - ``init_qp_minus26``
> -
> @@ -3077,6 +3083,14 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
> - 0x0004
> -
> + * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> + - 0x0008
> + - Specifies the presence of deblocking filter control syntax elements
> in
> + the PPS
> + * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> + - 0x0010
> + - Specifies that tile column boundaries and likewise tile row
> boundaries
> + are distributed uniformly across the picture
>
> .. raw:: latex
>
> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> index b4cb2ef02f17..003f819ecb26 100644
> --- a/include/media/hevc-ctrls.h
> +++ b/include/media/hevc-ctrls.h
> @@ -100,10 +100,14 @@ struct v4l2_ctrl_hevc_sps {
> #define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
> #define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
> #define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL <<
> 18)
> +#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
> +#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
>
> struct v4l2_ctrl_hevc_pps {
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
> __u8num_extra_slice_header_bits;
> + __u8num_ref_idx_l0_default_active_minus1;
> + __u8num_ref_idx_l1_default_active_minus1;
> __s8init_qp_minus26;
> __u8diff_cu_qp_delta_depth;
> __s8pps_cb_qp_offset;
[PATCH v4 9/9] media: uapi: Move MPEG-2 stateless controls out of staging
Until now, the MPEG-2 V4L2 API was not exported as a public API,
and only defined in a private media header (media/mpeg2-ctrls.h).
After reviewing the MPEG-2 specification in detail, and reworking
the controls so they match the MPEG-2 semantics properly,
we can consider it ready.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec-stateless.rst | 218 +
.../media/v4l/ext-ctrls-codec.rst | 219 --
.../media/v4l/pixfmt-compressed.rst | 10 +-
.../media/v4l/vidioc-g-ext-ctrls.rst | 12 +
drivers/media/v4l2-core/v4l2-ctrls.c | 12 +-
drivers/staging/media/hantro/hantro_drv.c | 6 +-
.../media/hantro/hantro_g1_mpeg2_dec.c| 6 +-
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 6 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 6 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 6 +-
include/media/mpeg2-ctrls.h | 128 --
include/media/v4l2-ctrls.h| 1 -
include/uapi/linux/v4l2-controls.h| 114 +
include/uapi/linux/videodev2.h| 3 +
14 files changed, 373 insertions(+), 374 deletions(-)
delete mode 100644 include/media/mpeg2-ctrls.h
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 3fc04daa9ffb..d8c0c3f5c676 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -1244,3 +1244,221 @@ FWHT Flags
* - __u8
- ``padding[3]``
- Applications and drivers must set this to zero.
+
+.. _v4l2-codec-stateless-mpeg2:
+
+``V4L2_CID_STATELESS_MPEG2_SEQUENCE (struct)``
+Specifies the sequence parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the sequence header and sequence extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. c:type:: v4l2_ctrl_mpeg2_sequence
+
+.. raw:: latex
+
+\small
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{1.4cm}|p{6.5cm}|p{9.4cm}|
+
+.. flat-table:: struct v4l2_ctrl_mpeg2_sequence
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u16
+ - ``horizontal_size``
+ - The width of the displayable part of the frame's luminance component.
+* - __u16
+ - ``vertical_size``
+ - The height of the displayable part of the frame's luminance component.
+* - __u32
+ - ``vbv_buffer_size``
+ - Used to calculate the required size of the video buffering verifier,
+ defined (in bits) as: 16 * 1024 * vbv_buffer_size.
+* - __u16
+ - ``profile_and_level_indication``
+ - The current profile and level indication as extracted from the
+ bitstream.
+* - __u8
+ - ``chroma_format``
+ - The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4).
+* - __u8
+ - ``reserved``
+ - Applications and drivers must set this to zero.
+* - __u32
+ - ``flags``
+ - See :ref:`MPEG-2 Sequence Flags `.
+
+.. _mpeg2_sequence_flags:
+
+``MPEG-2 Sequence Flags``
+
+.. cssclass:: longtable
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - ``V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE``
+ - 0x0001
+ - Indication that all the frames for the sequence are progressive instead
+ of interlaced.
+
+.. raw:: latex
+
+\normalsize
+
+``V4L2_CID_STATELESS_MPEG2_PICTURE (struct)``
+Specifies the picture parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the picture header and picture coding extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. c:type:: v4l2_ctrl_mpeg2_picture
+
+.. raw:: latex
+
+\small
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{1.0cm}|p{5.6cm}|p{10.7cm}|
+
+.. flat-table:: struct v4l2_ctrl_mpeg2_picture
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u64
+ - ``backward_ref_ts``
+ - Timestamp of the V4L2 capture buffer to use as backward reference, used
+with B-coded and P-coded frames. The timestamp refers to the
+ ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
+ :c:func:`v4l2_timeval_to_ns()` function to convert the struct
+ :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``forward_ref_ts``
+ - Timestamp for the V4L2 capture buffer to use as forward reference, used
+with B-coded frames. The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer
[PATCH v4 7/9] media: controls: Log MPEG-2 stateless control in .std_log
Simply print the type of the control.
Signed-off-by: Ezequiel Garcia
---
drivers/media/v4l2-core/v4l2-ctrls.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 28657f2f679b..063a373c8965 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1826,6 +1826,15 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_VP8_FRAME:
pr_cont("VP8_FRAME");
break;
+ case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION:
+ pr_cont("MPEG2_QUANTIZATION");
+ break;
+ case V4L2_CTRL_TYPE_MPEG2_SEQUENCE:
+ pr_cont("MPEG2_SEQUENCE");
+ break;
+ case V4L2_CTRL_TYPE_MPEG2_PICTURE:
+ pr_cont("MPEG2_PICTURE");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
--
2.30.0
[PATCH v4 6/9] media: uapi: mpeg2: Remove V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS
The Hantro and Cedrus drivers work in frame-mode,
meaning they expect all the slices in a picture (either frame
or field structure) to be passed in each OUTPUT buffer.
These two are the only V4L2 MPEG-2 stateless decoders currently
supported. Given the VA-API drivers also work per-frame,
coalescing all the MPEG-2 slices in a buffer before the decoding
operation, it makes sense to not expect slice-mode drivers and
therefore remove V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS.
This is done to avoid carrying an unused interface. If needed,
this control can be added without breaking backwards compatibility.
Note that this would mean introducing a enumerator control to
specify the decoding mode (see V4L2_CID_STATELESS_H264_DECODE_MODE).
Signed-off-by: Ezequiel Garcia
Co-developed-by: Nicolas Dufresne
Signed-off-by: Nicolas Dufresne
---
.../media/v4l/ext-ctrls-codec.rst | 35 ---
.../media/v4l/pixfmt-compressed.rst | 6 ++--
.../media/v4l/vidioc-queryctrl.rst| 6
.../media/videodev2.h.rst.exceptions | 1 -
drivers/media/v4l2-core/v4l2-ctrls.c | 19 --
drivers/staging/media/hantro/hantro_drv.c | 5 ---
.../media/hantro/hantro_g1_mpeg2_dec.c| 9 ++---
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 8 ++---
drivers/staging/media/sunxi/cedrus/cedrus.c | 6
drivers/staging/media/sunxi/cedrus/cedrus.h | 1 -
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 --
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 2 --
include/media/mpeg2-ctrls.h | 16 -
include/media/v4l2-ctrls.h| 2 --
14 files changed, 7 insertions(+), 111 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index c0cabae3e8ee..76b3652a6444 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1575,41 +1575,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. _v4l2-mpeg-mpeg2:
-``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS (struct)``
-Specifies the slice parameters (as extracted from the bitstream) for the
-associated MPEG-2 slice data. This includes the necessary parameters for
-configuring a stateless hardware decoding pipeline for MPEG-2.
-The bitstream parameters are defined according to :ref:`mpeg2part2`.
-
-.. note::
-
- This compound control is not yet part of the public kernel API and
- it is expected to change.
-
-.. c:type:: v4l2_ctrl_mpeg2_slice_params
-
-.. tabularcolumns:: |p{5.6cm}|p{4.6cm}|p{7.1cm}|
-
-.. cssclass:: longtable
-
-.. flat-table:: struct v4l2_ctrl_mpeg2_slice_params
-:header-rows: 0
-:stub-columns: 0
-:widths: 1 1 2
-
-* - __u32
- - ``bit_size``
- - Size (in bits) of the current slice data.
-* - __u32
- - ``data_bit_offset``
- - Offset (in bits) to the video data in the current slice data.
-* - __u32
- - ``quantiser_scale_code``
- - Code used to determine the quantization scale to use for the IDCT.
-* - __u8
- - ``reserved``
- - Applications and drivers must set this to zero.
-
``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE (struct)``
Specifies the sequence parameters (as extracted from the bitstream) for the
associated MPEG-2 slice data. This includes fields matching the syntax
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 229728fcc117..e6b7d3c7188f 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -114,9 +114,9 @@ Compressed Formats
This format is adapted for stateless video decoders that implement a
MPEG-2 pipeline (using the :ref:`mem2mem` and :ref:`media-request-api`).
Metadata associated with the frame to decode is required to be passed
- through the ``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE``,
-``V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE``, and
``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS``
-controls. Quantization matrices can optionally be specified through the
+ through the ``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE`` and
+``V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE`` controls.
+Quantization matrices can optionally be specified through the
``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION`` control.
See the :ref:`associated Codec Control IDs `.
Exactly one output and one capture buffer must be provided for use with
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 233cdfa38f3e..9550a4dfed16 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -417,12 +417,6
[PATCH v4 8/9] media: uapi: Move the MPEG-2 stateless control type out of staging
Move the MPEG-2 stateless control types out of staging,
and re-number it to avoid any confusion.
Signed-off-by: Ezequiel Garcia
---
include/media/mpeg2-ctrls.h| 4
include/uapi/linux/videodev2.h | 4
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/media/mpeg2-ctrls.h b/include/media/mpeg2-ctrls.h
index 62a8a49c1e8b..9237e1a4befe 100644
--- a/include/media/mpeg2-ctrls.h
+++ b/include/media/mpeg2-ctrls.h
@@ -16,10 +16,6 @@
#define V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE
(V4L2_CID_CODEC_BASE+253)
/* enum v4l2_ctrl_type type values */
-#define V4L2_CTRL_TYPE_MPEG2_QUANTIZATION 0x0131
-#define V4L2_CTRL_TYPE_MPEG2_SEQUENCE 0x0132
-#define V4L2_CTRL_TYPE_MPEG2_PICTURE 0x0133
-
#define V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE0x0001
/**
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8d15f6ccc4b4..558dfc8cb8d8 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1804,6 +1804,10 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_FWHT_PARAMS = 0x0220,
V4L2_CTRL_TYPE_VP8_FRAME= 0x0240,
+
+ V4L2_CTRL_TYPE_MPEG2_QUANTIZATION = 0x0250,
+ V4L2_CTRL_TYPE_MPEG2_SEQUENCE = 0x0251,
+ V4L2_CTRL_TYPE_MPEG2_PICTURE= 0x0252,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.30.0
[PATCH v4 4/9] media: uapi: mpeg2: Move reference buffer fields
The forward and backwards references are specified per-picture
and not per-slice. Move it to V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec.rst | 28 +--
.../media/hantro/hantro_g1_mpeg2_dec.c| 6 ++--
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 6 ++--
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 4 +--
include/media/mpeg2-ctrls.h | 16 +--
5 files changed, 28 insertions(+), 32 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 35d8845d396c..c0cabae3e8ee 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1603,20 +1603,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u32
- ``data_bit_offset``
- Offset (in bits) to the video data in the current slice data.
-* - __u64
- - ``backward_ref_ts``
- - Timestamp of the V4L2 capture buffer to use as backward reference, used
-with B-coded and P-coded frames. The timestamp refers to the
- ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
- :c:func:`v4l2_timeval_to_ns()` function to convert the struct
- :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
-* - __u64
- - ``forward_ref_ts``
- - Timestamp for the V4L2 capture buffer to use as forward reference, used
-with B-coded frames. The timestamp refers to the ``timestamp`` field in
- struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
- function to convert the struct :c:type:`timeval` in struct
- :c:type:`v4l2_buffer` to a __u64.
* - __u32
- ``quantiser_scale_code``
- Code used to determine the quantization scale to use for the IDCT.
@@ -1712,6 +1698,20 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
+* - __u64
+ - ``backward_ref_ts``
+ - Timestamp of the V4L2 capture buffer to use as backward reference, used
+with B-coded and P-coded frames. The timestamp refers to the
+ ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the
+ :c:func:`v4l2_timeval_to_ns()` function to convert the struct
+ :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``forward_ref_ts``
+ - Timestamp for the V4L2 capture buffer to use as forward reference, used
+with B-coded frames. The timestamp refers to the ``timestamp`` field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
* - __u8
- ``picture_coding_type``
- Picture coding type for the frame covered by the current slice
diff --git a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
index 6f48dd3870d8..627451df43d3 100644
--- a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c
@@ -104,12 +104,10 @@ hantro_g1_mpeg2_dec_set_buffers(struct hantro_dev *vpu,
struct hantro_ctx *ctx,
switch (pic->picture_coding_type) {
case V4L2_MPEG2_PIC_CODING_TYPE_B:
- backward_addr = hantro_get_ref(ctx,
- slice_params->backward_ref_ts);
+ backward_addr = hantro_get_ref(ctx, pic->backward_ref_ts);
fallthrough;
case V4L2_MPEG2_PIC_CODING_TYPE_P:
- forward_addr = hantro_get_ref(ctx,
- slice_params->forward_ref_ts);
+ forward_addr = hantro_get_ref(ctx, pic->forward_ref_ts);
}
/* Source bitstream buffer */
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
index 95a50013b289..e0f0e20d6854 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c
@@ -106,12 +106,10 @@ rk3399_vpu_mpeg2_dec_set_buffers(struct hantro_dev *vpu,
switch (pic->picture_coding_type) {
case V4L2_MPEG2_PIC_CODING_TYPE_B:
- backward_addr = hantro_get_ref(ctx,
- slice_params->backward_ref_ts);
+ backward_addr = hantro_get_ref(ctx, pic->backward_ref_ts);
fallthrough;
case V4L2_MPEG2_PIC_CODING_TYPE_P:
- forward_addr = hantro_get_ref(ctx,
- slice_params->forward_ref_ts);
+ forward_addr = hantro_get_ref(ctx, pic->forward_ref_ts);
}
/* Source bitstream buffer */
diff --git
[PATCH v4 5/9] media: hantro/cedrus: Remove unneeded slice size and slice offset
The MPEG2_SLICE_PARAMS control is designed to refer to a single slice. However, the Hantro and Cedrus drivers operate in per-frame mode, and so does the current Ffmpeg and GStreamer implementations that are tested with these two drivers. In other words, the drivers are expecting all the slices in a picture (with either frame or field structure) to be contained in the OUTPUT buffer, which means the slice size and offset shouldn't be used. Signed-off-by: Ezequiel Garcia --- drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c | 4 ++-- drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c | 4 ++-- drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c | 7 +++ 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c index 627451df43d3..a4e87b7cc9f1 100644 --- a/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_mpeg2_dec.c @@ -206,7 +206,7 @@ void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx) G1_REG_TOPFIELDFIRST_E(pic->flags & V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST); vdpu_write_relaxed(vpu, reg, G1_SWREG(4)); - reg = G1_REG_STRM_START_BIT(slice_params->data_bit_offset) | + reg = G1_REG_STRM_START_BIT(0) | G1_REG_QSCALE_TYPE(pic->flags & V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE) | G1_REG_CON_MV_E(pic->flags & V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV) | G1_REG_INTRA_DC_PREC(pic->intra_dc_precision) | @@ -215,7 +215,7 @@ void hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx) vdpu_write_relaxed(vpu, reg, G1_SWREG(5)); reg = G1_REG_INIT_QP(1) | - G1_REG_STREAM_LEN(slice_params->bit_size >> 3); + G1_REG_STREAM_LEN(vb2_get_plane_payload(_buf->vb2_buf, 0)); vdpu_write_relaxed(vpu, reg, G1_SWREG(6)); reg = G1_REG_ALT_SCAN_FLAG_E(pic->flags & V4L2_MPEG2_PIC_FLAG_ALT_SCAN) | diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c index e0f0e20d6854..0c2207a1d8c0 100644 --- a/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c +++ b/drivers/staging/media/hantro/rk3399_vpu_hw_mpeg2_dec.c @@ -179,7 +179,7 @@ void rk3399_vpu_mpeg2_dec_run(struct hantro_ctx *ctx) vdpu_write_relaxed(vpu, reg, VDPU_SWREG(50)); reg = VDPU_REG_INIT_QP(1) | - VDPU_REG_STREAM_LEN(slice_params->bit_size >> 3); + VDPU_REG_STREAM_LEN(vb2_get_plane_payload(_buf->vb2_buf, 0)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(51)); reg = VDPU_REG_APF_THRESHOLD(8) | @@ -222,7 +222,7 @@ void rk3399_vpu_mpeg2_dec_run(struct hantro_ctx *ctx) VDPU_REG_TOPFIELDFIRST_E(pic->flags & V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(120)); - reg = VDPU_REG_STRM_START_BIT(slice_params->data_bit_offset) | + reg = VDPU_REG_STRM_START_BIT(0) | VDPU_REG_QSCALE_TYPE(pic->flags & V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE) | VDPU_REG_CON_MV_E(pic->flags & V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV) | VDPU_REG_INTRA_DC_PREC(pic->intra_dc_precision) | diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c index 0332eba390cb..b256f42f1b61 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c @@ -152,10 +152,9 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) /* Source offset and length in bits. */ - cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, -slice_params->data_bit_offset); + cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, 0); - reg = slice_params->bit_size - slice_params->data_bit_offset; + reg = vb2_get_plane_payload(>src->vb2_buf, 0) * 8; cedrus_write(dev, VE_DEC_MPEG_VLD_LEN, reg); /* Source beginning and end addresses. */ @@ -169,7 +168,7 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) cedrus_write(dev, VE_DEC_MPEG_VLD_ADDR, reg); - reg = src_buf_addr + DIV_ROUND_UP(slice_params->bit_size, 8); + reg = src_buf_addr + vb2_get_plane_payload(>src->vb2_buf, 0); cedrus_write(dev, VE_DEC_MPEG_VLD_END_ADDR, reg); /* Macroblock address: start at the beginning. */ -- 2.30.0
[PATCH v4 3/9] media: uapi: mpeg2: Split sequence and picture parameters
Typically, bitstreams are composed of a sequence header,
followed by a number of picture header and picture coding extension
headers. Each picture can be composed by a number of slices.
Let's split the MPEG-2 uAPI to follow these semantics more closely,
allowing more usage flexibility. Having these controls split
allows applications to set a sequence control at the beginning
of a sequence, and then set a picture control for each frame.
While here add padding fields where needed, and document
the uAPI header thoroughly.
Note that the V4L2_CTRL_TYPE_{} defines had to be moved because
it clashes with existing ones. This is not really an issue
since they will be re-defined when the controls are moved
out of staging.
Signed-off-by: Ezequiel Garcia
Tested-by: Jonas Karlman
---
.../media/v4l/ext-ctrls-codec.rst | 47 ++---
.../media/v4l/pixfmt-compressed.rst | 5 +-
.../media/v4l/vidioc-queryctrl.rst| 12 +++
.../media/videodev2.h.rst.exceptions | 2 +
drivers/media/v4l2-core/v4l2-ctrls.c | 57 ---
drivers/staging/media/hantro/hantro_drv.c | 10 ++
.../media/hantro/hantro_g1_mpeg2_dec.c| 14 +--
.../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 14 +--
drivers/staging/media/sunxi/cedrus/cedrus.c | 12 +++
drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +
.../staging/media/sunxi/cedrus/cedrus_dec.c | 4 +
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 8 +-
include/media/mpeg2-ctrls.h | 97 +++
include/media/v4l2-ctrls.h| 4 +
14 files changed, 228 insertions(+), 60 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 2289e1279fa5..35d8845d396c 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1603,14 +1603,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u32
- ``data_bit_offset``
- Offset (in bits) to the video data in the current slice data.
-* - struct :c:type:`v4l2_mpeg2_sequence`
- - ``sequence``
- - Structure with MPEG-2 sequence metadata, merging relevant fields from
- the sequence header and sequence extension parts of the bitstream.
-* - struct :c:type:`v4l2_mpeg2_picture`
- - ``picture``
- - Structure with MPEG-2 picture metadata, merging relevant fields from
- the picture header and picture coding extension parts of the bitstream.
* - __u64
- ``backward_ref_ts``
- Timestamp of the V4L2 capture buffer to use as backward reference, used
@@ -1628,14 +1620,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u32
- ``quantiser_scale_code``
- Code used to determine the quantization scale to use for the IDCT.
+* - __u8
+ - ``reserved``
+ - Applications and drivers must set this to zero.
-.. c:type:: v4l2_mpeg2_sequence
+``V4L2_CID_MPEG_VIDEO_MPEG2_SEQUENCE (struct)``
+Specifies the sequence parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the sequence header and sequence extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. note::
+
+ This compound control is not yet part of the public kernel API and
+ it is expected to change.
+
+.. c:type:: v4l2_ctrl_mpeg2_sequence
.. cssclass:: longtable
.. tabularcolumns:: |p{1.4cm}|p{6.5cm}|p{9.4cm}|
-.. flat-table:: struct v4l2_mpeg2_sequence
+.. flat-table:: struct v4l2_ctrl_mpeg2_sequence
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
@@ -1657,6 +1663,9 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - __u8
- ``chroma_format``
- The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4).
+* - __u8
+ - ``reserved``
+ - Applications and drivers must set this to zero.
* - __u32
- ``flags``
- See :ref:`MPEG-2 Sequence Flags `.
@@ -1677,7 +1686,18 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- Indication that all the frames for the sequence are progressive instead
of interlaced.
-.. c:type:: v4l2_mpeg2_picture
+``V4L2_CID_MPEG_VIDEO_MPEG2_PICTURE (struct)``
+Specifies the picture parameters (as extracted from the bitstream) for the
+associated MPEG-2 slice data. This includes fields matching the syntax
+elements from the picture header and picture coding extension parts of the
+bitstream as specified by :ref:`mpeg2part2`.
+
+.. note::
+
+ This compound control is not yet part of the public kernel API and
+ it is expected to change.
+
+.. c:type:: v4l2_ctrl_mpeg2_picture
.. raw:: latex
@@ -1687,7 +1707,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. tabularcolumns:: |p{1.0cm}|p{5.6cm}|p
[PATCH v4 2/9] media: uapi: mpeg2: Cleanup flags
Our current MPEG-2 uAPI uses 1-byte fields for MPEG-2 boolean syntax elements. Clean these by adding a 'flags' field and flag macro for each boolean syntax element. A follow-up change will refactor this uAPI so we don't need to add padding fields just yet. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- .../media/v4l/ext-ctrls-codec.rst | 77 +-- drivers/media/v4l2-core/v4l2-ctrls.c | 14 ++-- .../media/hantro/hantro_g1_mpeg2_dec.c| 76 +- .../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 76 +- .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 38 - include/media/mpeg2-ctrls.h | 36 + 6 files changed, 175 insertions(+), 142 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index 29d017654141..2289e1279fa5 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1654,13 +1654,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``profile_and_level_indication`` - The current profile and level indication as extracted from the bitstream. -* - __u8 - - ``progressive_sequence`` - - Indication that all the frames for the sequence are progressive instead - of interlaced. * - __u8 - ``chroma_format`` - The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4). +* - __u32 + - ``flags`` + - See :ref:`MPEG-2 Sequence Flags `. + +.. _mpeg2_sequence_flags: + +``MPEG-2 Sequence Flags`` + +.. cssclass:: longtable + +.. flat-table:: +:header-rows: 0 +:stub-columns: 0 +:widths: 1 1 2 + +* - ``V4L2_MPEG2_SEQ_FLAG_PROGRESSIVE`` + - 0x0001 + - Indication that all the frames for the sequence are progressive instead + of interlaced. .. c:type:: v4l2_mpeg2_picture @@ -1693,29 +1708,45 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``picture_structure`` - Picture structure (1: interlaced top field, 2: interlaced bottom field, 3: progressive frame). -* - __u8 - - ``top_field_first`` - - If set to 1 and interlaced stream, top field is output first. -* - __u8 - - ``frame_pred_frame_dct`` - - If set to 1, only frame-DCT and frame prediction are used. -* - __u8 - - ``concealment_motion_vectors`` - - If set to 1, motion vectors are coded for intra macroblocks. -* - __u8 - - ``q_scale_type`` +* - __u32 + - ``flags`` + - See :ref:`MPEG-2 Picture Flags `. + + +.. _mpeg2_picture_flags: + +``MPEG-2 Picture Flags`` + +.. cssclass:: longtable + +.. flat-table:: +:header-rows: 0 +:stub-columns: 0 +:widths: 1 1 2 + +* - ``V4L2_MPEG2_PIC_FLAG_TOP_FIELD_FIRST`` + - 0x0001 + - If set and it's an interlaced stream, top field is output first. +* - ``V4L2_MPEG2_PIC_FLAG_FRAME_PRED_DCT`` + - 0x0002 + - If set only frame-DCT and frame prediction are used. +* - ``V4L2_MPEG2_PIC_FLAG_CONCEALMENT_MV`` + - 0x0004 + - If set motion vectors are coded for intra macroblocks. +* - ``V4L2_MPEG2_PIC_FLAG_Q_SCALE_TYPE`` + - 0x0008 - This flag affects the inverse quantization process. -* - __u8 - - ``intra_vlc_format`` +* - ``V4L2_MPEG2_PIC_FLAG_INTRA_VLC`` + - 0x0010 - This flag affects the decoding of transform coefficient data. -* - __u8 - - ``alternate_scan`` +* - ``V4L2_MPEG2_PIC_FLAG_ALT_SCAN`` + - 0x0020 - This flag affects the decoding of transform coefficient data. -* - __u8 - - ``repeat_first_field`` +* - ``V4L2_MPEG2_PIC_FLAG_REPEAT_FIRST`` + - 0x0040 - This flag affects the decoding process of progressive frames. -* - __u16 - - ``progressive_frame`` +* - ``V4L2_MPEG2_PIC_FLAG_PROGRESSIVE`` + - 0x0080 - Indicates whether the current frame is progressive. .. raw:: latex diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index 6da4f994e1fc..a265574eea5c 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -1691,7 +1691,7 @@ static void std_init_compound(const struct v4l2_ctrl *ctrl, u32 idx, /* interlaced top field */ p_mpeg2_slice_params->picture.picture_structure = 1; p_mpeg2_slice_params->picture.picture_coding_type = - V4L2_MPEG2_PICTURE_CODING_TYPE_I; + V4L2_MPEG2_PIC_CODING_TYPE_I; break; case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: p_mpeg2_quant = p; @@ -1901,18 +1901,18 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u
[PATCH v4 1/9] media: uapi: mpeg2: Rework quantization matrices semantics
As stated in the MPEG-2 specification, section 6.3.7 "Quant matrix
extension":
Each quantisation matrix has a default set of values. When a
sequence_header_code is decoded all matrices shall be reset to
their default values. User defined matrices may be downloaded
and this can occur in a sequence_header() or in a
quant_matrix_extension().
The load_intra_quantiser_matrix syntax elements are transmitted
in the bistream headers, signalling that a quantization matrix
needs to be loaded and used for pictures transmitted afterwards
(until the matrices are reset).
These "load" semantics are implemented in the V4L2 interface
without the need of any "load" flags: passing the control
is effectively a load.
Therefore, rework the V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION
semantics to match the MPEG-2 semantics. Quantization matrices
values are now initialized by the V4L2 control core to their
reset default value, and applications are expected to reset
their values as specified.
The quantization control is therefore optional, and used to
load bitstream-defined values in the quantization matrices.
Signed-off-by: Ezequiel Garcia
Co-developed-by: Nicolas Dufresne
Signed-off-by: Nicolas Dufresne
---
.../media/v4l/ext-ctrls-codec.rst | 17 -
drivers/media/v4l2-core/v4l2-ctrls.c | 21 ++
.../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 38 +--
include/media/mpeg2-ctrls.h | 5 ---
4 files changed, 23 insertions(+), 58 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 188aef8e40d0..29d017654141 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1746,23 +1746,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
-* - __u8
- - ``load_intra_quantiser_matrix``
- - One bit to indicate whether to load the ``intra_quantiser_matrix``
data.
-* - __u8
- - ``load_non_intra_quantiser_matrix``
- - One bit to indicate whether to load the ``non_intra_quantiser_matrix``
- data.
-* - __u8
- - ``load_chroma_intra_quantiser_matrix``
- - One bit to indicate whether to load the
- ``chroma_intra_quantiser_matrix`` data, only relevant for non-4:2:0 YUV
- formats.
-* - __u8
- - ``load_chroma_non_intra_quantiser_matrix``
- - One bit to indicate whether to load the
- ``chroma_non_intra_quantiser_matrix`` data, only relevant for non-4:2:0
- YUV formats.
* - __u8
- ``intra_quantiser_matrix[64]``
- The quantization matrix coefficients for intra-coded frames, in zigzag
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 39038c6ad8fb..6da4f994e1fc 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -57,6 +57,18 @@ static bool is_new_manual(const struct v4l2_ctrl *master)
return master->is_auto && master->val == master->manual_mode_value;
}
+/* Default MPEG-2 quantization coefficients, from the specification. */
+static const u8 mpeg2_intra_quant_matrix[64] = {
+ 8, 16, 16, 19, 16, 19, 22, 22,
+ 22, 22, 22, 22, 26, 24, 26, 27,
+ 27, 27, 26, 26, 26, 26, 27, 27,
+ 27, 29, 29, 29, 34, 34, 34, 29,
+ 29, 29, 27, 27, 29, 29, 32, 32,
+ 34, 34, 37, 38, 37, 35, 35, 34,
+ 35, 38, 38, 40, 40, 40, 48, 48,
+ 46, 46, 56, 56, 58, 69, 69, 83
+};
+
/* Returns NULL or a character pointer array containing the menu for
the given control ID. The pointer array ends with a NULL pointer.
An empty string signifies a menu entry that is invalid. This allows
@@ -1656,6 +1668,7 @@ static void std_init_compound(const struct v4l2_ctrl
*ctrl, u32 idx,
union v4l2_ctrl_ptr ptr)
{
struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params;
+ struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quant;
struct v4l2_ctrl_vp8_frame *p_vp8_frame;
struct v4l2_ctrl_fwht_params *p_fwht_params;
void *p = ptr.p + idx * ctrl->elem_size;
@@ -1680,6 +1693,14 @@ static void std_init_compound(const struct v4l2_ctrl
*ctrl, u32 idx,
p_mpeg2_slice_params->picture.picture_coding_type =
V4L2_MPEG2_PICTURE_CODING_TYPE_I;
break;
+ case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION:
+ p_mpeg2_quant = p;
+
+ memcpy(p_mpeg2_quant->intra_quantiser_matrix,
+ mpeg2_intra_quant_matrix,
+ ARRAY_SIZE(mpeg2_intra_quant_matrix));
+ memset(p_mpeg2_quant->non_intra_quantiser_matrix, 16, 64);
+ break;
case V4L2_CTRL_TYPE_VP8_FRAME:
p_vp
[PATCH v4 0/9] MPEG-2 stateless API cleanup and destaging
Hi everyone, Over the last few weeks Daniel Almeida and Nicolas Dufresne have been working on GStreamer v4l2codecs support for stateless MPEG-2 decoding. This allowed us to re-review the MPEG-2 specification and re-discuss the API with some additional insight. The API now looks quite cleaner, and hopefully ready for destaging. This series is mostly thanks to Daniel and Nicolas! Patches 1 to 6 cleanup the API, and patches 7 to 9 move it out of staging. This is tested on i.MX8M and RK3399 platforms, using GStreamer v4l2codecs, which will be upstream very soon. v4: * Rework and clarify quantization matrices control semantics. * Move reference buffer fields to the picture parameter control. * Remove slice parameters control. This can be added back in the future if needed, but for now it's not used. See patch 6/9 for details. * Destage the API. v3: * No API changes, just minor boilerplate fixes for the new controls to be properly exposed, initialized and validated. v2: * Fixed bad use of boolean negation in a flag, which was fortunately reported by 0day bot. Ezequiel Garcia (9): media: uapi: mpeg2: Rework quantization matrices semantics media: uapi: mpeg2: Cleanup flags media: uapi: mpeg2: Split sequence and picture parameters media: uapi: mpeg2: Move reference buffer fields media: hantro/cedrus: Remove unneeded slice size and slice offset media: uapi: mpeg2: Remove V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS media: controls: Log MPEG-2 stateless control in .std_log media: uapi: Move the MPEG-2 stateless control type out of staging media: uapi: Move MPEG-2 stateless controls out of staging .../media/v4l/ext-ctrls-codec-stateless.rst | 218 ++ .../media/v4l/ext-ctrls-codec.rst | 217 - .../media/v4l/pixfmt-compressed.rst | 11 +- .../media/v4l/vidioc-g-ext-ctrls.rst | 12 + .../media/v4l/vidioc-queryctrl.rst| 18 +- .../media/videodev2.h.rst.exceptions | 3 +- drivers/media/v4l2-core/v4l2-ctrls.c | 110 ++--- drivers/staging/media/hantro/hantro_drv.c | 9 +- .../media/hantro/hantro_g1_mpeg2_dec.c| 95 .../media/hantro/rk3399_vpu_hw_mpeg2_dec.c| 94 drivers/staging/media/sunxi/cedrus/cedrus.c | 10 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 3 +- .../staging/media/sunxi/cedrus/cedrus_dec.c | 8 +- .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 89 +++ include/media/mpeg2-ctrls.h | 82 --- include/media/v4l2-ctrls.h| 7 +- include/uapi/linux/v4l2-controls.h| 114 + include/uapi/linux/videodev2.h| 7 + 18 files changed, 586 insertions(+), 521 deletions(-) delete mode 100644 include/media/mpeg2-ctrls.h -- 2.30.0
Re: [PATCH v6 13/13] arm64: dts: imx8mq: Add node to G2 hardware
On Fri, 2021-03-26 at 15:33 +0100, Benjamin Gaignard wrote:
>
> Le 26/03/2021 à 15:24, Philipp Zabel a écrit :
> > On Thu, Mar 18, 2021 at 09:20:46AM +0100, Benjamin Gaignard wrote:
> > > Split VPU node in two: one for G1 and one for G2 since they are
> > > different hardware blocks.
> > > Add syscon for hardware control block.
> > > Remove reg-names property that is useless.
> > > Each VPU node only need one interrupt.
> > >
> > > Signed-off-by: Benjamin Gaignard
> > > ---
> > > version 5:
> > > - use syscon instead of VPU reset
> > >
> > > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 43 ++-
> > > 1 file changed, 34 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> > > b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> > > index 17c449e12c2e..b537d153ebbd 100644
> > > --- a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> > > +++ b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> > > @@ -1329,15 +1329,16 @@ usb3_phy1: usb-phy@382f0040 {
> > > status = "disabled";
> > > };
> > >
> > > - vpu: video-codec@3830 {
> > > + vpu_ctrl: syscon@3832 {
> > > + compatible = "nxp,imx8mq-vpu-ctrl", "syscon";
> > > + reg = <0x3832 0x1>;
> > > + };
> > > +
> > > + vpu_g1: video-codec@3830 {
> > > compatible = "nxp,imx8mq-vpu";
> > > - reg = <0x3830 0x1>,
> > > - <0x3831 0x1>,
> > > - <0x3832 0x1>;
> > > - reg-names = "g1", "g2", "ctrl";
> > > - interrupts = ,
> > > - ;
> > > - interrupt-names = "g1", "g2";
> > > + reg = <0x3830 0x1>;
> > > + interrupts = ;
> > > + interrupt-names = "g1";
> > > clocks = < IMX8MQ_CLK_VPU_G1_ROOT>,
> > > < IMX8MQ_CLK_VPU_G2_ROOT>,
> > > < IMX8MQ_CLK_VPU_DEC_ROOT>;
> > > @@ -1350,9 +1351,33 @@ vpu: video-codec@3830 {
> > > <
> > > IMX8MQ_VPU_PLL_OUT>,
> > > <
> > > IMX8MQ_SYS1_PLL_800M>,
> > > < IMX8MQ_VPU_PLL>;
> > > - assigned-clock-rates = <6>, <6>,
> > > + assigned-clock-rates = <6>, <3>,
> > I'd like to see this mentioned in the commit message.
>
> Yes I would do that.
> The value comes from the datasheet.
>
> >
> > > + <8>, <0>;
> > > + power-domains = <_vpu>;
> > > + nxp,imx8mq-vpu-ctrl = <_ctrl>;
> > > + };
> > > +
> > > + vpu_g2: video-codec@3831 {
> > > + compatible = "nxp,imx8mq-vpu-g2";
> > > + reg = <0x3831 0x1>;
> > > + interrupts = ;
> > > + interrupt-names = "g2";
> > > + clocks = < IMX8MQ_CLK_VPU_G1_ROOT>,
> > > + < IMX8MQ_CLK_VPU_G2_ROOT>,
> > > + < IMX8MQ_CLK_VPU_DEC_ROOT>;
> > > + clock-names = "g1", "g2", "bus";
> > > + assigned-clocks = < IMX8MQ_CLK_VPU_G1>,
> > Can the G1 clock configuration be dropped from the G2 device node and
> > the G2 clock configuration from the G1 device node? It looks weird that
> > these devices configure each other's clocks.
>
> No because if only one device node is enabled we need to configure the both
> clocks anyway.
>
Since this is akward, how about adding a comment here in the dtsi to clarify it?
Thanks,
Ezequiel
Re: [PATCH v4 07/11] media: hantro: Introduce G2/HEVC decoder
On Tue, 2021-03-16 at 21:19 +0100, Benjamin Gaignard wrote:
>
> Le 16/03/2021 à 19:46, Ezequiel Garcia a écrit :
> > Hi Benjamin,
> >
> > The series is looking really good. Some comments below.
> >
> > On Wed, 2021-03-03 at 12:39 +0100, Benjamin Gaignard wrote:
> > > Implement all the logic to get G2 hardware decoding HEVC frames.
> > > It support up level 5.1 HEVC stream.
> > > It doesn't support yet 10 bits formats or scaling feature.
> > >
> > > Add HANTRO HEVC dedicated control to skip some bits at the beginning
> > > of the slice header. That is very specific to this hardware so can't
> > > go into uapi structures. Compute the needed value is complex and require
> > > information from the stream that only the userland knows so let it
> > > provide the correct value to the driver.
> > >
> > > Signed-off-by: Benjamin Gaignard
> > > ---
> > > version 4:
> > > - fix Ezequiel comments
> > > - use dedicated control as an integer
> > > - change hantro_g2_hevc_dec_run prototype to return errors
> > >
> > > version 2:
> > > - squash multiple commits in this one.
> > > - fix the comments done by Ezequiel about dma_alloc_coherent usage
> > > - fix Dan's comments about control copy, reverse the test logic
> > > in tile_buffer_reallocate, rework some goto and return cases.
> > >
> > > drivers/staging/media/hantro/Makefile | 2 +
> > > drivers/staging/media/hantro/hantro.h | 18 +
> > > drivers/staging/media/hantro/hantro_drv.c | 53 ++
> > > .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++
> > > drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++
> > > drivers/staging/media/hantro/hantro_hevc.c | 321 ++
> > > drivers/staging/media/hantro/hantro_hw.h | 49 ++
> > > 7 files changed, 1228 insertions(+)
> > > create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> > > create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> > > create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
> > >
> > > diff --git a/drivers/staging/media/hantro/Makefile
> > > b/drivers/staging/media/hantro/Makefile
> > > index 743ce08eb184..0357f1772267 100644
> > > --- a/drivers/staging/media/hantro/Makefile
> > > +++ b/drivers/staging/media/hantro/Makefile
> > > @@ -9,12 +9,14 @@ hantro-vpu-y += \
> > > hantro_h1_jpeg_enc.o \
> > > hantro_g1_h264_dec.o \
> > > hantro_g1_mpeg2_dec.o \
> > > + hantro_g2_hevc_dec.o \
> > > hantro_g1_vp8_dec.o \
> > > rk3399_vpu_hw_jpeg_enc.o \
> > > rk3399_vpu_hw_mpeg2_dec.o \
> > > rk3399_vpu_hw_vp8_dec.o \
> > > hantro_jpeg.o \
> > > hantro_h264.o \
> > > + hantro_hevc.o \
> > > hantro_mpeg2.o \
> > > hantro_vp8.o
> > >
> > > diff --git a/drivers/staging/media/hantro/hantro.h
> > > b/drivers/staging/media/hantro/hantro.h
> > > index 05876e426419..a9b80b2c9124 100644
> > > --- a/drivers/staging/media/hantro/hantro.h
> > > +++ b/drivers/staging/media/hantro/hantro.h
> > > @@ -225,6 +225,7 @@ struct hantro_dev {
> > > * @jpeg_enc: JPEG-encoding context.
> > > * @mpeg2_dec: MPEG-2-decoding context.
> > > * @vp8_dec: VP8-decoding context.
> > > + * @hevc_dec: HEVC-decoding context.
> > > */
> > > struct hantro_ctx {
> > > struct hantro_dev *dev;
> > > @@ -251,6 +252,7 @@ struct hantro_ctx {
> > > struct hantro_jpeg_enc_hw_ctx jpeg_enc;
> > > struct hantro_mpeg2_dec_hw_ctx mpeg2_dec;
> > > struct hantro_vp8_dec_hw_ctx vp8_dec;
> > > + struct hantro_hevc_dec_hw_ctx hevc_dec;
> > > };
> > > };
> > >
> > > @@ -428,6 +430,22 @@ hantro_get_dec_buf_addr(struct hantro_ctx *ctx,
> > > struct vb2_buffer *vb)
> > > return vb2_dma_contig_plane_dma_addr(vb, 0);
> > > }
> > >
> > > +static inline size_t
> > > +hantro_get_dec_buf_size(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> > > +{
> > > + if (hantro_needs_postproc(ctx, ctx->vpu_ds
Re: [PATCH v4 07/11] media: hantro: Introduce G2/HEVC decoder
Hi Benjamin,
The series is looking really good. Some comments below.
On Wed, 2021-03-03 at 12:39 +0100, Benjamin Gaignard wrote:
> Implement all the logic to get G2 hardware decoding HEVC frames.
> It support up level 5.1 HEVC stream.
> It doesn't support yet 10 bits formats or scaling feature.
>
> Add HANTRO HEVC dedicated control to skip some bits at the beginning
> of the slice header. That is very specific to this hardware so can't
> go into uapi structures. Compute the needed value is complex and require
> information from the stream that only the userland knows so let it
> provide the correct value to the driver.
>
> Signed-off-by: Benjamin Gaignard
> ---
> version 4:
> - fix Ezequiel comments
> - use dedicated control as an integer
> - change hantro_g2_hevc_dec_run prototype to return errors
>
> version 2:
> - squash multiple commits in this one.
> - fix the comments done by Ezequiel about dma_alloc_coherent usage
> - fix Dan's comments about control copy, reverse the test logic
> in tile_buffer_reallocate, rework some goto and return cases.
>
> drivers/staging/media/hantro/Makefile | 2 +
> drivers/staging/media/hantro/hantro.h | 18 +
> drivers/staging/media/hantro/hantro_drv.c | 53 ++
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++
> drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++
> drivers/staging/media/hantro/hantro_hevc.c | 321 ++
> drivers/staging/media/hantro/hantro_hw.h | 49 ++
> 7 files changed, 1228 insertions(+)
> create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
>
> diff --git a/drivers/staging/media/hantro/Makefile
> b/drivers/staging/media/hantro/Makefile
> index 743ce08eb184..0357f1772267 100644
> --- a/drivers/staging/media/hantro/Makefile
> +++ b/drivers/staging/media/hantro/Makefile
> @@ -9,12 +9,14 @@ hantro-vpu-y += \
> hantro_h1_jpeg_enc.o \
> hantro_g1_h264_dec.o \
> hantro_g1_mpeg2_dec.o \
> + hantro_g2_hevc_dec.o \
> hantro_g1_vp8_dec.o \
> rk3399_vpu_hw_jpeg_enc.o \
> rk3399_vpu_hw_mpeg2_dec.o \
> rk3399_vpu_hw_vp8_dec.o \
> hantro_jpeg.o \
> hantro_h264.o \
> + hantro_hevc.o \
> hantro_mpeg2.o \
> hantro_vp8.o
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index 05876e426419..a9b80b2c9124 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -225,6 +225,7 @@ struct hantro_dev {
> * @jpeg_enc: JPEG-encoding context.
> * @mpeg2_dec: MPEG-2-decoding context.
> * @vp8_dec: VP8-decoding context.
> + * @hevc_dec: HEVC-decoding context.
> */
> struct hantro_ctx {
> struct hantro_dev *dev;
> @@ -251,6 +252,7 @@ struct hantro_ctx {
> struct hantro_jpeg_enc_hw_ctx jpeg_enc;
> struct hantro_mpeg2_dec_hw_ctx mpeg2_dec;
> struct hantro_vp8_dec_hw_ctx vp8_dec;
> + struct hantro_hevc_dec_hw_ctx hevc_dec;
> };
> };
>
> @@ -428,6 +430,22 @@ hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct
> vb2_buffer *vb)
> return vb2_dma_contig_plane_dma_addr(vb, 0);
> }
>
> +static inline size_t
> +hantro_get_dec_buf_size(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> +{
> + if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
> + return ctx->postproc.dec_q[vb->index].size;
> + return vb2_plane_size(vb, 0);
> +}
> +
> +static inline void *
> +hantro_get_dec_buf(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> +{
> + if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
> + return ctx->postproc.dec_q[vb->index].cpu;
> + return vb2_plane_vaddr(vb, 0);
> +}
> +
Seems hantro_get_dec_buf_size and hantro_get_dec_buf are not used?
> void hantro_postproc_disable(struct hantro_ctx *ctx);
> void hantro_postproc_enable(struct hantro_ctx *ctx);
> void hantro_postproc_free(struct hantro_ctx *ctx);
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index e3e6df28f470..bc90a52f4d3d 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -30,6 +30,13 @@
>
> #define DRIVER_NAME "hantro-vpu"
>
> +/*
> + * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
> + * the number of data (in bits) to skip in the
> + * slice segment header syntax after 'slice type' token
> + */
I think we need to document this better, so applications can
correctly use the control. From i.MX reference code, it seems
this needs to be used as follows:
If non-IDR, the bits to be skipped go from syntax
Re: [PATCH v3 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding
Hi Alex,
On Mon, 15 Mar 2021 at 08:28, Alexandre Courbot wrote:
>
> Hi Ezequiel,
>
> On Thu, Mar 4, 2021 at 6:47 AM Ezequiel Garcia
> wrote:
> >
> > Hi Alex,
> >
> > Thanks for the patch.
> >
> > On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot
> > wrote:
> > >
> > > From: Yunfei Dong
> > >
> > > Add support for H.264 decoding using the stateless API, as supported by
> > > MT8183. This support takes advantage of the V4L2 H.264 reference list
> > > builders.
> > >
> > > Signed-off-by: Yunfei Dong
> > > [acourbot: refactor, cleanup and split]
> > > Co-developed-by: Alexandre Courbot
> > > Signed-off-by: Alexandre Courbot
> > > ---
> > > drivers/media/platform/Kconfig| 1 +
> > > drivers/media/platform/mtk-vcodec/Makefile| 1 +
> > > .../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++
> > > .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 +
> > > .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 +
> > > 5 files changed, 813 insertions(+)
> > > create mode 100644
> > > drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > >
> > > diff --git a/drivers/media/platform/Kconfig
> > > b/drivers/media/platform/Kconfig
> > > index fd1831e97b22..c27db5643712 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> > > @@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC
> > > select V4L2_MEM2MEM_DEV
> > > select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
> > > select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
> > > + select V4L2_H264
> > > help
> > > Mediatek video codec driver provides HW capability to
> > > encode and decode in a range of video formats on MT8173
> > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile
> > > b/drivers/media/platform/mtk-vcodec/Makefile
> > > index 4ba93d838ab6..ca8e9e7a9c4e 100644
> > > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > > @@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
> > > mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> > > vdec/vdec_vp8_if.o \
> > > vdec/vdec_vp9_if.o \
> > > + vdec/vdec_h264_req_if.o \
> > > mtk_vcodec_dec_drv.o \
> > > vdec_drv_if.o \
> > > vdec_vpu_if.o \
> > > diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > > b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > > new file mode 100644
> > > index ..2fbbfbbcfbec
> > > --- /dev/null
> > > +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > > @@ -0,0 +1,807 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +
> > > +#include "../vdec_drv_if.h"
> > > +#include "../mtk_vcodec_util.h"
> > > +#include "../mtk_vcodec_dec.h"
> > > +#include "../mtk_vcodec_intr.h"
> > > +#include "../vdec_vpu_if.h"
> > > +#include "../vdec_drv_base.h"
> > > +
> > > +#define NAL_NON_IDR_SLICE 0x01
> > > +#define NAL_IDR_SLICE 0x05
> > > +#define NAL_H264_PPS 0x08
> >
> > Not used?
> >
> > > +#define NAL_TYPE(value)((value) & 0x1F)
> > > +
> >
> > I believe you may not need the NAL type.
>
> True, removed this block of defines.
>
> >
> > > +#define BUF_PREDICTION_SZ (64 * 4096)
> > > +#define MB_UNIT_LEN16
> > > +
> > > +/* get used parameters for sps/pps */
> > > +#define GET_MTK_VDEC_FLAG(cond, flag) \
> > > + { dst_param->cond = ((src_param->flags & flag) ? (1) : (0)); }
> > > +#define GET_MTK_VDEC_PARAM(param) \
> > > + { dst_param->param = src_param->param; }
> > > +/* motion vector size (bytes) for every macro block */
> > > +#define HW_MB_STORE_SZ 64
> > > +
> > > +#define H264_MAX_FB_NUM
Re: [PATCH v3 05/15] media: mtk-vcodec: vdec: support stateless API
Hi Alexandre,
On Mon, 15 Mar 2021 at 08:28, Alexandre Courbot wrote:
>
> Hi Ezequiel, thanks for the feedback!
>
> On Thu, Mar 4, 2021 at 6:30 AM Ezequiel Garcia
> wrote:
> >
> > Hello Alex,
> >
> > Thanks for the patch.
> >
> > On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot
> > wrote:
> > >
> > > From: Yunfei Dong
> > >
> > > Support the stateless codec API that will be used by MT8183.
> > >
> > > Signed-off-by: Yunfei Dong
> > > [acourbot: refactor, cleanup and split]
> > > Co-developed-by: Alexandre Courbot
> > > Signed-off-by: Alexandre Courbot
> > > ---
> > > drivers/media/platform/mtk-vcodec/Makefile| 1 +
> > > .../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 ++-
> > > .../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +-
> > > .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 ++
> > > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 3 +
> > > 5 files changed, 503 insertions(+), 3 deletions(-)
> > > create mode 100644
> > > drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
> > >
> > [..]
> >
> > > +
> > > +static const struct mtk_stateless_control mtk_stateless_controls[] = {
> > > + {
> > > + .cfg = {
> > > + .id = V4L2_CID_STATELESS_H264_SPS,
> > > + },
> > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > + .needed_in_request = true,
> >
> > This "needed_in_request" is not really required, as controls
> > are not volatile, and their value is stored per-context (per-fd).
> >
> > It's perfectly valid for an application to pass the SPS control
> > at the beginning of the sequence, and then omit it
> > in further requests.
>
> If I understand how v4l2_ctrl_request_hdl_ctrl_find() works with
> requests, this boolean only checks that the control has been provided
> at least once, and not that it is provided with every request. Without
> it we could send a frame to the firmware without e.g. setting an SPS,
> which would be a problem.
>
As Nicolas points out, in V4L2 controls have an initial value,
so no control can be unset.
> >
> > > + },
> > > + {
> > > + .cfg = {
> > > + .id = V4L2_CID_STATELESS_H264_PPS,
> > > + },
> > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > + .needed_in_request = true,
> > > + },
> > > + {
> > > + .cfg = {
> > > + .id = V4L2_CID_STATELESS_H264_SCALING_MATRIX,
> > > + },
> > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > + .needed_in_request = true,
> > > + },
> > > + {
> > > + .cfg = {
> > > + .id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
> > > + },
> > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > + .needed_in_request = true,
> > > + },
> > > + {
> > > + .cfg = {
> > > + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > > + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> > > + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > > + .menu_skip_mask =
> > > +
> > > BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
> > > +
> > > BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
> > > + },
> > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > + },
> > > + {
> > > + .cfg = {
> > > + .id = V4L2_CID_STATELESS_H264_DECODE_MODE,
> > > + .min =
> > > V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> > > + .def =
> > > V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> > > + .max =
> > > V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> > > + },
> > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > + },
> > > +};
> >
> > Applications also need to know which V4L2_CID_STATELESS_H264_START_CODE
> > the driver supports. From a n
[PATCH v3 0/8] V4L2 stateless VP8 de-staging
After close scrutiny of the VP8 specification, it seems the VP8 stateless API is ready. This series contains a series of clean-ups and improvement: renaming symbols for consistency, documenting things for clarity, and then moves the control to proper V4L2 headers. It must be noted that, unlike parsed H.264, V4L2_PIX_FMT_VP8_FRAME buffers must contain the VP8 frame header. In others words a VP8 parsed frame buffer is: ++-+-++ | tag 3B | extra 7B | header | MB data | DCT size | DCT part 0 | ... | DCT part N | ++-+-++ Hopefully the pixel format documentation is now clear about that. Support in Cedrus and Hantro is available upstream, and Mediatek support is available downstream [1] which uses the API as-is. This was tested with GStreamer v4l2codec [2] element and Fluster [3] conformance test, on a Hantro G1 device. Changelog: v3: * Rebased on latest media/master branch. * Rename "seg" field to "segment" as noted by Alex, (thanks!) v2: * Rename VP8_FRAME_IS_KEY_FRAME [1] https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2614338/21/ [2] https://gitlab.freedesktop.org/ezequielgarcia/gst-plugins-bad/-/tree/vp8-api-update [3] https://github.com/fluendo/fluster Ezequiel Garcia (8): media: uapi: vp8: Remove "header" from symbol names and macros media: uapi: vp8: Rename v4l2_vp8_loopfilter to v4l2_vp8_loop_filter media: uapi: vp8: Add proper kernel-doc documentation media: uapi: Move parsed VP8 pixel format out of staging media: uapi: Move the VP8 stateless control type out of staging media: controls: Log VP8 stateless control in .std_log media: vp8: Rename V4L2 VP8 specific macro to V4L2_VP8_ media: uapi: move VP8 stateless controls out of staging .../media/v4l/ext-ctrls-codec-stateless.rst | 334 + .../media/v4l/ext-ctrls-codec.rst | 339 -- .../media/v4l/pixfmt-compressed.rst | 15 +- .../media/v4l/vidioc-g-ext-ctrls.rst | 4 + .../media/v4l/vidioc-queryctrl.rst| 6 + .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ctrls.c | 39 +- drivers/staging/media/hantro/hantro_drv.c | 2 +- .../staging/media/hantro/hantro_g1_vp8_dec.c | 55 ++- drivers/staging/media/hantro/hantro_hw.h | 2 +- drivers/staging/media/hantro/hantro_vp8.c | 10 +- .../media/hantro/rk3399_vpu_hw_vp8_dec.c | 55 ++- drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +- .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +- .../staging/media/sunxi/cedrus/cedrus_vp8.c | 113 +++--- include/media/v4l2-ctrls.h| 5 +- include/media/vp8-ctrls.h | 114 -- include/uapi/linux/v4l2-controls.h| 195 ++ include/uapi/linux/videodev2.h| 4 + 20 files changed, 693 insertions(+), 606 deletions(-) delete mode 100644 include/media/vp8-ctrls.h -- 2.30.0
[PATCH v3 8/8] media: uapi: move VP8 stateless controls out of staging
Until now, the VP8 V4L2 API was not exported as a public API,
and only defined in a private media header (media/vp8-ctrls.h).
The reason for this was a concern about the API not complete
and ready to support VP8 decoding hardware accelerators.
After reviewing the VP8 specification in detail, and now
that the API is able to support Cedrus and Hantro G1,
we can consider this ready.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec-stateless.rst | 334 +
.../media/v4l/ext-ctrls-codec.rst | 339 --
.../media/v4l/pixfmt-compressed.rst | 10 +-
.../media/v4l/vidioc-g-ext-ctrls.rst | 4 +
.../media/v4l/vidioc-queryctrl.rst| 6 +
.../media/videodev2.h.rst.exceptions | 1 +
drivers/media/v4l2-core/v4l2-ctrls.c | 4 +-
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../staging/media/hantro/hantro_g1_vp8_dec.c | 3 +-
.../media/hantro/rk3399_vpu_hw_vp8_dec.c | 3 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_vp8.c | 3 +-
include/media/v4l2-ctrls.h| 1 -
include/media/vp8-ctrls.h | 210 ---
include/uapi/linux/v4l2-controls.h| 195 ++
include/uapi/linux/videodev2.h| 1 +
17 files changed, 555 insertions(+), 565 deletions(-)
delete mode 100644 include/media/vp8-ctrls.h
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 681f59e0a8f5..6628be22e670 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -908,3 +908,337 @@ FWHT Flags
.. raw:: latex
\normalsize
+
+.. _v4l2-codec-stateless-vp8:
+
+``V4L2_CID_STATELESS_VP8_FRAME (struct)``
+Specifies the frame parameters for the associated VP8 parsed frame data.
+This includes the necessary parameters for
+configuring a stateless hardware decoding pipeline for VP8.
+The bitstream parameters are defined according to :ref:`vp8`.
+
+.. c:type:: v4l2_ctrl_vp8_frame
+
+.. raw:: latex
+
+\small
+
+.. tabularcolumns:: |p{7.0cm}|p{4.6cm}|p{5.7cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_vp8_frame
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - struct :c:type:`v4l2_vp8_segment`
+ - ``segment``
+ - Structure with segment-based adjustments metadata.
+* - struct :c:type:`v4l2_vp8_loop_filter`
+ - ``lf``
+ - Structure with loop filter level adjustments metadata.
+* - struct :c:type:`v4l2_vp8_quantization`
+ - ``quant``
+ - Structure with VP8 dequantization indices metadata.
+* - struct :c:type:`v4l2_vp8_entropy`
+ - ``entropy``
+ - Structure with VP8 entropy coder probabilities metadata.
+* - struct :c:type:`v4l2_vp8_entropy_coder_state`
+ - ``coder_state``
+ - Structure with VP8 entropy coder state.
+* - __u16
+ - ``width``
+ - The width of the frame. Must be set for all frames.
+* - __u16
+ - ``height``
+ - The height of the frame. Must be set for all frames.
+* - __u8
+ - ``horizontal_scale``
+ - Horizontal scaling factor.
+* - __u8
+ - ``vertical_scaling factor``
+ - Vertical scale.
+* - __u8
+ - ``version``
+ - Bitstream version.
+* - __u8
+ - ``prob_skip_false``
+ - Indicates the probability that the macroblock is not skipped.
+* - __u8
+ - ``prob_intra``
+ - Indicates the probability that a macroblock is intra-predicted.
+* - __u8
+ - ``prob_last``
+ - Indicates the probability that the last reference frame is used
+for inter-prediction
+* - __u8
+ - ``prob_gf``
+ - Indicates the probability that the golden reference frame is used
+for inter-prediction
+* - __u8
+ - ``num_dct_parts``
+ - Number of DCT coefficients partitions. Must be one of: 1, 2, 4, or 8.
+* - __u32
+ - ``first_part_size``
+ - Size of the first partition, i.e. the control partition.
+* - __u32
+ - ``first_part_header_bits``
+ - Size in bits of the first partition header portion.
+* - __u32
+ - ``dct_part_sizes[8]``
+ - DCT coefficients sizes.
+* - __u64
+ - ``last_frame_ts``
+ - Timestamp for the V4L2 capture buffer to use as last reference frame,
used
+with inter-coded frames. The timestamp refers to the ``timestamp``
field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``golden_frame_ts``
+ - Timestamp for the V4L2 capture buffer
[PATCH v3 3/8] media: uapi: vp8: Add proper kernel-doc documentation
In preparation for making the interface public,
document all the structures.
Signed-off-by: Ezequiel Garcia
---
include/media/vp8-ctrls.h | 99 +++
1 file changed, 99 insertions(+)
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 3afaf6ea4bc9..ab46d775788b 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -23,6 +23,20 @@
#define V4L2_VP8_SEGMENT_FLAG_UPDATE_FEATURE_DATA 0x04
#define V4L2_VP8_SEGMENT_FLAG_DELTA_VALUE_MODE 0x08
+/**
+ * struct v4l2_vp8_segment - VP8 segment-based adjustments parameters
+ *
+ * @quant_update: update values for the segment quantizer.
+ * @lf_update: update values for the loop filter level.
+ * @segment_probs: branch probabilities of the segment_id decoding tree.
+ * @padding: padding field. Should be zeroed by applications.
+ * @flags: see V4L2_VP8_SEGMENT_FLAG_{}.
+ *
+ * This structure contains segment-based adjustments related parameters.
+ * See the 'update_segmentation()' part of the frame header syntax,
+ * and section '9.3. Segment-Based Adjustments' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_segment {
__s8 quant_update[4];
__s8 lf_update[4];
@@ -34,6 +48,22 @@ struct v4l2_vp8_segment {
#define V4L2_VP8_LF_ADJ_ENABLE 0x01
#define V4L2_VP8_LF_DELTA_UPDATE 0x02
#define V4L2_VP8_LF_FILTER_TYPE_SIMPLE 0x04
+
+/**
+ * struct v4l2_vp8_loop_filter - VP8 loop filter parameters
+ *
+ * @ref_frm_delta: Reference frame signed delta values.
+ * @mb_mode_delta: MB prediction mode signed delta values.
+ * @sharpness_level: matches sharpness_level syntax element.
+ * @level: matches loop_filter_level syntax element.
+ * @padding: padding field. Should be zeroed by applications.
+ * @flags: see V4L2_VP8_LF_FLAG_{}.
+ *
+ * This structure contains loop filter related parameters.
+ * See the 'mb_lf_adjustments()' part of the frame header syntax,
+ * and section '9.4. Loop Filter Type and Levels' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_loop_filter {
__s8 ref_frm_delta[4];
__s8 mb_mode_delta[4];
@@ -43,6 +73,22 @@ struct v4l2_vp8_loop_filter {
__u32 flags;
};
+/**
+ * struct v4l2_vp8_quantization - VP8 quantizattion indices
+ *
+ * @y_ac_qi: luma AC coefficient table index.
+ * @y_dc_delta: luma DC delta vaue.
+ * @y2_dc_delta: y2 block DC delta value.
+ * @y2_ac_delta: y2 block AC delta value.
+ * @uv_dc_delta: chroma DC delta value.
+ * @uv_ac_delta: chroma AC delta value.
+ * @padding: padding field. Should be zeroed by applications.
+
+ * This structure contains the quantization indices present
+ * in 'quant_indices()' part of the frame header syntax.
+ * See section '9.6. Dequantization Indices' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_quantization {
__u8 y_ac_qi;
__s8 y_dc_delta;
@@ -55,6 +101,21 @@ struct v4l2_vp8_quantization {
#define V4L2_VP8_COEFF_PROB_CNT 11
#define V4L2_VP8_MV_PROB_CNT 19
+
+/**
+ * struct v4l2_vp8_entropy - VP8 update probabilities
+ *
+ * @coeff_probs: coefficient probability update values.
+ * @y_mode_probs: luma intra-prediction probabilities.
+ * @uv_mode_probs: chroma intra-prediction probabilities.
+ * @mv_probs: mv decoding probability.
+ * @padding: padding field. Should be zeroed by applications.
+ *
+ * This structure contains the update probabilities present in
+ * 'token_prob_update()' and 'mv_prob_update()' part of the frame header.
+ * See section '17.2. Probability Updates' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_entropy {
__u8 coeff_probs[4][8][3][V4L2_VP8_COEFF_PROB_CNT];
__u8 y_mode_probs[4];
@@ -63,6 +124,17 @@ struct v4l2_vp8_entropy {
__u8 padding[3];
};
+/**
+ * struct v4l2_vp8_entropy_coder_state - VP8 boolean coder state
+ *
+ * @range: coder state value for "Range"
+ * @value: coder state value for "Value"
+ * @bit_count: number of bits left in range "Value".
+ * @padding: padding field. Should be zeroed by applications.
+ *
+ * This structure contains the state for the boolean coder, as
+ * explained in section '7. Boolean Entropy Decoder' of the VP8 specification.
+ */
struct v4l2_vp8_entropy_coder_state {
__u8 range;
__u8 value;
@@ -80,6 +152,33 @@ struct v4l2_vp8_entropy_coder_state {
#define VP8_FRAME_IS_KEY_FRAME(hdr) \
(!!((hdr)->flags & V4L2_VP8_FRAME_FLAG_KEY_FRAME))
+/**
+ * struct v4l2_vp8_frame - VP8 frame parameters
+ *
+ * @seg: segmentation parameters. See _vp8_segment for more details
+ * @lf: loop filter parameters. See _vp8_loop_filter for more details
+ * @quant: quantization parameters. See _vp8_quantization for more details
+ * @probs: probabilities. See _vp9_probabilities for more details
+ * @width: frame width.
+ * @height: frame height.
+ * @horizontal_scale: horizontal scaling factor.
+ * @vertical_scale: vertical scaling fa
[PATCH v3 5/8] media: uapi: Move the VP8 stateless control type out of staging
Move the VP8 stateless control types out of staging,
and re-number it to avoid any confusion.
Signed-off-by: Ezequiel Garcia
---
include/media/vp8-ctrls.h | 1 -
include/uapi/linux/videodev2.h | 2 ++
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 50c92c5ed0eb..409a87f9e609 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -14,7 +14,6 @@
#include
#define V4L2_CID_MPEG_VIDEO_VP8_FRAME (V4L2_CID_CODEC_BASE + 2000)
-#define V4L2_CTRL_TYPE_VP8_FRAME 0x301
#define V4L2_VP8_SEGMENT_FLAG_ENABLED 0x01
#define V4L2_VP8_SEGMENT_FLAG_UPDATE_MAP 0x02
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a1d903c6f9f0..611b75df7f17 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1793,6 +1793,8 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_H264_PRED_WEIGHTS= 0x0205,
V4L2_CTRL_TYPE_FWHT_PARAMS = 0x0220,
+
+ V4L2_CTRL_TYPE_VP8_FRAME= 0x0240,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.30.0
[PATCH v3 7/8] media: vp8: Rename V4L2 VP8 specific macro to V4L2_VP8_
Before we move the VP8 controls to a stable public header,
rename this macro so it's clearly V4L2-specific.
Signed-off-by: Ezequiel Garcia
---
drivers/staging/media/hantro/hantro_g1_vp8_dec.c | 6 +++---
drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c | 6 +++---
drivers/staging/media/sunxi/cedrus/cedrus_vp8.c | 12 ++--
include/media/vp8-ctrls.h| 2 +-
4 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
index cbe1d087f103..7d6270e26297 100644
--- a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
@@ -234,7 +234,7 @@ static void cfg_parts(struct hantro_ctx *ctx,
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *vb2_src;
- u32 first_part_offset = VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
+ u32 first_part_offset = V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
u32 mb_size, mb_offset_bytes, mb_offset_bits, mb_start_bits;
u32 dct_size_part_size, dct_part_offset;
struct hantro_reg reg;
@@ -442,7 +442,7 @@ void hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
return;
/* Reset segment_map buffer in keyframe */
- if (VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
+ if (V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
memset(ctx->vp8_dec.segment_map.cpu, 0,
ctx->vp8_dec.segment_map.size);
@@ -460,7 +460,7 @@ void hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
vdpu_write_relaxed(vpu, reg, G1_REG_CONFIG);
reg = G1_REG_DEC_CTRL0_DEC_MODE(10);
- if (!VP8_FRAME_IS_KEY_FRAME(hdr))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(hdr))
reg |= G1_REG_DEC_CTRL0_PIC_INTER_E;
if (!(hdr->flags & V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF))
reg |= G1_REG_DEC_CTRL0_SKIP_MODE;
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
index 1a0d7fef4723..caa1435008d4 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
@@ -347,7 +347,7 @@ static void cfg_parts(struct hantro_ctx *ctx,
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *vb2_src;
- u32 first_part_offset = VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
+ u32 first_part_offset = V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
u32 mb_size, mb_offset_bytes, mb_offset_bits, mb_start_bits;
u32 dct_size_part_size, dct_part_offset;
dma_addr_t src_dma;
@@ -520,7 +520,7 @@ void rk3399_vpu_vp8_dec_run(struct hantro_ctx *ctx)
return;
/* Reset segment_map buffer in keyframe */
- if (VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
+ if (V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
memset(ctx->vp8_dec.segment_map.cpu, 0,
ctx->vp8_dec.segment_map.size);
@@ -537,7 +537,7 @@ void rk3399_vpu_vp8_dec_run(struct hantro_ctx *ctx)
reg = VDPU_REG_CONFIG_DEC_TIMEOUT_E
| VDPU_REG_CONFIG_DEC_CLK_GATE_E;
- if (!VP8_FRAME_IS_KEY_FRAME(hdr))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(hdr))
reg |= VDPU_REG_DEC_CTRL0_PIC_INTER_E;
vdpu_write_relaxed(vpu, reg, VDPU_REG_EN_FLAGS);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
b/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
index 64149481584c..12b233d28db3 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
@@ -526,7 +526,7 @@ static void cedrus_read_header(struct cedrus_dev *dev,
{
int i, j;
- if (VP8_FRAME_IS_KEY_FRAME(slice)) {
+ if (V4L2_VP8_FRAME_IS_KEY_FRAME(slice)) {
read_bits(dev, 1, VP8_PROB_HALF);
read_bits(dev, 1, VP8_PROB_HALF);
}
@@ -550,12 +550,12 @@ static void cedrus_read_header(struct cedrus_dev *dev,
for (i = 0; i < QUANT_DELTA_COUNT; i++)
get_delta_q(dev);
- if (!VP8_FRAME_IS_KEY_FRAME(slice))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(slice))
process_ref_frame_info(dev);
read_bits(dev, 1, VP8_PROB_HALF);
- if (!VP8_FRAME_IS_KEY_FRAME(slice))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(slice))
read_bits(dev, 1, VP8_PROB_HALF);
cedrus_write(dev, VE_H264_TRIGGER_TYPE,
VE_H264_TRIGGER_TYPE_VP8_UPDATE_COEF);
@@ -565,7 +565,7 @@ static void cedrus_read_header(struct cedrus_dev *dev,
if (read_bits(dev, 1, VP8_PROB_HALF))
read_bits(dev, 8, VP8_PROB_HALF);
- if (!VP8_FRAME_IS_KEY_FRAME(slice)) {
+ if (!V4L2_VP8_FRAM
[PATCH v3 2/8] media: uapi: vp8: Rename v4l2_vp8_loopfilter to v4l2_vp8_loop_filter
Rename to struct v4l2_vp8_loop_filter, which is closer to the
VP8 syntax. This change is only cosmetics, but since this API
is exposed to userspace, it's worth the trouble to get it consistent.
Signed-off-by: Ezequiel Garcia
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 12 ++--
drivers/staging/media/hantro/hantro_g1_vp8_dec.c | 2 +-
drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c | 2 +-
include/media/vp8-ctrls.h| 4 ++--
4 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index c391ce4683e5..1795f079aca2 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1809,7 +1809,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - struct :c:type:`v4l2_vp8_segment`
- ``segment``
- Structure with segment-based adjustments metadata.
-* - struct :c:type:`v4l2_vp8_loopfilter`
+* - struct :c:type:`v4l2_vp8_loop_filter`
- ``lf``
- Structure with loop filter level adjustments metadata.
* - struct :c:type:`v4l2_vp8_quantization`
@@ -2007,13 +2007,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
\normalsize
-.. c:type:: v4l2_vp8_loopfilter
+.. c:type:: v4l2_vp8_loop_filter
.. cssclass:: longtable
.. tabularcolumns:: |p{1.5cm}|p{3.9cm}|p{11.9cm}|
-.. flat-table:: struct v4l2_vp8_loopfilter
+.. flat-table:: struct v4l2_vp8_loop_filter
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
@@ -2035,11 +2035,11 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- Applications and drivers must set this to zero.
* - __u32
- ``flags``
- - See :ref:`Loopfilter Flags `
+ - See :ref:`Loop Filter Flags `
-.. _vp8_loopfilter_flags:
+.. _vp8_loop_filter_flags:
-``Loopfilter Flags``
+``Loop Filter Flags``
.. tabularcolumns:: |p{7.0cm}|p{1.2cm}|p{9.1cm}|
diff --git a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
index 07acb4ea5dd0..cbe1d087f103 100644
--- a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
@@ -136,7 +136,7 @@ static void cfg_lf(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr)
{
const struct v4l2_vp8_segment *seg = >segment;
- const struct v4l2_vp8_loopfilter *lf = >lf;
+ const struct v4l2_vp8_loop_filter *lf = >lf;
struct hantro_dev *vpu = ctx->dev;
unsigned int i;
u32 reg;
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
index 4dfd4dddbcac..1a0d7fef4723 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
@@ -277,7 +277,7 @@ static void cfg_lf(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr)
{
const struct v4l2_vp8_segment *seg = >segment;
- const struct v4l2_vp8_loopfilter *lf = >lf;
+ const struct v4l2_vp8_loop_filter *lf = >lf;
struct hantro_dev *vpu = ctx->dev;
unsigned int i;
u32 reg;
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 3f74012e23ff..3afaf6ea4bc9 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -34,7 +34,7 @@ struct v4l2_vp8_segment {
#define V4L2_VP8_LF_ADJ_ENABLE 0x01
#define V4L2_VP8_LF_DELTA_UPDATE 0x02
#define V4L2_VP8_LF_FILTER_TYPE_SIMPLE 0x04
-struct v4l2_vp8_loopfilter {
+struct v4l2_vp8_loop_filter {
__s8 ref_frm_delta[4];
__s8 mb_mode_delta[4];
__u8 sharpness_level;
@@ -82,7 +82,7 @@ struct v4l2_vp8_entropy_coder_state {
struct v4l2_ctrl_vp8_frame {
struct v4l2_vp8_segment segment;
- struct v4l2_vp8_loopfilter lf;
+ struct v4l2_vp8_loop_filter lf;
struct v4l2_vp8_quantization quant;
struct v4l2_vp8_entropy entropy;
struct v4l2_vp8_entropy_coder_state coder_state;
--
2.30.0
[PATCH v3 6/8] media: controls: Log VP8 stateless control in .std_log
Simply print the type of the control.
Signed-off-by: Ezequiel Garcia
---
drivers/media/v4l2-core/v4l2-ctrls.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 49bbc06d2bb3..658621b961d5 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1781,6 +1781,9 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_FWHT_PARAMS:
pr_cont("FWHT_PARAMS");
break;
+ case V4L2_CTRL_TYPE_VP8_FRAME:
+ pr_cont("VP8_FRAME");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
--
2.30.0
[PATCH v3 4/8] media: uapi: Move parsed VP8 pixel format out of staging
Since we are ready to stabilize the VP8 stateless API,
move the parsed VP8 pixel format.
Signed-off-by: Ezequiel Garcia
---
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 5 -
include/media/vp8-ctrls.h | 2 --
include/uapi/linux/videodev2.h | 1 +
3 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 7341c9b4516b..0232d0808519 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -164,11 +164,6 @@ Compressed Formats
of macroblocks to decode a full corresponding frame to the matching
capture buffer.
- .. note::
-
- This format is not yet part of the public kernel API and it
- is expected to change.
-
* .. _V4L2-PIX-FMT-VP9:
- ``V4L2_PIX_FMT_VP9``
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index ab46d775788b..50c92c5ed0eb 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -13,8 +13,6 @@
#include
-#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F')
-
#define V4L2_CID_MPEG_VIDEO_VP8_FRAME (V4L2_CID_CODEC_BASE + 2000)
#define V4L2_CTRL_TYPE_VP8_FRAME 0x301
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..a1d903c6f9f0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -694,6 +694,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_VC1_ANNEX_G v4l2_fourcc('V', 'C', '1', 'G') /* SMPTE 421M
Annex G compliant stream */
#define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M
Annex L compliant stream */
#define V4L2_PIX_FMT_VP8 v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
+#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F') /* VP8 parsed
frame */
#define V4L2_PIX_FMT_VP9 v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
#define V4L2_PIX_FMT_HEVC v4l2_fourcc('H', 'E', 'V', 'C') /* HEVC aka
H.265 */
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh
Hadamard Transform (vicodec) */
--
2.30.0
[PATCH v3 1/8] media: uapi: vp8: Remove "header" from symbol names and macros
It doesn't seem to add any clarity to have a "header" suffix in controls,
struct names and flags.
Since this just makes names too long without any benefit, just drop it.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec.rst | 80 +++
.../media/v4l/pixfmt-compressed.rst | 2 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 36 +++
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../staging/media/hantro/hantro_g1_vp8_dec.c | 48 -
drivers/staging/media/hantro/hantro_hw.h | 2 +-
drivers/staging/media/hantro/hantro_vp8.c | 10 +-
.../media/hantro/rk3399_vpu_hw_vp8_dec.c | 48 -
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_vp8.c | 98 +--
include/media/v4l2-ctrls.h| 4 +-
include/media/vp8-ctrls.h | 48 -
14 files changed, 192 insertions(+), 192 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 76ebf86b9d0b..c391ce4683e5 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1780,7 +1780,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. _v4l2-mpeg-vp8:
-``V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER (struct)``
+``V4L2_CID_MPEG_VIDEO_VP8_FRAME (struct)``
Specifies the frame parameters for the associated VP8 parsed frame data.
This includes the necessary parameters for
configuring a stateless hardware decoding pipeline for VP8.
@@ -1791,7 +1791,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
This compound control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_ctrl_vp8_frame_header
+.. c:type:: v4l2_ctrl_vp8_frame
.. raw:: latex
@@ -1801,22 +1801,22 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. cssclass:: longtable
-.. flat-table:: struct v4l2_ctrl_vp8_frame_header
+.. flat-table:: struct v4l2_ctrl_vp8_frame
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
-* - struct :c:type:`v4l2_vp8_segment_header`
- - ``segment_header``
+* - struct :c:type:`v4l2_vp8_segment`
+ - ``segment``
- Structure with segment-based adjustments metadata.
-* - struct :c:type:`v4l2_vp8_loopfilter_header`
- - ``loopfilter_header``
+* - struct :c:type:`v4l2_vp8_loopfilter`
+ - ``lf``
- Structure with loop filter level adjustments metadata.
-* - struct :c:type:`v4l2_vp8_quantization_header`
- - ``quant_header``
+* - struct :c:type:`v4l2_vp8_quantization`
+ - ``quant``
- Structure with VP8 dequantization indices metadata.
-* - struct :c:type:`v4l2_vp8_entropy_header`
- - ``entropy_header``
+* - struct :c:type:`v4l2_vp8_entropy`
+ - ``entropy``
- Structure with VP8 entropy coder probabilities metadata.
* - struct :c:type:`v4l2_vp8_entropy_coder_state`
- ``coder_state``
@@ -1885,15 +1885,15 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:c:type:`v4l2_buffer` to a __u64.
* - __u64
- ``flags``
- - See :ref:`Frame Header Flags `
+ - See :ref:`Frame Flags `
.. raw:: latex
\normalsize
-.. _vp8_frame_header_flags:
+.. _vp8_frame_flags:
-``Frame Header Flags``
+``Frame Flags``
.. tabularcolumns:: |p{9.8cm}|p{0.8cm}|p{6.7cm}|
@@ -1904,22 +1904,22 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_KEY_FRAME``
+* - ``V4L2_VP8_FRAME_FLAG_KEY_FRAME``
- 0x01
- Indicates if the frame is a key frame.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_EXPERIMENTAL``
+* - ``V4L2_VP8_FRAME_FLAG_EXPERIMENTAL``
- 0x02
- Experimental bitstream.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SHOW_FRAME``
+* - ``V4L2_VP8_FRAME_FLAG_SHOW_FRAME``
- 0x04
- Show frame flag, indicates if the frame is for display.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_MB_NO_SKIP_COEFF``
+* - ``V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF``
- 0x08
- Enable/disable skipping of macroblocks with no non-zero coefficients.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_GOLDEN``
+* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_GOLDEN``
- 0x10
- Sign of motion vectors when the golden frame is referenced.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_ALT``
+* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_ALT``
- 0x20
- Sign of motion vectors when the alt frame is referenced.
@@ -1947,13 +1947,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- ``padding``
- Applications and drivers must set this to zero.
Re: [PATCH v3 05/15] media: mtk-vcodec: vdec: support stateless API
Hello Alex,
Thanks for the patch.
On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot wrote:
>
> From: Yunfei Dong
>
> Support the stateless codec API that will be used by MT8183.
>
> Signed-off-by: Yunfei Dong
> [acourbot: refactor, cleanup and split]
> Co-developed-by: Alexandre Courbot
> Signed-off-by: Alexandre Courbot
> ---
> drivers/media/platform/mtk-vcodec/Makefile| 1 +
> .../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 ++-
> .../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +-
> .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 ++
> .../platform/mtk-vcodec/mtk_vcodec_drv.h | 3 +
> 5 files changed, 503 insertions(+), 3 deletions(-)
> create mode 100644
> drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
>
[..]
> +
> +static const struct mtk_stateless_control mtk_stateless_controls[] = {
> + {
> + .cfg = {
> + .id = V4L2_CID_STATELESS_H264_SPS,
> + },
> + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> + .needed_in_request = true,
This "needed_in_request" is not really required, as controls
are not volatile, and their value is stored per-context (per-fd).
It's perfectly valid for an application to pass the SPS control
at the beginning of the sequence, and then omit it
in further requests.
> + },
> + {
> + .cfg = {
> + .id = V4L2_CID_STATELESS_H264_PPS,
> + },
> + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> + .needed_in_request = true,
> + },
> + {
> + .cfg = {
> + .id = V4L2_CID_STATELESS_H264_SCALING_MATRIX,
> + },
> + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> + .needed_in_request = true,
> + },
> + {
> + .cfg = {
> + .id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
> + },
> + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> + .needed_in_request = true,
> + },
> + {
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> + .menu_skip_mask =
> + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
> + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
> + },
> + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> + },
> + {
> + .cfg = {
> + .id = V4L2_CID_STATELESS_H264_DECODE_MODE,
> + .min = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> + .def = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> + .max = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> + },
> + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> + },
> +};
Applications also need to know which V4L2_CID_STATELESS_H264_START_CODE
the driver supports. From a next patch, this case seems to be
V4L2_STATELESS_H264_START_CODE_ANNEX_B.
> +#define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
> +
> +static const struct mtk_video_fmt mtk_video_formats[] = {
> + {
> + .fourcc = V4L2_PIX_FMT_H264_SLICE,
> + .type = MTK_FMT_DEC,
> + .num_planes = 1,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_MM21,
> + .type = MTK_FMT_FRAME,
> + .num_planes = 2,
> + },
> +};
> +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> +#define DEFAULT_OUT_FMT_IDX0
> +#define DEFAULT_CAP_FMT_IDX1
> +
> +static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
> + {
> + .fourcc = V4L2_PIX_FMT_H264_SLICE,
> + .stepwise = {
> + MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
> + MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16,
> + },
> + },
> +};
> +
> +#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
> +
> +static void mtk_vdec_stateless_set_dst_payload(struct mtk_vcodec_ctx *ctx,
> + struct vdec_fb *fb)
> +{
> + struct mtk_video_dec_buf *vdec_frame_buf =
> + container_of(fb, struct mtk_video_dec_buf, frame_buffer);
> + struct vb2_v4l2_buffer *vb = _frame_buf->m2m_buf.vb;
> + unsigned int cap_y_size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[0];
> +
> + vb2_set_plane_payload(>vb2_buf, 0, cap_y_size);
> + if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2) {
> + unsigned int cap_c_size =
> + ctx->q_data[MTK_Q_DATA_DST].sizeimage[1];
> +
> + vb2_set_plane_payload(>vb2_buf, 1, cap_c_size);
> + }
> +}
> +
> +static struct vdec_fb
Re: [PATCH v3 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding
Hi Alex,
Thanks for the patch.
On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot wrote:
>
> From: Yunfei Dong
>
> Add support for H.264 decoding using the stateless API, as supported by
> MT8183. This support takes advantage of the V4L2 H.264 reference list
> builders.
>
> Signed-off-by: Yunfei Dong
> [acourbot: refactor, cleanup and split]
> Co-developed-by: Alexandre Courbot
> Signed-off-by: Alexandre Courbot
> ---
> drivers/media/platform/Kconfig| 1 +
> drivers/media/platform/mtk-vcodec/Makefile| 1 +
> .../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++
> .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 +
> .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 +
> 5 files changed, 813 insertions(+)
> create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index fd1831e97b22..c27db5643712 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC
> select V4L2_MEM2MEM_DEV
> select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
> select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
> + select V4L2_H264
> help
> Mediatek video codec driver provides HW capability to
> encode and decode in a range of video formats on MT8173
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile
> b/drivers/media/platform/mtk-vcodec/Makefile
> index 4ba93d838ab6..ca8e9e7a9c4e 100644
> --- a/drivers/media/platform/mtk-vcodec/Makefile
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
> mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> vdec/vdec_vp8_if.o \
> vdec/vdec_vp9_if.o \
> + vdec/vdec_h264_req_if.o \
> mtk_vcodec_dec_drv.o \
> vdec_drv_if.o \
> vdec_vpu_if.o \
> diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> new file mode 100644
> index ..2fbbfbbcfbec
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> @@ -0,0 +1,807 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +
> +#include "../vdec_drv_if.h"
> +#include "../mtk_vcodec_util.h"
> +#include "../mtk_vcodec_dec.h"
> +#include "../mtk_vcodec_intr.h"
> +#include "../vdec_vpu_if.h"
> +#include "../vdec_drv_base.h"
> +
> +#define NAL_NON_IDR_SLICE 0x01
> +#define NAL_IDR_SLICE 0x05
> +#define NAL_H264_PPS 0x08
Not used?
> +#define NAL_TYPE(value)((value) & 0x1F)
> +
I believe you may not need the NAL type.
> +#define BUF_PREDICTION_SZ (64 * 4096)
> +#define MB_UNIT_LEN16
> +
> +/* get used parameters for sps/pps */
> +#define GET_MTK_VDEC_FLAG(cond, flag) \
> + { dst_param->cond = ((src_param->flags & flag) ? (1) : (0)); }
> +#define GET_MTK_VDEC_PARAM(param) \
> + { dst_param->param = src_param->param; }
> +/* motion vector size (bytes) for every macro block */
> +#define HW_MB_STORE_SZ 64
> +
> +#define H264_MAX_FB_NUM17
> +#define H264_MAX_MV_NUM32
> +#define HDR_PARSING_BUF_SZ 1024
> +
> +/**
> + * struct mtk_h264_dpb_info - h264 dpb information
> + * @y_dma_addr: Y bitstream physical address
> + * @c_dma_addr: CbCr bitstream physical address
> + * @reference_flag: reference picture flag (short/long term reference
> picture)
> + * @field: field picture flag
> + */
> +struct mtk_h264_dpb_info {
> + dma_addr_t y_dma_addr;
> + dma_addr_t c_dma_addr;
> + int reference_flag;
> + int field;
> +};
> +
> +/**
> + * struct mtk_h264_sps_param - parameters for sps
> + */
> +struct mtk_h264_sps_param {
> + unsigned char chroma_format_idc;
> + unsigned char bit_depth_luma_minus8;
> + unsigned char bit_depth_chroma_minus8;
> + unsigned char log2_max_frame_num_minus4;
> + unsigned char pic_order_cnt_type;
> + unsigned char log2_max_pic_order_cnt_lsb_minus4;
> + unsigned char max_num_ref_frames;
> + unsigned char separate_colour_plane_flag;
> + unsigned short pic_width_in_mbs_minus1;
> + unsigned short pic_height_in_map_units_minus1;
> + unsigned int max_frame_nums;
> + unsigned char qpprime_y_zero_transform_bypass_flag;
> + unsigned char delta_pic_order_always_zero_flag;
> + unsigned char frame_mbs_only_flag;
> + unsigned char mb_adaptive_frame_field_flag;
> + unsigned char direct_8x8_inference_flag;
> + unsigned char reserved[3];
> +};
> +
>
Re: [PATCH v4 03/11] media: hantro: change hantro_codec_ops run prototype to return errors
On Wed, 2021-03-03 at 12:39 +0100, Benjamin Gaignard wrote:
> Change hantro_codec_ops run prototype from 'void' to 'int'.
> This allow to cancel the job if an error occur while configuring
> the hardware.
>
> Signed-off-by: Benjamin Gaignard
> ---
> drivers/staging/media/hantro/hantro_drv.c | 4 +++-
> .../staging/media/hantro/hantro_g1_h264_dec.c | 6 --
> .../media/hantro/hantro_g1_mpeg2_dec.c | 4 +++-
> .../staging/media/hantro/hantro_g1_vp8_dec.c | 6 --
> .../staging/media/hantro/hantro_h1_jpeg_enc.c | 4 +++-
> drivers/staging/media/hantro/hantro_hw.h | 19 ++-
> .../media/hantro/rk3399_vpu_hw_jpeg_enc.c | 4 +++-
> .../media/hantro/rk3399_vpu_hw_mpeg2_dec.c | 4 +++-
> .../media/hantro/rk3399_vpu_hw_vp8_dec.c | 6 --
> 9 files changed, 37 insertions(+), 20 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index e5f200e64993..ac1429f00b33 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -161,7 +161,9 @@ static void device_run(void *priv)
>
> v4l2_m2m_buf_copy_metadata(src, dst, true);
>
> - ctx->codec_ops->run(ctx);
> + if (ctx->codec_ops->run(ctx))
> + goto err_cancel_job;
> +
> return;
>
> err_cancel_job:
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index 845bef73d218..fcd4db13c9fe 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -273,13 +273,13 @@ static void set_buffers(struct hantro_ctx *ctx)
> vdpu_write_relaxed(vpu, ctx->h264_dec.priv.dma, G1_REG_ADDR_QTABLE);
> }
>
> -void hantro_g1_h264_dec_run(struct hantro_ctx *ctx)
> +int hantro_g1_h264_dec_run(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
>
> /* Prepare the H264 decoder context. */
> if (hantro_h264_dec_prepare_run(ctx))
> - return;
> + return -EINVAL;
This should be returning the value from hantro_h264_dec_prepare_run.
Thanks!
Ezequiel
Re: [PATCH v4 05/11] media: hantro: Add a field to distinguish the hardware versions
On Wed, 2021-03-03 at 12:39 +0100, Benjamin Gaignard wrote:
> Decoders hardware blocks could exist in multiple versions: add
> a field to distinguish them at runtime.
> G2 hardware block doesn't have postprocessor hantro_needs_postproc
> function should always returns false in for this hardware.
> hantro_needs_postproc function becoming to much complex to
> stay inline in .h file move it to .c file.
>
Note that I already questioned this patch before:
https://lkml.org/lkml/2021/2/17/722
I think it's better to rely on of_device_id.data for this
type of thing.
In particular, I was expecting that just using
hantro_variant.postproc_regs would be enough.
Can you try if that works and avoid reading swreg(0)
and probing the hardware core?
Thanks!
Ezequiel
> Keep the default behavoir to be G1 hardware.
>
> Signed-off-by: Benjamin Gaignard
> ---
> drivers/staging/media/hantro/hantro.h | 13 +++--
> drivers/staging/media/hantro/hantro_drv.c | 2 ++
> drivers/staging/media/hantro/hantro_postproc.c | 17 +
> 3 files changed, 26 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index a76a0d79db9f..05876e426419 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -37,6 +37,9 @@ struct hantro_codec_ops;
> #define HANTRO_HEVC_DECODERBIT(19)
> #define HANTRO_DECODERS0x
>
> +#define HANTRO_G1_REV 0x6731
> +#define HANTRO_G2_REV 0x6732
> +
> /**
> * struct hantro_irq - irq handler and name
> *
> @@ -171,6 +174,7 @@ hantro_vdev_to_func(struct video_device *vdev)
> * @enc_base: Mapped address of VPU encoder register for
> convenience.
> * @dec_base: Mapped address of VPU decoder register for
> convenience.
> * @ctrl_base: Mapped address of VPU control block.
> + * @core_hw_dec_revRuntime detected HW decoder core revision
> * @vpu_mutex: Mutex to synchronize V4L2 calls.
> * @irqlock: Spinlock to synchronize access to data structures
> * shared with interrupt handlers.
> @@ -190,6 +194,7 @@ struct hantro_dev {
> void __iomem *enc_base;
> void __iomem *dec_base;
> void __iomem *ctrl_base;
> + u32 core_hw_dec_rev;
>
> struct mutex vpu_mutex; /* video_device lock */
> spinlock_t irqlock;
> @@ -412,12 +417,8 @@ hantro_get_dst_buf(struct hantro_ctx *ctx)
> return v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
> }
>
> -static inline bool
> -hantro_needs_postproc(const struct hantro_ctx *ctx,
> - const struct hantro_fmt *fmt)
> -{
> - return !ctx->is_encoder && fmt->fourcc != V4L2_PIX_FMT_NV12;
> -}
> +bool hantro_needs_postproc(const struct hantro_ctx *ctx,
> + const struct hantro_fmt *fmt);
>
> static inline dma_addr_t
> hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index f0b68e16fcc0..e3e6df28f470 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -836,6 +836,8 @@ static int hantro_probe(struct platform_device *pdev)
> }
> vpu->enc_base = vpu->reg_bases[0] + vpu->variant->enc_offset;
> vpu->dec_base = vpu->reg_bases[0] + vpu->variant->dec_offset;
> + /* by default decoder is G1 */
> + vpu->core_hw_dec_rev = HANTRO_G1_REV;
>
> ret = dma_set_coherent_mask(vpu->dev, DMA_BIT_MASK(32));
> if (ret) {
> diff --git a/drivers/staging/media/hantro/hantro_postproc.c
> b/drivers/staging/media/hantro/hantro_postproc.c
> index 6d2a8f2a8f0b..050880f720d6 100644
> --- a/drivers/staging/media/hantro/hantro_postproc.c
> +++ b/drivers/staging/media/hantro/hantro_postproc.c
> @@ -50,6 +50,23 @@ const struct hantro_postproc_regs hantro_g1_postproc_regs
> = {
> .display_width = {G1_REG_PP_DISPLAY_WIDTH, 0, 0xfff},
> };
>
> +bool hantro_needs_postproc(const struct hantro_ctx *ctx,
> + const struct hantro_fmt *fmt)
> +{
> + struct hantro_dev *vpu = ctx->dev;
> +
> + if (ctx->is_encoder)
> + return false;
> +
> + if (vpu->core_hw_dec_rev == HANTRO_G1_REV):q
> + return fmt->fourcc != V4L2_PIX_FMT_NV12;
> +
> + if (vpu->core_hw_dec_rev == HANTRO_G2_REV)
> + return false;
> +
> + return false;
> +}
> +
> void hantro_postproc_enable(struct hantro_ctx *ctx)
> {
> struct hantro_dev *vpu = ctx->dev;
Re: [PATCH v4 09/11] media: hantro: IMX8M: add variant for G2/HEVC codec
On Wed, 2021-03-03 at 12:39 +0100, Benjamin Gaignard wrote:
> Add variant to IMX8M to enable G2/HEVC codec.
> Define the capabilities for the hardware up to 3840x2160.
> Retrieve the hardware version at init to distinguish G1 from G2.
>
> Signed-off-by: Benjamin Gaignard
> ---
> version 2:
> - remove useless clocks
>
> drivers/staging/media/hantro/hantro_drv.c | 1 +
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> drivers/staging/media/hantro/imx8m_vpu_hw.c | 95 -
> 3 files changed, 93 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index bc90a52f4d3d..976be7b6ecfb 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -591,6 +591,7 @@ static const struct of_device_id of_hantro_match[] = {
> #endif
> #ifdef CONFIG_VIDEO_HANTRO_IMX8M
> { .compatible = "nxp,imx8mq-vpu", .data = _vpu_variant, },
> + { .compatible = "nxp,imx8mq-vpu-g2", .data = _vpu_g2_variant },
> #endif
> { /* sentinel */ }
> };
> diff --git a/drivers/staging/media/hantro/hantro_hw.h
> b/drivers/staging/media/hantro/hantro_hw.h
> index dade3b0769c1..f61f58da05fe 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -193,6 +193,7 @@ extern const struct hantro_variant rk3399_vpu_variant;
> extern const struct hantro_variant rk3328_vpu_variant;
> extern const struct hantro_variant rk3288_vpu_variant;
> extern const struct hantro_variant imx8mq_vpu_variant;
> +extern const struct hantro_variant imx8mq_vpu_g2_variant;
>
> extern const struct hantro_postproc_regs hantro_g1_postproc_regs;
>
> diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> index d5b4312b9391..46b33531be85 100644
> --- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
> +++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
> @@ -12,6 +12,7 @@
> #include "hantro.h"
> #include "hantro_jpeg.h"
> #include "hantro_g1_regs.h"
> +#include "hantro_g2_regs.h"
>
> static int imx8mq_runtime_resume(struct hantro_dev *vpu)
> {
> @@ -90,6 +91,26 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
> },
> };
>
> +static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
> + {
> + .fourcc = V4L2_PIX_FMT_NV12,
> + .codec_mode = HANTRO_MODE_NONE,
> + },
> + {
> + .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
> + .codec_mode = HANTRO_MODE_HEVC_DEC,
> + .max_depth = 2,
> + .frmsize = {
> + .min_width = 48,
> + .max_width = 3840,
> + .step_width = MB_DIM,
> + .min_height = 48,
> + .max_height = 2160,
> + .step_height = MB_DIM,
> + },
> + },
> +};
> +
> static irqreturn_t imx8m_vpu_g1_irq(int irq, void *dev_id)
> {
> struct hantro_dev *vpu = dev_id;
> @@ -108,9 +129,42 @@ static irqreturn_t imx8m_vpu_g1_irq(int irq, void
> *dev_id)
> return IRQ_HANDLED;
> }
>
> +static irqreturn_t imx8m_vpu_g2_irq(int irq, void *dev_id)
> +{
> + struct hantro_dev *vpu = dev_id;
> + enum vb2_buffer_state state;
> + u32 status;
> +
> + status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
> + state = (status & HEVC_REG_INTERRUPT_DEC_RDY_INT) ?
> + VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
> +
> + vdpu_write(vpu, 0, HEVC_REG_INTERRUPT);
> + vdpu_write(vpu, HEVC_REG_CONFIG_DEC_CLK_GATE_E, HEVC_REG_CONFIG);
Is this clock gate enable needed on each interrupt?
> +
> + hantro_irq_done(vpu, state);
> +
> + return IRQ_HANDLED;
> +}
> +
> static int imx8mq_vpu_hw_init(struct hantro_dev *vpu)
> {
> - vpu->dec_base = vpu->reg_bases[0];
> + int ret;
> +
> + /* Check variant version */
> + ret = clk_bulk_prepare_enable(vpu->variant->num_clocks, vpu->clocks);
> + if (ret) {
> + dev_err(vpu->dev, "Failed to enable clocks\n");
> + return ret;
> + }
> +
> + /* Make that the device has been reset before read it id */
> + ret = device_reset(vpu->dev);
> + if (ret)
> + dev_err(vpu->dev, "Failed to reset Hantro VPU\n");
> +
> + vpu->core_hw_dec_rev = (vdpu_read(vpu, HEVC_REG_VERSION) >> 16) &
> 0x;
> + clk_bulk_disable_unprepare(vpu->variant->num_clocks, vpu->clocks);
>
> return 0;
> }
> @@ -149,17 +203,32 @@ static const struct hantro_codec_ops
> imx8mq_vpu_codec_ops[] = {
> },
> };
>
> +static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
> + [HANTRO_MODE_HEVC_DEC] = {
> + .run = hantro_g2_hevc_dec_run,
> + .reset = imx8mq_vpu_reset,
> + .init = hantro_hevc_dec_init,
> +
[PATCH v2 8/8] media: uapi: move VP8 stateless controls out of staging
Until now, the VP8 V4L2 API was not exported as a public API,
and only defined in a private media header (media/vp8-ctrls.h).
The reason for this was a concern about the API not complete
and ready to support VP8 decoding hardware accelerators.
After reviewing the VP8 specification in detail, and now
that the API is able to support Cedrus and Hantro G1,
we can consider this ready.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec-stateless.rst | 318 +
.../media/v4l/ext-ctrls-codec.rst | 323 --
.../media/v4l/pixfmt-compressed.rst | 10 +-
.../media/v4l/vidioc-g-ext-ctrls.rst | 4 +
.../media/v4l/vidioc-queryctrl.rst| 6 +
.../media/videodev2.h.rst.exceptions | 1 +
drivers/media/v4l2-core/v4l2-ctrls.c | 4 +-
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../staging/media/hantro/hantro_g1_vp8_dec.c | 3 +-
.../media/hantro/rk3399_vpu_hw_vp8_dec.c | 3 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_vp8.c | 3 +-
include/media/v4l2-ctrls.h| 1 -
include/media/vp8-ctrls.h | 210
include/uapi/linux/v4l2-controls.h| 195 +++
include/uapi/linux/videodev2.h| 1 +
17 files changed, 539 insertions(+), 549 deletions(-)
delete mode 100644 include/media/vp8-ctrls.h
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 01e3b1a3fb99..62147b1d341e 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -791,3 +791,321 @@ FWHT Flags
* - ``V4L2_FWHT_FL_PIXENC_HSV``
- 0x0018
- Set if the pixel encoding is HSV.
+
+.. _v4l2-codec-stateless-vp8:
+
+``V4L2_CID_STATELESS_VP8_FRAME (struct)``
+Specifies the frame parameters for the associated VP8 parsed frame data.
+This includes the necessary parameters for
+configuring a stateless hardware decoding pipeline for VP8.
+The bitstream parameters are defined according to :ref:`vp8`.
+
+.. c:type:: v4l2_ctrl_vp8_frame
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
+
+.. flat-table:: struct v4l2_ctrl_vp8_frame
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - struct :c:type:`v4l2_vp8_segment`
+ - ``seg``
+ - Structure with segment-based adjustments metadata.
+* - struct :c:type:`v4l2_vp8_loop_filter`
+ - ``lf``
+ - Structure with loop filter level adjustments metadata.
+* - struct :c:type:`v4l2_vp8_quantization`
+ - ``quant``
+ - Structure with VP8 dequantization indices metadata.
+* - struct :c:type:`v4l2_vp8_entropy`
+ - ``entropy``
+ - Structure with VP8 entropy coder probabilities metadata.
+* - struct :c:type:`v4l2_vp8_entropy_coder_state`
+ - ``coder_state``
+ - Structure with VP8 entropy coder state.
+* - __u16
+ - ``width``
+ - The width of the frame. Must be set for all frames.
+* - __u16
+ - ``height``
+ - The height of the frame. Must be set for all frames.
+* - __u8
+ - ``horizontal_scale``
+ - Horizontal scaling factor.
+* - __u8
+ - ``vertical_scaling factor``
+ - Vertical scale.
+* - __u8
+ - ``version``
+ - Bitstream version.
+* - __u8
+ - ``prob_skip_false``
+ - Indicates the probability that the macroblock is not skipped.
+* - __u8
+ - ``prob_intra``
+ - Indicates the probability that a macroblock is intra-predicted.
+* - __u8
+ - ``prob_last``
+ - Indicates the probability that the last reference frame is used
+for inter-prediction
+* - __u8
+ - ``prob_gf``
+ - Indicates the probability that the golden reference frame is used
+for inter-prediction
+* - __u8
+ - ``num_dct_parts``
+ - Number of DCT coefficients partitions. Must be one of: 1, 2, 4, or 8.
+* - __u32
+ - ``first_part_size``
+ - Size of the first partition, i.e. the control partition.
+* - __u32
+ - ``first_part_header_bits``
+ - Size in bits of the first partition header portion.
+* - __u32
+ - ``dct_part_sizes[8]``
+ - DCT coefficients sizes.
+* - __u64
+ - ``last_frame_ts``
+ - Timestamp for the V4L2 capture buffer to use as last reference frame,
used
+with inter-coded frames. The timestamp refers to the ``timestamp``
field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``golden_frame_ts``
+ - Timestamp
[PATCH v2 7/8] media: vp8: Rename V4L2 VP8 specific macro to V4L2_VP8_
Before we move the VP8 controls to a stable public header,
rename this macro so it's clearly V4L2-specific.
Signed-off-by: Ezequiel Garcia
---
drivers/staging/media/hantro/hantro_g1_vp8_dec.c | 6 +++---
drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c | 6 +++---
drivers/staging/media/sunxi/cedrus/cedrus_vp8.c | 12 ++--
include/media/vp8-ctrls.h| 2 +-
4 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
index 2de80fef07f4..03360fdb9341 100644
--- a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
@@ -234,7 +234,7 @@ static void cfg_parts(struct hantro_ctx *ctx,
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *vb2_src;
- u32 first_part_offset = VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
+ u32 first_part_offset = V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
u32 mb_size, mb_offset_bytes, mb_offset_bits, mb_start_bits;
u32 dct_size_part_size, dct_part_offset;
struct hantro_reg reg;
@@ -442,7 +442,7 @@ void hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
return;
/* Reset segment_map buffer in keyframe */
- if (VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
+ if (V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
memset(ctx->vp8_dec.segment_map.cpu, 0,
ctx->vp8_dec.segment_map.size);
@@ -460,7 +460,7 @@ void hantro_g1_vp8_dec_run(struct hantro_ctx *ctx)
vdpu_write_relaxed(vpu, reg, G1_REG_CONFIG);
reg = G1_REG_DEC_CTRL0_DEC_MODE(10);
- if (!VP8_FRAME_IS_KEY_FRAME(hdr))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(hdr))
reg |= G1_REG_DEC_CTRL0_PIC_INTER_E;
if (!(hdr->flags & V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF))
reg |= G1_REG_DEC_CTRL0_SKIP_MODE;
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
index cbca123db746..bb9f5d3c63c7 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
@@ -347,7 +347,7 @@ static void cfg_parts(struct hantro_ctx *ctx,
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *vb2_src;
- u32 first_part_offset = VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
+ u32 first_part_offset = V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) ? 10 : 3;
u32 mb_size, mb_offset_bytes, mb_offset_bits, mb_start_bits;
u32 dct_size_part_size, dct_part_offset;
dma_addr_t src_dma;
@@ -520,7 +520,7 @@ void rk3399_vpu_vp8_dec_run(struct hantro_ctx *ctx)
return;
/* Reset segment_map buffer in keyframe */
- if (VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
+ if (V4L2_VP8_FRAME_IS_KEY_FRAME(hdr) && ctx->vp8_dec.segment_map.cpu)
memset(ctx->vp8_dec.segment_map.cpu, 0,
ctx->vp8_dec.segment_map.size);
@@ -537,7 +537,7 @@ void rk3399_vpu_vp8_dec_run(struct hantro_ctx *ctx)
reg = VDPU_REG_CONFIG_DEC_TIMEOUT_E
| VDPU_REG_CONFIG_DEC_CLK_GATE_E;
- if (!VP8_FRAME_IS_KEY_FRAME(hdr))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(hdr))
reg |= VDPU_REG_DEC_CTRL0_PIC_INTER_E;
vdpu_write_relaxed(vpu, reg, VDPU_REG_EN_FLAGS);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
b/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
index cab0770515c5..1e6f3086e400 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_vp8.c
@@ -526,7 +526,7 @@ static void cedrus_read_header(struct cedrus_dev *dev,
{
int i, j;
- if (VP8_FRAME_IS_KEY_FRAME(slice)) {
+ if (V4L2_VP8_FRAME_IS_KEY_FRAME(slice)) {
read_bits(dev, 1, VP8_PROB_HALF);
read_bits(dev, 1, VP8_PROB_HALF);
}
@@ -550,12 +550,12 @@ static void cedrus_read_header(struct cedrus_dev *dev,
for (i = 0; i < QUANT_DELTA_COUNT; i++)
get_delta_q(dev);
- if (!VP8_FRAME_IS_KEY_FRAME(slice))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(slice))
process_ref_frame_info(dev);
read_bits(dev, 1, VP8_PROB_HALF);
- if (!VP8_FRAME_IS_KEY_FRAME(slice))
+ if (!V4L2_VP8_FRAME_IS_KEY_FRAME(slice))
read_bits(dev, 1, VP8_PROB_HALF);
cedrus_write(dev, VE_H264_TRIGGER_TYPE,
VE_H264_TRIGGER_TYPE_VP8_UPDATE_COEF);
@@ -565,7 +565,7 @@ static void cedrus_read_header(struct cedrus_dev *dev,
if (read_bits(dev, 1, VP8_PROB_HALF))
read_bits(dev, 8, VP8_PROB_HALF);
- if (!VP8_FRAME_IS_KEY_FRAME(slice)) {
+ if (!V4L2_VP8_FRAM
[PATCH v2 6/8] media: controls: Log VP8 stateless control in .std_log
Simply print the type of the control.
Signed-off-by: Ezequiel Garcia
---
drivers/media/v4l2-core/v4l2-ctrls.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 1a683f4fa978..0fc7180a47e9 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1781,6 +1781,9 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_FWHT_PARAMS:
pr_cont("FWHT_PARAMS");
break;
+ case V4L2_CTRL_TYPE_VP8_FRAME:
+ pr_cont("VP8_FRAME");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
--
2.30.0
[PATCH v2 5/8] media: uapi: Move the VP8 stateless control type out of staging
Move the VP8 stateless control types out of staging,
and re-number it to avoid any confusion.
Signed-off-by: Ezequiel Garcia
---
include/media/vp8-ctrls.h | 1 -
include/uapi/linux/videodev2.h | 2 ++
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 1a5a3d8ab444..051700f33863 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -14,7 +14,6 @@
#include
#define V4L2_CID_MPEG_VIDEO_VP8_FRAME (V4L2_CID_CODEC_BASE + 2000)
-#define V4L2_CTRL_TYPE_VP8_FRAME 0x301
#define V4L2_VP8_SEGMENT_FLAG_ENABLED 0x01
#define V4L2_VP8_SEGMENT_FLAG_UPDATE_MAP 0x02
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a1d903c6f9f0..611b75df7f17 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1793,6 +1793,8 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_H264_PRED_WEIGHTS= 0x0205,
V4L2_CTRL_TYPE_FWHT_PARAMS = 0x0220,
+
+ V4L2_CTRL_TYPE_VP8_FRAME= 0x0240,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.30.0
[PATCH v2 4/8] media: uapi: Move parsed VP8 pixel format out of staging
Since we are ready to stabilize the VP8 stateless API,
move the parsed VP8 pixel format.
Signed-off-by: Ezequiel Garcia
---
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 5 -
include/media/vp8-ctrls.h | 2 --
include/uapi/linux/videodev2.h | 1 +
3 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 2e07e48053d6..e91bc54fa286 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -158,11 +158,6 @@ Compressed Formats
of macroblocks to decode a full corresponding frame to the matching
capture buffer.
- .. note::
-
- This format is not yet part of the public kernel API and it
- is expected to change.
-
* .. _V4L2-PIX-FMT-VP9:
- ``V4L2_PIX_FMT_VP9``
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 9f6cb40088a6..1a5a3d8ab444 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -13,8 +13,6 @@
#include
-#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F')
-
#define V4L2_CID_MPEG_VIDEO_VP8_FRAME (V4L2_CID_CODEC_BASE + 2000)
#define V4L2_CTRL_TYPE_VP8_FRAME 0x301
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..a1d903c6f9f0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -694,6 +694,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_VC1_ANNEX_G v4l2_fourcc('V', 'C', '1', 'G') /* SMPTE 421M
Annex G compliant stream */
#define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M
Annex L compliant stream */
#define V4L2_PIX_FMT_VP8 v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
+#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F') /* VP8 parsed
frame */
#define V4L2_PIX_FMT_VP9 v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
#define V4L2_PIX_FMT_HEVC v4l2_fourcc('H', 'E', 'V', 'C') /* HEVC aka
H.265 */
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh
Hadamard Transform (vicodec) */
--
2.30.0
[PATCH v2 3/8] media: uapi: vp8: Add proper kernel-doc documentation
In preparation for making the interface public,
document all the structures.
Signed-off-by: Ezequiel Garcia
---
include/media/vp8-ctrls.h | 99 +++
1 file changed, 99 insertions(+)
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index f20117e825ef..9f6cb40088a6 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -23,6 +23,20 @@
#define V4L2_VP8_SEGMENT_FLAG_UPDATE_FEATURE_DATA 0x04
#define V4L2_VP8_SEGMENT_FLAG_DELTA_VALUE_MODE 0x08
+/**
+ * struct v4l2_vp8_segment - VP8 segment-based adjustments parameters
+ *
+ * @quant_update: update values for the segment quantizer.
+ * @lf_update: update values for the loop filter level.
+ * @segment_probs: branch probabilities of the segment_id decoding tree.
+ * @padding: padding field. Should be zeroed by applications.
+ * @flags: see V4L2_VP8_SEGMENT_FLAG_{}.
+ *
+ * This structure contains segment-based adjustments related parameters.
+ * See the 'update_segmentation()' part of the frame header syntax,
+ * and section '9.3. Segment-Based Adjustments' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_segment {
__s8 quant_update[4];
__s8 lf_update[4];
@@ -34,6 +48,22 @@ struct v4l2_vp8_segment {
#define V4L2_VP8_LF_ADJ_ENABLE 0x01
#define V4L2_VP8_LF_DELTA_UPDATE 0x02
#define V4L2_VP8_LF_FILTER_TYPE_SIMPLE 0x04
+
+/**
+ * struct v4l2_vp8_loop_filter - VP8 loop filter parameters
+ *
+ * @ref_frm_delta: Reference frame signed delta values.
+ * @mb_mode_delta: MB prediction mode signed delta values.
+ * @sharpness_level: matches sharpness_level syntax element.
+ * @level: matches loop_filter_level syntax element.
+ * @padding: padding field. Should be zeroed by applications.
+ * @flags: see V4L2_VP8_LF_FLAG_{}.
+ *
+ * This structure contains loop filter related parameters.
+ * See the 'mb_lf_adjustments()' part of the frame header syntax,
+ * and section '9.4. Loop Filter Type and Levels' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_loop_filter {
__s8 ref_frm_delta[4];
__s8 mb_mode_delta[4];
@@ -43,6 +73,22 @@ struct v4l2_vp8_loop_filter {
__u32 flags;
};
+/**
+ * struct v4l2_vp8_quantization - VP8 quantizattion indices
+ *
+ * @y_ac_qi: luma AC coefficient table index.
+ * @y_dc_delta: luma DC delta vaue.
+ * @y2_dc_delta: y2 block DC delta value.
+ * @y2_ac_delta: y2 block AC delta value.
+ * @uv_dc_delta: chroma DC delta value.
+ * @uv_ac_delta: chroma AC delta value.
+ * @padding: padding field. Should be zeroed by applications.
+
+ * This structure contains the quantization indices present
+ * in 'quant_indices()' part of the frame header syntax.
+ * See section '9.6. Dequantization Indices' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_quantization {
__u8 y_ac_qi;
__s8 y_dc_delta;
@@ -55,6 +101,21 @@ struct v4l2_vp8_quantization {
#define V4L2_VP8_COEFF_PROB_CNT 11
#define V4L2_VP8_MV_PROB_CNT 19
+
+/**
+ * struct v4l2_vp8_entropy - VP8 update probabilities
+ *
+ * @coeff_probs: coefficient probability update values.
+ * @y_mode_probs: luma intra-prediction probabilities.
+ * @uv_mode_probs: chroma intra-prediction probabilities.
+ * @mv_probs: mv decoding probability.
+ * @padding: padding field. Should be zeroed by applications.
+ *
+ * This structure contains the update probabilities present in
+ * 'token_prob_update()' and 'mv_prob_update()' part of the frame header.
+ * See section '17.2. Probability Updates' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_entropy {
__u8 coeff_probs[4][8][3][V4L2_VP8_COEFF_PROB_CNT];
__u8 y_mode_probs[4];
@@ -63,6 +124,17 @@ struct v4l2_vp8_entropy {
__u8 padding[3];
};
+/**
+ * struct v4l2_vp8_entropy_coder_state - VP8 boolean coder state
+ *
+ * @range: coder state value for "Range"
+ * @value: coder state value for "Value"
+ * @bit_count: number of bits left in range "Value".
+ * @padding: padding field. Should be zeroed by applications.
+ *
+ * This structure contains the state for the boolean coder, as
+ * explained in section '7. Boolean Entropy Decoder' of the VP8 specification.
+ */
struct v4l2_vp8_entropy_coder_state {
__u8 range;
__u8 value;
@@ -80,6 +152,33 @@ struct v4l2_vp8_entropy_coder_state {
#define VP8_FRAME_IS_KEY_FRAME(hdr) \
(!!((hdr)->flags & V4L2_VP8_FRAME_FLAG_KEY_FRAME))
+/**
+ * struct v4l2_vp8_frame - VP8 frame parameters
+ *
+ * @seg: segmentation parameters. See _vp8_segment for more details
+ * @lf: loop filter parameters. See _vp8_loop_filter for more details
+ * @quant: quantization parameters. See _vp8_quantization for more details
+ * @probs: probabilities. See _vp9_probabilities for more details
+ * @width: frame width.
+ * @height: frame height.
+ * @horizontal_scale: horizontal scaling factor.
+ * @vertical_scale: vertical scaling fa
[PATCH v2 1/8] media: uapi: vp8: Remove "header" from symbol names and macros
It doesn't seem to add any clarity to have a "header" suffix in controls,
struct names and flags.
Since this just makes names too long without any benefit, just drop it.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec.rst | 80 +++
.../media/v4l/pixfmt-compressed.rst | 2 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 36 +++
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../staging/media/hantro/hantro_g1_vp8_dec.c | 48 -
drivers/staging/media/hantro/hantro_hw.h | 2 +-
drivers/staging/media/hantro/hantro_vp8.c | 10 +-
.../media/hantro/rk3399_vpu_hw_vp8_dec.c | 48 -
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_vp8.c | 98 +--
include/media/v4l2-ctrls.h| 4 +-
include/media/vp8-ctrls.h | 48 -
14 files changed, 192 insertions(+), 192 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 00944e97d638..006301e88554 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1749,7 +1749,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. _v4l2-mpeg-vp8:
-``V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER (struct)``
+``V4L2_CID_MPEG_VIDEO_VP8_FRAME (struct)``
Specifies the frame parameters for the associated VP8 parsed frame data.
This includes the necessary parameters for
configuring a stateless hardware decoding pipeline for VP8.
@@ -1760,28 +1760,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
This compound control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_ctrl_vp8_frame_header
+.. c:type:: v4l2_ctrl_vp8_frame
.. cssclass:: longtable
.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
-.. flat-table:: struct v4l2_ctrl_vp8_frame_header
+.. flat-table:: struct v4l2_ctrl_vp8_frame
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
-* - struct :c:type:`v4l2_vp8_segment_header`
- - ``segment_header``
+* - struct :c:type:`v4l2_vp8_segment`
+ - ``seg``
- Structure with segment-based adjustments metadata.
-* - struct :c:type:`v4l2_vp8_loopfilter_header`
- - ``loopfilter_header``
+* - struct :c:type:`v4l2_vp8_loopfilter`
+ - ``lf``
- Structure with loop filter level adjustments metadata.
-* - struct :c:type:`v4l2_vp8_quantization_header`
- - ``quant_header``
+* - struct :c:type:`v4l2_vp8_quantization`
+ - ``quant``
- Structure with VP8 dequantization indices metadata.
-* - struct :c:type:`v4l2_vp8_entropy_header`
- - ``entropy_header``
+* - struct :c:type:`v4l2_vp8_entropy`
+ - ``entropy``
- Structure with VP8 entropy coder probabilities metadata.
* - struct :c:type:`v4l2_vp8_entropy_coder_state`
- ``coder_state``
@@ -1850,11 +1850,11 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:c:type:`v4l2_buffer` to a __u64.
* - __u64
- ``flags``
- - See :ref:`Frame Header Flags `
+ - See :ref:`Frame Flags `
-.. _vp8_frame_header_flags:
+.. _vp8_frame_flags:
-``Frame Header Flags``
+``Frame Flags``
.. cssclass:: longtable
@@ -1863,22 +1863,22 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_KEY_FRAME``
+* - ``V4L2_VP8_FRAME_FLAG_KEY_FRAME``
- 0x01
- Indicates if the frame is a key frame.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_EXPERIMENTAL``
+* - ``V4L2_VP8_FRAME_FLAG_EXPERIMENTAL``
- 0x02
- Experimental bitstream.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SHOW_FRAME``
+* - ``V4L2_VP8_FRAME_FLAG_SHOW_FRAME``
- 0x04
- Show frame flag, indicates if the frame is for display.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_MB_NO_SKIP_COEFF``
+* - ``V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF``
- 0x08
- Enable/disable skipping of macroblocks with no non-zero coefficients.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_GOLDEN``
+* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_GOLDEN``
- 0x10
- Sign of motion vectors when the golden frame is referenced.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_ALT``
+* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_ALT``
- 0x20
- Sign of motion vectors when the alt frame is referenced.
@@ -1906,13 +1906,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- ``padding``
- Applications and drivers must set this to zero.
-.. c:type:: v4l2_vp8_segment_header
+.. c:type:: v4l2_vp8_segment
.. cssclass:: longtable
.. tabularcolumns:
[PATCH v2 2/8] media: uapi: vp8: Rename v4l2_vp8_loopfilter to v4l2_vp8_loop_filter
Rename to struct v4l2_vp8_loop_filter, which is closer to the
VP8 syntax. This change is only cosmetics, but since this API
is exposed to userspace, it's worth the trouble to get it consistent.
Signed-off-by: Ezequiel Garcia
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 12 ++--
drivers/staging/media/hantro/hantro_g1_vp8_dec.c | 2 +-
drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c | 2 +-
include/media/vp8-ctrls.h| 4 ++--
4 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 006301e88554..8bbfea44a2ca 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1774,7 +1774,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - struct :c:type:`v4l2_vp8_segment`
- ``seg``
- Structure with segment-based adjustments metadata.
-* - struct :c:type:`v4l2_vp8_loopfilter`
+* - struct :c:type:`v4l2_vp8_loop_filter`
- ``lf``
- Structure with loop filter level adjustments metadata.
* - struct :c:type:`v4l2_vp8_quantization`
@@ -1958,13 +1958,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- If is set, the segment feature data mode is delta-value.
If cleared, it's absolute-value.
-.. c:type:: v4l2_vp8_loopfilter
+.. c:type:: v4l2_vp8_loop_filter
.. cssclass:: longtable
.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}|
-.. flat-table:: struct v4l2_vp8_loopfilter
+.. flat-table:: struct v4l2_vp8_loop_filter
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
@@ -1986,11 +1986,11 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- Applications and drivers must set this to zero.
* - __u32
- ``flags``
- - See :ref:`Loopfilter Flags `
+ - See :ref:`Loop Filter Flags `
-.. _vp8_loopfilter_flags:
+.. _vp8_loop_filter_flags:
-``Loopfilter Flags``
+``Loop Filter Flags``
.. cssclass:: longtable
diff --git a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
index 3f4311c1dab8..2de80fef07f4 100644
--- a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
@@ -136,7 +136,7 @@ static void cfg_lf(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr)
{
const struct v4l2_vp8_segment *seg = >seg;
- const struct v4l2_vp8_loopfilter *lf = >lf;
+ const struct v4l2_vp8_loop_filter *lf = >lf;
struct hantro_dev *vpu = ctx->dev;
unsigned int i;
u32 reg;
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
index 5c43f59ad531..cbca123db746 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
@@ -277,7 +277,7 @@ static void cfg_lf(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr)
{
const struct v4l2_vp8_segment *seg = >seg;
- const struct v4l2_vp8_loopfilter *lf = >lf;
+ const struct v4l2_vp8_loop_filter *lf = >lf;
struct hantro_dev *vpu = ctx->dev;
unsigned int i;
u32 reg;
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 86bb6e774604..f20117e825ef 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -34,7 +34,7 @@ struct v4l2_vp8_segment {
#define V4L2_VP8_LF_ADJ_ENABLE 0x01
#define V4L2_VP8_LF_DELTA_UPDATE 0x02
#define V4L2_VP8_LF_FILTER_TYPE_SIMPLE 0x04
-struct v4l2_vp8_loopfilter {
+struct v4l2_vp8_loop_filter {
__s8 ref_frm_delta[4];
__s8 mb_mode_delta[4];
__u8 sharpness_level;
@@ -82,7 +82,7 @@ struct v4l2_vp8_entropy_coder_state {
struct v4l2_ctrl_vp8_frame {
struct v4l2_vp8_segment seg;
- struct v4l2_vp8_loopfilter lf;
+ struct v4l2_vp8_loop_filter lf;
struct v4l2_vp8_quantization quant;
struct v4l2_vp8_entropy entropy;
struct v4l2_vp8_entropy_coder_state coder_state;
--
2.30.0
[PATCH v2 0/8] V4L2 stateless VP8 de-staging
After close scrutiny of the VP8 specification, it seems the VP8 stateless API is ready. This series contains a series of clean-ups and improvement: renaming symbols for consistency, documenting things for clarity, and then moves the control to proper V4L2 headers. It must be noted that, unlike parsed H.264, V4L2_PIX_FMT_VP8_FRAME buffers must contain the VP8 frame header. In others words a VP8 parsed frame buffer is: ++-+-++ | tag 3B | extra 7B | header | MB data | DCT size | DCT part 0 | ... | DCT part N | ++-+-++ Hopefully the pixel format documentation is now clear about that. Support in Cedrus and Hantro is available upstream, and Mediatek support is available downstream [1] which uses the API as-is. This was tested with GStreamer v4l2codec element and Fluster [2] conformance test, on a Hantro G1 device. Changelog: v2: * Rename VP8_FRAME_IS_KEY_FRAME [1] https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2614338/21/ [2] https://github.com/fluendo/fluster Ezequiel Garcia (8): media: uapi: vp8: Remove "header" from symbol names and macros media: uapi: vp8: Rename v4l2_vp8_loopfilter to v4l2_vp8_loop_filter media: uapi: vp8: Add proper kernel-doc documentation media: uapi: Move parsed VP8 pixel format out of staging media: uapi: Move the VP8 stateless control type out of staging media: controls: Log VP8 stateless control in .std_log media: uapi: Rename V4L2 VP8 specific macro to V4L2_VP8_ media: uapi: move VP8 stateless controls out of staging .../media/v4l/ext-ctrls-codec-stateless.rst | 318 + .../media/v4l/ext-ctrls-codec.rst | 323 -- .../media/v4l/pixfmt-compressed.rst | 15 +- .../media/v4l/vidioc-g-ext-ctrls.rst | 4 + .../media/v4l/vidioc-queryctrl.rst| 6 + .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ctrls.c | 39 ++- drivers/staging/media/hantro/hantro_drv.c | 2 +- .../staging/media/hantro/hantro_g1_vp8_dec.c | 55 ++- drivers/staging/media/hantro/hantro_hw.h | 2 +- drivers/staging/media/hantro/hantro_vp8.c | 10 +- .../media/hantro/rk3399_vpu_hw_vp8_dec.c | 55 ++- drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +- .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +- .../staging/media/sunxi/cedrus/cedrus_vp8.c | 113 +++--- include/media/v4l2-ctrls.h| 5 +- include/media/vp8-ctrls.h | 114 --- include/uapi/linux/v4l2-controls.h| 195 +++ include/uapi/linux/videodev2.h| 4 + 20 files changed, 677 insertions(+), 590 deletions(-) delete mode 100644 include/media/vp8-ctrls.h -- 2.30.0
[PATCH 6/7] media: controls: Log VP8 stateless control in .std_log
Simply print the type of the control.
Signed-off-by: Ezequiel Garcia
---
drivers/media/v4l2-core/v4l2-ctrls.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 1a683f4fa978..0fc7180a47e9 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1781,6 +1781,9 @@ static void std_log(const struct v4l2_ctrl *ctrl)
case V4L2_CTRL_TYPE_FWHT_PARAMS:
pr_cont("FWHT_PARAMS");
break;
+ case V4L2_CTRL_TYPE_VP8_FRAME:
+ pr_cont("VP8_FRAME");
+ break;
default:
pr_cont("unknown type %d", ctrl->type);
break;
--
2.30.0
[PATCH 7/7] media: uapi: move VP8 stateless controls out of staging
Until now, the VP8 V4L2 API was not exported as a public API,
and only defined in a private media header (media/vp8-ctrls.h).
The reason for this was a concern about the API not complete
and ready to support VP8 decoding hardware accelerators.
After reviewing the VP8 specification in detail, and now
that the API is able to support Cedrus and Hantro G1,
we can consider this ready.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec-stateless.rst | 318 +
.../media/v4l/ext-ctrls-codec.rst | 323 --
.../media/v4l/pixfmt-compressed.rst | 10 +-
.../media/v4l/vidioc-g-ext-ctrls.rst | 4 +
.../media/v4l/vidioc-queryctrl.rst| 6 +
.../media/videodev2.h.rst.exceptions | 1 +
drivers/media/v4l2-core/v4l2-ctrls.c | 4 +-
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../staging/media/hantro/hantro_g1_vp8_dec.c | 3 +-
.../media/hantro/rk3399_vpu_hw_vp8_dec.c | 3 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_vp8.c | 3 +-
include/media/v4l2-ctrls.h| 1 -
include/media/vp8-ctrls.h | 210
include/uapi/linux/v4l2-controls.h| 195 +++
include/uapi/linux/videodev2.h| 1 +
17 files changed, 539 insertions(+), 549 deletions(-)
delete mode 100644 include/media/vp8-ctrls.h
diff --git
a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 01e3b1a3fb99..62147b1d341e 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -791,3 +791,321 @@ FWHT Flags
* - ``V4L2_FWHT_FL_PIXENC_HSV``
- 0x0018
- Set if the pixel encoding is HSV.
+
+.. _v4l2-codec-stateless-vp8:
+
+``V4L2_CID_STATELESS_VP8_FRAME (struct)``
+Specifies the frame parameters for the associated VP8 parsed frame data.
+This includes the necessary parameters for
+configuring a stateless hardware decoding pipeline for VP8.
+The bitstream parameters are defined according to :ref:`vp8`.
+
+.. c:type:: v4l2_ctrl_vp8_frame
+
+.. cssclass:: longtable
+
+.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
+
+.. flat-table:: struct v4l2_ctrl_vp8_frame
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - struct :c:type:`v4l2_vp8_segment`
+ - ``seg``
+ - Structure with segment-based adjustments metadata.
+* - struct :c:type:`v4l2_vp8_loop_filter`
+ - ``lf``
+ - Structure with loop filter level adjustments metadata.
+* - struct :c:type:`v4l2_vp8_quantization`
+ - ``quant``
+ - Structure with VP8 dequantization indices metadata.
+* - struct :c:type:`v4l2_vp8_entropy`
+ - ``entropy``
+ - Structure with VP8 entropy coder probabilities metadata.
+* - struct :c:type:`v4l2_vp8_entropy_coder_state`
+ - ``coder_state``
+ - Structure with VP8 entropy coder state.
+* - __u16
+ - ``width``
+ - The width of the frame. Must be set for all frames.
+* - __u16
+ - ``height``
+ - The height of the frame. Must be set for all frames.
+* - __u8
+ - ``horizontal_scale``
+ - Horizontal scaling factor.
+* - __u8
+ - ``vertical_scaling factor``
+ - Vertical scale.
+* - __u8
+ - ``version``
+ - Bitstream version.
+* - __u8
+ - ``prob_skip_false``
+ - Indicates the probability that the macroblock is not skipped.
+* - __u8
+ - ``prob_intra``
+ - Indicates the probability that a macroblock is intra-predicted.
+* - __u8
+ - ``prob_last``
+ - Indicates the probability that the last reference frame is used
+for inter-prediction
+* - __u8
+ - ``prob_gf``
+ - Indicates the probability that the golden reference frame is used
+for inter-prediction
+* - __u8
+ - ``num_dct_parts``
+ - Number of DCT coefficients partitions. Must be one of: 1, 2, 4, or 8.
+* - __u32
+ - ``first_part_size``
+ - Size of the first partition, i.e. the control partition.
+* - __u32
+ - ``first_part_header_bits``
+ - Size in bits of the first partition header portion.
+* - __u32
+ - ``dct_part_sizes[8]``
+ - DCT coefficients sizes.
+* - __u64
+ - ``last_frame_ts``
+ - Timestamp for the V4L2 capture buffer to use as last reference frame,
used
+with inter-coded frames. The timestamp refers to the ``timestamp``
field in
+ struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()`
+ function to convert the struct :c:type:`timeval` in struct
+ :c:type:`v4l2_buffer` to a __u64.
+* - __u64
+ - ``golden_frame_ts``
+ - Timestamp
[PATCH 3/7] media: uapi: vp8: Add proper kernel-doc documentation
In preparation for making the interface public,
document all the structures.
Signed-off-by: Ezequiel Garcia
---
include/media/vp8-ctrls.h | 99 +++
1 file changed, 99 insertions(+)
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index f20117e825ef..9f6cb40088a6 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -23,6 +23,20 @@
#define V4L2_VP8_SEGMENT_FLAG_UPDATE_FEATURE_DATA 0x04
#define V4L2_VP8_SEGMENT_FLAG_DELTA_VALUE_MODE 0x08
+/**
+ * struct v4l2_vp8_segment - VP8 segment-based adjustments parameters
+ *
+ * @quant_update: update values for the segment quantizer.
+ * @lf_update: update values for the loop filter level.
+ * @segment_probs: branch probabilities of the segment_id decoding tree.
+ * @padding: padding field. Should be zeroed by applications.
+ * @flags: see V4L2_VP8_SEGMENT_FLAG_{}.
+ *
+ * This structure contains segment-based adjustments related parameters.
+ * See the 'update_segmentation()' part of the frame header syntax,
+ * and section '9.3. Segment-Based Adjustments' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_segment {
__s8 quant_update[4];
__s8 lf_update[4];
@@ -34,6 +48,22 @@ struct v4l2_vp8_segment {
#define V4L2_VP8_LF_ADJ_ENABLE 0x01
#define V4L2_VP8_LF_DELTA_UPDATE 0x02
#define V4L2_VP8_LF_FILTER_TYPE_SIMPLE 0x04
+
+/**
+ * struct v4l2_vp8_loop_filter - VP8 loop filter parameters
+ *
+ * @ref_frm_delta: Reference frame signed delta values.
+ * @mb_mode_delta: MB prediction mode signed delta values.
+ * @sharpness_level: matches sharpness_level syntax element.
+ * @level: matches loop_filter_level syntax element.
+ * @padding: padding field. Should be zeroed by applications.
+ * @flags: see V4L2_VP8_LF_FLAG_{}.
+ *
+ * This structure contains loop filter related parameters.
+ * See the 'mb_lf_adjustments()' part of the frame header syntax,
+ * and section '9.4. Loop Filter Type and Levels' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_loop_filter {
__s8 ref_frm_delta[4];
__s8 mb_mode_delta[4];
@@ -43,6 +73,22 @@ struct v4l2_vp8_loop_filter {
__u32 flags;
};
+/**
+ * struct v4l2_vp8_quantization - VP8 quantizattion indices
+ *
+ * @y_ac_qi: luma AC coefficient table index.
+ * @y_dc_delta: luma DC delta vaue.
+ * @y2_dc_delta: y2 block DC delta value.
+ * @y2_ac_delta: y2 block AC delta value.
+ * @uv_dc_delta: chroma DC delta value.
+ * @uv_ac_delta: chroma AC delta value.
+ * @padding: padding field. Should be zeroed by applications.
+
+ * This structure contains the quantization indices present
+ * in 'quant_indices()' part of the frame header syntax.
+ * See section '9.6. Dequantization Indices' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_quantization {
__u8 y_ac_qi;
__s8 y_dc_delta;
@@ -55,6 +101,21 @@ struct v4l2_vp8_quantization {
#define V4L2_VP8_COEFF_PROB_CNT 11
#define V4L2_VP8_MV_PROB_CNT 19
+
+/**
+ * struct v4l2_vp8_entropy - VP8 update probabilities
+ *
+ * @coeff_probs: coefficient probability update values.
+ * @y_mode_probs: luma intra-prediction probabilities.
+ * @uv_mode_probs: chroma intra-prediction probabilities.
+ * @mv_probs: mv decoding probability.
+ * @padding: padding field. Should be zeroed by applications.
+ *
+ * This structure contains the update probabilities present in
+ * 'token_prob_update()' and 'mv_prob_update()' part of the frame header.
+ * See section '17.2. Probability Updates' of the VP8 specification
+ * for more details.
+ */
struct v4l2_vp8_entropy {
__u8 coeff_probs[4][8][3][V4L2_VP8_COEFF_PROB_CNT];
__u8 y_mode_probs[4];
@@ -63,6 +124,17 @@ struct v4l2_vp8_entropy {
__u8 padding[3];
};
+/**
+ * struct v4l2_vp8_entropy_coder_state - VP8 boolean coder state
+ *
+ * @range: coder state value for "Range"
+ * @value: coder state value for "Value"
+ * @bit_count: number of bits left in range "Value".
+ * @padding: padding field. Should be zeroed by applications.
+ *
+ * This structure contains the state for the boolean coder, as
+ * explained in section '7. Boolean Entropy Decoder' of the VP8 specification.
+ */
struct v4l2_vp8_entropy_coder_state {
__u8 range;
__u8 value;
@@ -80,6 +152,33 @@ struct v4l2_vp8_entropy_coder_state {
#define VP8_FRAME_IS_KEY_FRAME(hdr) \
(!!((hdr)->flags & V4L2_VP8_FRAME_FLAG_KEY_FRAME))
+/**
+ * struct v4l2_vp8_frame - VP8 frame parameters
+ *
+ * @seg: segmentation parameters. See _vp8_segment for more details
+ * @lf: loop filter parameters. See _vp8_loop_filter for more details
+ * @quant: quantization parameters. See _vp8_quantization for more details
+ * @probs: probabilities. See _vp9_probabilities for more details
+ * @width: frame width.
+ * @height: frame height.
+ * @horizontal_scale: horizontal scaling factor.
+ * @vertical_scale: vertical scaling fa
[PATCH 2/7] media: uapi: vp8: Rename v4l2_vp8_loopfilter to v4l2_vp8_loop_filter
Rename to struct v4l2_vp8_loop_filter, which is closer to the
VP8 syntax. This change is only cosmetics, but since this API
is exposed to userspace, it's worth the trouble to get it consistent.
Signed-off-by: Ezequiel Garcia
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 12 ++--
drivers/staging/media/hantro/hantro_g1_vp8_dec.c | 2 +-
drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c | 2 +-
include/media/vp8-ctrls.h| 4 ++--
4 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 006301e88554..8bbfea44a2ca 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1774,7 +1774,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
* - struct :c:type:`v4l2_vp8_segment`
- ``seg``
- Structure with segment-based adjustments metadata.
-* - struct :c:type:`v4l2_vp8_loopfilter`
+* - struct :c:type:`v4l2_vp8_loop_filter`
- ``lf``
- Structure with loop filter level adjustments metadata.
* - struct :c:type:`v4l2_vp8_quantization`
@@ -1958,13 +1958,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- If is set, the segment feature data mode is delta-value.
If cleared, it's absolute-value.
-.. c:type:: v4l2_vp8_loopfilter
+.. c:type:: v4l2_vp8_loop_filter
.. cssclass:: longtable
.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}|
-.. flat-table:: struct v4l2_vp8_loopfilter
+.. flat-table:: struct v4l2_vp8_loop_filter
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
@@ -1986,11 +1986,11 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- Applications and drivers must set this to zero.
* - __u32
- ``flags``
- - See :ref:`Loopfilter Flags `
+ - See :ref:`Loop Filter Flags `
-.. _vp8_loopfilter_flags:
+.. _vp8_loop_filter_flags:
-``Loopfilter Flags``
+``Loop Filter Flags``
.. cssclass:: longtable
diff --git a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
index 3f4311c1dab8..2de80fef07f4 100644
--- a/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_vp8_dec.c
@@ -136,7 +136,7 @@ static void cfg_lf(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr)
{
const struct v4l2_vp8_segment *seg = >seg;
- const struct v4l2_vp8_loopfilter *lf = >lf;
+ const struct v4l2_vp8_loop_filter *lf = >lf;
struct hantro_dev *vpu = ctx->dev;
unsigned int i;
u32 reg;
diff --git a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
index 5c43f59ad531..cbca123db746 100644
--- a/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
+++ b/drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c
@@ -277,7 +277,7 @@ static void cfg_lf(struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp8_frame *hdr)
{
const struct v4l2_vp8_segment *seg = >seg;
- const struct v4l2_vp8_loopfilter *lf = >lf;
+ const struct v4l2_vp8_loop_filter *lf = >lf;
struct hantro_dev *vpu = ctx->dev;
unsigned int i;
u32 reg;
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 86bb6e774604..f20117e825ef 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -34,7 +34,7 @@ struct v4l2_vp8_segment {
#define V4L2_VP8_LF_ADJ_ENABLE 0x01
#define V4L2_VP8_LF_DELTA_UPDATE 0x02
#define V4L2_VP8_LF_FILTER_TYPE_SIMPLE 0x04
-struct v4l2_vp8_loopfilter {
+struct v4l2_vp8_loop_filter {
__s8 ref_frm_delta[4];
__s8 mb_mode_delta[4];
__u8 sharpness_level;
@@ -82,7 +82,7 @@ struct v4l2_vp8_entropy_coder_state {
struct v4l2_ctrl_vp8_frame {
struct v4l2_vp8_segment seg;
- struct v4l2_vp8_loopfilter lf;
+ struct v4l2_vp8_loop_filter lf;
struct v4l2_vp8_quantization quant;
struct v4l2_vp8_entropy entropy;
struct v4l2_vp8_entropy_coder_state coder_state;
--
2.30.0
[PATCH 5/7] media: uapi: Move the VP8 stateless control type out of staging
Move the VP8 stateless control types out of staging,
and re-number it to avoid any confusion.
Signed-off-by: Ezequiel Garcia
---
include/media/vp8-ctrls.h | 1 -
include/uapi/linux/videodev2.h | 2 ++
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 1a5a3d8ab444..051700f33863 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -14,7 +14,6 @@
#include
#define V4L2_CID_MPEG_VIDEO_VP8_FRAME (V4L2_CID_CODEC_BASE + 2000)
-#define V4L2_CTRL_TYPE_VP8_FRAME 0x301
#define V4L2_VP8_SEGMENT_FLAG_ENABLED 0x01
#define V4L2_VP8_SEGMENT_FLAG_UPDATE_MAP 0x02
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a1d903c6f9f0..611b75df7f17 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1793,6 +1793,8 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_H264_PRED_WEIGHTS= 0x0205,
V4L2_CTRL_TYPE_FWHT_PARAMS = 0x0220,
+
+ V4L2_CTRL_TYPE_VP8_FRAME= 0x0240,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.30.0
[PATCH 4/7] media: uapi: Move parsed VP8 pixel format out of staging
Since we are ready to stabilize the VP8 stateless API,
move the parsed VP8 pixel format.
Signed-off-by: Ezequiel Garcia
---
Documentation/userspace-api/media/v4l/pixfmt-compressed.rst | 5 -
include/media/vp8-ctrls.h | 2 --
include/uapi/linux/videodev2.h | 1 +
3 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
index 2e07e48053d6..e91bc54fa286 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst
@@ -158,11 +158,6 @@ Compressed Formats
of macroblocks to decode a full corresponding frame to the matching
capture buffer.
- .. note::
-
- This format is not yet part of the public kernel API and it
- is expected to change.
-
* .. _V4L2-PIX-FMT-VP9:
- ``V4L2_PIX_FMT_VP9``
diff --git a/include/media/vp8-ctrls.h b/include/media/vp8-ctrls.h
index 9f6cb40088a6..1a5a3d8ab444 100644
--- a/include/media/vp8-ctrls.h
+++ b/include/media/vp8-ctrls.h
@@ -13,8 +13,6 @@
#include
-#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F')
-
#define V4L2_CID_MPEG_VIDEO_VP8_FRAME (V4L2_CID_CODEC_BASE + 2000)
#define V4L2_CTRL_TYPE_VP8_FRAME 0x301
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..a1d903c6f9f0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -694,6 +694,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_VC1_ANNEX_G v4l2_fourcc('V', 'C', '1', 'G') /* SMPTE 421M
Annex G compliant stream */
#define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M
Annex L compliant stream */
#define V4L2_PIX_FMT_VP8 v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
+#define V4L2_PIX_FMT_VP8_FRAME v4l2_fourcc('V', 'P', '8', 'F') /* VP8 parsed
frame */
#define V4L2_PIX_FMT_VP9 v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
#define V4L2_PIX_FMT_HEVC v4l2_fourcc('H', 'E', 'V', 'C') /* HEVC aka
H.265 */
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh
Hadamard Transform (vicodec) */
--
2.30.0
[PATCH 0/7] V4L2 stateless VP8 de-staging
After close scrutiny of the VP8 specification, it seems the VP8 stateless API is ready. This series contains a series of clean-ups and improvement: renaming symbols for consistency, documenting things for clarity, and then moves the control to proper V4L2 headers. It must be noted that, unlike parsed H.264, V4L2_PIX_FMT_VP8_FRAME buffers must contain the VP8 frame header. In others words a VP8 parsed frame buffer is: ++-+-++ | tag 3B | extra 7B | header | MB data | DCT size | DCT part 0 | ... | DCT part N | ++-+-++ Hopefully the pixel format documentation is now clear about that. Support in Cedrus and Hantro is available upstream, and Mediatek support is available downstream [1] which uses the API as-is. This was tested with GStreamer v4l2codec element and Fluster [2] conformance test, on a Hantro G1 device. [1] https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2614338/21/ [2] https://github.com/fluendo/fluster Ezequiel Garcia (7): media: uapi: vp8: Remove "header" from symbol names and macros media: uapi: vp8: Rename v4l2_vp8_loopfilter to v4l2_vp8_loop_filter media: uapi: vp8: Add proper kernel-doc documentation media: uapi: Move parsed VP8 pixel format out of staging media: uapi: Move the VP8 stateless control type out of staging media: controls: Log VP8 stateless control in .std_log media: uapi: move VP8 stateless controls out of staging .../media/v4l/ext-ctrls-codec-stateless.rst | 318 + .../media/v4l/ext-ctrls-codec.rst | 323 -- .../media/v4l/pixfmt-compressed.rst | 15 +- .../media/v4l/vidioc-g-ext-ctrls.rst | 4 + .../media/v4l/vidioc-queryctrl.rst| 6 + .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ctrls.c | 39 ++- drivers/staging/media/hantro/hantro_drv.c | 2 +- .../staging/media/hantro/hantro_g1_vp8_dec.c | 49 ++- drivers/staging/media/hantro/hantro_hw.h | 2 +- drivers/staging/media/hantro/hantro_vp8.c | 10 +- .../media/hantro/rk3399_vpu_hw_vp8_dec.c | 49 ++- drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +- .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +- .../staging/media/sunxi/cedrus/cedrus_vp8.c | 101 +++--- include/media/v4l2-ctrls.h| 5 +- include/media/vp8-ctrls.h | 114 --- include/uapi/linux/v4l2-controls.h| 195 +++ include/uapi/linux/videodev2.h| 4 + 20 files changed, 665 insertions(+), 578 deletions(-) delete mode 100644 include/media/vp8-ctrls.h -- 2.30.0
[PATCH 1/7] media: uapi: vp8: Remove "header" from symbol names and macros
It doesn't seem to add any clarity to have a "header" suffix in controls,
struct names and flags.
Since this just makes names too long without any benefit, just drop it.
Signed-off-by: Ezequiel Garcia
---
.../media/v4l/ext-ctrls-codec.rst | 80 +++
.../media/v4l/pixfmt-compressed.rst | 2 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 36 +++
drivers/staging/media/hantro/hantro_drv.c | 2 +-
.../staging/media/hantro/hantro_g1_vp8_dec.c | 48 -
drivers/staging/media/hantro/hantro_hw.h | 2 +-
drivers/staging/media/hantro/hantro_vp8.c | 10 +-
.../media/hantro/rk3399_vpu_hw_vp8_dec.c | 48 -
drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
.../staging/media/sunxi/cedrus/cedrus_vp8.c | 98 +--
include/media/v4l2-ctrls.h| 4 +-
include/media/vp8-ctrls.h | 48 -
14 files changed, 192 insertions(+), 192 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 00944e97d638..006301e88554 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1749,7 +1749,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
.. _v4l2-mpeg-vp8:
-``V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER (struct)``
+``V4L2_CID_MPEG_VIDEO_VP8_FRAME (struct)``
Specifies the frame parameters for the associated VP8 parsed frame data.
This includes the necessary parameters for
configuring a stateless hardware decoding pipeline for VP8.
@@ -1760,28 +1760,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
This compound control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_ctrl_vp8_frame_header
+.. c:type:: v4l2_ctrl_vp8_frame
.. cssclass:: longtable
.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
-.. flat-table:: struct v4l2_ctrl_vp8_frame_header
+.. flat-table:: struct v4l2_ctrl_vp8_frame
:header-rows: 0
:stub-columns: 0
:widths: 1 1 2
-* - struct :c:type:`v4l2_vp8_segment_header`
- - ``segment_header``
+* - struct :c:type:`v4l2_vp8_segment`
+ - ``seg``
- Structure with segment-based adjustments metadata.
-* - struct :c:type:`v4l2_vp8_loopfilter_header`
- - ``loopfilter_header``
+* - struct :c:type:`v4l2_vp8_loopfilter`
+ - ``lf``
- Structure with loop filter level adjustments metadata.
-* - struct :c:type:`v4l2_vp8_quantization_header`
- - ``quant_header``
+* - struct :c:type:`v4l2_vp8_quantization`
+ - ``quant``
- Structure with VP8 dequantization indices metadata.
-* - struct :c:type:`v4l2_vp8_entropy_header`
- - ``entropy_header``
+* - struct :c:type:`v4l2_vp8_entropy`
+ - ``entropy``
- Structure with VP8 entropy coder probabilities metadata.
* - struct :c:type:`v4l2_vp8_entropy_coder_state`
- ``coder_state``
@@ -1850,11 +1850,11 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:c:type:`v4l2_buffer` to a __u64.
* - __u64
- ``flags``
- - See :ref:`Frame Header Flags `
+ - See :ref:`Frame Flags `
-.. _vp8_frame_header_flags:
+.. _vp8_frame_flags:
-``Frame Header Flags``
+``Frame Flags``
.. cssclass:: longtable
@@ -1863,22 +1863,22 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
:stub-columns: 0
:widths: 1 1 2
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_KEY_FRAME``
+* - ``V4L2_VP8_FRAME_FLAG_KEY_FRAME``
- 0x01
- Indicates if the frame is a key frame.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_EXPERIMENTAL``
+* - ``V4L2_VP8_FRAME_FLAG_EXPERIMENTAL``
- 0x02
- Experimental bitstream.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SHOW_FRAME``
+* - ``V4L2_VP8_FRAME_FLAG_SHOW_FRAME``
- 0x04
- Show frame flag, indicates if the frame is for display.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_MB_NO_SKIP_COEFF``
+* - ``V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF``
- 0x08
- Enable/disable skipping of macroblocks with no non-zero coefficients.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_GOLDEN``
+* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_GOLDEN``
- 0x10
- Sign of motion vectors when the golden frame is referenced.
-* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_ALT``
+* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_ALT``
- 0x20
- Sign of motion vectors when the alt frame is referenced.
@@ -1906,13 +1906,13 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
- ``padding``
- Applications and drivers must set this to zero.
-.. c:type:: v4l2_vp8_segment_header
+.. c:type:: v4l2_vp8_segment
.. cssclass:: longtable
.. tabularcolumns:
Re: [PATCH 4/7] misc: Add driver for DAB IP found on Renesas R-Car devices
Hello everyone, On Mon, 1 Mar 2021 at 14:36, Fabrizio Castro wrote: > > Hi Arnd, > > Thanks for your feedback! > > > From: Arnd Bergmann > > Sent: 26 February 2021 10:38 > > Subject: Re: [PATCH 4/7] misc: Add driver for DAB IP found on Renesas R- > > Car devices > > > > On Thu, Feb 25, 2021 at 11:51 PM Fabrizio Castro > > wrote: > > > > > > The DAB hardware accelerator found on R-Car E3 and R-Car M3-N devices is > > > a hardware accelerator for software DAB demodulators. > > > It consists of one FFT (Fast Fourier Transform) module and one decoder > > > module, compatible with DAB specification (ETSI EN 300 401 and > > > ETSI TS 102 563). > > > The decoder module can perform FIC decoding and MSC decoding processing > > > from de-puncture to final decoded result. > > > > > > This patch adds a device driver to support the FFT module only. > > > > > > Signed-off-by: Fabrizio Castro > > > --- > > > MAINTAINERS | 7 ++ > > > drivers/misc/Kconfig | 1 + > > > drivers/misc/Makefile| 1 + > > > drivers/misc/rcar_dab/Kconfig| 11 ++ > > > drivers/misc/rcar_dab/Makefile | 8 ++ > > > drivers/misc/rcar_dab/rcar_dev.c | 176 +++ > > > drivers/misc/rcar_dab/rcar_dev.h | 116 > > > drivers/misc/rcar_dab/rcar_fft.c | 160 > > > include/uapi/linux/rcar_dab.h| 35 ++ > > > > Can you explain why this is not in drivers/media/? > > I wasn't entirely sure were to put it to be honest as I couldn't find > anything similar, that's why "misc" for v1, to mainly get a feedback > and advice. > > > > > I don't think we want a custom ioctl interface for a device that > > implements > > a generic specification. My first feeling would be that this should not > > have a user-level API but instead get called by the DAB radio driver. > > I hear you, the trouble is that the modules of this IP should be seen > as part of a SW and HW processing pipeline. > Some of the SW components of the pipeline can be proprietary, and > unfortunately we don't have access (yet) to a framework that allows us to > test and demonstrate the whole pipeline, for the moment we can only test > things in isolation. It'll take us a while to come up with a full solution > (or to have access to one), and in the meantime our SoCs come with an IP > that can't be used because there is no driver for it (yet). > > After discussing things further with Geert and Laurent, I think they > are right in saying that the best path for this is probably to add this > driver under "drivers/staging" as an interim solution, so that the IP will > be accessible by developers, and when we have everything we need (a fully > fledged processing framework), we will able to integrate it better with > the other components (if possible). > > > > > What is the intended usage model here? I assume the idea is to > > use it in an application that receives audio or metadata from DAB. > > What driver do you use for that? > > This IP is similar to a DMA to some extent, in the sense that it takes > input data from a buffer in memory and it writes output data onto a buffer > in memory, and of course it does some processing in between. That sounds like something that can fit V4L2 MEM2MEM driver. You queue two buffers and an operation, and then run a job. > It's not directly connected to any other Radio related IP. > The user space application can read DAB IQ samples from the R-Car DRIF > interface (via driver drivers/media/platform/rcar_drif.c, or from other > sources since this IP is decoupled from DRIF), and then when it's time > to accelerate FFT, FIC, or MSC, it can request services to the DAB IP, so > that the app can go on and use the processor to do some work, while the DAB > IP processes things. > A framework to connect and exchange processing blocks (either SW or HW) and > interfaces is therefore vital to properly place a kernel implementation > in the great scheme of things, in its absence a simple driver can help I'm not entirely sure we are missing a framework. What's missing in V4L2 MEM2MEM? Before considering drivers/staging, it would be interesting to figure out if V4L2 can do it as-is, and if not, discuss what's missing. Thanks, Ezequiel
Re: [PATCH v3 5/9] media: hantro: Introduce G2/HEVC decoder
On Fri, 2021-02-26 at 10:47 +0100, Benjamin Gaignard wrote:
>
> Le 25/02/2021 à 18:55, Ezequiel Garcia a écrit :
> > Hi Benjamin,
> >
> > Thanks for the good work!
> > I mostly have two concerns with this implementation,
> > the tiled output and the allocation path.
> >
> > More below.
> >
> > On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
> > > Implement all the logic to get G2 hardware decoding HEVC frames.
> > > It support up level 5.1 HEVC stream.
> > > It doesn't support yet 10 bits formats or scaling feature.
> > >
> > > Add HANTRO HEVC dedicated control to skip some bits at the beginning
> > > of the slice header. That is very specific to this hardware so can't
> > > go into uapi structures. Compute the needed value is complex and require
> > > information from the stream that only the userland knows so let it
> > > provide the correct value to the driver.
> > >
> > > Signed-off-by: Benjamin Gaignard
> > > ---
> > > version 2:
> > > - squash multiple commits in this one.
> > > - fix the comments done by Ezequiel about dma_alloc_coherent usage
> > > - fix Dan's comments about control copy, reverse the test logic
> > > in tile_buffer_reallocate, rework some goto and return cases.
> > >
> > > drivers/staging/media/hantro/Makefile | 2 +
> > > drivers/staging/media/hantro/hantro.h | 19 +
> > > drivers/staging/media/hantro/hantro_drv.c | 42 ++
> > > .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++
> > > drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++
> > > drivers/staging/media/hantro/hantro_hevc.c | 321 ++
> > > drivers/staging/media/hantro/hantro_hw.h | 47 ++
> > > 7 files changed, 1216 insertions(+)
> > > create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> > > create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> > > create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
> > >
> > >
> > [snip]
> > > +
> > > +static void set_buffers(struct hantro_ctx *ctx)
> > > +{
> > > + struct vb2_v4l2_buffer *src_buf, *dst_buf;
> > > + struct hantro_dev *vpu = ctx->dev;
> > > + const struct hantro_hevc_dec_ctrls *ctrls = >hevc_dec.ctrls;
> > > + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
> > > + size_t cr_offset = hantro_hevc_chroma_offset(sps);
> > > + dma_addr_t src_dma, dst_dma;
> > > + u32 src_len, src_buf_len;
> > > +
> > > + src_buf = hantro_get_src_buf(ctx);
> > > + dst_buf = hantro_get_dst_buf(ctx);
> > > +
> > > + /* Source (stream) buffer. */
> > > + src_dma = vb2_dma_contig_plane_dma_addr(_buf->vb2_buf, 0);
> > > + src_len = vb2_get_plane_payload(_buf->vb2_buf, 0);
> > > + src_buf_len = vb2_plane_size(_buf->vb2_buf, 0);
> > > +
> > > + hantro_write_addr(vpu, HEVC_ADDR_STR, src_dma);
> > > + hantro_reg_write(vpu, hevc_stream_len, src_len);
> > > + hantro_reg_write(vpu, hevc_strm_buffer_len, src_buf_len);
> > > + hantro_reg_write(vpu, hevc_strm_start_offset, 0);
> > > + hantro_reg_write(vpu, hevc_write_mvs_e, 1);
> > > +
> > > + /* Destination (decoded frame) buffer. */
> > > + dst_dma = hantro_get_dec_buf_addr(ctx, _buf->vb2_buf);
> > > +
> > > + hantro_write_addr(vpu, HEVC_RASTER_SCAN, dst_dma);
> > > + hantro_write_addr(vpu, HEVC_RASTER_SCAN_CHR, dst_dma + cr_offset);
> > The way this is done the raster-scan output is the only
> > output, and the tiled ouput it kept entirely internal, in
> > hantro_hevc_dec_hw_ctx.ref_bufs.
> >
> > This means there's no way to expose tiled NV12 in i.MX8M VPU tiled format
> > to userspace, which could be desirable for some use cases.
> >
> > I'm not suggesting to actually expose tiled NV12 in this patch, but to
> > prepare
> > the driver to be able to support that easily in the future.
> >
> > It should be possible to consider this detiling as
> > post-processing, adding some code in hantro_postproc.c
> > leveraging the existing post-proc support.
> >
> > IOW, hantro_postproc_ctx.dec_q would hold the tiled output,
> > hantro_postproc_enable() writes the raster-scan
> > buffer destination address, and so on.
>
> Well the G2 can't use th
Re: [PATCH] media: hantro: Auto generate the AXI ID to avoid conflicts
Hi Enric, Thanks a lot for the patch. On Thu, 25 Feb 2021 at 09:08, Enric Balletbo i Serra wrote: > > The AXI ID is an AXI bus configuration for improve bus performance. If > read and write operations use different ID the operations can be > paralleled, whereas when they have the same ID the operations will be > serialized. Right now, the write ID is fixed to 0 but we can set it to > 0xff to get auto generated ID to avoid possible conflicts. > > This change has no functional changes, but seems reasonable to let the > hardware to autogenerate the ID instead of hardcoding in software. > > Signed-off-by: Enric Balletbo i Serra > --- > > drivers/staging/media/hantro/hantro_g1_h264_dec.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c > b/drivers/staging/media/hantro/hantro_g1_h264_dec.c > index 845bef73d218..090088cd98ea 100644 > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c > @@ -30,7 +30,7 @@ static void set_params(struct hantro_ctx *ctx) > u32 reg; > > /* Decoder control register 0. */ > - reg = G1_REG_DEC_CTRL0_DEC_AXI_WR_ID(0x0); > + reg = G1_REG_DEC_CTRL0_DEC_AXI_WR_ID(0xff); Can we define a macro to avoid this magic number, and add some comments explaining what's 0xff for? Given this is AXI configuration, I'd expect it's CODEC-agnostic. Maybe we could move CODEC-agnostic path to avoid duplicating the code? Does this change apply to the rkvdec driver? Thanks, Ezequiel
Re: [PATCH v3 5/9] media: hantro: Introduce G2/HEVC decoder
Hi Benjamin,
Thanks for the good work!
I mostly have two concerns with this implementation,
the tiled output and the allocation path.
More below.
On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
> Implement all the logic to get G2 hardware decoding HEVC frames.
> It support up level 5.1 HEVC stream.
> It doesn't support yet 10 bits formats or scaling feature.
>
> Add HANTRO HEVC dedicated control to skip some bits at the beginning
> of the slice header. That is very specific to this hardware so can't
> go into uapi structures. Compute the needed value is complex and require
> information from the stream that only the userland knows so let it
> provide the correct value to the driver.
>
> Signed-off-by: Benjamin Gaignard
> ---
> version 2:
> - squash multiple commits in this one.
> - fix the comments done by Ezequiel about dma_alloc_coherent usage
> - fix Dan's comments about control copy, reverse the test logic
> in tile_buffer_reallocate, rework some goto and return cases.
>
> drivers/staging/media/hantro/Makefile | 2 +
> drivers/staging/media/hantro/hantro.h | 19 +
> drivers/staging/media/hantro/hantro_drv.c | 42 ++
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++
> drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++
> drivers/staging/media/hantro/hantro_hevc.c | 321 ++
> drivers/staging/media/hantro/hantro_hw.h | 47 ++
> 7 files changed, 1216 insertions(+)
> create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
>
>
[snip]
> +
> +static void set_buffers(struct hantro_ctx *ctx)
> +{
> + struct vb2_v4l2_buffer *src_buf, *dst_buf;
> + struct hantro_dev *vpu = ctx->dev;
> + const struct hantro_hevc_dec_ctrls *ctrls = >hevc_dec.ctrls;
> + const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
> + size_t cr_offset = hantro_hevc_chroma_offset(sps);
> + dma_addr_t src_dma, dst_dma;
> + u32 src_len, src_buf_len;
> +
> + src_buf = hantro_get_src_buf(ctx);
> + dst_buf = hantro_get_dst_buf(ctx);
> +
> + /* Source (stream) buffer. */
> + src_dma = vb2_dma_contig_plane_dma_addr(_buf->vb2_buf, 0);
> + src_len = vb2_get_plane_payload(_buf->vb2_buf, 0);
> + src_buf_len = vb2_plane_size(_buf->vb2_buf, 0);
> +
> + hantro_write_addr(vpu, HEVC_ADDR_STR, src_dma);
> + hantro_reg_write(vpu, hevc_stream_len, src_len);
> + hantro_reg_write(vpu, hevc_strm_buffer_len, src_buf_len);
> + hantro_reg_write(vpu, hevc_strm_start_offset, 0);
> + hantro_reg_write(vpu, hevc_write_mvs_e, 1);
> +
> + /* Destination (decoded frame) buffer. */
> + dst_dma = hantro_get_dec_buf_addr(ctx, _buf->vb2_buf);
> +
> + hantro_write_addr(vpu, HEVC_RASTER_SCAN, dst_dma);
> + hantro_write_addr(vpu, HEVC_RASTER_SCAN_CHR, dst_dma + cr_offset);
The way this is done the raster-scan output is the only
output, and the tiled ouput it kept entirely internal, in
hantro_hevc_dec_hw_ctx.ref_bufs.
This means there's no way to expose tiled NV12 in i.MX8M VPU tiled format
to userspace, which could be desirable for some use cases.
I'm not suggesting to actually expose tiled NV12 in this patch, but to prepare
the driver to be able to support that easily in the future.
It should be possible to consider this detiling as
post-processing, adding some code in hantro_postproc.c
leveraging the existing post-proc support.
IOW, hantro_postproc_ctx.dec_q would hold the tiled output,
hantro_postproc_enable() writes the raster-scan
buffer destination address, and so on.
> + hantro_write_addr(vpu, HEVC_ADDR_TILE_SIZE,
> ctx->hevc_dec.tile_sizes.dma);
> + hantro_write_addr(vpu, HEVC_TILE_FILTER,
> ctx->hevc_dec.tile_filter.dma);
> + hantro_write_addr(vpu, HEVC_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
> + hantro_write_addr(vpu, HEVC_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
> +}
> +
> +void hantro_g2_check_idle(struct hantro_dev *vpu)
> +{
> + int i;
> +
> + for (i = 0; i < 3; i++) {
> + u32 status;
> +
> + /* Make sure the VPU is idle */
> + status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
> + if (status & HEVC_REG_INTERRUPT_DEC_E) {
> + pr_warn("%s: still enabled!!! resetting.\n",
> __func__);
> + status |= HEVC_REG_INTERRUPT_DEC_ABORT_E |
> HEVC_REG_INTERRUPT_DEC_IRQ_DIS;
> + vdpu_write(vpu, status, HEVC_REG_INTERRUPT);
> + }
> + }
> +}
> +
> +void hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
> +{
> + struct hantro_dev *vpu = ctx->dev;
> +
> + hantro_g2_check_idle(vpu);
> +
> + /* Prepare HEVC decoder context. */
> + if (hantro_hevc_dec_prepare_run(ctx))
> + return;
> +
> + /* Configure
Re: Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec
Hey Jernej, On Thu, 2021-02-25 at 18:01 +0100, Jernej Škrabec wrote: > Hi Ezequiel, > > Dne četrtek, 25. februar 2021 ob 14:09:52 CET je Ezequiel Garcia napisal(a): > > Hi Benjamin, > > > > Thanks for the good work. > > > > On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote: > > > The H.265 ITU specification (section 7.4) define the general > > > slice segment header semantics. > > > Modified/added fields are: > > > - video_parameter_set_id: (7.4.3.1) identifies the VPS for > > > reference by other syntax elements. > > > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of > > > the vps_video_parameter_set_id of the active VPS. > > > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling > > > relative to the luma sampling > > > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for > > > reference by other syntax elements > > > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies > > > the inferred value of num_ref_idx_l0_active_minus1 > > > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies > > > the inferred value of num_ref_idx_l1_active_minus1 > > > - slice_segment_addr: (7.4.7.1) specifies the address of > > > the first coding tree block in the slice segment > > > - num_entry_point_offsets: (7.4.7.1) specifies the number of > > > entry_point_offset_minus1[ i ] syntax elements in the slice header > > > > > > Add HEVC decode params contains the information used in section > > > "8.3 Slice decoding process" of the specification to let the hardware > > > perform decoding of a slices. > > > > > > Adapt Cedrus driver according to these changes. > > > > > > Signed-off-by: Benjamin Gaignard > > > --- > > > version 3: > > > - Add documentation about the new structuers and fields. > > > > > > version 2: > > > - remove all change related to scaling > > > - squash commits to a coherent split > > > - be more verbose about the added fields > > > > > > .../media/v4l/ext-ctrls-codec.rst | 126 +++--- > > > .../media/v4l/vidioc-queryctrl.rst | 6 + > > > drivers/media/v4l2-core/v4l2-ctrls.c | 26 +++- > > > drivers/staging/media/sunxi/cedrus/cedrus.c | 6 + > > > drivers/staging/media/sunxi/cedrus/cedrus.h | 1 + > > > .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 + > > > .../staging/media/sunxi/cedrus/cedrus_h265.c | 6 +- > > > include/media/hevc-ctrls.h | 45 +-- > > > 8 files changed, 186 insertions(+), 32 deletions(-) > > > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/ > Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > index 00944e97d638..5e6d77e858c0 100644 > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > > > :stub-columns: 0 > > > :widths: 1 1 2 > > > > > > + * - __u8 > > > + - ``video_parameter_set_id`` > > > + - Identifies the VPS for reference by other syntax elements > > > + * - __u8 > > > + - ``seq_parameter_set_id̀`` > > > + - Specifies the value of the vps_video_parameter_set_id of the > active VPS > > > + * - __u8 > > > + - ``chroma_format_idc`` > > > + - Specifies the chroma sampling relative to the luma sampling > > > > None of these fields seem needed for the Hantro G2 driver, > > so I suggest you drop them for now. > > > > > * - __u16 > > > - ``pic_width_in_luma_samples`` > > > - > > > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > > > * - __u8 > > > - ``chroma_format_idc`` > > > - > > > + * - __u8 > > > + - ``num_slices`` > > > + > > > > Not used, but also doesn't seem part of the SPS syntax. If we have to > > pass the number of slices, we'll need another mechanism. > > > > > - > > > * - __u64 > > > - ``flags`` > > > - See :ref:`Sequence Parameter Set Flags ` > > > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > > > :stub-co
Re: [PATCH v3 4/9] media: uapi: Add a control for HANTRO driver
Hi Benjamin,
On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
> The HEVC HANTRO driver needs to know the number of bits to skip at
s/HANTRO/Hantro
> the beginning of the slice header.
As discussed in a different thread, we should describe exactly
what the hardware is expecting, so applications can parse that
and pass a correct value.
> That is a hardware specific requirement so create a dedicated control
> that this purpose.
>
> Signed-off-by: Benjamin Gaignard
> ---
> version 3:
> - Fix typo in field name
>
> include/uapi/linux/hantro-v4l2-controls.h | 20
> include/uapi/linux/v4l2-controls.h | 5 +
> 2 files changed, 25 insertions(+)
> create mode 100644 include/uapi/linux/hantro-v4l2-controls.h
>
> diff --git a/include/uapi/linux/hantro-v4l2-controls.h
> b/include/uapi/linux/hantro-v4l2-controls.h
> new file mode 100644
> index ..a8dfd6b1a2a9
> --- /dev/null
> +++ b/include/uapi/linux/hantro-v4l2-controls.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +
> +#ifndef __UAPI_HANTRO_V4L2_CONYTROLS_H__
> +#define __UAPI_HANTRO_V4L2_CONYTROLS_H__
> +
> +#include
> +#include
> +
> +#define V4L2_CID_HANTRO_HEVC_EXTRA_DECODE_PARAMS
> (V4L2_CID_USER_HANTRO_BASE + 0)
> +
> +/**
> + * struct hantro_hevc_extra_decode_params - extra decode parameters for
> hantro driver
> + * @hevc_hdr_skip_length: header first bits offset
> + */
> +struct hantro_hevc_extra_decode_params {
> + __u32 hevc_hdr_skip_length;
> + __u8padding[4];
> +};
> +
I think we can get away with a simpler solution. Since it's just one integer
we need, there's no need for a compound control. Something like this:
.codec = HANTRO_HEVC_DECODER,
.cfg = {
.id = V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP,
.name = "Hantro HEVC slice header skip bytes",
.type = V4L2_CTRL_TYPE_INTEGER,
.min = 0,
.max = 0x7fff,
.step = 1,
},
Also see V4L2_CID_CODA_MB_ERR_CNT which is defined in
drivers/media/platform/coda/coda.h.
The control is sufficiently special that it could be kept in an internal driver
header.
> +#endif
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index 039c0d7add1b..ced7486c7f46 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -209,6 +209,11 @@ enum v4l2_colorfx {
> * We reserve 128 controls for this driver.
> */
> #define V4L2_CID_USER_CCS_BASE (V4L2_CID_USER_BASE + 0x10f0)
> +/*
> + * The base for HANTRO driver controls.
> + * We reserve 32 controls for this driver.
> + */
> +#define V4L2_CID_USER_HANTRO_BASE (V4L2_CID_USER_BASE + 0x1170)
>
> /* MPEG-class control IDs */
> /* The MPEG controls are applicable to all codec controls
Thanks,
Ezequiel
Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec
Hi Benjamin, Thanks for the good work. On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote: > The H.265 ITU specification (section 7.4) define the general > slice segment header semantics. > Modified/added fields are: > - video_parameter_set_id: (7.4.3.1) identifies the VPS for > reference by other syntax elements. > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of > the vps_video_parameter_set_id of the active VPS. > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling > relative to the luma sampling > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for > reference by other syntax elements > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies > the inferred value of num_ref_idx_l0_active_minus1 > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies > the inferred value of num_ref_idx_l1_active_minus1 > - slice_segment_addr: (7.4.7.1) specifies the address of > the first coding tree block in the slice segment > - num_entry_point_offsets: (7.4.7.1) specifies the number of > entry_point_offset_minus1[ i ] syntax elements in the slice header > > Add HEVC decode params contains the information used in section > "8.3 Slice decoding process" of the specification to let the hardware > perform decoding of a slices. > > Adapt Cedrus driver according to these changes. > > Signed-off-by: Benjamin Gaignard > --- > version 3: > - Add documentation about the new structuers and fields. > > version 2: > - remove all change related to scaling > - squash commits to a coherent split > - be more verbose about the added fields > > .../media/v4l/ext-ctrls-codec.rst | 126 +++--- > .../media/v4l/vidioc-queryctrl.rst | 6 + > drivers/media/v4l2-core/v4l2-ctrls.c | 26 +++- > drivers/staging/media/sunxi/cedrus/cedrus.c | 6 + > drivers/staging/media/sunxi/cedrus/cedrus.h | 1 + > .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 + > .../staging/media/sunxi/cedrus/cedrus_h265.c | 6 +- > include/media/hevc-ctrls.h | 45 +-- > 8 files changed, 186 insertions(+), 32 deletions(-) > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > index 00944e97d638..5e6d77e858c0 100644 > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > :stub-columns: 0 > :widths: 1 1 2 > > + * - __u8 > + - ``video_parameter_set_id`` > + - Identifies the VPS for reference by other syntax elements > + * - __u8 > + - ``seq_parameter_set_id̀`` > + - Specifies the value of the vps_video_parameter_set_id of the active > VPS > + * - __u8 > + - ``chroma_format_idc`` > + - Specifies the chroma sampling relative to the luma sampling None of these fields seem needed for the Hantro G2 driver, so I suggest you drop them for now. > * - __u16 > - ``pic_width_in_luma_samples`` > - > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > * - __u8 > - ``chroma_format_idc`` > - > + * - __u8 > + - ``num_slices`` > + Not used, but also doesn't seem part of the SPS syntax. If we have to pass the number of slices, we'll need another mechanism. > - > * - __u64 > - ``flags`` > - See :ref:`Sequence Parameter Set Flags ` > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > :stub-columns: 0 > :widths: 1 1 2 > > + * - __u8 > + - ``pic_parameter_set_id`` > + - Identifies the PPS for reference by other syntax elements Not used. > * - __u8 > - ``num_extra_slice_header_bits`` > - > + * - __u8 > + - ``num_ref_idx_l0_default_active_minus1`` > + - Specifies the inferred value of num_ref_idx_l0_active_minus1 > + * - __u8 > + - ``num_ref_idx_l1_default_active_minus1`` > + - Specifies the inferred value of num_ref_idx_l1_active_minus1 > * - __s8 > - ``init_qp_minus26`` > - > @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field - > * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT`` > - 0x0004 > - > + * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT`` > + - 0x0008 > + - > + * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING`` > + - 0x0010 > + - > I suggest to do all the PPS control changes in a separate patch, feels easier to review and cleaner as you can explain the changes with more detail in the commit description. Looking at the PPS syntax for tiles, I'm wondering if these deserve their own control, which would be used if tiles are enabled, i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set. __u8num_tile_columns_minus1;
Re: [PATCH v3 2/9] media: hantro: Define HEVC codec profiles and supported features
On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> Define which HEVC profiles (up to level 5.1) and features
> (no scaling, no 10 bits) are supported by the driver.
>
> Signed-off-by: Benjamin Gaignard
> ---
> drivers/staging/media/hantro/hantro.h | 2 +
> drivers/staging/media/hantro/hantro_drv.c | 58 +++
> 2 files changed, 60 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index 65f9f7ea7dcf..bde65231f22f 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -99,6 +99,7 @@ struct hantro_variant {
> * @HANTRO_MODE_H264_DEC: H264 decoder.
> * @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
> * @HANTRO_MODE_VP8_DEC: VP8 decoder.
> + * @HANTRO_MODE_HEVC_DEC: HEVC decoder.
> */
> enum hantro_codec_mode {
> HANTRO_MODE_NONE = -1,
> @@ -106,6 +107,7 @@ enum hantro_codec_mode {
> HANTRO_MODE_H264_DEC,
> HANTRO_MODE_MPEG2_DEC,
> HANTRO_MODE_VP8_DEC,
> + HANTRO_MODE_HEVC_DEC,
> };
>
> /*
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index e5f200e64993..d86e322a5980 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -243,6 +243,18 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
> + } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
> + const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
> +
> + if (sps->bit_depth_luma_minus8 !=
> sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
> + /* No scaling support */
> + return -EINVAL;
> }
> return 0;
> }
> @@ -349,6 +361,52 @@ static const struct hantro_ctrl controls[] = {
> .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> }
> }, {
> + .codec = HANTRO_HEVC_DECODER,
Silly nitpick. Looks like this is not defined yet?
I'm getting:
drivers/staging/media/hantro/hantro_drv.c:364:12: error: ‘HANTRO_HEVC_DECODER’
undeclared here (not in a function); did you mean
‘HANTRO_H264_DECODER’?
364 | .codec = HANTRO_HEVC_DECODER,
|^~~
|HANTRO_H264_DECODER
I'll review the G2 driver soon :-)
Thanks,
Ezequiel
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
> + .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
> + .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
> + .min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> + .max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
> + .def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
> + .min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
> + .max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
> + .ops = _ctrl_ops,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> + },
> },
> };
>
Re: [PATCH v3 0/9] Add HANTRO G2/HEVC decoder support for IMX8MQ
Hi Benjamin, On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote: > The IMX8MQ got two VPUs but until now only G1 has been enabled. > This series aim to add the second VPU (aka G2) and provide basic > HEVC decoding support. > > To be able to decode HEVC it is needed to add/update some of the > structures in the uapi. In addition of them one HANTRO dedicated > control is required to inform the driver of the numbre of bits to skip > at the beginning of the slice header. > The hardware require to allocate few auxiliary buffers to store the > references frame or tile size data. > > The driver has been tested with fluster test suite stream. > For example with this command: ./fluster.py run -ts JCT-VC-HEVC_V1 -d > GStreamer-H.265-V4L2SL-Gst1.0 > > This series depends of the reset rework posted here: > https://www.spinics.net/lists/arm-kernel/msg875766.html > > Finally the both VPUs will have a node the device-tree and be > independent from v4l2 point of view. > > A branch with all the dev is available here: > https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/upstream_g2_v2 > > version 3: > - Fix typo in Hantro v4l2 dedicated control > - Add documentation for the new structures and fields > - Rebased on top of media_tree for-linus-5.12-rc1 tag > > version 2: > - remove all change related to scaling > - squash commits to a coherent split > - be more verbose about the added fields > - fix the comments done by Ezequiel about dma_alloc_coherent usage > - fix Dan's comments about control copy, reverse the test logic > in tile_buffer_reallocate, rework some goto and return cases. > - be more verbose about why I change the bindings > - remove all sign-off expect mime since it is confusing I don't think it's about having less confusing commits, but trying to describe fairly accurately the origin of the work and the authorship of changes. In this series I believe we have three cases, please correct me if I'm wrong: * Changes that you have authored, for instance it would seem to me that "dt-bindings: media: nxp,imx8mq-vpu: Update bindings", and "media: uapi: Add a control for HANTRO driver" would be in that category. * Changes that Adrian and/or me have initially authored, where both Adrian and me did further work, and also where you did significant work on as well. IOW, changes where the three of us did significant work. I guess in this case, the original author could be the author of the patch (git's Author tag), and additionally authorship would be indicated with Co-developed-by tags. commit 494adacd844e5656b570895a82bc343438b23023 Author: Adrian Ratiu Date: Thu Feb 18 12:17:37 2021 +0100 media: hantro: Introduce G2/HEVC decoder ... Signed-off-by: Adrian Ratiu Co-developed-by: Ezequiel Garcia Signed-off-by: Ezequiel Garcia Co-developed-by: Benjamin Gaignard Signed-off-by: Benjamin Gaignard * And finally changes with an original author (Adrian or me), which you haven't changed, but you are just submitting (or you did minor work). Maybe with the current way of splitting commits we don't have these type of patches, but just for the sake of it, let's say there's a commit authored by Adrian that you are mostly picking up, it would be: commit 896776d3bcd032808e4d5772e6749da5dd4eec42 Author: Adrian Ratiu Date: Fri Feb 5 12:14:16 2021 +0100 media: hantro: Some change ... Signed-off-by: Adrian Ratiu Signed-off-by: Benjamin Gaignard Thanks a lot! Ezequiel > - remove useless clocks in VPUs nodes > > Benjamin > > Benjamin Gaignard (9): > media: hevc: Modify structures to follow H265 ITU spec > media: hantro: Define HEVC codec profiles and supported features > media: hantro: Add a field to distinguish the hardware versions > media: uapi: Add a control for HANTRO driver > media: hantro: Introduce G2/HEVC decoder > media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control > media: hantro: IMX8M: add variant for G2/HEVC codec > dt-bindings: media: nxp,imx8mq-vpu: Update bindings > arm64: dts: imx8mq: Add node to G2 hardware > > .../bindings/media/nxp,imx8mq-vpu.yaml | 54 +- > .../media/v4l/ext-ctrls-codec.rst | 126 +++- > .../media/v4l/vidioc-queryctrl.rst | 6 + > arch/arm64/boot/dts/freescale/imx8mq.dtsi | 41 +- > drivers/media/v4l2-core/v4l2-ctrls.c | 26 +- > drivers/staging/media/hantro/Makefile | 2 + > drivers/staging/media/hantro/hantro.h | 34 +- > drivers/staging/media/hantro/hantro_drv.c | 103 +++ > .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++ > drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++ > drivers/staging/media/hantro/hantro_hevc.c | 321 ++ >
Re: [PATCH v3 8/9] dt-bindings: media: nxp,imx8mq-vpu: Update bindings
Hi Rob, On Tue, 2021-02-23 at 08:31 -0600, Rob Herring wrote: > On Tue, Feb 23, 2021 at 2:04 AM Benjamin Gaignard > wrote: > > > > > > Le 23/02/2021 à 01:34, Rob Herring a écrit : > > > On Mon, Feb 22, 2021 at 01:24:05PM +0100, Benjamin Gaignard wrote: > > > > The current bindings seem to make the assumption that the > > > > two VPUs hardware blocks (G1 and G2) are only one set of > > > > registers. > > > > After implementing the VPU reset driver and G2 decoder driver > > > > it shows that all the VPUs are independent and don't need to > > > > know about the registers of the other blocks. > > > > Remove from the bindings the need to set all blocks register > > > > but keep reg-names property because removing it from the driver > > > > may affect other variants. > > > > > > > > Signed-off-by: Benjamin Gaignard > > > > --- > > > > version 2: > > > > - be more verbose about why I change the bindings > > > > Keep in mind that series comes after: > > > > https://www.spinics.net/lists/arm-kernel/msg875766.html > > > > without that review and ack it won't work > > > Better, but you've still mentioned nothing about breaking compatibility. > > > Why is that okay? > > Indeed, the commit description should be clearer about breaking compatibility. > > Because this reg-names wasn't used before for this variant so remove it > > won't change anything. > > It is the reset changes in the driver that break. The driver > previously got the 'ctrl' registers whether it went by name or index, > right? With an old DTB and a kernel with the changes (and vice-versa), > you'll have nothing to handle the VPU resets because the VPU reset > node doesn't exist. It could work if the default state is not held in > reset. > > At least the removal of 'ctrl' registers belongs in the reset changes series. > > Considering that FFMPEG patches that are required to support this driver are still floating around, and GStreamer's implementation is also still a bit under discussion, we are certain there aren't many upstreams users (leaving ChromiumOS aside which mostly care for Rockchip variants). So, given the driver is in staging, and that there aren't users of the i.MX8MQ G1 variant just yet, I think we are safe breaking the compatibility (and I'm not taking it lightly). It would be important to detect an old devicetree and do some pr_warn about the driver not supporting it. Thanks, Ezequiel
Re: [PATCH v2 4/9] media: uapi: Add a control for HANTRO driver
On Mon, 2021-02-22 at 17:28 +0100, Benjamin Gaignard wrote:
>
> Le 22/02/2021 à 17:16, John Cox a écrit :
> > > The HEVC HANTRO driver needs to know the number of bits to skip at
> > > the beginning of the slice header.
> > > That is a hardware specific requirement so create a dedicated control
> > > that this purpose.
> > >
> > > Signed-off-by: Benjamin Gaignard
> > > ---
> > > include/uapi/linux/hantro-v4l2-controls.h | 20
> > > include/uapi/linux/v4l2-controls.h | 5 +
> > > 2 files changed, 25 insertions(+)
> > > create mode 100644 include/uapi/linux/hantro-v4l2-controls.h
> > >
> > > diff --git a/include/uapi/linux/hantro-v4l2-controls.h
> > > b/include/uapi/linux/hantro-v4l2-controls.h
> > > new file mode 100644
> > > index ..30b1999b7af3
> > > --- /dev/null
> > > +++ b/include/uapi/linux/hantro-v4l2-controls.h
> > > @@ -0,0 +1,20 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> > > +
> > > +#ifndef __UAPI_HANTRO_V4L2_CONYTROLS_H__
> > > +#define __UAPI_HANTRO_V4L2_CONYTROLS_H__
> > > +
> > > +#include
> > > +#include
> > > +
> > > +#define V4L2_CID_HANTRO_HEVC_EXTRA_DECODE_PARAMS
> > > (V4L2_CID_USER_HANTRO_BASE + 0)
> > > +
> > > +/**
> > > + * struct hantro_hevc_extra_decode_params - extra decode parameters for
> > > hantro driver
> > > + * @hevc_hdr_skip_lenght: header first bits offset
> > > + */
> > > +struct hantro_hevc_extra_decode_params {
> > > + __u32 hevc_hdr_skip_lenght;
> > > + __u8padding[4];
> > > +};
> > Can you clarify how hevc_hdr_skip_length differs from
> > v4l2_ctrl_hevc_slice_params.data_bit_offset? At first sight they would
> > appear to be very similar.
>
> hevc_hdr_skip_length is the difference between the start positions of 2 nals.
> v4l2_ctrl_hevc_slice_params.data_bit_offset is the offset of the data in the
> nal.
>
I think the hardware is weird enough that we should have detailed
documentation to the exact expectation for this control, i.e. detailing
exactly what syntax elements userspace is expected to pass the distance.
Maybe documenting this somewhere in Documentation/.../media/something,
and then linking that in the kernel-doc comment for
hantro_hevc_extra_decode_params.
Thanks,
Ezequiel
Re: [PATCH] drm/panel: kd35t133: allow using non-continuous dsi clock
On Sat, 6 Feb 2021 at 10:50, Heiko Stuebner wrote:
>
> The panel is able to work when dsi clock is non-continuous, thus
> the system power consumption can be reduced using such feature.
>
> Add MIPI_DSI_CLOCK_NON_CONTINUOUS to panel's mode_flags.
>
> Also the flag actually becomes necessary after
> commit c6d94e37bdbb ("drm/bridge/synopsys: dsi: add support for
> non-continuous HS clock")
> and without it the panel only emits stripes instead of output.
>
Looks like c6d94e37bdbb landed in v5.10, so we should add:
Fixes: c6d94e37bdbb ("drm/bridge/synopsys: dsi: add support for
non-continuous HS clock")
Cc: sta...@vger.kernel.org # 5.10+
With this fix, my Odroid Go2 has display.
Tested-by: Ezequiel Garcia
Thanks,
Ezequiel
> Signed-off-by: Heiko Stuebner
> ---
> drivers/gpu/drm/panel/panel-elida-kd35t133.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/panel/panel-elida-kd35t133.c
> b/drivers/gpu/drm/panel/panel-elida-kd35t133.c
> index bc36aa3c1123..fe5ac3ef9018 100644
> --- a/drivers/gpu/drm/panel/panel-elida-kd35t133.c
> +++ b/drivers/gpu/drm/panel/panel-elida-kd35t133.c
> @@ -265,7 +265,8 @@ static int kd35t133_probe(struct mipi_dsi_device *dsi)
> dsi->lanes = 1;
> dsi->format = MIPI_DSI_FMT_RGB888;
> dsi->mode_flags = MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_BURST |
> - MIPI_DSI_MODE_LPM | MIPI_DSI_MODE_EOT_PACKET;
> + MIPI_DSI_MODE_LPM | MIPI_DSI_MODE_EOT_PACKET |
> + MIPI_DSI_CLOCK_NON_CONTINUOUS;
>
> drm_panel_init(>panel, >dev, _funcs,
>DRM_MODE_CONNECTOR_DSI);
> --
> 2.29.2
>
> ___
> dri-devel mailing list
> dri-de...@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH v1 13/18] media: hantro: Introduce G2/HEVC decoder
Hi Benjamin,
Before I review the implementation in detail,
there's one thing that looks suspicious.
On Wed, 2021-02-17 at 09:03 +0100, Benjamin Gaignard wrote:
> Implement all the logic to get G2 hardware decoding HEVC frames.
> It support up level 5.1 HEVC stream.
> It doesn't support yet 10 bits formats or scaling feature.
>
> Add HANTRO HEVC dedicated control to skip some bits at the beginning
> of the slice header. That is very specific to this hardware so can't
> go into uapi structures. Compute the needed value is complex and require
> information from the stream that only the userland knows so let it
> provide the correct value to the driver.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/Makefile | 2 +
> drivers/staging/media/hantro/hantro_drv.c | 41 ++
> .../staging/media/hantro/hantro_g2_hevc_dec.c | 637 ++
> drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++
> drivers/staging/media/hantro/hantro_hevc.c | 274
> drivers/staging/media/hantro/hantro_hw.h | 14 +
> 6 files changed, 1166 insertions(+)
> create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
>
> diff --git a/drivers/staging/media/hantro/Makefile
> b/drivers/staging/media/hantro/Makefile
> index 743ce08eb184..0357f1772267 100644
> --- a/drivers/staging/media/hantro/Makefile
> +++ b/drivers/staging/media/hantro/Makefile
> @@ -9,12 +9,14 @@ hantro-vpu-y += \
> hantro_h1_jpeg_enc.o \
> hantro_g1_h264_dec.o \
> hantro_g1_mpeg2_dec.o \
> + hantro_g2_hevc_dec.o \
> hantro_g1_vp8_dec.o \
> rk3399_vpu_hw_jpeg_enc.o \
> rk3399_vpu_hw_mpeg2_dec.o \
> rk3399_vpu_hw_vp8_dec.o \
> hantro_jpeg.o \
> hantro_h264.o \
> + hantro_hevc.o \
> hantro_mpeg2.o \
> hantro_vp8.o
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index e1443c394f62..d171fb80876a 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -280,6 +280,20 @@ static int hantro_jpeg_s_ctrl(struct v4l2_ctrl *ctrl)
> return 0;
> }
>
> +static int hantro_extra_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + const struct hantro_hevc_extra_decode_params *extra_params;
> + struct hantro_ctx *ctx;
> +
> + ctx = container_of(ctrl->handler,
> + struct hantro_ctx, ctrl_handler);
> + extra_params = >hevc_dec.ctrls.extra_params;
> +
> + memcpy((void *)extra_params, ctrl->p_new.p_u8, sizeof(extra_params));
> +
> + return 0;
> +}
> +
> static const struct v4l2_ctrl_ops hantro_ctrl_ops = {
> .try_ctrl = hantro_try_ctrl,
> };
> @@ -288,6 +302,10 @@ static const struct v4l2_ctrl_ops hantro_jpeg_ctrl_ops =
> {
> .s_ctrl = hantro_jpeg_s_ctrl,
> };
>
> +static const struct v4l2_ctrl_ops hantro_extra_ctrl_ops = {
> + .s_ctrl = hantro_extra_s_ctrl,
> +};
> +
> static const struct hantro_ctrl controls[] = {
> {
> .codec = HANTRO_JPEG_ENCODER,
> @@ -413,6 +431,29 @@ static const struct hantro_ctrl controls[] = {
> .cfg = {
> .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_HANTRO_HEVC_EXTRA_DECODE_PARAMS,
> + .name = "HANTRO extra decode params",
> + .type = V4L2_CTRL_TYPE_U8,
> + .min = 0,
> + .def = 0,
> + .max = 255,
> + .step = 1,
> + .dims = { sizeof(struct
> hantro_hevc_extra_decode_params) },
> + .ops = _extra_ctrl_ops,
> + },
> + }, {
> + .codec = HANTRO_JPEG_ENCODER | HANTRO_MPEG2_DECODER |
> + HANTRO_VP8_DECODER | HANTRO_H264_DECODER |
> + HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_USER_CLASS,
Are you sure you need to expose the V4L2_CID_USER_CLASS?
Maybe I'm missing something, but this looks odd.
Thanks,
Ezequiel
Re: [PATCH v1 18/18] arm64: dts: imx8mq: Add node to G2 hardware
On Wed, 2021-02-17 at 09:03 +0100, Benjamin Gaignard wrote:
> Split VPU node in two: one for G1 and one for G2 since they are
> different hardware blocks.
>
> Signed-off-by: Benjamin Gaignard
> ---
> arch/arm64/boot/dts/freescale/imx8mq.dtsi | 43 +--
> 1 file changed, 33 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> index d9d9efc8592d..3cab3f0b9131 100644
> --- a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
> @@ -1287,17 +1287,16 @@ vpu_reset: vpu-reset@3832 {
> #reset-cells = <1>;
> };
>
> - vpu: video-codec@3830 {
> + vpu_g1: video-codec@3830 {
> compatible = "nxp,imx8mq-vpu";
> - reg = <0x3830 0x1>,
> - <0x3831 0x1>;
> - reg-names = "g1", "g2";
> - interrupts = ,
> - ;
> - interrupt-names = "g1", "g2";
> + reg = <0x3830 0x1>;
> + reg-names = "g1";
> + interrupts = ;
> + interrupt-names = "g1";
> clocks = < IMX8MQ_CLK_VPU_G1_ROOT>,
> - < IMX8MQ_CLK_VPU_G2_ROOT>;
> - clock-names = "g1", "g2";
> + < IMX8MQ_CLK_VPU_G2_ROOT>,
> + < IMX8MQ_CLK_VPU_DEC_ROOT>;
> + clock-names = "g1", "g2", "bus";
How come the G1 block needs the G2 clock?
> assigned-clocks = < IMX8MQ_CLK_VPU_G1>,
> < IMX8MQ_CLK_VPU_G2>,
> < IMX8MQ_CLK_VPU_BUS>,
> @@ -1306,12 +1305,36 @@ vpu: video-codec@3830 {
> < IMX8MQ_VPU_PLL_OUT>,
> < IMX8MQ_SYS1_PLL_800M>,
> < IMX8MQ_VPU_PLL>;
> - assigned-clock-rates = <6>, <6>,
> + assigned-clock-rates = <6>, <3>,
> <8>, <0>;
> resets = <_reset IMX8MQ_RESET_VPU_RESET_G1>;
> power-domains = <_vpu>;
> };
>
> + vpu_g2: video-codec@3831 {
> + compatible = "nxp,imx8mq-vpu-g2";
> + reg = <0x3831 0x1>;
> + reg-names = "g2";
> + interrupts = ;
> + interrupt-names = "g2";
> + clocks = < IMX8MQ_CLK_VPU_G1_ROOT>,
> + < IMX8MQ_CLK_VPU_G2_ROOT>,
Ditto, the G2 block needs the G1 clock?
Thanks,
Ezequiel
Re: [PATCH v1 11/18] media: hantro: Add helper function for auxiliary buffers allocation
Hi Benjamin,
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Add helper functions to allocate and free auxiliary buffers.
> These buffers aren't for frames but are needed by the hardware
> to store scaling matrix, tiles size, border filters etc...
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/hantro.h | 24
> 1 file changed, 24 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index a9b80b2c9124..7f842edbc341 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -446,6 +446,30 @@ hantro_get_dec_buf(struct hantro_ctx *ctx, struct
> vb2_buffer *vb)
> return vb2_plane_vaddr(vb, 0);
> }
>
> +static inline int
> +hantro_aux_buf_alloc(struct hantro_dev *vpu,
> + struct hantro_aux_buf *buf, size_t size)
> +{
Can you add convert the dma_alloc_ calls in the driver,
and squash it in this patch?
I.e. hantro_h264_dec_init, hantro_vp8_dec_init, etc.
Thanks!
Ezequiel
> + buf->cpu = dma_alloc_coherent(vpu->dev, size, >dma, GFP_KERNEL);
> + if (!buf->cpu)
> + return -ENOMEM;
> +
> + buf->size = size;
> + return 0;
> +}
> +
> +static inline void
> +hantro_aux_buf_free(struct hantro_dev *vpu,
> + struct hantro_aux_buf *buf)
> +{
> + if (buf->cpu)
> + dma_free_coherent(vpu->dev, buf->size, buf->cpu, buf->dma);
> +
> + buf->cpu = NULL;
> + buf->dma = 0;
> + buf->size = 0;
> +}
> +
> void hantro_postproc_disable(struct hantro_ctx *ctx);
> void hantro_postproc_enable(struct hantro_ctx *ctx);
> void hantro_postproc_free(struct hantro_ctx *ctx);
Re: [PATCH v1 10/18] media: hantro: Add helper functions for buffer information
Hi Benjamin,
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Add help functions to retrieve buffer address and size.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/hantro.h | 16
> 1 file changed, 16 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index 2523c0d010df..a9b80b2c9124 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -430,6 +430,22 @@ hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct
> vb2_buffer *vb)
> return vb2_dma_contig_plane_dma_addr(vb, 0);
> }
>
> +static inline size_t
> +hantro_get_dec_buf_size(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> +{
> + if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
> + return ctx->postproc.dec_q[vb->index].size;
> + return vb2_plane_size(vb, 0);
> +}
> +
> +static inline void *
> +hantro_get_dec_buf(struct hantro_ctx *ctx, struct vb2_buffer *vb)
> +{
> + if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
> + return ctx->postproc.dec_q[vb->index].cpu;
> + return vb2_plane_vaddr(vb, 0);
> +}
> +
It may sound like a nitpick but I think you could just squash this
change where it's needed. That way it's easier to review and
see why this was added.
Thanks,
Ezequiel
Re: [PATCH v1 07/18] media: hantro: Add a field to distinguish the hardware versions
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Decoders hardware blocks could exist in multiple versions: add
> a field to distinguish them at runtime.
> Keep the default behavoir to be G1 hardware.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/hantro.h | 5 +
> drivers/staging/media/hantro/hantro_drv.c | 2 ++
> 2 files changed, 7 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index bde65231f22f..2a566dfc2fe3 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -36,6 +36,9 @@ struct hantro_codec_ops;
> #define HANTRO_H264_DECODERBIT(18)
> #define HANTRO_DECODERS0x
>
> +#define HANTRO_G1_REV 0x6731
> +#define HANTRO_G2_REV 0x6732
> +
> /**
> * struct hantro_irq - irq handler and name
> *
> @@ -170,6 +173,7 @@ hantro_vdev_to_func(struct video_device *vdev)
> * @enc_base: Mapped address of VPU encoder register for
> convenience.
> * @dec_base: Mapped address of VPU decoder register for
> convenience.
> * @ctrl_base: Mapped address of VPU control block.
> + * @core_hw_dec_revRuntime detected HW decoder core revision
> * @vpu_mutex: Mutex to synchronize V4L2 calls.
> * @irqlock: Spinlock to synchronize access to data structures
> * shared with interrupt handlers.
> @@ -189,6 +193,7 @@ struct hantro_dev {
> void __iomem *enc_base;
> void __iomem *dec_base;
> void __iomem *ctrl_base;
> + u32 core_hw_dec_rev;
>
> struct mutex vpu_mutex; /* video_device lock */
> spinlock_t irqlock;
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index 0570047c7fa0..e1443c394f62 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -840,6 +840,8 @@ static int hantro_probe(struct platform_device *pdev)
> }
> vpu->enc_base = vpu->reg_bases[0] + vpu->variant->enc_offset;
> vpu->dec_base = vpu->reg_bases[0] + vpu->variant->dec_offset;
> + /* by default decoder is G1 */
> + vpu->core_hw_dec_rev = HANTRO_G1_REV;
>
What's the use of this field? Can't we simply rely on the compatible string?
Thanks,
Ezequiel
Re: [PATCH v1 15/18] media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control
On Wed, 2021-02-17 at 09:03 +0100, Benjamin Gaignard wrote:
> Make sure that V4L2_PIX_FMT_HEVC_SLICE is correctly handle by v4l2
> of the driver.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/hantro_v4l2.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_v4l2.c
> b/drivers/staging/media/hantro/hantro_v4l2.c
> index 1bc118e375a1..1608a3f78eaa 100644
> --- a/drivers/staging/media/hantro/hantro_v4l2.c
> +++ b/drivers/staging/media/hantro/hantro_v4l2.c
> @@ -232,7 +232,7 @@ static int vidioc_g_fmt_cap_mplane(struct file *file,
> void *priv,
> return 0;
> }
>
> -static int hantro_try_fmt(const struct hantro_ctx *ctx,
> +static int hantro_try_fmt(struct hantro_ctx *ctx,
Spurious change?
> struct v4l2_pix_format_mplane *pix_mp,
> enum v4l2_buf_type type)
> {
> @@ -390,6 +390,7 @@ hantro_update_requires_request(struct hantro_ctx *ctx,
> u32 fourcc)
> case V4L2_PIX_FMT_MPEG2_SLICE:
> case V4L2_PIX_FMT_VP8_FRAME:
> case V4L2_PIX_FMT_H264_SLICE:
> + case V4L2_PIX_FMT_HEVC_SLICE:
> ctx->fh.m2m_ctx->out_q_ctx.q.requires_requests = true;
> break;
> default:
Otherwise looks good.
Thanks,
Ezequiel
Re: [PATCH v1 06/18] media: hantro: Make sure that ctx->codex_ops is set
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Do not try to call ctx->codec_ops->done if ctx->codec_ops is not set.
>
If codec_ops is not set for a codec variant,
things will go south really fast. See hantro_start_streaming
for instance.
I think you can just drop this patch.
Thanks,
Ezequiel
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/hantro_drv.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index 0d58209fc55c..0570047c7fa0 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -93,7 +93,8 @@ void hantro_irq_done(struct hantro_dev *vpu,
> * and will take care of finishing the job.
> */
> if (cancel_delayed_work(>watchdog_work)) {
> - if (result == VB2_BUF_STATE_DONE && ctx->codec_ops->done)
> + if (result == VB2_BUF_STATE_DONE &&
> + ctx->codec_ops && ctx->codec_ops->done)
> ctx->codec_ops->done(ctx);
> hantro_job_finish(vpu, ctx, result);
> }
Re: [PATCH v1 05/18] media: controls: Add control for HEVC codec
Hi Benjamin,
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Add HEVC decode params and scaling matrix controls.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 36 ++--
> 1 file changed, 29 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 016cf6204cbb..5e45333fd862 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -1028,6 +1028,8 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC
> Sequence Parameter Set";
> case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC
> Picture Parameter Set";
> case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC
> Slice Parameters";
> + case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:return "HEVC
> Decode Parameters";
> + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC
> Scaling Matrix";
I would move all the SCALING_MATRIX changes to their own patches.
> case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC
> Decode Mode";
> case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC
> Start Code";
>
> @@ -1482,6 +1484,12 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
> v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
> break;
> + case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
> + *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
> + break;
> case V4L2_CID_UNIT_CELL_SIZE:
> *type = V4L2_CTRL_TYPE_AREA;
> *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> @@ -1833,6 +1841,7 @@ static int std_validate_compound(const struct v4l2_ctrl
> *ctrl, u32 idx,
> struct v4l2_ctrl_hevc_sps *p_hevc_sps;
> struct v4l2_ctrl_hevc_pps *p_hevc_pps;
> struct v4l2_ctrl_hevc_slice_params *p_hevc_slice_params;
> + struct v4l2_ctrl_hevc_decode_params *p_hevc_decode_params;
> struct v4l2_area *area;
> void *p = ptr.p + idx * ctrl->elem_size;
> unsigned int i;
> @@ -2108,26 +2117,33 @@ static int std_validate_compound(const struct
> v4l2_ctrl *ctrl, u32 idx,
> zero_padding(*p_hevc_pps);
> break;
>
> - case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
> - p_hevc_slice_params = p;
> + case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
> + p_hevc_decode_params = p;
>
> - if (p_hevc_slice_params->num_active_dpb_entries >
> + if (p_hevc_decode_params->num_active_dpb_entries >
I suspect this change should be squashed with the patch that moves
num_active_dpb_entries from the slice control, or otherwise
this code won't compile.
> V4L2_HEVC_DPB_ENTRIES_NUM_MAX)
> return -EINVAL;
>
> - zero_padding(p_hevc_slice_params->pred_weight_table);
> -
> - for (i = 0; i < p_hevc_slice_params->num_active_dpb_entries;
> + for (i = 0; i < p_hevc_decode_params->num_active_dpb_entries;
> i++) {
> struct v4l2_hevc_dpb_entry *dpb_entry =
> - _hevc_slice_params->dpb[i];
> + _hevc_decode_params->dpb[i];
>
Ditto.
Thanks,
Ezequiel
Re: [PATCH v1 04/18] media: hevc: add structures for hevc codec
Hi Benjamin,
Thanks a lot for picking this up.
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Define additional structures to be used by HEVC codecs.
> This will allow to provide the needed information to the
> hardware block.
> Adapt Cedrus driver to use these new structures
>
So this commit description needs some more information.
See commit d9358563179a7f01f9020ebbe201c7e54ba3af48
Author: Ezequiel Garcia
Date: Tue Aug 25 05:52:36 2020 +0200
media: uapi: h264: Clean slice invariants syntax elements
which explains why it's OK to move some fields out of the slice control,
and which also explains which fields can be moved.
See 7.4.7.1 General slice segment header semantics, in the H.265 ITU
specification.
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/sunxi/cedrus/cedrus.c | 6 +++
> drivers/staging/media/sunxi/cedrus/cedrus.h | 1 +
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +
> .../staging/media/sunxi/cedrus/cedrus_h265.c | 6 ++-
> include/media/hevc-ctrls.h | 52 ---
> 5 files changed, 57 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c
> b/drivers/staging/media/sunxi/cedrus/cedrus.c
> index 7bd9291c8d5f..4cd3cab1a257 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
> @@ -151,6 +151,12 @@ static const struct cedrus_control cedrus_controls[] = {
> },
> .codec = CEDRUS_CODEC_VP8,
> },
> + {
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> + },
> + .codec = CEDRUS_CODEC_H265,
> + },
> };
>
> #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls)
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h
> b/drivers/staging/media/sunxi/cedrus/cedrus.h
> index 251a6a660351..c18b7f7a2820 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.h
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h
> @@ -76,6 +76,7 @@ struct cedrus_h265_run {
> const struct v4l2_ctrl_hevc_sps *sps;
> const struct v4l2_ctrl_hevc_pps *pps;
> const struct v4l2_ctrl_hevc_slice_params*slice_params;
> + const struct v4l2_ctrl_hevc_decode_params *decode_params;
> };
>
> struct cedrus_vp8_run {
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> index a9090daf626a..cd821f417a14 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> @@ -68,6 +68,8 @@ void cedrus_device_run(void *priv)
> V4L2_CID_MPEG_VIDEO_HEVC_PPS);
> run.h265.slice_params = cedrus_find_control_data(ctx,
> V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS);
> + run.h265.decode_params = cedrus_find_control_data(ctx,
> + V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
> break;
>
> case V4L2_PIX_FMT_VP8_FRAME:
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> index ce497d0197df..dce5db6be13a 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> @@ -245,6 +245,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
> const struct v4l2_ctrl_hevc_sps *sps;
> const struct v4l2_ctrl_hevc_pps *pps;
> const struct v4l2_ctrl_hevc_slice_params *slice_params;
> + const struct v4l2_ctrl_hevc_decode_params *decode_params;
> const struct v4l2_hevc_pred_weight_table *pred_weight_table;
> dma_addr_t src_buf_addr;
> dma_addr_t src_buf_end_addr;
> @@ -256,6 +257,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
> sps = run->h265.sps;
> pps = run->h265.pps;
> slice_params = run->h265.slice_params;
> + decode_params = run->h265.decode_params;
> pred_weight_table = _params->pred_weight_table;
>
> /* MV column buffer size and allocation. */
> @@ -487,7 +489,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>
> reg =
> VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_TC_OFFSET_DIV2(slice_params->slice_tc_offset_div2)
> |
>
> VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_BETA_OFFSET_DIV2(slice_params->slice_beta_offset_div2)
> |
> -
> VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_POC_BIG
Re: [PATCH v1 03/18] arm64: dts: imx8mq-evk: add reserve memory node for CMA region
Hi Benjamin,
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Define allocation range for the default CMA region.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
Despite it seems like I signed-off this one...
> Signed-off-by: Adrian Ratiu
> ---
> arch/arm64/boot/dts/freescale/imx8mq-evk.dts | 15 +++
> 1 file changed, 15 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
> b/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
> index 85b045253a0e..047dfd4a1ffd 100644
> --- a/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
> +++ b/arch/arm64/boot/dts/freescale/imx8mq-evk.dts
> @@ -21,6 +21,21 @@ memory@4000 {
> reg = <0x 0x4000 0 0xc000>;
> };
>
>
> + resmem: reserved-memory {
> + #address-cells = <2>;
> + #size-cells = <2>;
> + ranges;
> +
> + /* global autoconfigured region for contiguous allocations */
> + linux,cma {
> + compatible = "shared-dma-pool";
> + reusable;
> + size = <0 0x3c00>;
> + alloc-ranges = <0 0x4000 0 0x4000>;
> + linux,cma-default;
> + };
... I'm not a fan of the change :)
Hopefully someone from NXP can provide some insight here?
If it's absolutely needed for the VPU, then I guess it should
be 1) very well documented and 2) moved to the top-lovel dtsi.
But if we can drop it, that'd be nicer.
Thanks,
Ezequiel
Re: [PATCH v1 02/18] media: hantro: Define HEVC codec profiles and supported features
Hello Benjamin,
Thanks a lot for the patch.
On Wed, 2021-02-17 at 09:02 +0100, Benjamin Gaignard wrote:
> Define which HEVC profiles (up to level 5.1) and features
> (no scaling, no 10 bits) are supported by the driver.
>
> Signed-off-by: Benjamin Gaignard
> Signed-off-by: Ezequiel Garcia
> Signed-off-by: Adrian Ratiu
> ---
> drivers/staging/media/hantro/hantro.h | 2 +
> drivers/staging/media/hantro/hantro_drv.c | 63 +++
> 2 files changed, 65 insertions(+)
>
> diff --git a/drivers/staging/media/hantro/hantro.h
> b/drivers/staging/media/hantro/hantro.h
> index 65f9f7ea7dcf..bde65231f22f 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -99,6 +99,7 @@ struct hantro_variant {
> * @HANTRO_MODE_H264_DEC: H264 decoder.
> * @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
> * @HANTRO_MODE_VP8_DEC: VP8 decoder.
> + * @HANTRO_MODE_HEVC_DEC: HEVC decoder.
> */
> enum hantro_codec_mode {
> HANTRO_MODE_NONE = -1,
> @@ -106,6 +107,7 @@ enum hantro_codec_mode {
> HANTRO_MODE_H264_DEC,
> HANTRO_MODE_MPEG2_DEC,
> HANTRO_MODE_VP8_DEC,
> + HANTRO_MODE_HEVC_DEC,
> };
>
> /*
> diff --git a/drivers/staging/media/hantro/hantro_drv.c
> b/drivers/staging/media/hantro/hantro_drv.c
> index e5f200e64993..0d58209fc55c 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -243,6 +243,18 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
> + } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
> + const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
> +
> + if (sps->bit_depth_luma_minus8 !=
> sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
Note that you are rejecting scaling lists here, butx you are adding the
SCALING_MATRIX control, and there's some support in patch 13/18
(see prepare_scaling_list_buffer).
Either drop all the scaling list support for now (and its extra controls)
or allow it properly (and make sure it's working as expected).
Thanks,
Ezequiel
> + /* No scaling support */
> + return -EINVAL;
> }
> return 0;
> }
> @@ -349,6 +361,57 @@ static const struct hantro_ctrl controls[] = {
> .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> }
> }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
> + .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
> + .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
> + .min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> + .max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
> + .def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
> + .min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
> + .max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> + .cfg = {
> + .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
> + .ops = _ctrl_ops,
> + },
> + }, {
> + .codec = HANTRO_HEVC_DECODER,
> +
Re: linux-next: build warning after merge of the v4l-dvb tree
On Mon, 2021-02-08 at 18:40 +0100, Mauro Carvalho Chehab wrote:
> Em Mon, 08 Feb 2021 13:57:56 -0300
> Ezequiel Garcia escreveu:
>
> > On Mon, 2021-02-08 at 18:46 +0200, Sakari Ailus wrote:
> > > Hi Ezequiel,
> > >
> > > Thanks for addressing this.
> > >
> > > On Mon, Feb 08, 2021 at 01:42:21PM -0300, Ezequiel Garcia wrote:
> > > > Hi Stephen,
> > > >
> > > > On Mon, 2021-02-08 at 23:37 +1100, Stephen Rothwell wrote:
> > > > > Hi all,
> > > > >
> > > > > After merging the v4l-dvb tree, today's linux-next build (htmldocs)
> > > > > produced this warning:
> > > > >
> > > > > include/media/v4l2-async.h:178: warning: expecting prototype for
> > > > > v4l2_async_notifier_add_fwnode_subdev(). Prototype was for
> > > > > __v4l2_async_notifier_add_fwnode_subdev() instead
> > > > > include/media/v4l2-async.h:207: warning: expecting prototype for
> > > > > v4l2_async_notifier_add_fwnode_remote_subdev(). Prototype was for
> > > > > __v4l2_async_notifier_add_fwnode_remote_subdev() instead
> > > > > include/media/v4l2-async.h:230: warning: expecting prototype for
> > > > > v4l2_async_notifier_add_i2c_subdev(). Prototype was for
> > > > > __v4l2_async_notifier_add_i2c_subdev() instead
> > > > >
> > > > > Maybe introduced by commit
> > > > >
> > > > > c1cc23625062 ("media: v4l2-async: Discourage use of
> > > > > v4l2_async_notifier_add_subdev")
> > > > >
> > > >
> > > > Thanks for spotting this. Should be fixed by:
> > > >
> > > > diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h
> > > > index 6f22daa6f067..3785445282fc 100644
> > > > --- a/include/media/v4l2-async.h
> > > > +++ b/include/media/v4l2-async.h
> > > > @@ -157,7 +157,7 @@ int __v4l2_async_notifier_add_subdev(struct
> > > > v4l2_async_notifier *notifier,
> > > > struct v4l2_async_subdev *asd);
> > > >
> > > > /**
> > > > - * v4l2_async_notifier_add_fwnode_subdev - Allocate and add a fwnode
> > > > async
> > > > + * __v4l2_async_notifier_add_fwnode_subdev - Allocate and add a fwnode
> > > > async
> > >
> > > The problem with the approach is that this no longer documents the API
> > > that
> > > drivers are intended to use, but the intermediate one.
>
> Yep. the better would be to keep documenting what will be used.
>
Is there a way to silence/ignore the warning for a specific function(s)?
Ezequiel
Re: linux-next: build warning after merge of the v4l-dvb tree
On Mon, 2021-02-08 at 18:46 +0200, Sakari Ailus wrote:
> Hi Ezequiel,
>
> Thanks for addressing this.
>
> On Mon, Feb 08, 2021 at 01:42:21PM -0300, Ezequiel Garcia wrote:
> > Hi Stephen,
> >
> > On Mon, 2021-02-08 at 23:37 +1100, Stephen Rothwell wrote:
> > > Hi all,
> > >
> > > After merging the v4l-dvb tree, today's linux-next build (htmldocs)
> > > produced this warning:
> > >
> > > include/media/v4l2-async.h:178: warning: expecting prototype for
> > > v4l2_async_notifier_add_fwnode_subdev(). Prototype was for
> > > __v4l2_async_notifier_add_fwnode_subdev() instead
> > > include/media/v4l2-async.h:207: warning: expecting prototype for
> > > v4l2_async_notifier_add_fwnode_remote_subdev(). Prototype was for
> > > __v4l2_async_notifier_add_fwnode_remote_subdev() instead
> > > include/media/v4l2-async.h:230: warning: expecting prototype for
> > > v4l2_async_notifier_add_i2c_subdev(). Prototype was for
> > > __v4l2_async_notifier_add_i2c_subdev() instead
> > >
> > > Maybe introduced by commit
> > >
> > > c1cc23625062 ("media: v4l2-async: Discourage use of
> > > v4l2_async_notifier_add_subdev")
> > >
> >
> > Thanks for spotting this. Should be fixed by:
> >
> > diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h
> > index 6f22daa6f067..3785445282fc 100644
> > --- a/include/media/v4l2-async.h
> > +++ b/include/media/v4l2-async.h
> > @@ -157,7 +157,7 @@ int __v4l2_async_notifier_add_subdev(struct
> > v4l2_async_notifier *notifier,
> > struct v4l2_async_subdev *asd);
> >
> > /**
> > - * v4l2_async_notifier_add_fwnode_subdev - Allocate and add a fwnode async
> > + * __v4l2_async_notifier_add_fwnode_subdev - Allocate and add a fwnode
> > async
>
> The problem with the approach is that this no longer documents the API that
> drivers are intended to use, but the intermediate one. I guess fixing
> this properly could require changes to kerneldoc so I have no objections to
> the approach.
>
Right, but do we have any other solution here?
> > * subdev to the notifier's master asd_list.
> > *
> > * @notifier: pointer to v4l2_async_notifier
> > @@ -181,7 +181,7 @@ __v4l2_async_notifier_add_fwnode_subdev(struct
> > v4l2_async_notifier *notifier,
> > sizeof(__type)))
> >
> > /**
> > - * v4l2_async_notifier_add_fwnode_remote_subdev - Allocate and add a fwnode
> > + * __v4l2_async_notifier_add_fwnode_remote_subdev - Allocate and add a
> > fwnode
> > * remote async subdev to the
> > * notifier's master
> > asd_list.
> > *
> > @@ -210,7 +210,7 @@ __v4l2_async_notifier_add_fwnode_remote_subdev(struct
> > v4l2_async_notifier *notif
> > sizeof(__type)))
> >
> > /**
> > - * v4l2_async_notifier_add_i2c_subdev - Allocate and add an i2c async
> > + * __v4l2_async_notifier_add_i2c_subdev - Allocate and add an i2c async
> > * subdev to the notifier's master asd_list.
> > *
> > * @notifier: pointer to v4l2_async_notifier
> > @@ -228,7 +228,7 @@ struct v4l2_async_subdev *
> > __v4l2_async_notifier_add_i2c_subdev(struct v4l2_async_notifier *notifier,
> > int adapter_id, unsigned short address,
> > unsigned int asd_struct_size);
> > -#define v4l2_async_notifier_add_i2c_subdev(__notifier, __adap, __addr,
> > __type) \
> > +#define v4l2_async_notifier_i2c(__notifier, __adap, __addr, __type)\
>
> I guess this change was not intentional?
>
Indeed :)
Thanks,
Ezequiel
Re: linux-next: build warning after merge of the v4l-dvb tree
Hi Stephen,
On Mon, 2021-02-08 at 23:37 +1100, Stephen Rothwell wrote:
> Hi all,
>
> After merging the v4l-dvb tree, today's linux-next build (htmldocs)
> produced this warning:
>
> include/media/v4l2-async.h:178: warning: expecting prototype for
> v4l2_async_notifier_add_fwnode_subdev(). Prototype was for
> __v4l2_async_notifier_add_fwnode_subdev() instead
> include/media/v4l2-async.h:207: warning: expecting prototype for
> v4l2_async_notifier_add_fwnode_remote_subdev(). Prototype was for
> __v4l2_async_notifier_add_fwnode_remote_subdev() instead
> include/media/v4l2-async.h:230: warning: expecting prototype for
> v4l2_async_notifier_add_i2c_subdev(). Prototype was for
> __v4l2_async_notifier_add_i2c_subdev() instead
>
> Maybe introduced by commit
>
> c1cc23625062 ("media: v4l2-async: Discourage use of
> v4l2_async_notifier_add_subdev")
>
Thanks for spotting this. Should be fixed by:
diff --git a/include/media/v4l2-async.h b/include/media/v4l2-async.h
index 6f22daa6f067..3785445282fc 100644
--- a/include/media/v4l2-async.h
+++ b/include/media/v4l2-async.h
@@ -157,7 +157,7 @@ int __v4l2_async_notifier_add_subdev(struct
v4l2_async_notifier *notifier,
struct v4l2_async_subdev *asd);
/**
- * v4l2_async_notifier_add_fwnode_subdev - Allocate and add a fwnode async
+ * __v4l2_async_notifier_add_fwnode_subdev - Allocate and add a fwnode async
* subdev to the notifier's master asd_list.
*
* @notifier: pointer to v4l2_async_notifier
@@ -181,7 +181,7 @@ __v4l2_async_notifier_add_fwnode_subdev(struct
v4l2_async_notifier *notifier,
sizeof(__type)))
/**
- * v4l2_async_notifier_add_fwnode_remote_subdev - Allocate and add a fwnode
+ * __v4l2_async_notifier_add_fwnode_remote_subdev - Allocate and add a fwnode
* remote async subdev to the
* notifier's master asd_list.
*
@@ -210,7 +210,7 @@ __v4l2_async_notifier_add_fwnode_remote_subdev(struct
v4l2_async_notifier *notif
sizeof(__type)))
/**
- * v4l2_async_notifier_add_i2c_subdev - Allocate and add an i2c async
+ * __v4l2_async_notifier_add_i2c_subdev - Allocate and add an i2c async
* subdev to the notifier's master asd_list.
*
* @notifier: pointer to v4l2_async_notifier
@@ -228,7 +228,7 @@ struct v4l2_async_subdev *
__v4l2_async_notifier_add_i2c_subdev(struct v4l2_async_notifier *notifier,
int adapter_id, unsigned short address,
unsigned int asd_struct_size);
-#define v4l2_async_notifier_add_i2c_subdev(__notifier, __adap, __addr, __type)
\
+#define v4l2_async_notifier_i2c(__notifier, __adap, __addr, __type)\
((__type *)__v4l2_async_notifier_add_i2c_subdev(__notifier, __adap, __addr,
\
sizeof(__type)))
I'll send a fix for Mauro.
Best regards,
Ezequiel
