Re: [PATCH v3 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding
On Tue, Mar 16, 2021 at 12:21 AM Nicolas Dufresne wrote: > > Le lundi 15 mars 2021 à 20:28 +0900, Alexandre Courbot a écrit : > > Hi Ezequiel, > > > > On Thu, Mar 4, 2021 at 6:47 AM Ezequiel Garcia > > wrote: > > > > > > Hi Alex, > > > > > > Thanks for the patch. > > > > > > On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot > > > wrote: > > > > > > > > From: Yunfei Dong > > > > > > > > Add support for H.264 decoding using the stateless API, as supported by > > > > MT8183. This support takes advantage of the V4L2 H.264 reference list > > > > builders. > > > > > > > > Signed-off-by: Yunfei Dong > > > > [acourbot: refactor, cleanup and split] > > > > Co-developed-by: Alexandre Courbot > > > > Signed-off-by: Alexandre Courbot > > > > --- > > > > drivers/media/platform/Kconfig| 1 + > > > > drivers/media/platform/mtk-vcodec/Makefile| 1 + > > > > .../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++ > > > > .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + > > > > .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + > > > > 5 files changed, 813 insertions(+) > > > > create mode 100644 drivers/media/platform/mtk- > > > > vcodec/vdec/vdec_h264_req_if.c > > > > > > > > diff --git a/drivers/media/platform/Kconfig > > > > b/drivers/media/platform/Kconfig > > > > index fd1831e97b22..c27db5643712 100644 > > > > --- a/drivers/media/platform/Kconfig > > > > +++ b/drivers/media/platform/Kconfig > > > > @@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC > > > > select V4L2_MEM2MEM_DEV > > > > select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU > > > > select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP > > > > + select V4L2_H264 > > > > help > > > > Mediatek video codec driver provides HW capability to > > > > encode and decode in a range of video formats on MT8173 > > > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile > > > > b/drivers/media/platform/mtk-vcodec/Makefile > > > > index 4ba93d838ab6..ca8e9e7a9c4e 100644 > > > > --- a/drivers/media/platform/mtk-vcodec/Makefile > > > > +++ b/drivers/media/platform/mtk-vcodec/Makefile > > > > @@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o > > > > \ > > > > mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ > > > > vdec/vdec_vp8_if.o \ > > > > vdec/vdec_vp9_if.o \ > > > > + vdec/vdec_h264_req_if.o \ > > > > mtk_vcodec_dec_drv.o \ > > > > vdec_drv_if.o \ > > > > vdec_vpu_if.o \ > > > > diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > new file mode 100644 > > > > index ..2fbbfbbcfbec > > > > --- /dev/null > > > > +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > @@ -0,0 +1,807 @@ > > > > +// SPDX-License-Identifier: GPL-2.0 > > > > + > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > + > > > > +#include "../vdec_drv_if.h" > > > > +#include "../mtk_vcodec_util.h" > > > > +#include "../mtk_vcodec_dec.h" > > > > +#include "../mtk_vcodec_intr.h" > > > > +#include "../vdec_vpu_if.h" > > > > +#include "../vdec_drv_base.h" > > > > + > > > > +#define NAL_NON_IDR_SLICE 0x01 > > > > +#define NAL_IDR_SLICE 0x05 > > > > +#define NAL_H264_PPS 0x08 > > > > > > Not used? > > > > > > > +#define NAL_TYPE(value)((value) & 0x1F) > > > > + > > > > > > I believe you may not need the NAL type. > > > > True, removed this block of defines. > > > > > > > > > +#define BUF_PREDICTION_SZ (64 * 4096) > > > > +#define MB_UNIT_LEN
Re: [PATCH v3 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding
On Tue, Mar 16, 2021 at 7:08 AM Ezequiel Garcia wrote: > > Hi Alex, > > On Mon, 15 Mar 2021 at 08:28, Alexandre Courbot wrote: > > > > Hi Ezequiel, > > > > On Thu, Mar 4, 2021 at 6:47 AM Ezequiel Garcia > > wrote: > > > > > > Hi Alex, > > > > > > Thanks for the patch. > > > > > > On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot > > > wrote: > > > > > > > > From: Yunfei Dong > > > > > > > > Add support for H.264 decoding using the stateless API, as supported by > > > > MT8183. This support takes advantage of the V4L2 H.264 reference list > > > > builders. > > > > > > > > Signed-off-by: Yunfei Dong > > > > [acourbot: refactor, cleanup and split] > > > > Co-developed-by: Alexandre Courbot > > > > Signed-off-by: Alexandre Courbot > > > > --- > > > > drivers/media/platform/Kconfig| 1 + > > > > drivers/media/platform/mtk-vcodec/Makefile| 1 + > > > > .../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++ > > > > .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + > > > > .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + > > > > 5 files changed, 813 insertions(+) > > > > create mode 100644 > > > > drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > > > > > diff --git a/drivers/media/platform/Kconfig > > > > b/drivers/media/platform/Kconfig > > > > index fd1831e97b22..c27db5643712 100644 > > > > --- a/drivers/media/platform/Kconfig > > > > +++ b/drivers/media/platform/Kconfig > > > > @@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC > > > > select V4L2_MEM2MEM_DEV > > > > select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU > > > > select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP > > > > + select V4L2_H264 > > > > help > > > > Mediatek video codec driver provides HW capability to > > > > encode and decode in a range of video formats on MT8173 > > > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile > > > > b/drivers/media/platform/mtk-vcodec/Makefile > > > > index 4ba93d838ab6..ca8e9e7a9c4e 100644 > > > > --- a/drivers/media/platform/mtk-vcodec/Makefile > > > > +++ b/drivers/media/platform/mtk-vcodec/Makefile > > > > @@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o > > > > \ > > > > mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ > > > > vdec/vdec_vp8_if.o \ > > > > vdec/vdec_vp9_if.o \ > > > > + vdec/vdec_h264_req_if.o \ > > > > mtk_vcodec_dec_drv.o \ > > > > vdec_drv_if.o \ > > > > vdec_vpu_if.o \ > > > > diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > new file mode 100644 > > > > index ..2fbbfbbcfbec > > > > --- /dev/null > > > > +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > > > @@ -0,0 +1,807 @@ > > > > +// SPDX-License-Identifier: GPL-2.0 > > > > + > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > +#include > > > > + > > > > +#include "../vdec_drv_if.h" > > > > +#include "../mtk_vcodec_util.h" > > > > +#include "../mtk_vcodec_dec.h" > > > > +#include "../mtk_vcodec_intr.h" > > > > +#include "../vdec_vpu_if.h" > > > > +#include "../vdec_drv_base.h" > > > > + > > > > +#define NAL_NON_IDR_SLICE 0x01 > > > > +#define NAL_IDR_SLICE 0x05 > > > > +#define NAL_H264_PPS 0x08 > > > > > > Not used? > > > > > > > +#define NAL_TYPE(value)((value) & 0x1F) > > > > + > > > > > > I believe you may not need the NAL type. > > > > True, removed this block of defines. > > > > > > > > > +#define BUF_PREDICTION_SZ (64 * 4096) > > > > +#define MB_UN
Re: [PATCH v3 05/15] media: mtk-vcodec: vdec: support stateless API
On Tue, Mar 16, 2021 at 6:45 AM Ezequiel Garcia
wrote:
>
> Hi Alexandre,
>
> On Mon, 15 Mar 2021 at 08:28, Alexandre Courbot wrote:
> >
> > Hi Ezequiel, thanks for the feedback!
> >
> > On Thu, Mar 4, 2021 at 6:30 AM Ezequiel Garcia
> > wrote:
> > >
> > > Hello Alex,
> > >
> > > Thanks for the patch.
> > >
> > > On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot
> > > wrote:
> > > >
> > > > From: Yunfei Dong
> > > >
> > > > Support the stateless codec API that will be used by MT8183.
> > > >
> > > > Signed-off-by: Yunfei Dong
> > > > [acourbot: refactor, cleanup and split]
> > > > Co-developed-by: Alexandre Courbot
> > > > Signed-off-by: Alexandre Courbot
> > > > ---
> > > > drivers/media/platform/mtk-vcodec/Makefile| 1 +
> > > > .../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 ++-
> > > > .../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +-
> > > > .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 ++
> > > > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 3 +
> > > > 5 files changed, 503 insertions(+), 3 deletions(-)
> > > > create mode 100644
> > > > drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
> > > >
> > > [..]
> > >
> > > > +
> > > > +static const struct mtk_stateless_control mtk_stateless_controls[] = {
> > > > + {
> > > > + .cfg = {
> > > > + .id = V4L2_CID_STATELESS_H264_SPS,
> > > > + },
> > > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > > + .needed_in_request = true,
> > >
> > > This "needed_in_request" is not really required, as controls
> > > are not volatile, and their value is stored per-context (per-fd).
> > >
> > > It's perfectly valid for an application to pass the SPS control
> > > at the beginning of the sequence, and then omit it
> > > in further requests.
> >
> > If I understand how v4l2_ctrl_request_hdl_ctrl_find() works with
> > requests, this boolean only checks that the control has been provided
> > at least once, and not that it is provided with every request. Without
> > it we could send a frame to the firmware without e.g. setting an SPS,
> > which would be a problem.
> >
>
> As Nicolas points out, in V4L2 controls have an initial value,
> so no control can be unset.
I see. So I guess the expectation is that failure will occur later as
the firmware reports it cannot decode properly (or returns a corrupted
frame). Thanks for the precision.
>
> > >
> > > > + },
> > > > + {
> > > > + .cfg = {
> > > > + .id = V4L2_CID_STATELESS_H264_PPS,
> > > > + },
> > > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > > + .needed_in_request = true,
> > > > + },
> > > > + {
> > > > + .cfg = {
> > > > + .id = V4L2_CID_STATELESS_H264_SCALING_MATRIX,
> > > > + },
> > > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > > + .needed_in_request = true,
> > > > + },
> > > > + {
> > > > + .cfg = {
> > > > + .id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
> > > > + },
> > > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > > + .needed_in_request = true,
> > > > + },
> > > > + {
> > > > + .cfg = {
> > > > + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > > > + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> > > > + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > > > + .menu_skip_mask =
> > > > +
> > > > BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
> > > > +
> > > > BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
> > > > + },
> > > > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > > > + },
> > &g
Re: [PATCH v3 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding
Hi Ezequiel,
On Thu, Mar 4, 2021 at 6:47 AM Ezequiel Garcia
wrote:
>
> Hi Alex,
>
> Thanks for the patch.
>
> On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot wrote:
> >
> > From: Yunfei Dong
> >
> > Add support for H.264 decoding using the stateless API, as supported by
> > MT8183. This support takes advantage of the V4L2 H.264 reference list
> > builders.
> >
> > Signed-off-by: Yunfei Dong
> > [acourbot: refactor, cleanup and split]
> > Co-developed-by: Alexandre Courbot
> > Signed-off-by: Alexandre Courbot
> > ---
> > drivers/media/platform/Kconfig| 1 +
> > drivers/media/platform/mtk-vcodec/Makefile| 1 +
> > .../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++
> > .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 +
> > .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 +
> > 5 files changed, 813 insertions(+)
> > create mode 100644
> > drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index fd1831e97b22..c27db5643712 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC
> > select V4L2_MEM2MEM_DEV
> > select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
> > select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
> > + select V4L2_H264
> > help
> > Mediatek video codec driver provides HW capability to
> > encode and decode in a range of video formats on MT8173
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile
> > b/drivers/media/platform/mtk-vcodec/Makefile
> > index 4ba93d838ab6..ca8e9e7a9c4e 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
> > mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> > vdec/vdec_vp8_if.o \
> > vdec/vdec_vp9_if.o \
> > + vdec/vdec_h264_req_if.o \
> > mtk_vcodec_dec_drv.o \
> > vdec_drv_if.o \
> > vdec_vpu_if.o \
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > new file mode 100644
> > index ..2fbbfbbcfbec
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
> > @@ -0,0 +1,807 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "../vdec_drv_if.h"
> > +#include "../mtk_vcodec_util.h"
> > +#include "../mtk_vcodec_dec.h"
> > +#include "../mtk_vcodec_intr.h"
> > +#include "../vdec_vpu_if.h"
> > +#include "../vdec_drv_base.h"
> > +
> > +#define NAL_NON_IDR_SLICE 0x01
> > +#define NAL_IDR_SLICE 0x05
> > +#define NAL_H264_PPS 0x08
>
> Not used?
>
> > +#define NAL_TYPE(value)((value) & 0x1F)
> > +
>
> I believe you may not need the NAL type.
True, removed this block of defines.
>
> > +#define BUF_PREDICTION_SZ (64 * 4096)
> > +#define MB_UNIT_LEN16
> > +
> > +/* get used parameters for sps/pps */
> > +#define GET_MTK_VDEC_FLAG(cond, flag) \
> > + { dst_param->cond = ((src_param->flags & flag) ? (1) : (0)); }
> > +#define GET_MTK_VDEC_PARAM(param) \
> > + { dst_param->param = src_param->param; }
> > +/* motion vector size (bytes) for every macro block */
> > +#define HW_MB_STORE_SZ 64
> > +
> > +#define H264_MAX_FB_NUM17
> > +#define H264_MAX_MV_NUM32
> > +#define HDR_PARSING_BUF_SZ 1024
> > +
> > +/**
> > + * struct mtk_h264_dpb_info - h264 dpb information
> > + * @y_dma_addr: Y bitstream physical address
> > + * @c_dma_addr: CbCr bitstream physical address
> > + * @reference_flag: reference picture flag (short/long term reference
> > picture)
> > + * @field: field picture flag
> > + */
> > +struct mtk_h264_dpb_info {
> > + dma_addr_t y_
Re: [PATCH v3 05/15] media: mtk-vcodec: vdec: support stateless API
Hi Ezequiel, thanks for the feedback!
On Thu, Mar 4, 2021 at 6:30 AM Ezequiel Garcia
wrote:
>
> Hello Alex,
>
> Thanks for the patch.
>
> On Fri, 26 Feb 2021 at 07:06, Alexandre Courbot wrote:
> >
> > From: Yunfei Dong
> >
> > Support the stateless codec API that will be used by MT8183.
> >
> > Signed-off-by: Yunfei Dong
> > [acourbot: refactor, cleanup and split]
> > Co-developed-by: Alexandre Courbot
> > Signed-off-by: Alexandre Courbot
> > ---
> > drivers/media/platform/mtk-vcodec/Makefile| 1 +
> > .../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 ++-
> > .../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +-
> > .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 ++
> > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 3 +
> > 5 files changed, 503 insertions(+), 3 deletions(-)
> > create mode 100644
> > drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
> >
> [..]
>
> > +
> > +static const struct mtk_stateless_control mtk_stateless_controls[] = {
> > + {
> > + .cfg = {
> > + .id = V4L2_CID_STATELESS_H264_SPS,
> > + },
> > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > + .needed_in_request = true,
>
> This "needed_in_request" is not really required, as controls
> are not volatile, and their value is stored per-context (per-fd).
>
> It's perfectly valid for an application to pass the SPS control
> at the beginning of the sequence, and then omit it
> in further requests.
If I understand how v4l2_ctrl_request_hdl_ctrl_find() works with
requests, this boolean only checks that the control has been provided
at least once, and not that it is provided with every request. Without
it we could send a frame to the firmware without e.g. setting an SPS,
which would be a problem.
>
> > + },
> > + {
> > + .cfg = {
> > + .id = V4L2_CID_STATELESS_H264_PPS,
> > + },
> > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > + .needed_in_request = true,
> > + },
> > + {
> > + .cfg = {
> > + .id = V4L2_CID_STATELESS_H264_SCALING_MATRIX,
> > + },
> > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > + .needed_in_request = true,
> > + },
> > + {
> > + .cfg = {
> > + .id = V4L2_CID_STATELESS_H264_DECODE_PARAMS,
> > + },
> > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > + .needed_in_request = true,
> > + },
> > + {
> > + .cfg = {
> > + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> > + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
> > + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> > + .menu_skip_mask =
> > + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
> > + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
> > + },
> > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > + },
> > + {
> > + .cfg = {
> > + .id = V4L2_CID_STATELESS_H264_DECODE_MODE,
> > + .min = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> > + .def = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> > + .max = V4L2_STATELESS_H264_DECODE_MODE_FRAME_BASED,
> > + },
> > + .codec_type = V4L2_PIX_FMT_H264_SLICE,
> > + },
> > +};
>
> Applications also need to know which V4L2_CID_STATELESS_H264_START_CODE
> the driver supports. From a next patch, this case seems to be
> V4L2_STATELESS_H264_START_CODE_ANNEX_B.
Indeed - I've added the control, thanks for catching this!
>
> > +#define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
> > +
> > +static const struct mtk_video_fmt mtk_video_formats[] = {
> > + {
> > + .fourcc = V4L2_PIX_FMT_H264_SLICE,
> > + .type = MTK_FMT_DEC,
> > + .num_planes = 1,
> > + },
> > + {
> > + .fourcc = V4L2_PIX_FMT_MM21,
> > + .type = MTK_FMT_FRAME,
> > + .num_planes = 2,
> > + },
> > +};
> > +#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
> &g
Re: [PATCH v2 1/8] media: uapi: vp8: Remove "header" from symbol names and macros
On Thu, Mar 4, 2021 at 5:27 AM Ezequiel Garcia wrote:
>
> It doesn't seem to add any clarity to have a "header" suffix in controls,
> struct names and flags.
>
> Since this just makes names too long without any benefit, just drop it.
>
> Signed-off-by: Ezequiel Garcia
> ---
> .../media/v4l/ext-ctrls-codec.rst | 80 +++
> .../media/v4l/pixfmt-compressed.rst | 2 +-
> drivers/media/v4l2-core/v4l2-ctrls.c | 36 +++
> drivers/staging/media/hantro/hantro_drv.c | 2 +-
> .../staging/media/hantro/hantro_g1_vp8_dec.c | 48 -
> drivers/staging/media/hantro/hantro_hw.h | 2 +-
> drivers/staging/media/hantro/hantro_vp8.c | 10 +-
> .../media/hantro/rk3399_vpu_hw_vp8_dec.c | 48 -
> drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
> drivers/staging/media/sunxi/cedrus/cedrus.h | 2 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
> .../staging/media/sunxi/cedrus/cedrus_vp8.c | 98 +--
> include/media/v4l2-ctrls.h| 4 +-
> include/media/vp8-ctrls.h | 48 -
> 14 files changed, 192 insertions(+), 192 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 00944e97d638..006301e88554 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -1749,7 +1749,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>
> .. _v4l2-mpeg-vp8:
>
> -``V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER (struct)``
> +``V4L2_CID_MPEG_VIDEO_VP8_FRAME (struct)``
> Specifies the frame parameters for the associated VP8 parsed frame data.
> This includes the necessary parameters for
> configuring a stateless hardware decoding pipeline for VP8.
> @@ -1760,28 +1760,28 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> This compound control is not yet part of the public kernel API and
> it is expected to change.
>
> -.. c:type:: v4l2_ctrl_vp8_frame_header
> +.. c:type:: v4l2_ctrl_vp8_frame
>
> .. cssclass:: longtable
>
> .. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
>
> -.. flat-table:: struct v4l2_ctrl_vp8_frame_header
> +.. flat-table:: struct v4l2_ctrl_vp8_frame
> :header-rows: 0
> :stub-columns: 0
> :widths: 1 1 2
>
> -* - struct :c:type:`v4l2_vp8_segment_header`
> - - ``segment_header``
> +* - struct :c:type:`v4l2_vp8_segment`
> + - ``seg``
>- Structure with segment-based adjustments metadata.
> -* - struct :c:type:`v4l2_vp8_loopfilter_header`
> - - ``loopfilter_header``
> +* - struct :c:type:`v4l2_vp8_loopfilter`
> + - ``lf``
>- Structure with loop filter level adjustments metadata.
> -* - struct :c:type:`v4l2_vp8_quantization_header`
> - - ``quant_header``
> +* - struct :c:type:`v4l2_vp8_quantization`
> + - ``quant``
>- Structure with VP8 dequantization indices metadata.
> -* - struct :c:type:`v4l2_vp8_entropy_header`
> - - ``entropy_header``
> +* - struct :c:type:`v4l2_vp8_entropy`
> + - ``entropy``
>- Structure with VP8 entropy coder probabilities metadata.
> * - struct :c:type:`v4l2_vp8_entropy_coder_state`
>- ``coder_state``
> @@ -1850,11 +1850,11 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> :c:type:`v4l2_buffer` to a __u64.
> * - __u64
>- ``flags``
> - - See :ref:`Frame Header Flags `
> + - See :ref:`Frame Flags `
>
> -.. _vp8_frame_header_flags:
> +.. _vp8_frame_flags:
>
> -``Frame Header Flags``
> +``Frame Flags``
>
> .. cssclass:: longtable
>
> @@ -1863,22 +1863,22 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> :stub-columns: 0
> :widths: 1 1 2
>
> -* - ``V4L2_VP8_FRAME_HEADER_FLAG_KEY_FRAME``
> +* - ``V4L2_VP8_FRAME_FLAG_KEY_FRAME``
>- 0x01
>- Indicates if the frame is a key frame.
> -* - ``V4L2_VP8_FRAME_HEADER_FLAG_EXPERIMENTAL``
> +* - ``V4L2_VP8_FRAME_FLAG_EXPERIMENTAL``
>- 0x02
>- Experimental bitstream.
> -* - ``V4L2_VP8_FRAME_HEADER_FLAG_SHOW_FRAME``
> +* - ``V4L2_VP8_FRAME_FLAG_SHOW_FRAME``
>- 0x04
>- Show frame flag, indicates if the frame is for display.
> -* - ``V4L2_VP8_FRAME_HEADER_FLAG_MB_NO_SKIP_COEFF``
> +* - ``V4L2_VP8_FRAME_FLAG_MB_NO_SKIP_COEFF``
>- 0x08
>- Enable/disable skipping of macroblocks with no non-zero coefficients.
> -* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_GOLDEN``
> +* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_GOLDEN``
>- 0x10
>- Sign of motion vectors when the golden frame is referenced.
> -* - ``V4L2_VP8_FRAME_HEADER_FLAG_SIGN_BIAS_ALT``
> +* - ``V4L2_VP8_FRAME_FLAG_SIGN_BIAS_ALT``
>- 0x20
>- Sign of motion vectors when the alt frame is referenced.
>
> @@ -1906,13
Re: [PATCH v2 3/8] media: add Mediatek's MM21 format
Hi Nicolas,
On Wed, Mar 3, 2021 at 2:38 AM Nicolas Dufresne wrote:
>
> Le jeudi 25 février 2021 à 19:16 +0900, Alexandre Courbot a écrit :
> > Add Mediatek's non-compressed 8 bit block video mode. This format is
> > produced by the MT8183 codec and can be converted to a non-proprietary
> > format by the MDP3 component.
> >
> > Signed-off-by: Alexandre Courbot
> > ---
> > Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 7 +++
> > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> > include/uapi/linux/videodev2.h| 1 +
> > 3 files changed, 9 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> > b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> > index c9231e18859b..187ea89f7a25 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> > @@ -242,6 +242,13 @@ please make a proposal on the linux-media mailing list.
> > It is an opaque intermediate format and the MDP hardware must be
> > used to convert ``V4L2_PIX_FMT_MT21C`` to ``V4L2_PIX_FMT_NV12M``,
> > ``V4L2_PIX_FMT_YUV420M`` or ``V4L2_PIX_FMT_YVU420``.
> > +* .. _V4L2-PIX-FMT-MM21:
> > +
> > + - ``V4L2_PIX_FMT_MM21``
> > + - 'MM21'
> > + - Non-compressed, tiled two-planar format used by Mediatek MT8183.
> > + This is an opaque intermediate format and the MDP3 hardware can be
> > + used to convert it to other formats.
> > * .. _V4L2-PIX-FMT-SUNXI-TILED-NV12:
>
> The SUNXI one was a mistake, it's linear layout 32x32 tiles. The problem with
> calling this a vendor format, is that other vendor might be using it too. But
> they won't know and the format might endup duplicated, even if it's the same
> one.
>
> So here's my request, have you tried to understand a bit more what the tiling
> layout is ? Could be tiled + SAND, could use zigzag layout like Samsung do. I
> think if we can avoid vendor formats, we can preserve the pixel format list
> sanity here. Most of the HW I've encoutered was very easy to reverse, even if
> undocumented (except the compressed one).
Unfortunately I don't think I can look too closely into that, for
non-technical reasons. But if MTK could come forward and document
their format, that would be indeed ideal.
>
> If not possible, I would like to suggest:
>
> V4L2_PIX_FMT_MTK_NV21
>
> The important part is to add a clear seperation for the vendor name, it easy
> to
> recognize then.
The MTK prefix makes sense, but I do not know if NV21 is a correct
denomination for that format. Note that we have another MTK-only
format currently named V4L2_PIX_FMT_MT21C, which is already public. It
would be nice to rename it as well, but since it is part of the public
API I'm afraid the cat is already out of the bag for that one...
>
> >
> >- ``V4L2_PIX_FMT_SUNXI_TILED_NV12``
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-
> > core/v4l2-ioctl.c
> > index 31d1342e61e8..0b85b2bbc628 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1384,6 +1384,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > case V4L2_PIX_FMT_TM6000: descr = "A/V + VBI Mux Packet";
> > break;
> > case V4L2_PIX_FMT_CIT_YYVYUY: descr = "GSPCA CIT YYVYUY"; break;
> > case V4L2_PIX_FMT_KONICA420:descr = "GSPCA KONICA420"; break;
> > + case V4L2_PIX_FMT_MM21: descr = "Mediatek 8-bit block
> > format";
> > break;
> > case V4L2_PIX_FMT_HSV24:descr = "24-bit HSV 8-8-8"; break;
> > case V4L2_PIX_FMT_HSV32:descr = "32-bit XHSV 8-8-8-8";
> > break;
> > case V4L2_SDR_FMT_CU8: descr = "Complex U8"; break;
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 79dbde3bcf8d..e6890dae76ec 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -731,6 +731,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_Y12I v4l2_fourcc('Y', '1', '2', 'I') /* Greyscale
> > 12-bit L/R interleaved */
> > #define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
> > 16-bit */
> > #define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /* Mediatek
> > compressed block mode */
> > +#define V4L2_PIX_FMT_MM21 v4l2_fourcc('M', 'M', '2', '1') /* Mediatek
> > 8-
> > bit block mode, two non-contiguous planes */
> > #define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel
> > Planar
> > Greyscale 10-bit and Depth 16-bit */
> > #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /*
> > Sunxi Tiled NV12 Format */
> > #define V4L2_PIX_FMT_CNF4 v4l2_fourcc('C', 'N', 'F', '4') /* Intel
> > 4-bit
> > packed depth confidence information */
>
>
Re: [PATCH v4 2/3] media: uapi: Add VP9 stateless decoder controls
On Tue, Mar 2, 2021 at 3:43 AM Nicolas Dufresne wrote: > > Le dimanche 28 février 2021 à 15:13 +0900, Alexandre Courbot a écrit : > > Hi Nicolas, > > > > On Thu, Feb 25, 2021 at 6:08 AM Nicolas Dufresne > > wrote: > > > > > > Le jeudi 10 septembre 2020 à 15:04 +0900, Alexandre Courbot a écrit : > > > > Hi Ezequiel, sorry for the late review! > > > > > > > > On Tue, May 19, 2020 at 2:40 AM Ezequiel Garcia > > > > wrote: > > > > > > > > > > From: Boris Brezillon > > > > > > > > > > Add the VP9 stateless decoder controls plus the documentation that > > > > > goes > > > > > with it. > > > > > > > > > > Signed-off-by: Boris Brezillon > > > > > Signed-off-by: Ezequiel Garcia > > > > > --- > > > > > .../userspace-api/media/v4l/biblio.rst| 10 + > > > > > .../media/v4l/ext-ctrls-codec.rst | 550 > > > > > ++ > > > > > drivers/media/v4l2-core/v4l2-ctrls.c | 239 > > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > > > > > include/media/v4l2-ctrls.h| 1 + > > > > > include/media/vp9-ctrls.h | 485 +++ > > > > > 6 files changed, 1286 insertions(+) > > > > > create mode 100644 include/media/vp9-ctrls.h > > > > > > > > > > diff --git a/Documentation/userspace-api/media/v4l/biblio.rst > > > > > b/Documentation/userspace-api/media/v4l/biblio.rst > > > > > index 3c9634173e82..e09102e572fd 100644 > > > > > --- a/Documentation/userspace-api/media/v4l/biblio.rst > > > > > +++ b/Documentation/userspace-api/media/v4l/biblio.rst > > > > > @@ -414,3 +414,13 @@ VP8 > > > > > :title: RFC 6386: "VP8 Data Format and Decoding Guide" > > > > > > > > > > :author:J. Bankoski et al. > > > > > + > > > > > +.. _vp9: > > > > > + > > > > > +VP9 > > > > > +=== > > > > > + > > > > > + > > > > > +:title: VP9 Bitstream & Decoding Process Specification > > > > > + > > > > > +:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), > > > > > Jonathan > > > > > Hunt (Argon Design) > > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > > > b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > > > index d0d506a444b1..5c5f7dd868da 100644 > > > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > > > @@ -2668,6 +2668,556 @@ enum > > > > > v4l2_mpeg_video_h264_hierarchical_coding_type - > > > > >- ``padding[3]`` > > > > >- Applications and drivers must set this to zero. > > > > > > > > > > +.. _v4l2-mpeg-vp9: > > > > > + > > > > > +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT(0..3) (struct)`` > > > > > +Stores VP9 probabilities attached to a specific frame context. > > > > > The > > > > > VP9 > > > > > +specification allows using a maximum of 4 contexts. Each frame > > > > > being > > > > > +decoded refers to one of those context. See section '7.1.2 > > > > > Refresh > > > > > +probs semantics' section of :ref:`vp9` for more details about > > > > > these > > > > > +contexts. > > > > > + > > > > > +This control is bi-directional: > > > > > + > > > > > +* all 4 contexts must be initialized by userspace just after the > > > > > + stream is started and before the first decoding request is > > > > > submitted. > > > > > +* the referenced context might be read by the kernel when a > > > > > decoding > > > > > + request is submitted, and will be updated after the decoder is > > > > > done > > > > > + decoding the frame if the > > > > > `V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX` > > > > > flag > > > > > + is set. > > > > &
Re: [PATCH v4 2/3] media: uapi: Add VP9 stateless decoder controls
Hi Nicolas, On Thu, Feb 25, 2021 at 6:08 AM Nicolas Dufresne wrote: > > Le jeudi 10 septembre 2020 à 15:04 +0900, Alexandre Courbot a écrit : > > Hi Ezequiel, sorry for the late review! > > > > On Tue, May 19, 2020 at 2:40 AM Ezequiel Garcia > > wrote: > > > > > > From: Boris Brezillon > > > > > > Add the VP9 stateless decoder controls plus the documentation that goes > > > with it. > > > > > > Signed-off-by: Boris Brezillon > > > Signed-off-by: Ezequiel Garcia > > > --- > > > .../userspace-api/media/v4l/biblio.rst| 10 + > > > .../media/v4l/ext-ctrls-codec.rst | 550 ++ > > > drivers/media/v4l2-core/v4l2-ctrls.c | 239 > > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > > > include/media/v4l2-ctrls.h| 1 + > > > include/media/vp9-ctrls.h | 485 +++ > > > 6 files changed, 1286 insertions(+) > > > create mode 100644 include/media/vp9-ctrls.h > > > > > > diff --git a/Documentation/userspace-api/media/v4l/biblio.rst > > > b/Documentation/userspace-api/media/v4l/biblio.rst > > > index 3c9634173e82..e09102e572fd 100644 > > > --- a/Documentation/userspace-api/media/v4l/biblio.rst > > > +++ b/Documentation/userspace-api/media/v4l/biblio.rst > > > @@ -414,3 +414,13 @@ VP8 > > > :title: RFC 6386: "VP8 Data Format and Decoding Guide" > > > > > > :author:J. Bankoski et al. > > > + > > > +.. _vp9: > > > + > > > +VP9 > > > +=== > > > + > > > + > > > +:title: VP9 Bitstream & Decoding Process Specification > > > + > > > +:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), > > > Jonathan > > > Hunt (Argon Design) > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > index d0d506a444b1..5c5f7dd868da 100644 > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst > > > @@ -2668,6 +2668,556 @@ enum > > > v4l2_mpeg_video_h264_hierarchical_coding_type - > > >- ``padding[3]`` > > >- Applications and drivers must set this to zero. > > > > > > +.. _v4l2-mpeg-vp9: > > > + > > > +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT(0..3) (struct)`` > > > +Stores VP9 probabilities attached to a specific frame context. The > > > VP9 > > > +specification allows using a maximum of 4 contexts. Each frame being > > > +decoded refers to one of those context. See section '7.1.2 Refresh > > > +probs semantics' section of :ref:`vp9` for more details about these > > > +contexts. > > > + > > > +This control is bi-directional: > > > + > > > +* all 4 contexts must be initialized by userspace just after the > > > + stream is started and before the first decoding request is > > > submitted. > > > +* the referenced context might be read by the kernel when a decoding > > > + request is submitted, and will be updated after the decoder is done > > > + decoding the frame if the `V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX` > > > flag > > > + is set. > > > +* contexts will be read back by user space before each decoding > > > request > > > + to retrieve the updated probabilities. > > > +* userspace will re-initialize the context to their default values > > > when > > > + a reset context is required. > > > > Just to make sure I understand this part correctly, it means that if > > frame A and B use the same context, and frame A has > > V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX set, then user-space must wait > > for frame A to get dequeued and read back this control from the > > completed request before it can submit frame B? > > We are preparing a new version, we believe it was an API mistake to try and > share the probability update between kernel and userspace. It's actually > worst, > you really need to push/pull in a lock step, regardless of the frame context > id. > > As you may know, the probabilities are used to parse the compressed part of > the > stream (also know as entropy decoding). They are probability of a bitstream > symbol of being 0 or 1. On RK3399, the
[PATCH v3 15/15] media: mtk-vcodec: venc: make sure buffer exists in list before removing
From: Hsin-Yi Wang
It is possible that empty_flush_buf is removed in mtk_venc_worker() and
then again in vb2ops_venc_stop_streaming(). However, there's no empty
list check in v4l2_m2m_buf_remove_by_buf(). Double remove causes a
kernel crash.
Signed-off-by: Hsin-Yi Wang
[acourbot: fix commit log a bit]
Signed-off-by: Alexandre Courbot
---
.../media/platform/mtk-vcodec/mtk_vcodec_enc.c | 16 ++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 4de381b522ae..8af7e840b958 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -933,9 +933,21 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
}
/* STREAMOFF on the CAPTURE queue completes any ongoing flush */
if (ctx->is_flushing) {
+ struct v4l2_m2m_buffer *b, *n;
+
mtk_v4l2_debug(1, "STREAMOFF called while flushing");
- v4l2_m2m_buf_remove_by_buf(>m2m_ctx->out_q_ctx,
- >empty_flush_buf.vb);
+ /*
+* STREAMOFF could be called before the flush buffer is
+* dequeued. Check whether empty flush buf is still in
+* queue before removing it.
+*/
+ v4l2_m2m_for_each_src_buf_safe(ctx->m2m_ctx, b, n) {
+ if (b == >empty_flush_buf) {
+ v4l2_m2m_src_buf_remove_by_buf(
+ ctx->m2m_ctx, >vb);
+ break;
+ }
+ }
ctx->is_flushing = false;
}
} else {
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 14/15] media: mtk-vcodec: venc: support START and STOP commands
The V4L2 encoder specification requires encoders to support the
V4L2_ENC_CMD_START and V4L2_ENC_CMD_STOP commands. Add support for these
to the mtk-vcodec encoder by reusing the same flush buffer as used by
the decoder driver.
Signed-off-by: Alexandre Courbot
---
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 2 +
.../platform/mtk-vcodec/mtk_vcodec_enc.c | 123 +-
.../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 4 +
3 files changed, 122 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 8dab9f520283..73da6c7b69a8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -253,6 +253,7 @@ struct vdec_pic_info {
* @last_decoded_picinfo: pic information get from latest decode
* @empty_flush_buf: a fake size-0 capture buffer that indicates flush. Only
* to be used with encoder and stateful decoder.
+ * @is_flushing: set to true if flushing is in progress.
* @current_codec: current set input codec, in V4L2 pixel format
*
* @colorspace: enum v4l2_colorspace; supplemental to pixelformat
@@ -292,6 +293,7 @@ struct mtk_vcodec_ctx {
struct work_struct encode_work;
struct vdec_pic_info last_decoded_picinfo;
struct v4l2_m2m_buffer empty_flush_buf;
+ bool is_flushing;
u32 current_codec;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 8c917969c2f1..4de381b522ae 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -658,6 +658,7 @@ static int vidioc_venc_dqbuf(struct file *file, void *priv,
struct v4l2_buffer *buf)
{
struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ int ret;
if (ctx->state == MTK_STATE_ABORT) {
mtk_v4l2_err("[%d] Call on QBUF after unrecoverable error",
@@ -665,7 +666,77 @@ static int vidioc_venc_dqbuf(struct file *file, void *priv,
return -EIO;
}
- return v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+ ret = v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+ if (ret)
+ return ret;
+
+ /*
+* Complete flush if the user dequeued the 0-payload LAST buffer.
+* We check the payload because a buffer with the LAST flag can also
+* be seen during resolution changes. If we happen to be flushing at
+* that time, the last buffer before the resolution changes could be
+* misinterpreted for the buffer generated by the flush and terminate
+* it earlier than we want.
+*/
+ if (!V4L2_TYPE_IS_OUTPUT(buf->type) &&
+ buf->flags & V4L2_BUF_FLAG_LAST &&
+ buf->m.planes[0].bytesused == 0 &&
+ ctx->is_flushing) {
+ /*
+* Last CAPTURE buffer is dequeued, we can allow another flush
+* to take place.
+*/
+ ctx->is_flushing = false;
+ }
+
+ return 0;
+}
+
+static int vidioc_encoder_cmd(struct file *file, void *priv,
+ struct v4l2_encoder_cmd *cmd)
+{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+ struct vb2_queue *src_vq, *dst_vq;
+ int ret;
+
+ ret = v4l2_m2m_ioctl_try_encoder_cmd(file, priv, cmd);
+ if (ret)
+ return ret;
+
+ /* Calling START or STOP is invalid if a flush is in progress */
+ if (ctx->is_flushing)
+ return -EBUSY;
+
+ mtk_v4l2_debug(1, "encoder cmd=%u", cmd->cmd);
+
+ dst_vq = v4l2_m2m_get_vq(ctx->m2m_ctx,
+ V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+ switch (cmd->cmd) {
+ case V4L2_ENC_CMD_STOP:
+ src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx,
+ V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+ if (!vb2_is_streaming(src_vq)) {
+ mtk_v4l2_debug(1, "Output stream is off. No need to
flush.");
+ return 0;
+ }
+ if (!vb2_is_streaming(dst_vq)) {
+ mtk_v4l2_debug(1, "Capture stream is off. No need to
flush.");
+ return 0;
+ }
+ ctx->is_flushing = true;
+ v4l2_m2m_buf_queue(ctx->m2m_ctx, >empty_flush_buf.vb);
+ v4l2_m2m_try_schedule(ctx->m2m_ctx);
+ break;
+
+ case V4L2_ENC_CMD_START:
+ vb2_clear_last_buffer_dequeued(dst_vq);
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
}
const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
@@ -701,6 +772,9 @@
[PATCH v3 13/15] media: mtk-vcodec: make flush buffer reusable by encoder
The flush buffer is a special buffer that tells the decoder driver to
send an empty CAPTURE frame to the client with V4L2_BUF_FLAG_LAST set.
We need similar functionality for the encoder ; however currently the
flush buffer depends on decoder-specific structures and thus cannot be
reused with the encoder.
Fix this by testing for this buffer by its VB2 address, and not through
a dedicated flag stored in a higher-level decoder structure. This also
allows us to remove said flag and simplify the code a bit.
Since the flush buffer should never be used in the stateless decoder,
also add safeguards to check against it.
Signed-off-by: Alexandre Courbot
---
.../media/platform/mtk-vcodec/mtk_vcodec_dec.c| 9 ++---
.../media/platform/mtk-vcodec/mtk_vcodec_dec.h| 2 --
.../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 12 +---
.../platform/mtk-vcodec/mtk_vcodec_dec_stateful.c | 15 +--
.../media/platform/mtk-vcodec/mtk_vcodec_drv.h| 6 --
5 files changed, 16 insertions(+), 28 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 209ccf3d2d67..1c86130bb52d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -86,8 +86,7 @@ static int vidioc_decoder_cmd(struct file *file, void *priv,
mtk_v4l2_debug(1, "Capture stream is off. No need to
flush.");
return 0;
}
- v4l2_m2m_buf_queue(ctx->m2m_ctx,
- >empty_flush_buf->m2m_buf.vb);
+ v4l2_m2m_buf_queue(ctx->m2m_ctx, >empty_flush_buf.vb);
v4l2_m2m_try_schedule(ctx->m2m_ctx);
break;
@@ -779,8 +778,6 @@ int vb2ops_vdec_buf_init(struct vb2_buffer *vb)
if (vb->vb2_queue->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
buf->used = false;
buf->queued_in_v4l2 = false;
- } else {
- buf->lastframe = false;
}
return 0;
@@ -807,9 +804,7 @@ void vb2ops_vdec_stop_streaming(struct vb2_queue *q)
if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
while ((src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) {
- struct mtk_video_dec_buf *buf_info = container_of(
-src_buf, struct mtk_video_dec_buf, m2m_buf.vb);
- if (!buf_info->lastframe) {
+ if (src_buf != >empty_flush_buf.vb) {
struct media_request *req =
src_buf->vb2_buf.req_obj.req;
v4l2_m2m_buf_done(src_buf,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
index a2949e1bc7fe..a510e74251e6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.h
@@ -42,7 +42,6 @@ struct vdec_fb {
* @queued_in_vb2: Capture buffer is queue in vb2
* @queued_in_v4l2:Capture buffer is in v4l2 driver, but not in vb2
* queue yet
- * @lastframe: Intput buffer is last buffer - EOS
* @error: An unrecoverable error occurs on this buffer.
* @frame_buffer: Decode status, and buffer information of Capture buffer
* @bs_buffer: Output buffer info
@@ -55,7 +54,6 @@ struct mtk_video_dec_buf {
boolused;
boolqueued_in_vb2;
boolqueued_in_v4l2;
- boollastframe;
boolerror;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index e0526c0900c8..4789e669c258 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -82,21 +82,14 @@ static int fops_vcodec_open(struct file *file)
{
struct mtk_vcodec_dev *dev = video_drvdata(file);
struct mtk_vcodec_ctx *ctx = NULL;
- struct mtk_video_dec_buf *mtk_buf = NULL;
int ret = 0;
struct vb2_queue *src_vq;
ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
if (!ctx)
return -ENOMEM;
- mtk_buf = kzalloc(sizeof(*mtk_buf), GFP_KERNEL);
- if (!mtk_buf) {
- kfree(ctx);
- return -ENOMEM;
- }
mutex_lock(>dev_mutex);
- ctx->empty_flush_buf = mtk_buf;
ctx->id = dev->id_counter++;
v4l2_fh_init(>fh, video_devdata(file));
file->private_data = >fh;
@@ -122,8 +115,7 @@ static int fops_vcodec_open(struct file *file)
}
src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx,
V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
- ctx->empty_flush_buf-&
[PATCH v3 10/15] media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD
Let's use the dedicated helpers to make sure we get the expected
behavior on stateful decoders as well.
Signed-off-by: Alexandre Courbot
---
.../media/platform/mtk-vcodec/mtk_vcodec_dec.c | 16 ++--
1 file changed, 2 insertions(+), 14 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index c286cc0f239f..8bcff0b3626e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -52,22 +52,10 @@ static int vidioc_try_decoder_cmd(struct file *file, void
*priv,
if (ctx->dev->vdec_pdata->uses_stateless_api)
return v4l2_m2m_ioctl_stateless_try_decoder_cmd(file, priv,
cmd);
-
- switch (cmd->cmd) {
- case V4L2_DEC_CMD_STOP:
- case V4L2_DEC_CMD_START:
- if (cmd->flags != 0) {
- mtk_v4l2_err("cmd->flags=%u", cmd->flags);
- return -EINVAL;
- }
- break;
- default:
- return -EINVAL;
- }
- return 0;
+ else
+ return v4l2_m2m_ioctl_try_decoder_cmd(file, priv, cmd);
}
-
static int vidioc_decoder_cmd(struct file *file, void *priv,
struct v4l2_decoder_cmd *cmd)
{
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 12/15] media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits
Calling S_FMT or TRY_FMT on the OUTPUT queue should adjust the
resolution to the limits supported by the hardware. Until now this was
only done on the CAPTURE queue, which could make clients believe that
unsupported resolutions can be used when they set the coded size on the
OUTPUT queue.
In the case of the stateless decoder, the problem was even bigger since
subsequently calling G_FMT on the CAPTURE queue would result in the
unclamped resolution being returned, further inducing the client into
error.
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 8bcff0b3626e..209ccf3d2d67 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -223,19 +223,19 @@ static int vidioc_try_fmt(struct v4l2_format *f,
pix_fmt_mp->field = V4L2_FIELD_NONE;
+ pix_fmt_mp->width = clamp(pix_fmt_mp->width,
+ MTK_VDEC_MIN_W,
+ MTK_VDEC_MAX_W);
+ pix_fmt_mp->height = clamp(pix_fmt_mp->height,
+ MTK_VDEC_MIN_H,
+ MTK_VDEC_MAX_H);
+
if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
pix_fmt_mp->num_planes = 1;
pix_fmt_mp->plane_fmt[0].bytesperline = 0;
} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
int tmp_w, tmp_h;
- pix_fmt_mp->height = clamp(pix_fmt_mp->height,
- MTK_VDEC_MIN_H,
- MTK_VDEC_MAX_H);
- pix_fmt_mp->width = clamp(pix_fmt_mp->width,
- MTK_VDEC_MIN_W,
- MTK_VDEC_MAX_W);
-
/*
* Find next closer width align 64, heign align 64, size align
* 64 rectangle
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 11/15] media: mtk-vcodec: vdec: Support H264 profile control
From: Hirokazu Honda
Add H264 profiles supported by the MediaTek 8173 decoder.
Signed-off-by: Hirokazu Honda
[acourbot: fix commit log a bit]
Signed-off-by: Alexandre Courbot
---
.../platform/mtk-vcodec/mtk_vcodec_dec_stateful.c | 11 ++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
index f9db7ef19c28..3666c7e73bff 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
@@ -591,7 +591,16 @@ static int mtk_vcodec_dec_ctrls_setup(struct
mtk_vcodec_ctx *ctx)
V4L2_CID_MPEG_VIDEO_VP9_PROFILE,
V4L2_MPEG_VIDEO_VP9_PROFILE_0,
0, V4L2_MPEG_VIDEO_VP9_PROFILE_0);
-
+ /*
+* H264. Baseline / Extended decoding is not supported.
+*/
+ v4l2_ctrl_new_std_menu(>ctrl_hdl,
+ _vcodec_dec_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_H264_PROFILE,
+ V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) |
+ BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
+ V4L2_MPEG_VIDEO_H264_PROFILE_MAIN);
if (ctx->ctrl_hdl.error) {
mtk_v4l2_err("Adding control failed %d",
ctx->ctrl_hdl.error);
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 08/15] dt-bindings: media: document mediatek,mt8183-vcodec-dec
MT8183's decoder is instantiated similarly to MT8173's. Signed-off-by: Alexandre Courbot --- Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt index 8217424fd4bd..30c5888bf2d6 100644 --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt @@ -7,6 +7,7 @@ Required properties: - compatible : "mediatek,mt8173-vcodec-enc" for MT8173 encoder "mediatek,mt8183-vcodec-enc" for MT8183 encoder. "mediatek,mt8173-vcodec-dec" for MT8173 decoder. + "mediatek,mt8183-vcodec-dec" for MT8183 decoder. - reg : Physical base address of the video codec registers and length of memory mapped region. - interrupts : interrupt number to the cpu. -- 2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 07/15] media: mtk-vcodec: vdec: add media device if using stateless api
From: Yunfei Dong
The stateless API requires a media device for issuing requests. Add one
if we are being instantiated as a stateless decoder.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/Kconfig| 1 +
.../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 39 +++
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 2 +
3 files changed, 42 insertions(+)
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index c27db5643712..9d83b4223ecc 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -296,6 +296,7 @@ config VIDEO_MEDIATEK_VCODEC
select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
select V4L2_H264
+ select MEDIA_CONTROLLER
help
Mediatek video codec driver provides HW capability to
encode and decode in a range of video formats on MT8173
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 533781d4680a..e942e28f96fe 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -14,6 +14,7 @@
#include
#include
#include
+#include
#include "mtk_vcodec_drv.h"
#include "mtk_vcodec_dec.h"
@@ -324,6 +325,31 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
goto err_event_workq;
}
+ if (dev->vdec_pdata->uses_stateless_api) {
+ dev->mdev_dec.dev = >dev;
+ strscpy(dev->mdev_dec.model, MTK_VCODEC_DEC_NAME,
+ sizeof(dev->mdev_dec.model));
+
+ media_device_init(>mdev_dec);
+ dev->mdev_dec.ops = _vcodec_media_ops;
+ dev->v4l2_dev.mdev = >mdev_dec;
+
+ ret = v4l2_m2m_register_media_controller(dev->m2m_dev_dec,
+ dev->vfd_dec, MEDIA_ENT_F_PROC_VIDEO_DECODER);
+ if (ret) {
+ mtk_v4l2_err("Failed to register media controller");
+ goto err_reg_cont;
+ }
+
+ ret = media_device_register(>mdev_dec);
+ if (ret) {
+ mtk_v4l2_err("Failed to register media device");
+ goto err_media_reg;
+ }
+
+ mtk_v4l2_debug(0, "media registered as /dev/media%d",
+ vfd_dec->num);
+ }
ret = video_register_device(vfd_dec, VFL_TYPE_VIDEO, 0);
if (ret) {
mtk_v4l2_err("Failed to register video device");
@@ -336,6 +362,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
return 0;
err_dec_reg:
+ if (dev->vdec_pdata->uses_stateless_api)
+ media_device_unregister(>mdev_dec);
+err_media_reg:
+ if (dev->vdec_pdata->uses_stateless_api)
+ v4l2_m2m_unregister_media_controller(dev->m2m_dev_dec);
+err_reg_cont:
destroy_workqueue(dev->decode_workqueue);
err_event_workq:
v4l2_m2m_release(dev->m2m_dev_dec);
@@ -368,6 +400,13 @@ static int mtk_vcodec_dec_remove(struct platform_device
*pdev)
flush_workqueue(dev->decode_workqueue);
destroy_workqueue(dev->decode_workqueue);
+
+ if (media_devnode_is_registered(dev->mdev_dec.devnode)) {
+ media_device_unregister(>mdev_dec);
+ v4l2_m2m_unregister_media_controller(dev->m2m_dev_dec);
+ media_device_cleanup(>mdev_dec);
+ }
+
if (dev->m2m_dev_dec)
v4l2_m2m_release(dev->m2m_dev_dec);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 3b884a321883..79d6a1e6c916 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -384,6 +384,7 @@ struct mtk_vcodec_enc_pdata {
* struct mtk_vcodec_dev - driver data
* @v4l2_dev: V4L2 device to register video devices for.
* @vfd_dec: Video device for decoder
+ * @mdev_dec: Media device for decoder
* @vfd_enc: Video device for encoder.
*
* @m2m_dev_dec: m2m device for decoder
@@ -420,6 +421,7 @@ struct mtk_vcodec_enc_pdata {
struct mtk_vcodec_dev {
struct v4l2_device v4l2_dev;
struct video_device *vfd_dec;
+ struct media_device mdev_dec;
struct video_device *vfd_enc;
struct v4l2_m2m_dev *m2m_dev_dec;
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 09/15] media: mtk-vcodec: enable MT8183 decoder
From: Yunfei Dong
Now that all the supporting blocks are present, enable decoder for
MT8183.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index e942e28f96fe..e0526c0900c8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -383,12 +383,17 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
extern const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata;
+extern const struct mtk_vcodec_dec_pdata mtk_vdec_8183_pdata;
static const struct of_device_id mtk_vcodec_match[] = {
{
.compatible = "mediatek,mt8173-vcodec-dec",
.data = _vdec_8173_pdata,
},
+ {
+ .compatible = "mediatek,mt8183-vcodec-dec",
+ .data = _vdec_8183_pdata,
+ },
{},
};
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 06/15] media: mtk-vcodec: vdec: support stateless H.264 decoding
From: Yunfei Dong
Add support for H.264 decoding using the stateless API, as supported by
MT8183. This support takes advantage of the V4L2 H.264 reference list
builders.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/Kconfig| 1 +
drivers/media/platform/mtk-vcodec/Makefile| 1 +
.../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++
.../media/platform/mtk-vcodec/vdec_drv_if.c | 3 +
.../media/platform/mtk-vcodec/vdec_drv_if.h | 1 +
5 files changed, 813 insertions(+)
create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index fd1831e97b22..c27db5643712 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC
select V4L2_MEM2MEM_DEV
select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
+ select V4L2_H264
help
Mediatek video codec driver provides HW capability to
encode and decode in a range of video formats on MT8173
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 4ba93d838ab6..ca8e9e7a9c4e 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
vdec/vdec_vp8_if.o \
vdec/vdec_vp9_if.o \
+ vdec/vdec_h264_req_if.o \
mtk_vcodec_dec_drv.o \
vdec_drv_if.o \
vdec_vpu_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
new file mode 100644
index ..2fbbfbbcfbec
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
@@ -0,0 +1,807 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include
+#include
+#include
+#include
+#include
+
+#include "../vdec_drv_if.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_dec.h"
+#include "../mtk_vcodec_intr.h"
+#include "../vdec_vpu_if.h"
+#include "../vdec_drv_base.h"
+
+#define NAL_NON_IDR_SLICE 0x01
+#define NAL_IDR_SLICE 0x05
+#define NAL_H264_PPS 0x08
+#define NAL_TYPE(value)((value) & 0x1F)
+
+#define BUF_PREDICTION_SZ (64 * 4096)
+#define MB_UNIT_LEN16
+
+/* get used parameters for sps/pps */
+#define GET_MTK_VDEC_FLAG(cond, flag) \
+ { dst_param->cond = ((src_param->flags & flag) ? (1) : (0)); }
+#define GET_MTK_VDEC_PARAM(param) \
+ { dst_param->param = src_param->param; }
+/* motion vector size (bytes) for every macro block */
+#define HW_MB_STORE_SZ 64
+
+#define H264_MAX_FB_NUM17
+#define H264_MAX_MV_NUM32
+#define HDR_PARSING_BUF_SZ 1024
+
+/**
+ * struct mtk_h264_dpb_info - h264 dpb information
+ * @y_dma_addr: Y bitstream physical address
+ * @c_dma_addr: CbCr bitstream physical address
+ * @reference_flag: reference picture flag (short/long term reference picture)
+ * @field: field picture flag
+ */
+struct mtk_h264_dpb_info {
+ dma_addr_t y_dma_addr;
+ dma_addr_t c_dma_addr;
+ int reference_flag;
+ int field;
+};
+
+/**
+ * struct mtk_h264_sps_param - parameters for sps
+ */
+struct mtk_h264_sps_param {
+ unsigned char chroma_format_idc;
+ unsigned char bit_depth_luma_minus8;
+ unsigned char bit_depth_chroma_minus8;
+ unsigned char log2_max_frame_num_minus4;
+ unsigned char pic_order_cnt_type;
+ unsigned char log2_max_pic_order_cnt_lsb_minus4;
+ unsigned char max_num_ref_frames;
+ unsigned char separate_colour_plane_flag;
+ unsigned short pic_width_in_mbs_minus1;
+ unsigned short pic_height_in_map_units_minus1;
+ unsigned int max_frame_nums;
+ unsigned char qpprime_y_zero_transform_bypass_flag;
+ unsigned char delta_pic_order_always_zero_flag;
+ unsigned char frame_mbs_only_flag;
+ unsigned char mb_adaptive_frame_field_flag;
+ unsigned char direct_8x8_inference_flag;
+ unsigned char reserved[3];
+};
+
+/**
+ * struct mtk_h264_pps_param - parameters for pps
+ */
+struct mtk_h264_pps_param {
+ unsigned char num_ref_idx_l0_default_active_minus1;
+ unsigned char num_ref_idx_l1_default_active_minus1;
+ unsigned char weighted_bipred_idc;
+ char pic_init_qp_minus26;
+ char chroma_qp
[PATCH v3 05/15] media: mtk-vcodec: vdec: support stateless API
From: Yunfei Dong
Support the stateless codec API that will be used by MT8183.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/Makefile| 1 +
.../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 ++-
.../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +-
.../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 ++
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 3 +
5 files changed, 503 insertions(+), 3 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 9c3cbb5b800e..4ba93d838ab6 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -12,6 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
vdec_vpu_if.o \
mtk_vcodec_dec.o \
mtk_vcodec_dec_stateful.o \
+ mtk_vcodec_dec_stateless.o \
mtk_vcodec_dec_pm.o \
mtk-vcodec-enc-y := venc/venc_vp8_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 4a91d294002b..c286cc0f239f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -46,6 +46,13 @@ static struct mtk_q_data *mtk_vdec_get_q_data(struct
mtk_vcodec_ctx *ctx,
static int vidioc_try_decoder_cmd(struct file *file, void *priv,
struct v4l2_decoder_cmd *cmd)
{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ /* Use M2M stateless helper if relevant */
+ if (ctx->dev->vdec_pdata->uses_stateless_api)
+ return v4l2_m2m_ioctl_stateless_try_decoder_cmd(file, priv,
+ cmd);
+
switch (cmd->cmd) {
case V4L2_DEC_CMD_STOP:
case V4L2_DEC_CMD_START:
@@ -72,6 +79,10 @@ static int vidioc_decoder_cmd(struct file *file, void *priv,
if (ret)
return ret;
+ /* Use M2M stateless helper if relevant */
+ if (ctx->dev->vdec_pdata->uses_stateless_api)
+ return v4l2_m2m_ioctl_stateless_decoder_cmd(file, priv, cmd);
+
mtk_v4l2_debug(1, "decoder cmd=%u", cmd->cmd);
dst_vq = v4l2_m2m_get_vq(ctx->m2m_ctx,
V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
@@ -414,7 +425,8 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
* Setting OUTPUT format after OUTPUT buffers are allocated is invalid
* if using the stateful API.
*/
- if ((f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+ if (!dec_pdata->uses_stateless_api &&
+ (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
vb2_is_busy(>m2m_ctx->out_q_ctx.q)) {
mtk_v4l2_err("out_q_ctx buffers already requested");
ret = -EBUSY;
@@ -457,6 +469,7 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
ctx->quantization = pix_mp->quantization;
ctx->xfer_func = pix_mp->xfer_func;
+ ctx->current_codec = fmt->fourcc;
if (ctx->state == MTK_STATE_FREE) {
ret = vdec_if_init(ctx, q_data->fmt->fourcc);
if (ret) {
@@ -468,6 +481,49 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
}
}
+ /*
+* If using the stateless API, S_FMT should have the effect of setting
+* the CAPTURE queue resolution no matter which queue it was called on.
+*/
+ if (dec_pdata->uses_stateless_api) {
+ ctx->picinfo.pic_w = pix_mp->width;
+ ctx->picinfo.pic_h = pix_mp->height;
+
+ ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, >picinfo);
+ if (ret) {
+ mtk_v4l2_err("[%d]Error!! Get GET_PARAM_PICTURE_INFO
Fail",
+ ctx->id);
+ return -EINVAL;
+ }
+
+ ctx->last_decoded_picinfo = ctx->picinfo;
+
+ if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 1) {
+ ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
+ ctx->picinfo.fb_sz[0] +
+ ctx->picinfo.fb_sz[1];
+ ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] =
+ ctx->picinfo.buf_w;
+ } else {
+ ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
+ ctx->picinfo.fb_sz[0];
+
[PATCH v3 02/15] media: mtk-vcodec: vdec: handle firmware version field
Firmwares for decoders newer than MT8173 will include an ABI version
number in their initialization ack message. Add the capacity to manage
it and make initialization fail if the firmware ABI is of a version that
we don't support.
For MT8173, this ABI version field does not exist ; thus ignore it on
this chip. There should only be one firmware version available for it
anyway.
Signed-off-by: Alexandre Courbot
---
.../mtk-vcodec/mtk_vcodec_dec_stateful.c | 1 +
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 4
.../media/platform/mtk-vcodec/vdec_ipi_msg.h | 5 +
.../media/platform/mtk-vcodec/vdec_vpu_if.c | 21 +--
4 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
index 48b7524bc8fb..f9db7ef19c28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
@@ -620,6 +620,7 @@ static struct vb2_ops mtk_vdec_frame_vb2_ops = {
};
const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata = {
+ .chip = MTK_MT8173,
.init_vdec_params = mtk_init_vdec_params,
.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
.vdec_vb2_ops = _vdec_frame_vb2_ops,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 9221c17a176b..60bc39efa20d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -322,6 +322,8 @@ enum mtk_chip {
* @vdec_framesizes: supported video decoder frame sizes
* @num_framesizes: count of video decoder frame sizes
*
+ * @chip: chip this decoder is compatible with
+ *
* @uses_stateless_api: whether the decoder uses the stateless API with
requests
*/
@@ -341,6 +343,8 @@ struct mtk_vcodec_dec_pdata {
const struct mtk_codec_framesizes *vdec_framesizes;
const int num_framesizes;
+ enum mtk_chip chip;
+
bool uses_stateless_api;
};
diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
index 47a1c1c0fd04..eb66729fda63 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
@@ -83,12 +83,17 @@ struct vdec_ap_ipi_dec_start {
* @status : VPU exeuction result
* @ap_inst_addr : AP vcodec_vpu_inst instance address
* @vpu_inst_addr : VPU decoder instance address
+ * @vdec_abi_version: ABI version of the firmware. Kernel can use it to
+ * ensure that it is compatible with the firmware.
+ * This field is not valid for MT8173 and must not be
+ * accessed for this chip.
*/
struct vdec_vpu_ipi_init_ack {
uint32_t msg_id;
int32_t status;
uint64_t ap_inst_addr;
uint32_t vpu_inst_addr;
+ uint32_t vdec_abi_version;
};
#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index 58b0e6fa8fd2..203089213e67 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -24,6 +24,22 @@ static void handle_init_ack_msg(const struct
vdec_vpu_ipi_init_ack *msg)
vpu->inst_addr = msg->vpu_inst_addr;
mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
+
+ /* Firmware version field does not exist on MT8173. */
+ if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173)
+ return;
+
+ /* Check firmware version. */
+ mtk_vcodec_debug(vpu, "firmware version 0x%x\n", msg->vdec_abi_version);
+ switch (msg->vdec_abi_version) {
+ case 1:
+ break;
+ default:
+ mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n",
+ msg->vdec_abi_version);
+ vpu->failure = 1;
+ break;
+ }
}
/*
@@ -44,6 +60,9 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len,
void *priv)
mtk_vcodec_debug(vpu, "+ id=%X", msg->msg_id);
+ vpu->failure = msg->status;
+ vpu->signaled = 1;
+
if (msg->status == 0) {
switch (msg->msg_id) {
case VPU_IPIMSG_DEC_INIT_ACK:
@@ -63,8 +82,6 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len,
void *priv)
}
mtk_vcodec_debug(vpu, "- id=%X", msg->msg_id);
- vpu->failure = msg->status;
- vpu->signaled = 1;
}
static int vcodec_vpu_send_msg(struct vdec_vpu_inst *vpu, void *msg, int len)
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 04/15] media: add Mediatek's MM21 format
Add Mediatek's non-compressed 8 bit block video mode. This format is
produced by the MT8183 codec and can be converted to a non-proprietary
format by the MDP3 component.
Signed-off-by: Alexandre Courbot
---
Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 7 +++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h| 1 +
3 files changed, 9 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
index c9231e18859b..187ea89f7a25 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
@@ -242,6 +242,13 @@ please make a proposal on the linux-media mailing list.
It is an opaque intermediate format and the MDP hardware must be
used to convert ``V4L2_PIX_FMT_MT21C`` to ``V4L2_PIX_FMT_NV12M``,
``V4L2_PIX_FMT_YUV420M`` or ``V4L2_PIX_FMT_YVU420``.
+* .. _V4L2-PIX-FMT-MM21:
+
+ - ``V4L2_PIX_FMT_MM21``
+ - 'MM21'
+ - Non-compressed, tiled two-planar format used by Mediatek MT8183.
+ This is an opaque intermediate format and the MDP3 hardware can be
+ used to convert it to other formats.
* .. _V4L2-PIX-FMT-SUNXI-TILED-NV12:
- ``V4L2_PIX_FMT_SUNXI_TILED_NV12``
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 31d1342e61e8..0b85b2bbc628 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1384,6 +1384,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_TM6000: descr = "A/V + VBI Mux Packet"; break;
case V4L2_PIX_FMT_CIT_YYVYUY: descr = "GSPCA CIT YYVYUY"; break;
case V4L2_PIX_FMT_KONICA420:descr = "GSPCA KONICA420"; break;
+ case V4L2_PIX_FMT_MM21: descr = "Mediatek 8-bit block format";
break;
case V4L2_PIX_FMT_HSV24:descr = "24-bit HSV 8-8-8"; break;
case V4L2_PIX_FMT_HSV32:descr = "32-bit XHSV 8-8-8-8"; break;
case V4L2_SDR_FMT_CU8: descr = "Complex U8"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..e6890dae76ec 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -731,6 +731,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_Y12I v4l2_fourcc('Y', '1', '2', 'I') /* Greyscale
12-bit L/R interleaved */
#define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
16-bit */
#define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /* Mediatek
compressed block mode */
+#define V4L2_PIX_FMT_MM21 v4l2_fourcc('M', 'M', '2', '1') /* Mediatek
8-bit block mode, two non-contiguous planes */
#define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar
Greyscale 10-bit and Depth 16-bit */
#define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi
Tiled NV12 Format */
#define V4L2_PIX_FMT_CNF4 v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit
packed depth confidence information */
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 03/15] media: mtk-vcodec: support version 2 of decoder firmware ABI
Add support for decoder firmware version 2, which makes the kernel
responsible for managing the VSI context and is used for stateless
codecs.
Signed-off-by: Alexandre Courbot
---
.../media/platform/mtk-vcodec/vdec_ipi_msg.h | 18 +---
.../media/platform/mtk-vcodec/vdec_vpu_if.c | 28 +++
.../media/platform/mtk-vcodec/vdec_vpu_if.h | 5
3 files changed, 42 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
index eb66729fda63..a0e773ae3ab3 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
@@ -29,11 +29,15 @@ enum vdec_ipi_msgid {
/**
* struct vdec_ap_ipi_cmd - generic AP to VPU ipi command format
* @msg_id : vdec_ipi_msgid
- * @vpu_inst_addr : VPU decoder instance address
+ * @vpu_inst_addr : VPU decoder instance address. Used if ABI version < 2.
+ * @inst_id : instance ID. Used if the ABI version >= 2.
*/
struct vdec_ap_ipi_cmd {
uint32_t msg_id;
- uint32_t vpu_inst_addr;
+ union {
+ uint32_t vpu_inst_addr;
+ uint32_t inst_id;
+ };
};
/**
@@ -63,7 +67,8 @@ struct vdec_ap_ipi_init {
/**
* struct vdec_ap_ipi_dec_start - for AP_IPIMSG_DEC_START
* @msg_id : AP_IPIMSG_DEC_START
- * @vpu_inst_addr : VPU decoder instance address
+ * @vpu_inst_addr : VPU decoder instance address. Used if ABI version < 2.
+ * @inst_id : instance ID. Used if the ABI version >= 2.
* @data : Header info
* H264 decoder [0]:buf_sz [1]:nal_start
* VP8 decoder [0]:width/height
@@ -72,7 +77,10 @@ struct vdec_ap_ipi_init {
*/
struct vdec_ap_ipi_dec_start {
uint32_t msg_id;
- uint32_t vpu_inst_addr;
+ union {
+ uint32_t vpu_inst_addr;
+ uint32_t inst_id;
+ };
uint32_t data[3];
uint32_t reserved;
};
@@ -87,6 +95,7 @@ struct vdec_ap_ipi_dec_start {
* ensure that it is compatible with the firmware.
* This field is not valid for MT8173 and must not be
* accessed for this chip.
+ * @inst_id : instance ID. Valid only if the ABI version >= 2.
*/
struct vdec_vpu_ipi_init_ack {
uint32_t msg_id;
@@ -94,6 +103,7 @@ struct vdec_vpu_ipi_init_ack {
uint64_t ap_inst_addr;
uint32_t vpu_inst_addr;
uint32_t vdec_abi_version;
+ uint32_t inst_id;
};
#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index 203089213e67..5dffc459a33d 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -25,18 +25,30 @@ static void handle_init_ack_msg(const struct
vdec_vpu_ipi_init_ack *msg)
mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
+ /* Set default ABI version if dealing with unversioned firmware. */
+ vpu->fw_abi_version = 0;
+ /*
+* Instance ID is only used if ABI version >= 2. Initialize it with
+* garbage by default.
+*/
+ vpu->inst_id = 0xdeadbeef;
+
/* Firmware version field does not exist on MT8173. */
if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173)
return;
/* Check firmware version. */
- mtk_vcodec_debug(vpu, "firmware version 0x%x\n", msg->vdec_abi_version);
- switch (msg->vdec_abi_version) {
+ vpu->fw_abi_version = msg->vdec_abi_version;
+ mtk_vcodec_debug(vpu, "firmware version 0x%x\n", vpu->fw_abi_version);
+ switch (vpu->fw_abi_version) {
case 1:
break;
+ case 2:
+ vpu->inst_id = msg->inst_id;
+ break;
default:
mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n",
- msg->vdec_abi_version);
+ vpu->fw_abi_version);
vpu->failure = 1;
break;
}
@@ -113,7 +125,10 @@ static int vcodec_send_ap_ipi(struct vdec_vpu_inst *vpu,
unsigned int msg_id)
memset(, 0, sizeof(msg));
msg.msg_id = msg_id;
- msg.vpu_inst_addr = vpu->inst_addr;
+ if (vpu->fw_abi_version < 2)
+ msg.vpu_inst_addr = vpu->inst_addr;
+ else
+ msg.inst_id = vpu->inst_id;
err = vcodec_vpu_send_msg(vpu, , sizeof(msg));
mtk_vcodec_debug(vpu, "- id=%X ret=%d", msg_id, err);
@@ -163,7 +178,10 @@ int vpu_dec_start(struct vdec_vpu_inst *vpu, uint32_t
*data, unsigned int len)
memset(, 0, sizeof(msg));
msg.msg_id = AP_IPIMSG_DEC_START;
- msg.vpu_inst_addr = vpu->inst_addr;
+ if (vpu->fw_abi_version <
[PATCH v3 00/15] media: mtk-vcodec: support for MT8183 decoder
Please ignore the v2 of this series since I neglected a final check on it and realized later it did not compile. >_< This series adds support for the stateless API into mtk-vcodec, by first separating the stateful ops into their own source file, and introducing a new set of ops suitable for stateless decoding. As such, support for stateful decoders should remain completely unaffected. This series has been tested with both MT8183 and MT8173. Decoding was working for both chips, and in the case of MT8173 no regression has been spotted. Patches 1-9 add MT8183 support to the decoder using the stateless API. MT8183 only support H.264 acceleration. Patches 10-15 are follow-ups that further improve compliance for the decoder and encoder, by fixing support for commands on both. Patch 11 also makes sure that supported H.264 profiles are exported on MT8173. Changes since v2: * Actually compiles (duh), * Add follow-up patches fixing support for START/STOP commands for the encoder, and stateful decoder. Alexandre Courbot (8): media: mtk-vcodec: vdec: handle firmware version field media: mtk-vcodec: support version 2 of decoder firmware ABI media: add Mediatek's MM21 format dt-bindings: media: document mediatek,mt8183-vcodec-dec media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits media: mtk-vcodec: make flush buffer reusable by encoder media: mtk-vcodec: venc: support START and STOP commands Hirokazu Honda (1): media: mtk-vcodec: vdec: Support H264 profile control Hsin-Yi Wang (1): media: mtk-vcodec: venc: make sure buffer exists in list before removing Yunfei Dong (5): media: mtk-vcodec: vdec: move stateful ops into their own file media: mtk-vcodec: vdec: support stateless API media: mtk-vcodec: vdec: support stateless H.264 decoding media: mtk-vcodec: vdec: add media device if using stateless api media: mtk-vcodec: enable MT8183 decoder .../bindings/media/mediatek-vcodec.txt| 1 + .../media/v4l/pixfmt-reserved.rst | 7 + drivers/media/platform/Kconfig| 2 + drivers/media/platform/mtk-vcodec/Makefile| 3 + .../platform/mtk-vcodec/mtk_vcodec_dec.c | 800 +++-- .../platform/mtk-vcodec/mtk_vcodec_dec.h | 30 +- .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 66 +- .../mtk-vcodec/mtk_vcodec_dec_stateful.c | 647 ++ .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 + .../platform/mtk-vcodec/mtk_vcodec_drv.h | 58 +- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 135 ++- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 4 + .../mtk-vcodec/vdec/vdec_h264_req_if.c| 807 ++ .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 23 +- .../media/platform/mtk-vcodec/vdec_vpu_if.c | 43 +- .../media/platform/mtk-vcodec/vdec_vpu_if.h | 5 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h| 1 + 20 files changed, 2360 insertions(+), 704 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c -- 2.30.1.766.gb4fecdf3b7-goog
[PATCH v3 01/15] media: mtk-vcodec: vdec: move stateful ops into their own file
From: Yunfei Dong
We are planning to add support for stateless decoders to this driver.
Part of the driver will be shared between stateful and stateless
codecs, but a few ops need to be specialized for both. Extract the
stateful part of the driver and move it into its own file, accessible
through ops that the common driver parts can call.
This patch only moves code around and introduces a set of abstractions ;
the behavior of the driver should not be changed in any way.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/Makefile| 1 +
.../platform/mtk-vcodec/mtk_vcodec_dec.c | 699 ++
.../platform/mtk-vcodec/mtk_vcodec_dec.h | 19 +-
.../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 10 +-
.../mtk-vcodec/mtk_vcodec_dec_stateful.c | 634
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 41 +
6 files changed, 759 insertions(+), 645 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 4618d43dbbc8..9c3cbb5b800e 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -11,6 +11,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
vdec_drv_if.o \
vdec_vpu_if.o \
mtk_vcodec_dec.o \
+ mtk_vcodec_dec_stateful.o \
mtk_vcodec_dec_pm.o \
mtk-vcodec-enc-y := venc/venc_vp8_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 56d86e59421e..4a91d294002b 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -16,68 +16,17 @@
#include "vdec_drv_if.h"
#include "mtk_vcodec_dec_pm.h"
-#define OUT_FMT_IDX0
-#define CAP_FMT_IDX3
-
-#define MTK_VDEC_MIN_W 64U
-#define MTK_VDEC_MIN_H 64U
#define DFT_CFG_WIDTH MTK_VDEC_MIN_W
#define DFT_CFG_HEIGHT MTK_VDEC_MIN_H
-static const struct mtk_video_fmt mtk_video_formats[] = {
- {
- .fourcc = V4L2_PIX_FMT_H264,
- .type = MTK_FMT_DEC,
- .num_planes = 1,
- .flags = V4L2_FMT_FLAG_DYN_RESOLUTION,
- },
- {
- .fourcc = V4L2_PIX_FMT_VP8,
- .type = MTK_FMT_DEC,
- .num_planes = 1,
- .flags = V4L2_FMT_FLAG_DYN_RESOLUTION,
- },
- {
- .fourcc = V4L2_PIX_FMT_VP9,
- .type = MTK_FMT_DEC,
- .num_planes = 1,
- .flags = V4L2_FMT_FLAG_DYN_RESOLUTION,
- },
- {
- .fourcc = V4L2_PIX_FMT_MT21C,
- .type = MTK_FMT_FRAME,
- .num_planes = 2,
- },
-};
-
-static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
- {
- .fourcc = V4L2_PIX_FMT_H264,
- .stepwise = { MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
- MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
- },
- {
- .fourcc = V4L2_PIX_FMT_VP8,
- .stepwise = { MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
- MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
- },
- {
- .fourcc = V4L2_PIX_FMT_VP9,
- .stepwise = { MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
- MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
- },
-};
-
-#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
-#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
-
-static const struct mtk_video_fmt *mtk_vdec_find_format(struct v4l2_format *f)
+static const struct mtk_video_fmt *mtk_vdec_find_format(struct v4l2_format *f,
+ const struct mtk_vcodec_dec_pdata *dec_pdata)
{
const struct mtk_video_fmt *fmt;
unsigned int k;
- for (k = 0; k < NUM_FORMATS; k++) {
- fmt = _video_formats[k];
+ for (k = 0; k < dec_pdata->num_formats; k++) {
+ fmt = _pdata->vdec_formats[k];
if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
return fmt;
}
@@ -94,393 +43,6 @@ static struct mtk_q_data *mtk_vdec_get_q_data(struct
mtk_vcodec_ctx *ctx,
return >q_data[MTK_Q_DATA_DST];
}
-/*
- * This function tries to clean all display buffers, the buffers will return
- * in display order.
- * Note the buffers returned from codec driver may still be in driver's
- * reference list.
- */
-static struct vb2_buffer *get_display_buffer(struct mtk_vcodec_ctx *ctx)
-{
- struct vdec_fb *disp_frame_buffer = NULL;
- struct mtk_video_dec_buf *dstbuf;
- struct vb2_v4l2_buffer *vb;
-
- mtk_v4l2_debug(3, "[%d]
[PATCH] media: mtk-vcodec: venc: remove redundant code
vidioc_try_fmt() does clamp height and width when called on the OUTPUT
queue, so clamping them prior to calling this function is redundant. Set
the queue's parameters after calling vidioc_try_fmt() so we can use the
values it computed.
Signed-off-by: Alexandre Courbot
---
.../media/platform/mtk-vcodec/mtk_vcodec_enc.c | 16
1 file changed, 4 insertions(+), 12 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 8c917969c2f1..d16e1ec87a49 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -442,7 +442,6 @@ static int vidioc_venc_s_fmt_out(struct file *file, void
*priv,
struct mtk_q_data *q_data;
int ret, i;
const struct mtk_video_fmt *fmt;
- struct v4l2_pix_format_mplane *pix_fmt_mp = >fmt.pix_mp;
vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
if (!vq) {
@@ -467,20 +466,13 @@ static int vidioc_venc_s_fmt_out(struct file *file, void
*priv,
f->fmt.pix.pixelformat = fmt->fourcc;
}
- pix_fmt_mp->height = clamp(pix_fmt_mp->height,
- MTK_VENC_MIN_H,
- MTK_VENC_MAX_H);
- pix_fmt_mp->width = clamp(pix_fmt_mp->width,
- MTK_VENC_MIN_W,
- MTK_VENC_MAX_W);
-
- q_data->visible_width = f->fmt.pix_mp.width;
- q_data->visible_height = f->fmt.pix_mp.height;
- q_data->fmt = fmt;
- ret = vidioc_try_fmt(f, q_data->fmt);
+ ret = vidioc_try_fmt(f, fmt);
if (ret)
return ret;
+ q_data->fmt = fmt;
+ q_data->visible_width = f->fmt.pix_mp.width;
+ q_data->visible_height = f->fmt.pix_mp.height;
q_data->coded_width = f->fmt.pix_mp.width;
q_data->coded_height = f->fmt.pix_mp.height;
--
2.30.1.766.gb4fecdf3b7-goog
[PATCH v2 2/8] media: mtk-vcodec: support version 2 of decoder firmware ABI
Add support for decoder firmware version 2, which makes the kernel
responsible for managing the VSI context and is used for stateless
codecs.
Signed-off-by: Alexandre Courbot
---
.../media/platform/mtk-vcodec/vdec_ipi_msg.h | 18 +---
.../media/platform/mtk-vcodec/vdec_vpu_if.c | 28 +++
.../media/platform/mtk-vcodec/vdec_vpu_if.h | 5
3 files changed, 42 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
index eb66729fda63..a0e773ae3ab3 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
@@ -29,11 +29,15 @@ enum vdec_ipi_msgid {
/**
* struct vdec_ap_ipi_cmd - generic AP to VPU ipi command format
* @msg_id : vdec_ipi_msgid
- * @vpu_inst_addr : VPU decoder instance address
+ * @vpu_inst_addr : VPU decoder instance address. Used if ABI version < 2.
+ * @inst_id : instance ID. Used if the ABI version >= 2.
*/
struct vdec_ap_ipi_cmd {
uint32_t msg_id;
- uint32_t vpu_inst_addr;
+ union {
+ uint32_t vpu_inst_addr;
+ uint32_t inst_id;
+ };
};
/**
@@ -63,7 +67,8 @@ struct vdec_ap_ipi_init {
/**
* struct vdec_ap_ipi_dec_start - for AP_IPIMSG_DEC_START
* @msg_id : AP_IPIMSG_DEC_START
- * @vpu_inst_addr : VPU decoder instance address
+ * @vpu_inst_addr : VPU decoder instance address. Used if ABI version < 2.
+ * @inst_id : instance ID. Used if the ABI version >= 2.
* @data : Header info
* H264 decoder [0]:buf_sz [1]:nal_start
* VP8 decoder [0]:width/height
@@ -72,7 +77,10 @@ struct vdec_ap_ipi_init {
*/
struct vdec_ap_ipi_dec_start {
uint32_t msg_id;
- uint32_t vpu_inst_addr;
+ union {
+ uint32_t vpu_inst_addr;
+ uint32_t inst_id;
+ };
uint32_t data[3];
uint32_t reserved;
};
@@ -87,6 +95,7 @@ struct vdec_ap_ipi_dec_start {
* ensure that it is compatible with the firmware.
* This field is not valid for MT8173 and must not be
* accessed for this chip.
+ * @inst_id : instance ID. Valid only if the ABI version >= 2.
*/
struct vdec_vpu_ipi_init_ack {
uint32_t msg_id;
@@ -94,6 +103,7 @@ struct vdec_vpu_ipi_init_ack {
uint64_t ap_inst_addr;
uint32_t vpu_inst_addr;
uint32_t vdec_abi_version;
+ uint32_t inst_id;
};
#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index 203089213e67..5dffc459a33d 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -25,18 +25,30 @@ static void handle_init_ack_msg(const struct
vdec_vpu_ipi_init_ack *msg)
mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
+ /* Set default ABI version if dealing with unversioned firmware. */
+ vpu->fw_abi_version = 0;
+ /*
+* Instance ID is only used if ABI version >= 2. Initialize it with
+* garbage by default.
+*/
+ vpu->inst_id = 0xdeadbeef;
+
/* Firmware version field does not exist on MT8173. */
if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173)
return;
/* Check firmware version. */
- mtk_vcodec_debug(vpu, "firmware version 0x%x\n", msg->vdec_abi_version);
- switch (msg->vdec_abi_version) {
+ vpu->fw_abi_version = msg->vdec_abi_version;
+ mtk_vcodec_debug(vpu, "firmware version 0x%x\n", vpu->fw_abi_version);
+ switch (vpu->fw_abi_version) {
case 1:
break;
+ case 2:
+ vpu->inst_id = msg->inst_id;
+ break;
default:
mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n",
- msg->vdec_abi_version);
+ vpu->fw_abi_version);
vpu->failure = 1;
break;
}
@@ -113,7 +125,10 @@ static int vcodec_send_ap_ipi(struct vdec_vpu_inst *vpu,
unsigned int msg_id)
memset(, 0, sizeof(msg));
msg.msg_id = msg_id;
- msg.vpu_inst_addr = vpu->inst_addr;
+ if (vpu->fw_abi_version < 2)
+ msg.vpu_inst_addr = vpu->inst_addr;
+ else
+ msg.inst_id = vpu->inst_id;
err = vcodec_vpu_send_msg(vpu, , sizeof(msg));
mtk_vcodec_debug(vpu, "- id=%X ret=%d", msg_id, err);
@@ -163,7 +178,10 @@ int vpu_dec_start(struct vdec_vpu_inst *vpu, uint32_t
*data, unsigned int len)
memset(, 0, sizeof(msg));
msg.msg_id = AP_IPIMSG_DEC_START;
- msg.vpu_inst_addr = vpu->inst_addr;
+ if (vpu->fw_abi_version <
[PATCH v2 7/8] dt-bindings: media: document mediatek,mt8183-vcodec-dec
MT8183's decoder is instantiated similarly to MT8173's. Signed-off-by: Alexandre Courbot --- Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt index 8217424fd4bd..30c5888bf2d6 100644 --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt @@ -7,6 +7,7 @@ Required properties: - compatible : "mediatek,mt8173-vcodec-enc" for MT8173 encoder "mediatek,mt8183-vcodec-enc" for MT8183 encoder. "mediatek,mt8173-vcodec-dec" for MT8173 decoder. + "mediatek,mt8183-vcodec-dec" for MT8183 decoder. - reg : Physical base address of the video codec registers and length of memory mapped region. - interrupts : interrupt number to the cpu. -- 2.30.0.617.g56c4b15f3c-goog
[PATCH v2 6/8] media: mtk-vcodec: vdec: add media device if using stateless api
From: Yunfei Dong
The stateless API requires a media device for issuing requests. Add one
if we are being instantiated as a stateless decoder.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/Kconfig| 1 +
.../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 39 +++
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 2 +
3 files changed, 42 insertions(+)
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index c27db5643712..9d83b4223ecc 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -296,6 +296,7 @@ config VIDEO_MEDIATEK_VCODEC
select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
select V4L2_H264
+ select MEDIA_CONTROLLER
help
Mediatek video codec driver provides HW capability to
encode and decode in a range of video formats on MT8173
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 533781d4680a..e942e28f96fe 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -14,6 +14,7 @@
#include
#include
#include
+#include
#include "mtk_vcodec_drv.h"
#include "mtk_vcodec_dec.h"
@@ -324,6 +325,31 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
goto err_event_workq;
}
+ if (dev->vdec_pdata->uses_stateless_api) {
+ dev->mdev_dec.dev = >dev;
+ strscpy(dev->mdev_dec.model, MTK_VCODEC_DEC_NAME,
+ sizeof(dev->mdev_dec.model));
+
+ media_device_init(>mdev_dec);
+ dev->mdev_dec.ops = _vcodec_media_ops;
+ dev->v4l2_dev.mdev = >mdev_dec;
+
+ ret = v4l2_m2m_register_media_controller(dev->m2m_dev_dec,
+ dev->vfd_dec, MEDIA_ENT_F_PROC_VIDEO_DECODER);
+ if (ret) {
+ mtk_v4l2_err("Failed to register media controller");
+ goto err_reg_cont;
+ }
+
+ ret = media_device_register(>mdev_dec);
+ if (ret) {
+ mtk_v4l2_err("Failed to register media device");
+ goto err_media_reg;
+ }
+
+ mtk_v4l2_debug(0, "media registered as /dev/media%d",
+ vfd_dec->num);
+ }
ret = video_register_device(vfd_dec, VFL_TYPE_VIDEO, 0);
if (ret) {
mtk_v4l2_err("Failed to register video device");
@@ -336,6 +362,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
return 0;
err_dec_reg:
+ if (dev->vdec_pdata->uses_stateless_api)
+ media_device_unregister(>mdev_dec);
+err_media_reg:
+ if (dev->vdec_pdata->uses_stateless_api)
+ v4l2_m2m_unregister_media_controller(dev->m2m_dev_dec);
+err_reg_cont:
destroy_workqueue(dev->decode_workqueue);
err_event_workq:
v4l2_m2m_release(dev->m2m_dev_dec);
@@ -368,6 +400,13 @@ static int mtk_vcodec_dec_remove(struct platform_device
*pdev)
flush_workqueue(dev->decode_workqueue);
destroy_workqueue(dev->decode_workqueue);
+
+ if (media_devnode_is_registered(dev->mdev_dec.devnode)) {
+ media_device_unregister(>mdev_dec);
+ v4l2_m2m_unregister_media_controller(dev->m2m_dev_dec);
+ media_device_cleanup(>mdev_dec);
+ }
+
if (dev->m2m_dev_dec)
v4l2_m2m_release(dev->m2m_dev_dec);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 3b884a321883..79d6a1e6c916 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -384,6 +384,7 @@ struct mtk_vcodec_enc_pdata {
* struct mtk_vcodec_dev - driver data
* @v4l2_dev: V4L2 device to register video devices for.
* @vfd_dec: Video device for decoder
+ * @mdev_dec: Media device for decoder
* @vfd_enc: Video device for encoder.
*
* @m2m_dev_dec: m2m device for decoder
@@ -420,6 +421,7 @@ struct mtk_vcodec_enc_pdata {
struct mtk_vcodec_dev {
struct v4l2_device v4l2_dev;
struct video_device *vfd_dec;
+ struct media_device mdev_dec;
struct video_device *vfd_enc;
struct v4l2_m2m_dev *m2m_dev_dec;
--
2.30.0.617.g56c4b15f3c-goog
[PATCH v2 8/8] media: mtk-vcodec: enable MT8183 decoder
From: Yunfei Dong
Now that all the supporting blocks are present, enable decoder for
MT8183.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index e942e28f96fe..e0526c0900c8 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -383,12 +383,17 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
extern const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata;
+extern const struct mtk_vcodec_dec_pdata mtk_vdec_8183_pdata;
static const struct of_device_id mtk_vcodec_match[] = {
{
.compatible = "mediatek,mt8173-vcodec-dec",
.data = _vdec_8173_pdata,
},
+ {
+ .compatible = "mediatek,mt8183-vcodec-dec",
+ .data = _vdec_8183_pdata,
+ },
{},
};
--
2.30.0.617.g56c4b15f3c-goog
[PATCH v2 5/8] media: mtk-vcodec: vdec: support stateless H.264 decoding
From: Yunfei Dong
Add support for H.264 decoding using the stateless API, as supported by
MT8183. This support takes advantage of the V4L2 H.264 reference list
builders.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/Kconfig| 1 +
drivers/media/platform/mtk-vcodec/Makefile| 1 +
.../mtk-vcodec/vdec/vdec_h264_req_if.c| 808 ++
.../media/platform/mtk-vcodec/vdec_drv_if.c | 3 +
.../media/platform/mtk-vcodec/vdec_drv_if.h | 1 +
5 files changed, 814 insertions(+)
create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index fd1831e97b22..c27db5643712 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -295,6 +295,7 @@ config VIDEO_MEDIATEK_VCODEC
select V4L2_MEM2MEM_DEV
select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
+ select V4L2_H264
help
Mediatek video codec driver provides HW capability to
encode and decode in a range of video formats on MT8173
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 4ba93d838ab6..ca8e9e7a9c4e 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -7,6 +7,7 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
vdec/vdec_vp8_if.o \
vdec/vdec_vp9_if.o \
+ vdec/vdec_h264_req_if.o \
mtk_vcodec_dec_drv.o \
vdec_drv_if.o \
vdec_vpu_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
new file mode 100644
index ..8becc73fae35
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
@@ -0,0 +1,808 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "../vdec_drv_if.h"
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_dec.h"
+#include "../mtk_vcodec_intr.h"
+#include "../vdec_vpu_if.h"
+#include "../vdec_drv_base.h"
+
+#define NAL_NON_IDR_SLICE 0x01
+#define NAL_IDR_SLICE 0x05
+#define NAL_H264_PPS 0x08
+#define NAL_TYPE(value)((value) & 0x1F)
+
+#define BUF_PREDICTION_SZ (64 * 4096)
+#define MB_UNIT_LEN16
+
+/* get used parameters for sps/pps */
+#define GET_MTK_VDEC_FLAG(cond, flag) \
+ { dst_param->cond = ((src_param->flags & flag) ? (1) : (0)) }
+#define GET_MTK_VDEC_PARAM(param) \
+ { dst_param->param = src_param->param }
+/* motion vector size (bytes) for every macro block */
+#define HW_MB_STORE_SZ 64
+
+#define H264_MAX_FB_NUM17
+#define H264_MAX_MV_NUM32
+#define HDR_PARSING_BUF_SZ 1024
+
+/**
+ * struct mtk_h264_dpb_info - h264 dpb information
+ * @y_dma_addr: Y bitstream physical address
+ * @c_dma_addr: CbCr bitstream physical address
+ * @reference_flag: reference picture flag (short/long term reference picture)
+ * @field: field picture flag
+ */
+struct mtk_h264_dpb_info {
+ dma_addr_t y_dma_addr;
+ dma_addr_t c_dma_addr;
+ int reference_flag;
+ int field;
+};
+
+/**
+ * struct mtk_h264_sps_param - parameters for sps
+ */
+struct mtk_h264_sps_param {
+ unsigned char chroma_format_idc;
+ unsigned char bit_depth_luma_minus8;
+ unsigned char bit_depth_chroma_minus8;
+ unsigned char log2_max_frame_num_minus4;
+ unsigned char pic_order_cnt_type;
+ unsigned char log2_max_pic_order_cnt_lsb_minus4;
+ unsigned char max_num_ref_frames;
+ unsigned char separate_colour_plane_flag;
+ unsigned short pic_width_in_mbs_minus1;
+ unsigned short pic_height_in_map_units_minus1;
+ unsigned int max_frame_nums;
+ unsigned char qpprime_y_zero_transform_bypass_flag;
+ unsigned char delta_pic_order_always_zero_flag;
+ unsigned char frame_mbs_only_flag;
+ unsigned char mb_adaptive_frame_field_flag;
+ unsigned char direct_8x8_inference_flag;
+ unsigned char reserved[3];
+};
+
+/**
+ * struct mtk_h264_pps_param - parameters for pps
+ */
+struct mtk_h264_pps_param {
+ unsigned char num_ref_idx_l0_default_active_minus1;
+ unsigned char num_ref_idx_l1_default_active_minus1;
+ unsigned char weighted_bipred_idc;
+ char pic_init_qp_minus26;
+ char chro
[PATCH v2 4/8] media: mtk-vcodec: vdec: support stateless API
From: Yunfei Dong
Support the stateless codec API that will be used by MT8183.
Signed-off-by: Yunfei Dong
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/Makefile| 1 +
.../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 ++-
.../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +-
.../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 ++
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 3 +
5 files changed, 503 insertions(+), 3 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 9c3cbb5b800e..4ba93d838ab6 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -12,6 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
vdec_vpu_if.o \
mtk_vcodec_dec.o \
mtk_vcodec_dec_stateful.o \
+ mtk_vcodec_dec_stateless.o \
mtk_vcodec_dec_pm.o \
mtk-vcodec-enc-y := venc/venc_vp8_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 4a91d294002b..c286cc0f239f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -46,6 +46,13 @@ static struct mtk_q_data *mtk_vdec_get_q_data(struct
mtk_vcodec_ctx *ctx,
static int vidioc_try_decoder_cmd(struct file *file, void *priv,
struct v4l2_decoder_cmd *cmd)
{
+ struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+
+ /* Use M2M stateless helper if relevant */
+ if (ctx->dev->vdec_pdata->uses_stateless_api)
+ return v4l2_m2m_ioctl_stateless_try_decoder_cmd(file, priv,
+ cmd);
+
switch (cmd->cmd) {
case V4L2_DEC_CMD_STOP:
case V4L2_DEC_CMD_START:
@@ -72,6 +79,10 @@ static int vidioc_decoder_cmd(struct file *file, void *priv,
if (ret)
return ret;
+ /* Use M2M stateless helper if relevant */
+ if (ctx->dev->vdec_pdata->uses_stateless_api)
+ return v4l2_m2m_ioctl_stateless_decoder_cmd(file, priv, cmd);
+
mtk_v4l2_debug(1, "decoder cmd=%u", cmd->cmd);
dst_vq = v4l2_m2m_get_vq(ctx->m2m_ctx,
V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
@@ -414,7 +425,8 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
* Setting OUTPUT format after OUTPUT buffers are allocated is invalid
* if using the stateful API.
*/
- if ((f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
+ if (!dec_pdata->uses_stateless_api &&
+ (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) &&
vb2_is_busy(>m2m_ctx->out_q_ctx.q)) {
mtk_v4l2_err("out_q_ctx buffers already requested");
ret = -EBUSY;
@@ -457,6 +469,7 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
ctx->quantization = pix_mp->quantization;
ctx->xfer_func = pix_mp->xfer_func;
+ ctx->current_codec = fmt->fourcc;
if (ctx->state == MTK_STATE_FREE) {
ret = vdec_if_init(ctx, q_data->fmt->fourcc);
if (ret) {
@@ -468,6 +481,49 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
}
}
+ /*
+* If using the stateless API, S_FMT should have the effect of setting
+* the CAPTURE queue resolution no matter which queue it was called on.
+*/
+ if (dec_pdata->uses_stateless_api) {
+ ctx->picinfo.pic_w = pix_mp->width;
+ ctx->picinfo.pic_h = pix_mp->height;
+
+ ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, >picinfo);
+ if (ret) {
+ mtk_v4l2_err("[%d]Error!! Get GET_PARAM_PICTURE_INFO
Fail",
+ ctx->id);
+ return -EINVAL;
+ }
+
+ ctx->last_decoded_picinfo = ctx->picinfo;
+
+ if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 1) {
+ ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
+ ctx->picinfo.fb_sz[0] +
+ ctx->picinfo.fb_sz[1];
+ ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] =
+ ctx->picinfo.buf_w;
+ } else {
+ ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
+ ctx->picinfo.fb_sz[0];
+
[PATCH v2 3/8] media: add Mediatek's MM21 format
Add Mediatek's non-compressed 8 bit block video mode. This format is
produced by the MT8183 codec and can be converted to a non-proprietary
format by the MDP3 component.
Signed-off-by: Alexandre Courbot
---
Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 7 +++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h| 1 +
3 files changed, 9 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
index c9231e18859b..187ea89f7a25 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
@@ -242,6 +242,13 @@ please make a proposal on the linux-media mailing list.
It is an opaque intermediate format and the MDP hardware must be
used to convert ``V4L2_PIX_FMT_MT21C`` to ``V4L2_PIX_FMT_NV12M``,
``V4L2_PIX_FMT_YUV420M`` or ``V4L2_PIX_FMT_YVU420``.
+* .. _V4L2-PIX-FMT-MM21:
+
+ - ``V4L2_PIX_FMT_MM21``
+ - 'MM21'
+ - Non-compressed, tiled two-planar format used by Mediatek MT8183.
+ This is an opaque intermediate format and the MDP3 hardware can be
+ used to convert it to other formats.
* .. _V4L2-PIX-FMT-SUNXI-TILED-NV12:
- ``V4L2_PIX_FMT_SUNXI_TILED_NV12``
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 31d1342e61e8..0b85b2bbc628 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1384,6 +1384,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_TM6000: descr = "A/V + VBI Mux Packet"; break;
case V4L2_PIX_FMT_CIT_YYVYUY: descr = "GSPCA CIT YYVYUY"; break;
case V4L2_PIX_FMT_KONICA420:descr = "GSPCA KONICA420"; break;
+ case V4L2_PIX_FMT_MM21: descr = "Mediatek 8-bit block format";
break;
case V4L2_PIX_FMT_HSV24:descr = "24-bit HSV 8-8-8"; break;
case V4L2_PIX_FMT_HSV32:descr = "32-bit XHSV 8-8-8-8"; break;
case V4L2_SDR_FMT_CU8: descr = "Complex U8"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..e6890dae76ec 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -731,6 +731,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_Y12I v4l2_fourcc('Y', '1', '2', 'I') /* Greyscale
12-bit L/R interleaved */
#define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
16-bit */
#define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /* Mediatek
compressed block mode */
+#define V4L2_PIX_FMT_MM21 v4l2_fourcc('M', 'M', '2', '1') /* Mediatek
8-bit block mode, two non-contiguous planes */
#define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar
Greyscale 10-bit and Depth 16-bit */
#define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi
Tiled NV12 Format */
#define V4L2_PIX_FMT_CNF4 v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit
packed depth confidence information */
--
2.30.0.617.g56c4b15f3c-goog
[PATCH v2 1/8] media: mtk-vcodec: vdec: handle firmware version field
Firmwares for decoders newer than MT8173 will include an ABI version
number in their initialization ack message. Add the capacity to manage
it and make initialization fail if the firmware ABI is of a version that
we don't support.
For MT8173, this ABI version field does not exist ; thus ignore it on
this chip. There should only be one firmware version available for it
anyway.
Signed-off-by: Alexandre Courbot
---
.../mtk-vcodec/mtk_vcodec_dec_stateful.c | 1 +
.../platform/mtk-vcodec/mtk_vcodec_drv.h | 4
.../media/platform/mtk-vcodec/vdec_ipi_msg.h | 5 +
.../media/platform/mtk-vcodec/vdec_vpu_if.c | 21 +--
4 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
index 48b7524bc8fb..f9db7ef19c28 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
@@ -620,6 +620,7 @@ static struct vb2_ops mtk_vdec_frame_vb2_ops = {
};
const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata = {
+ .chip = MTK_MT8173,
.init_vdec_params = mtk_init_vdec_params,
.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
.vdec_vb2_ops = _vdec_frame_vb2_ops,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 9221c17a176b..60bc39efa20d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -322,6 +322,8 @@ enum mtk_chip {
* @vdec_framesizes: supported video decoder frame sizes
* @num_framesizes: count of video decoder frame sizes
*
+ * @chip: chip this decoder is compatible with
+ *
* @uses_stateless_api: whether the decoder uses the stateless API with
requests
*/
@@ -341,6 +343,8 @@ struct mtk_vcodec_dec_pdata {
const struct mtk_codec_framesizes *vdec_framesizes;
const int num_framesizes;
+ enum mtk_chip chip;
+
bool uses_stateless_api;
};
diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
index 47a1c1c0fd04..eb66729fda63 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
@@ -83,12 +83,17 @@ struct vdec_ap_ipi_dec_start {
* @status : VPU exeuction result
* @ap_inst_addr : AP vcodec_vpu_inst instance address
* @vpu_inst_addr : VPU decoder instance address
+ * @vdec_abi_version: ABI version of the firmware. Kernel can use it to
+ * ensure that it is compatible with the firmware.
+ * This field is not valid for MT8173 and must not be
+ * accessed for this chip.
*/
struct vdec_vpu_ipi_init_ack {
uint32_t msg_id;
int32_t status;
uint64_t ap_inst_addr;
uint32_t vpu_inst_addr;
+ uint32_t vdec_abi_version;
};
#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index 58b0e6fa8fd2..203089213e67 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -24,6 +24,22 @@ static void handle_init_ack_msg(const struct
vdec_vpu_ipi_init_ack *msg)
vpu->inst_addr = msg->vpu_inst_addr;
mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
+
+ /* Firmware version field does not exist on MT8173. */
+ if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173)
+ return;
+
+ /* Check firmware version. */
+ mtk_vcodec_debug(vpu, "firmware version 0x%x\n", msg->vdec_abi_version);
+ switch (msg->vdec_abi_version) {
+ case 1:
+ break;
+ default:
+ mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n",
+ msg->vdec_abi_version);
+ vpu->failure = 1;
+ break;
+ }
}
/*
@@ -44,6 +60,9 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len,
void *priv)
mtk_vcodec_debug(vpu, "+ id=%X", msg->msg_id);
+ vpu->failure = msg->status;
+ vpu->signaled = 1;
+
if (msg->status == 0) {
switch (msg->msg_id) {
case VPU_IPIMSG_DEC_INIT_ACK:
@@ -63,8 +82,6 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len,
void *priv)
}
mtk_vcodec_debug(vpu, "- id=%X", msg->msg_id);
- vpu->failure = msg->status;
- vpu->signaled = 1;
}
static int vcodec_vpu_send_msg(struct vdec_vpu_inst *vpu, void *msg, int len)
--
2.30.0.617.g56c4b15f3c-goog
[PATCH v2 0/8] media: mtk-vcodec: vdec: support for MT8183
Here is the second version is this patchset, although there has been such a delay after the first one that probably nobody remembers it has even been sent. This series adds support for the stateless API into mtk-vcodec, by first separating the stateful ops into their own source file, and introducing a new set of ops suitable for stateless decoding. As such, support for stateful decoders should remain completely unaffected. This series has been tested with both MT8183 and MT8173. Decoding was working for both chips, and in the case of MT8173 no regression has been spotted. v4l2-compliance is also passing, minor a problem when testing requests on MT8183: the test tries to call S_EXT_CTRLS on the first available control, which happens to be V4L2_CID_MIN_BUFFERS_FOR_CAPTURE. Since this control is read-only, the driver returns -EACCESS. I suppose this is a problem with v4l2-compliance and not the driver. Alexandre Courbot (4): media: mtk-vcodec: vdec: handle firmware version field media: mtk-vcodec: support version 2 of decoder firmware ABI media: add Mediatek's MM21 format dt-bindings: media: document mediatek,mt8183-vcodec-dec Yunfei Dong (4): media: mtk-vcodec: vdec: support stateless API media: mtk-vcodec: vdec: support stateless H.264 decoding media: mtk-vcodec: vdec: add media device if using stateless api media: mtk-vcodec: enable MT8183 decoder .../bindings/media/mediatek-vcodec.txt| 1 + .../media/v4l/pixfmt-reserved.rst | 7 + drivers/media/platform/Kconfig| 2 + drivers/media/platform/mtk-vcodec/Makefile| 2 + .../platform/mtk-vcodec/mtk_vcodec_dec.c | 66 +- .../platform/mtk-vcodec/mtk_vcodec_dec.h | 9 +- .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 44 + .../mtk-vcodec/mtk_vcodec_dec_stateful.c | 1 + .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 427 + .../platform/mtk-vcodec/mtk_vcodec_drv.h | 9 + .../mtk-vcodec/vdec/vdec_h264_req_if.c| 808 ++ .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 23 +- .../media/platform/mtk-vcodec/vdec_vpu_if.c | 43 +- .../media/platform/mtk-vcodec/vdec_vpu_if.h | 5 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h| 1 + 18 files changed, 1442 insertions(+), 11 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c -- 2.30.0.617.g56c4b15f3c-goog
Re: [PATCH 3/3] media: mtk-vcodec: Separating mtk encoder driver
On Sat, Feb 20, 2021 at 3:56 PM Irui Wang wrote:
>
> On Wed, 2021-02-03 at 19:44 +0900, Alexandre Courbot wrote:
> > Hi Irui,
> >
> > Thanks for pushing this forward. I had two small conflicts when
> > applying this patch to the media tree, so you may want to rebase
> > before sending the next version. Please see the comments inline.
> >
> > On Thu, Jan 21, 2021 at 3:18 PM Irui Wang wrote:
> > >
> > > MTK H264 Encoder(VENC_SYS) and VP8 Encoder(VENC_LT_SYS) are two
> > > independent hardware instance. They have their owner interrupt,
> > > register mapping, and special clocks.
> > >
> > > This patch seperates them into two drivers:
> >
> > seperates -> separates
> >
> > Also the patch does not result in two drivers, but two devices.
> >
> > > User Call "VIDIOC_QUERYCAP":
> > > H264 Encoder return driver name "mtk-vcodec-enc";
> > > VP8 Encoder return driver name "mtk-venc-vp8.
> >
> > I wonder if we need to use two different names? The driver is the
> > same, so it makes sense to me that both devices return
> > "mtk-vcodec-enc". Userspace can then list the formats on the CAPTURE
> > queue in order to query the supported codecs.
> >
> I'm afraid we can't, there is a symlink when chrome use the
> encoder(50-media.rules):
> ATTR{name} == "mtk-vcodec-enc", SYMLINK+="video-enc"
> ATTR{name} == "mtk-venc-vp8", SYMLINK+="video-enc0"
> if we use the same name,how userspace access the encoder? maybe there
> will be some modifications are needed in VEA(for example)?
Chrome OS can use a different udev rule to differentiate the two
nodes. Actually I already have a CL to support this:
https://chromium-review.googlesource.com/c/chromiumos/overlays/board-overlays/+/2673592
So both nodes being named the same won't be a problem for Chrome OS,
and makes more sense for an upstream merge anyway.
Cheers,
Alex.
Re: [PATCH v2] drm/nouveau/pmu: fix timeout on GP108
On Wed, Feb 17, 2021 at 1:20 AM Diego Viola wrote: > > This code times out on GP108, probably because the BIOS puts it into a > bad state. > > Since we reset the PMU on driver load anyway, we are at no risk from > missing a response from it since we are not waiting for one to begin > with. This looks safe to me, provided indeed that the PMU's reset is not called outside of initialization (which for GP108 is shouldn't be IIRC?). > > Signed-off-by: Diego Viola > --- > drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c | 6 +- > 1 file changed, 1 insertion(+), 5 deletions(-) > > diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c > b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c > index a0fe607c9c07..5c802f2d00cb 100644 > --- a/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c > +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.c > @@ -102,12 +102,8 @@ nvkm_pmu_reset(struct nvkm_pmu *pmu) > if (!pmu->func->enabled(pmu)) > return 0; > > - /* Inhibit interrupts, and wait for idle. */ > + /* Inhibit interrupts. */ > nvkm_wr32(device, 0x10a014, 0x); > - nvkm_msec(device, 2000, > - if (!nvkm_rd32(device, 0x10a04c)) > - break; > - ); > > /* Reset. */ > if (pmu->func->reset) > -- > 2.30.1 >
Re: [PATCH 3/3] media: mtk-vcodec: Separating mtk encoder driver
Hi Irui,
Thanks for pushing this forward. I had two small conflicts when
applying this patch to the media tree, so you may want to rebase
before sending the next version. Please see the comments inline.
On Thu, Jan 21, 2021 at 3:18 PM Irui Wang wrote:
>
> MTK H264 Encoder(VENC_SYS) and VP8 Encoder(VENC_LT_SYS) are two
> independent hardware instance. They have their owner interrupt,
> register mapping, and special clocks.
>
> This patch seperates them into two drivers:
seperates -> separates
Also the patch does not result in two drivers, but two devices.
> User Call "VIDIOC_QUERYCAP":
> H264 Encoder return driver name "mtk-vcodec-enc";
> VP8 Encoder return driver name "mtk-venc-vp8.
I wonder if we need to use two different names? The driver is the
same, so it makes sense to me that both devices return
"mtk-vcodec-enc". Userspace can then list the formats on the CAPTURE
queue in order to query the supported codecs.
>
> Signed-off-by: Hsin-Yi Wang
> Signed-off-by: Maoguang Meng
> Signed-off-by: Irui Wang
>
> ---
> .../platform/mtk-vcodec/mtk_vcodec_drv.h | 10 +-
> .../platform/mtk-vcodec/mtk_vcodec_enc.c | 23 +++-
> .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 121 +++---
> .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 40 +-
> .../platform/mtk-vcodec/venc/venc_vp8_if.c| 4 +-
> 5 files changed, 82 insertions(+), 116 deletions(-)
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> index 3dd010cba23e..1594edcc706d 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -19,6 +19,7 @@
> #define MTK_VCODEC_DRV_NAME"mtk_vcodec_drv"
> #define MTK_VCODEC_DEC_NAME"mtk-vcodec-dec"
> #define MTK_VCODEC_ENC_NAME"mtk-vcodec-enc"
> +#define MTK_VENC_VP8_NAME "mtk-venc-vp8"
> #define MTK_PLATFORM_STR "platform:mt8173"
>
> #define MTK_VCODEC_MAX_PLANES 3
> @@ -193,7 +194,6 @@ struct mtk_vcodec_pm {
>
> struct mtk_vcodec_clk venc_clk;
> struct device *larbvenc;
> - struct device *larbvenclt;
> struct device *dev;
> struct mtk_vcodec_dev *mtkdev;
> };
> @@ -311,25 +311,27 @@ enum mtk_chip {
> * @chip: chip this encoder is compatible with
> *
> * @uses_ext: whether the encoder uses the extended firmware messaging format
> - * @has_lt_irq: whether the encoder uses the LT irq
> + * @name: whether the encoder core is vp8
> * @min_birate: minimum supported encoding bitrate
> * @max_bitrate: maximum supported encoding bitrate
> * @capture_formats: array of supported capture formats
> * @num_capture_formats: number of entries in capture_formats
> * @output_formats: array of supported output formats
> * @num_output_formats: number of entries in output_formats
> + * @core_id: stand for h264 or vp8 encode index
> */
> struct mtk_vcodec_enc_pdata {
> enum mtk_chip chip;
>
> bool uses_ext;
> - bool has_lt_irq;
> + const char *name;
This new member can be removed if we use the same name for both devices.
> unsigned long min_bitrate;
> unsigned long max_bitrate;
> const struct mtk_video_fmt *capture_formats;
> size_t num_capture_formats;
> const struct mtk_video_fmt *output_formats;
> size_t num_output_formats;
> + int core_id;
> };
>
> #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
> @@ -361,7 +363,6 @@ struct mtk_vcodec_enc_pdata {
> *
> * @dec_irq: decoder irq resource
> * @enc_irq: h264 encoder irq resource
> - * @enc_lt_irq: vp8 encoder irq resource
> *
> * @dec_mutex: decoder hardware lock
> * @enc_mutex: encoder hardware lock.
> @@ -397,7 +398,6 @@ struct mtk_vcodec_dev {
>
> int dec_irq;
> int enc_irq;
> - int enc_lt_irq;
>
> struct mutex dec_mutex;
> struct mutex enc_mutex;
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> index 21de1431cfcb..0da6871b4b39 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -9,6 +9,7 @@
> #include
> #include
> #include
> +#include
>
> #include "mtk_vcodec_drv.h"
> #include "mtk_vcodec_enc.h"
> @@ -189,7 +190,10 @@ static int vidioc_enum_fmt_vid_out(struct file *file,
> void *priv,
> static int vidioc_venc_querycap(struct file *file, void *priv,
> struct v4l2_capability *cap)
> {
> - strscpy(cap->driver, MTK_VCODEC_ENC_NAME, sizeof(cap->driver));
> + const struct mtk_vcodec_enc_pdata *pdata =
> + fh_to_ctx(priv)->dev->venc_pdata;
> +
> + strscpy(cap->driver, pdata->name, sizeof(cap->driver));
> strscpy(cap->bus_info, MTK_PLATFORM_STR, sizeof(cap->bus_info));
> strscpy(cap->card, MTK_PLATFORM_STR,
Re: [PATCH 1/3] dt-bindings: media: mtk-vcodec: Separating mtk vcodec encoder node
On Thu, Jan 21, 2021 at 3:18 PM Irui Wang wrote:
>
> Updates binding document since the avc and vp8 hardware encoder in
> MT8173 are now separated. Separate "mediatek,mt8173-vcodec-enc" to
> "mediatek,mt8173-vcodec-vp8-enc" and "mediatek,mt8173-vcodec-avc-enc".
>
> Signed-off-by: Hsin-Yi Wang
> Signed-off-by: Maoguang Meng
> Signed-off-by: Irui Wang
>
> ---
> .../bindings/media/mediatek-vcodec.txt| 58 ++-
> 1 file changed, 31 insertions(+), 27 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> index 8217424fd4bd..f85276e629bf 100644
> --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -4,7 +4,9 @@ Mediatek Video Codec is the video codec hw present in
> Mediatek SoCs which
> supports high resolution encoding and decoding functionalities.
>
> Required properties:
> -- compatible : "mediatek,mt8173-vcodec-enc" for MT8173 encoder
> +- compatible : must be one of the following string:
> + "mediatek,mt8173-vcodec-vp8-enc" for mt8173 vp8 encoder.
> + "mediatek,mt8173-vcodec-avc-enc" for mt8173 avc encoder.
IMHO "mediatek,mt8173-vcodec-enc-vp8" and
"mediatek,mt8173-vcodec-enc-avc" would be more logical. Also to keep a
bit of backward compatibility, shall we also allow
"mediatek,mt8173-vcodec-enc" to be an alias for
"mediatek,mt8173-vcodec-enc-avc"? The line above would become
"mediatek,mt8173-vcodec-enc-avc" or "mediatek,mt8173-vcodec-enc" for
mt8173 avc encoder.
>"mediatek,mt8183-vcodec-enc" for MT8183 encoder.
>"mediatek,mt8173-vcodec-dec" for MT8173 decoder.
> - reg : Physical base address of the video codec registers and length of
> @@ -13,10 +15,11 @@ Required properties:
> - mediatek,larb : must contain the local arbiters in the current Socs.
> - clocks : list of clock specifiers, corresponding to entries in
>the clock-names property.
> -- clock-names: encoder must contain "venc_sel_src", "venc_sel",,
> - "venc_lt_sel_src", "venc_lt_sel", decoder must contain "vcodecpll",
> - "univpll_d2", "clk_cci400_sel", "vdec_sel", "vdecpll", "vencpll",
> - "venc_lt_sel", "vdec_bus_clk_src".
> +- clock-names:
> + avc venc must contain "venc_sel";
> + vp8 venc must contain "venc_lt_sel";
Can't we use "venc_sel" for both avc and vp8, since they are different
nodes now? That way we can just say
encoder must contain "venc_sel"
which is clearer and also simpler on the code side.
> + decoder must contain "vcodecpll", "univpll_d2", "clk_cci400_sel",
> + "vdec_sel", "vdecpll", "vencpll", "venc_lt_sel", "vdec_bus_clk_src".
> - iommus : should point to the respective IOMMU block with master port as
>argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
>for details.
> @@ -80,14 +83,10 @@ vcodec_dec: vcodec@1600 {
> assigned-clock-rates = <0>, <0>, <0>, <148200>, <8>;
>};
>
> - vcodec_enc: vcodec@18002000 {
> -compatible = "mediatek,mt8173-vcodec-enc";
> -reg = <0 0x18002000 0 0x1000>,/*VENC_SYS*/
> - <0 0x19002000 0 0x1000>;/*VENC_LT_SYS*/
> -interrupts = ,
> -;
> -mediatek,larb = <>,
> - <>;
> +vcodec_enc: vcodec@18002000 {
Let's use vcodec_enc_avc as a label?
> +compatible = "mediatek,mt8173-vcodec-avc-enc";
> +reg = <0 0x18002000 0 0x1000>;
> +interrupts = ;
> iommus = < M4U_PORT_VENC_RCPU>,
> < M4U_PORT_VENC_REC>,
> < M4U_PORT_VENC_BSDMA>,
> @@ -98,8 +97,20 @@ vcodec_dec: vcodec@1600 {
> < M4U_PORT_VENC_REF_LUMA>,
> < M4U_PORT_VENC_REF_CHROMA>,
> < M4U_PORT_VENC_NBM_RDMA>,
> - < M4U_PORT_VENC_NBM_WDMA>,
> - < M4U_PORT_VENC_RCPU_SET2>,
> + < M4U_PORT_VENC_NBM_WDMA>;
> +mediatek,larb = <>;
> +mediatek,vpu = <>;
> +clocks = < CLK_TOP_VENC_SEL>;
> +clock-names = "venc_sel";
> +assigned-clocks = < CLK_TOP_VENC_SEL>;
> +assigned-clock-parents = < CLK_TOP_VCODECPLL>;
> + };
> +
> +vcodec_enc_lt: vcodec@19002000 {
And here the label should probably be "vcodec_enc_vp8" for consistency.
> +compatible = "mediatek,mt8173-vcodec-vp8-enc";
> +reg = <0 0x19002000 0 0x1000>;/* VENC_LT_SYS */
> +interrupts = ;
> +iommus = < M4U_PORT_VENC_RCPU_SET2>,
> < M4U_PORT_VENC_REC_FRM_SET2>,
> < M4U_PORT_VENC_BSDMA_SET2>,
> < M4U_PORT_VENC_SV_COMA_SET2>,
> @@ -108,17 +119,10 @@ vcodec_dec: vcodec@1600 {
> < M4U_PORT_VENC_CUR_CHROMA_SET2>,
> < M4U_PORT_VENC_REF_LUMA_SET2>,
> < M4U_PORT_VENC_REC_CHROMA_SET2>;
> +mediatek,larb = <>;
> mediatek,vpu = <>;
> -clocks = < CLK_TOP_VENCPLL_D2>,
> - < CLK_TOP_VENC_SEL>,
> - < CLK_TOP_UNIVPLL1_D2>,
> - <
[PATCH] media: venus: use contig vb2 ops
This driver uses the SG vb2 ops, but effectively only ever accesses the first entry of the SG table, indicating that it expects a flat layout. Switch it to use the contiguous ops to make sure this expected invariant is always enforced. Since the device is supposed to be behind an IOMMU this should have little to none practical consequences beyond making the driver not rely on a particular behavior of the SG implementation. Reported-by: Tomasz Figa Signed-off-by: Alexandre Courbot --- Hi everyone, It probably doesn't hurt to fix this issue before some actual issue happens. I have tested this patch on Chrome OS and playback was just as fine as with the SG ops. drivers/media/platform/Kconfig | 2 +- drivers/media/platform/qcom/venus/helpers.c | 9 ++--- drivers/media/platform/qcom/venus/vdec.c| 6 +++--- drivers/media/platform/qcom/venus/venc.c| 6 +++--- 4 files changed, 9 insertions(+), 14 deletions(-) diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 35a18d388f3f..d9d7954111f2 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -533,7 +533,7 @@ config VIDEO_QCOM_VENUS depends on INTERCONNECT || !INTERCONNECT select QCOM_MDT_LOADER if ARCH_QCOM select QCOM_SCM if ARCH_QCOM - select VIDEOBUF2_DMA_SG + select VIDEOBUF2_DMA_CONTIG select V4L2_MEM2MEM_DEV help This is a V4L2 driver for Qualcomm Venus video accelerator diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c index 50439eb1ffea..859d260f002b 100644 --- a/drivers/media/platform/qcom/venus/helpers.c +++ b/drivers/media/platform/qcom/venus/helpers.c @@ -7,7 +7,7 @@ #include #include #include -#include +#include #include #include @@ -1284,14 +1284,9 @@ int venus_helper_vb2_buf_init(struct vb2_buffer *vb) struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue); struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); struct venus_buffer *buf = to_venus_buffer(vbuf); - struct sg_table *sgt; - - sgt = vb2_dma_sg_plane_desc(vb, 0); - if (!sgt) - return -EFAULT; buf->size = vb2_plane_size(vb, 0); - buf->dma_addr = sg_dma_address(sgt->sgl); + buf->dma_addr = vb2_dma_contig_plane_dma_addr(vb, 0); if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) list_add_tail(>reg_list, >registeredbufs); diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c index 8488411204c3..3fb277c81aca 100644 --- a/drivers/media/platform/qcom/venus/vdec.c +++ b/drivers/media/platform/qcom/venus/vdec.c @@ -13,7 +13,7 @@ #include #include #include -#include +#include #include "hfi_venus_io.h" #include "hfi_parser.h" @@ -1461,7 +1461,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->io_modes = VB2_MMAP | VB2_DMABUF; src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; src_vq->ops = _vb2_ops; - src_vq->mem_ops = _dma_sg_memops; + src_vq->mem_ops = _dma_contig_memops; src_vq->drv_priv = inst; src_vq->buf_struct_size = sizeof(struct venus_buffer); src_vq->allow_zero_bytesused = 1; @@ -1475,7 +1475,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq, dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; dst_vq->ops = _vb2_ops; - dst_vq->mem_ops = _dma_sg_memops; + dst_vq->mem_ops = _dma_contig_memops; dst_vq->drv_priv = inst; dst_vq->buf_struct_size = sizeof(struct venus_buffer); dst_vq->allow_zero_bytesused = 1; diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c index 1c61602c5de1..a09550cd1dba 100644 --- a/drivers/media/platform/qcom/venus/venc.c +++ b/drivers/media/platform/qcom/venus/venc.c @@ -10,7 +10,7 @@ #include #include #include -#include +#include #include #include #include @@ -1001,7 +1001,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF; src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; src_vq->ops = _vb2_ops; - src_vq->mem_ops = _dma_sg_memops; + src_vq->mem_ops = _dma_contig_memops; src_vq->drv_priv = inst; src_vq->buf_struct_size = sizeof(struct venus_buffer); src_vq->allow_zero_bytesused = 1; @@ -1017,7 +1017,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq, dst_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF; dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; dst_vq->ops = _vb2_ops; - dst_vq->mem_ops = _dma_sg
Re: [PATCH] media: mtk-vcodec: add remoteproc dependency
On Fri, Dec 4, 2020 at 8:15 AM Arnd Bergmann wrote:
>
> From: Arnd Bergmann
>
> The SCP firmware can only be built if CONFIG_REMOTEPROC is
> enabled:
>
> WARNING: unmet direct dependencies detected for MTK_SCP
> Depends on [n]: REMOTEPROC [=n] && (ARCH_MEDIATEK [=y] || COMPILE_TEST [=y])
> Selected by [y]:
> - VIDEO_MEDIATEK_VCODEC [=y] && MEDIA_SUPPORT [=y] &&
> MEDIA_PLATFORM_SUPPORT [=y] && V4L_MEM2MEM_DRIVERS [=y] && (MTK_IOMMU [=y] ||
> COMPILE_TEST [=y]) && VIDEO_DEV [=y] && VIDEO_V4L2 [=y] && (ARCH_MEDIATEK
> [=y] || COMPILE_TEST [=y])
Despite setting these same options I cannot reproduce this warning on
a merge of master + media. Which tree are you using?
>
> Add this as a dependency for mtk-vcodec.
>
> Fixes: c7244811b1c9 ("media: mtk-vcodec: add SCP firmware ops")
> Signed-off-by: Arnd Bergmann
> ---
> drivers/media/platform/Kconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index ef2267f4..295f74c3c04b 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -276,6 +276,7 @@ config VIDEO_MEDIATEK_VCODEC
> # our dependencies, to avoid missing symbols during link.
> depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
> depends on MTK_SCP || !MTK_SCP
> + depends on REMOTEPROC
> select VIDEOBUF2_DMA_CONTIG
> select V4L2_MEM2MEM_DEV
> select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
> --
> 2.27.0
>
Re: [PATCH] media: venus: preserve DRC state across seeks
On Wed, Dec 2, 2020 at 2:34 PM Alexandre Courbot wrote:
>
> DRC events can happen virtually at anytime, including when we are
> starting a seek. Should this happen, we must make sure to return to the
> DRC state, otherwise the firmware will expect buffers of the new
> resolution whereas userspace will still work with the old one.
>
> Returning to the DRC state upon resume for seeking makes sure that the
> client will get the DRC event and will reallocate the buffers to fit the
> firmware's expectations.
Oops, please ignore as this seems to depend on another patch... I will
repost once I can figure out the correct dependency chain, unless
Stanimir can find a better way to handle DRC during seek and flush.
>
> Signed-off-by: Alexandre Courbot
> ---
> drivers/media/platform/qcom/venus/vdec.c | 11 +--
> 1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
> index 8488411204c3..e3d0df7fd765 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -972,7 +972,10 @@ static int vdec_start_output(struct venus_inst *inst)
>
> if (inst->codec_state == VENUS_DEC_STATE_SEEK) {
> ret = venus_helper_process_initial_out_bufs(inst);
> - inst->codec_state = VENUS_DEC_STATE_DECODING;
> + if (inst->next_buf_last)
> + inst->codec_state = VENUS_DEC_STATE_DRC;
> + else
> + inst->codec_state = VENUS_DEC_STATE_DECODING;
> goto done;
> }
>
> @@ -1077,8 +1080,10 @@ static int vdec_stop_capture(struct venus_inst *inst)
> ret = hfi_session_flush(inst, HFI_FLUSH_ALL, true);
> fallthrough;
> case VENUS_DEC_STATE_DRAIN:
> - vdec_cancel_dst_buffers(inst);
> inst->codec_state = VENUS_DEC_STATE_STOPPED;
> + fallthrough;
> + case VENUS_DEC_STATE_SEEK:
> + vdec_cancel_dst_buffers(inst);
> break;
> case VENUS_DEC_STATE_DRC:
> WARN_ON(1);
> @@ -1102,6 +1107,7 @@ static int vdec_stop_output(struct venus_inst *inst)
> case VENUS_DEC_STATE_DECODING:
> case VENUS_DEC_STATE_DRAIN:
> case VENUS_DEC_STATE_STOPPED:
> + case VENUS_DEC_STATE_DRC:
> ret = hfi_session_flush(inst, HFI_FLUSH_ALL, true);
> inst->codec_state = VENUS_DEC_STATE_SEEK;
> break;
> @@ -1371,6 +1377,7 @@ static void vdec_event_change(struct venus_inst *inst,
> dev_dbg(dev, VDBGH "flush output error %d\n", ret);
> }
>
> + inst->next_buf_last = true;
> inst->reconfig = true;
> v4l2_event_queue_fh(>fh, );
> wake_up(>reconf_wait);
> --
> 2.29.2.454.gaff20da3a2-goog
>
Re: [PATCH v2] venus: vdec: Handle DRC after drain
On Wed, Dec 2, 2020 at 7:34 AM Stanimir Varbanov
wrote:
>
> Hi Fritz,
>
> On 12/1/20 6:23 AM, Fritz Koenig wrote:
> > If the DRC is near the end of the stream the client
> > may send a V4L2_DEC_CMD_STOP before the DRC occurs.
> > V4L2_DEC_CMD_STOP puts the driver into the
> > VENUS_DEC_STATE_DRAIN state. DRC must be aware so
> > that after the DRC event the state can be restored
> > correctly.
> >
> > Signed-off-by: Fritz Koenig
> > ---
> >
> > v2: remove TODO
> >
> > drivers/media/platform/qcom/venus/core.h | 1 +
> > drivers/media/platform/qcom/venus/vdec.c | 17 +++--
> > 2 files changed, 16 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/media/platform/qcom/venus/core.h
> > b/drivers/media/platform/qcom/venus/core.h
> > index 2b0899bf5b05f..1680c936c06fb 100644
> > --- a/drivers/media/platform/qcom/venus/core.h
> > +++ b/drivers/media/platform/qcom/venus/core.h
> > @@ -406,6 +406,7 @@ struct venus_inst {
> > unsigned int core_acquired: 1;
> > unsigned int bit_depth;
> > bool next_buf_last;
> > + bool drain_active;
>
> Could you introduce a new codec state instead of adding a flag for such
> corner case.
>
> I think that we will need to handle at least one more corner case (DRC
> during seek operation),
Just happen to have posted a patch for that. :)
https://lkml.org/lkml/2020/12/2/24
> so then we have to introduce another flag, and
> this is not a good solution to me. These additional flags will
> complicate the state handling and will make the code readability worst
>
> I'd introduce: VENUS_DEC_STATE_DRC_DURING_DRAIN or something similar.
I'm wondering what is the best approach here. As you see in my patch,
I did not have to introduce a new state but relied instead on the
state of next_buf_last (which may or may not be correct - maybe we can
think of a way to merge both patches into one?). Flushes, either
explicit or implicitly triggered by a DRC, are more than a state by
themselves but rather an extra dimension from which other states can
still apply. I'm afraid we already have many states as it is and
adding more might add complexity.
A lot of the recent issues we had came from that number of states,
notably from the fact that not all states are always tested when they
should (and fall back to the default: branch of a switch case that
does nothing). I think we could improve the robustness of this driver
if we mandate that each state check must be done using a switch
statement without a default: branch. That would force us to ensure
that each newly introduced state is considered in every situation
where it might be relevant.
>
> > };
> >
> > #define IS_V1(core) ((core)->res->hfi_version == HFI_VERSION_1XX)
> > diff --git a/drivers/media/platform/qcom/venus/vdec.c
> > b/drivers/media/platform/qcom/venus/vdec.c
> > index 5671cf3458a68..1d551b4d651a8 100644
> > --- a/drivers/media/platform/qcom/venus/vdec.c
> > +++ b/drivers/media/platform/qcom/venus/vdec.c
> > @@ -523,8 +523,10 @@ vdec_decoder_cmd(struct file *file, void *fh, struct
> > v4l2_decoder_cmd *cmd)
> >
> > ret = hfi_session_process_buf(inst, );
> >
> > - if (!ret && inst->codec_state == VENUS_DEC_STATE_DECODING)
> > + if (!ret && inst->codec_state == VENUS_DEC_STATE_DECODING) {
> > inst->codec_state = VENUS_DEC_STATE_DRAIN;
> > + inst->drain_active = true;
> > + }
> > }
> >
> > unlock:
> > @@ -976,10 +978,14 @@ static int vdec_start_capture(struct venus_inst *inst)
> >
> > inst->codec_state = VENUS_DEC_STATE_DECODING;
> >
> > + if (inst->drain_active)
> > + inst->codec_state = VENUS_DEC_STATE_DRAIN;
> > +
> > inst->streamon_cap = 1;
> > inst->sequence_cap = 0;
> > inst->reconfig = false;
> > inst->next_buf_last = false;
> > + inst->drain_active = false;
> >
> > return 0;
> >
> > @@ -1105,6 +,7 @@ static int vdec_stop_capture(struct venus_inst *inst)
> > /* fallthrough */
> > case VENUS_DEC_STATE_DRAIN:
> > inst->codec_state = VENUS_DEC_STATE_STOPPED;
> > + inst->drain_active = false;
> > /* fallthrough */
> > case VENUS_DEC_STATE_SEEK:
> > vdec_cancel_dst_buffers(inst);
> > @@ -1304,8 +1311,10 @@ static void vdec_buf_done(struct venus_inst *inst,
> > unsigned int buf_type,
> >
> > v4l2_event_queue_fh(>fh, );
> >
> > - if (inst->codec_state == VENUS_DEC_STATE_DRAIN)
> > + if (inst->codec_state == VENUS_DEC_STATE_DRAIN) {
> > + inst->drain_active = false;
> > inst->codec_state = VENUS_DEC_STATE_STOPPED;
> > + }
> > }
> >
> > if (!bytesused)
> > @@ -1429,9 +1438,13 @@ static void vdec_event_notify(struct venus_inst
> > *inst, u32 event,
> > case EVT_SYS_EVENT_CHANGE:
> >
[PATCH] media: venus: preserve DRC state across seeks
DRC events can happen virtually at anytime, including when we are
starting a seek. Should this happen, we must make sure to return to the
DRC state, otherwise the firmware will expect buffers of the new
resolution whereas userspace will still work with the old one.
Returning to the DRC state upon resume for seeking makes sure that the
client will get the DRC event and will reallocate the buffers to fit the
firmware's expectations.
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/qcom/venus/vdec.c | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/vdec.c
b/drivers/media/platform/qcom/venus/vdec.c
index 8488411204c3..e3d0df7fd765 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -972,7 +972,10 @@ static int vdec_start_output(struct venus_inst *inst)
if (inst->codec_state == VENUS_DEC_STATE_SEEK) {
ret = venus_helper_process_initial_out_bufs(inst);
- inst->codec_state = VENUS_DEC_STATE_DECODING;
+ if (inst->next_buf_last)
+ inst->codec_state = VENUS_DEC_STATE_DRC;
+ else
+ inst->codec_state = VENUS_DEC_STATE_DECODING;
goto done;
}
@@ -1077,8 +1080,10 @@ static int vdec_stop_capture(struct venus_inst *inst)
ret = hfi_session_flush(inst, HFI_FLUSH_ALL, true);
fallthrough;
case VENUS_DEC_STATE_DRAIN:
- vdec_cancel_dst_buffers(inst);
inst->codec_state = VENUS_DEC_STATE_STOPPED;
+ fallthrough;
+ case VENUS_DEC_STATE_SEEK:
+ vdec_cancel_dst_buffers(inst);
break;
case VENUS_DEC_STATE_DRC:
WARN_ON(1);
@@ -1102,6 +1107,7 @@ static int vdec_stop_output(struct venus_inst *inst)
case VENUS_DEC_STATE_DECODING:
case VENUS_DEC_STATE_DRAIN:
case VENUS_DEC_STATE_STOPPED:
+ case VENUS_DEC_STATE_DRC:
ret = hfi_session_flush(inst, HFI_FLUSH_ALL, true);
inst->codec_state = VENUS_DEC_STATE_SEEK;
break;
@@ -1371,6 +1377,7 @@ static void vdec_event_change(struct venus_inst *inst,
dev_dbg(dev, VDBGH "flush output error %d\n", ret);
}
+ inst->next_buf_last = true;
inst->reconfig = true;
v4l2_event_queue_fh(>fh, );
wake_up(>reconf_wait);
--
2.29.2.454.gaff20da3a2-goog
Re: [PATCH] venus: venc: Add VIDIOC_TRY_ENCODER_CMD support
On Tue, Dec 1, 2020 at 1:07 PM Fritz Koenig wrote:
>
> On Mon, Nov 30, 2020 at 7:24 PM Alexandre Courbot
> wrote:
> >
> > On Sun, Nov 29, 2020 at 3:05 PM Fritz Koenig wrote:
> > >
> > > V4L2_ENC_CMD_STOP and V4L2_ENC_CMD_START are already
> > > supported. Add a way to query for support.
> >
> > I think your Signed-off-by is missing (checkpatch.pl should warn you
> > about such problems).
> >
> > >
> > > ---
> > > drivers/media/platform/qcom/venus/venc.c | 26
> > > 1 file changed, 26 insertions(+)
> > >
> > > diff --git a/drivers/media/platform/qcom/venus/venc.c
> > > b/drivers/media/platform/qcom/venus/venc.c
> > > index 2ddfeddf98514..e05db3c4bfb24 100644
> > > --- a/drivers/media/platform/qcom/venus/venc.c
> > > +++ b/drivers/media/platform/qcom/venus/venc.c
> > > @@ -507,6 +507,27 @@ static int venc_enum_frameintervals(struct file
> > > *file, void *fh,
> > > return 0;
> > > }
> > >
> > > +static int
> > > +venc_try_encoder_cmd(struct file *file, void *fh, struct
> > > v4l2_encoder_cmd *cmd)
> > > +{
> > > + struct venus_inst *inst = to_inst(file);
> > > + struct device *dev = inst->core->dev_dec;
> > > +
> > > + switch (cmd->cmd) {
> > > + case V4L2_ENC_CMD_STOP:
> > > + case V4L2_ENC_CMD_START:
> > > + if (cmd->flags != 0) {
> > > + dev_dbg(dev, "flags=%u are not supported",
> > > cmd->flags);
> > > + return -EINVAL;
> > > + }
> > > + break;
> > > + default:
> > > + return -EINVAL;
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > static int
> > > venc_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd
> > > *cmd)
> >
> > I am not seeing venc_encoder_cmd() in the media tree, does this patch
> > depend on others that are not yet merged? If so they should be
> > submitted together as a series.
> >
>
> Sorry, I'm still a little unsure of procedures here. There is another
> patch set[1] posted
> and I thought it was missing this part. It turns out I had not
> applied the whole set to
> my tree.
>
> > > {
> > > @@ -514,6 +535,10 @@ venc_encoder_cmd(struct file *file, void *fh, struct
> > > v4l2_encoder_cmd *cmd)
> > > struct hfi_frame_data fdata = {0};
> > > int ret = 0;
> > >
> > > + ret = venc_try_encoder_cmd(file, fh, cmd);
> > > + if (ret < 0)
> > > + return ret;
> > > +
> >
> > v4l2_m2m_ioctl_try_encoder_cmd() is called right below, and AFAICT
> > does the same thing as the newly-defined venc_try_encoder_cmd(). So
> > IIUC this patch can be turned into a one-liner that does just the
> > following:
> >
> > @@ -575,6 +600,7 @@ static const struct v4l2_ioctl_ops venc_ioctl_ops = {
> > .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > .vidioc_encoder_cmd = venc_encoder_cmd,
> > + .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd,
> > };
> >
> Yes, that's how it is in the current patch[2], which is why I may have
> missed it.
> (I'm embarrassed because I reviewed that patch and then posted mine.)
Ah, ack - no worries, I am also far from being on top of everything
and missed the connection with that patchset. :)
I guess we can drop this one then.
>
> > > ret = v4l2_m2m_ioctl_try_encoder_cmd(file, fh, cmd);
> > > if (ret)
> > > return ret;
> > > @@ -575,6 +600,7 @@ static const struct v4l2_ioctl_ops venc_ioctl_ops = {
> > > .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> > > .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > > .vidioc_encoder_cmd = venc_encoder_cmd,
> > > + .vidioc_try_encoder_cmd = venc_try_encoder_cmd,
> > > };
> > >
> > > static int venc_set_properties(struct venus_inst *inst)
> > > --
> > > 2.29.2.454.gaff20da3a2-goog
> > >
>
> [1] : https://patchwork.kernel.org/project/linux-media/list/?series=382113
> [2]:
> https://patchwork.kernel.org/project/linux-media/patch/2020143755.24541-7-stanimir.varba...@linaro.org/
Re: [PATCH] venus: venc: Add VIDIOC_TRY_ENCODER_CMD support
On Sun, Nov 29, 2020 at 3:05 PM Fritz Koenig wrote:
>
> V4L2_ENC_CMD_STOP and V4L2_ENC_CMD_START are already
> supported. Add a way to query for support.
I think your Signed-off-by is missing (checkpatch.pl should warn you
about such problems).
>
> ---
> drivers/media/platform/qcom/venus/venc.c | 26
> 1 file changed, 26 insertions(+)
>
> diff --git a/drivers/media/platform/qcom/venus/venc.c
> b/drivers/media/platform/qcom/venus/venc.c
> index 2ddfeddf98514..e05db3c4bfb24 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -507,6 +507,27 @@ static int venc_enum_frameintervals(struct file *file,
> void *fh,
> return 0;
> }
>
> +static int
> +venc_try_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd
> *cmd)
> +{
> + struct venus_inst *inst = to_inst(file);
> + struct device *dev = inst->core->dev_dec;
> +
> + switch (cmd->cmd) {
> + case V4L2_ENC_CMD_STOP:
> + case V4L2_ENC_CMD_START:
> + if (cmd->flags != 0) {
> + dev_dbg(dev, "flags=%u are not supported",
> cmd->flags);
> + return -EINVAL;
> + }
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> static int
> venc_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd *cmd)
I am not seeing venc_encoder_cmd() in the media tree, does this patch
depend on others that are not yet merged? If so they should be
submitted together as a series.
> {
> @@ -514,6 +535,10 @@ venc_encoder_cmd(struct file *file, void *fh, struct
> v4l2_encoder_cmd *cmd)
> struct hfi_frame_data fdata = {0};
> int ret = 0;
>
> + ret = venc_try_encoder_cmd(file, fh, cmd);
> + if (ret < 0)
> + return ret;
> +
v4l2_m2m_ioctl_try_encoder_cmd() is called right below, and AFAICT
does the same thing as the newly-defined venc_try_encoder_cmd(). So
IIUC this patch can be turned into a one-liner that does just the
following:
@@ -575,6 +600,7 @@ static const struct v4l2_ioctl_ops venc_ioctl_ops = {
.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_encoder_cmd = venc_encoder_cmd,
+ .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd,
};
> ret = v4l2_m2m_ioctl_try_encoder_cmd(file, fh, cmd);
> if (ret)
> return ret;
> @@ -575,6 +600,7 @@ static const struct v4l2_ioctl_ops venc_ioctl_ops = {
> .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> .vidioc_encoder_cmd = venc_encoder_cmd,
> + .vidioc_try_encoder_cmd = venc_try_encoder_cmd,
> };
>
> static int venc_set_properties(struct venus_inst *inst)
> --
> 2.29.2.454.gaff20da3a2-goog
>
Re: [PATCH 2/3] venus: Limit HFI sessions to the maximum supported
On Fri, Nov 27, 2020 at 7:42 AM Stanimir Varbanov
wrote:
>
>
>
> On 11/26/20 8:28 AM, Alexandre Courbot wrote:
> > On Wed, Nov 25, 2020 at 10:01 PM Stanimir Varbanov
> > wrote:
> >>
> >>
> >>
> >> On 11/25/20 5:46 AM, Alexandre Courbot wrote:
> >>> On Fri, Nov 20, 2020 at 9:12 AM Stanimir Varbanov
> >>> wrote:
> >>>>
> >>>> Currently we rely on firmware to return error when we reach the maximum
> >>>> supported number of sessions. But this errors are happened at reqbuf
> >>>> time which is a bit later. The more reasonable way looks like is to
> >>>> return the error on driver open.
> >>>>
> >>>> To achieve that modify hfi_session_create to return error when we reach
> >>>> maximum count of sessions and thus refuse open.
> >>>>
> >>>> Signed-off-by: Stanimir Varbanov
> >>>> ---
> >>>> drivers/media/platform/qcom/venus/core.h | 1 +
> >>>> drivers/media/platform/qcom/venus/hfi.c | 19 +++
> >>>> .../media/platform/qcom/venus/hfi_parser.c| 3 +++
> >>>> 3 files changed, 19 insertions(+), 4 deletions(-)
> >>>>
> >>>> diff --git a/drivers/media/platform/qcom/venus/core.h
> >>>> b/drivers/media/platform/qcom/venus/core.h
> >>>> index db0e6738281e..3a477fcdd3a8 100644
> >>>> --- a/drivers/media/platform/qcom/venus/core.h
> >>>> +++ b/drivers/media/platform/qcom/venus/core.h
> >>>> @@ -96,6 +96,7 @@ struct venus_format {
> >>>> #define MAX_CAP_ENTRIES32
> >>>> #define MAX_ALLOC_MODE_ENTRIES 16
> >>>> #define MAX_CODEC_NUM 32
> >>>> +#define MAX_SESSIONS 16
> >>>>
> >>>> struct raw_formats {
> >>>> u32 buftype;
> >>>> diff --git a/drivers/media/platform/qcom/venus/hfi.c
> >>>> b/drivers/media/platform/qcom/venus/hfi.c
> >>>> index 638ed5cfe05e..8420be6d3991 100644
> >>>> --- a/drivers/media/platform/qcom/venus/hfi.c
> >>>> +++ b/drivers/media/platform/qcom/venus/hfi.c
> >>>> @@ -175,6 +175,7 @@ static int wait_session_msg(struct venus_inst *inst)
> >>>> int hfi_session_create(struct venus_inst *inst, const struct
> >>>> hfi_inst_ops *ops)
> >>>> {
> >>>> struct venus_core *core = inst->core;
> >>>> + int ret;
> >>>>
> >>>> if (!ops)
> >>>> return -EINVAL;
> >>>> @@ -183,12 +184,22 @@ int hfi_session_create(struct venus_inst *inst,
> >>>> const struct hfi_inst_ops *ops)
> >>>> init_completion(>done);
> >>>> inst->ops = ops;
> >>>>
> >>>> - mutex_lock(>lock);
> >>>> - list_add_tail(>list, >instances);
> >>>> - atomic_inc(>insts_count);
> >>>> + ret = mutex_lock_interruptible(>lock);
> >>>> + if (ret)
> >>>> + return ret;
> >>>
> >>> Why do we change to mutex_lock_interruptible() here? This makes this
> >>
> >> Because mutex_lock_interruptible is preferable in kernel docs, but I
> >> agree that changing mutex_lock with mutex_lock_interruptible should be
> >> subject of another lock related patches. I will drop this in next patch
> >> version.
> >>
> >>> function return an error even though we could obtain the lock just by
> >>> trying a bit harder.
> >>
> >> I didn't get that. The behavior of mutex_lock_interruptible is that same
> >> as mutex_lock, i.e. the it will sleep to acquire the lock. The
> >> difference is that the sleep could be interrupted by a signal. You might
> >> think about mutex_trylock?
> >
> > Unless that mutex can be held by someone else for a rather long time
> > (i.e. to the point where we may want to give priority to signals when
> > userspace opens the device, since that's where hfi_session_create() is
> > called), I am not convinced this change is necessary? It may confuse
>
> Exactly, if there is a case where the core->lock is taken (firmware
> recovery) and it is not unlocked for very long time (deadlock?) then
> client process cannot be interrupted with a signal.
>
> > userspace into thinking there was a serious error while there is none.
>
> The client should be able to handle EINTR, right?
>
> > Granted, userspace should manage this case, and from what I can see
> > this code is correct, but I'm not sure we would gain anything by
> > adding this extra complexity.
>
> The benefit is that if something wrong is happening in the driver the
> client process will be killable.
Ack, that definitely makes sense in that context, even though it
should probably be done separately from this patch series. :)
Cheers,
Alex.
Re: [PATCH 2/3] venus: Limit HFI sessions to the maximum supported
On Wed, Nov 25, 2020 at 10:01 PM Stanimir Varbanov
wrote:
>
>
>
> On 11/25/20 5:46 AM, Alexandre Courbot wrote:
> > On Fri, Nov 20, 2020 at 9:12 AM Stanimir Varbanov
> > wrote:
> >>
> >> Currently we rely on firmware to return error when we reach the maximum
> >> supported number of sessions. But this errors are happened at reqbuf
> >> time which is a bit later. The more reasonable way looks like is to
> >> return the error on driver open.
> >>
> >> To achieve that modify hfi_session_create to return error when we reach
> >> maximum count of sessions and thus refuse open.
> >>
> >> Signed-off-by: Stanimir Varbanov
> >> ---
> >> drivers/media/platform/qcom/venus/core.h | 1 +
> >> drivers/media/platform/qcom/venus/hfi.c | 19 +++
> >> .../media/platform/qcom/venus/hfi_parser.c| 3 +++
> >> 3 files changed, 19 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/qcom/venus/core.h
> >> b/drivers/media/platform/qcom/venus/core.h
> >> index db0e6738281e..3a477fcdd3a8 100644
> >> --- a/drivers/media/platform/qcom/venus/core.h
> >> +++ b/drivers/media/platform/qcom/venus/core.h
> >> @@ -96,6 +96,7 @@ struct venus_format {
> >> #define MAX_CAP_ENTRIES32
> >> #define MAX_ALLOC_MODE_ENTRIES 16
> >> #define MAX_CODEC_NUM 32
> >> +#define MAX_SESSIONS 16
> >>
> >> struct raw_formats {
> >> u32 buftype;
> >> diff --git a/drivers/media/platform/qcom/venus/hfi.c
> >> b/drivers/media/platform/qcom/venus/hfi.c
> >> index 638ed5cfe05e..8420be6d3991 100644
> >> --- a/drivers/media/platform/qcom/venus/hfi.c
> >> +++ b/drivers/media/platform/qcom/venus/hfi.c
> >> @@ -175,6 +175,7 @@ static int wait_session_msg(struct venus_inst *inst)
> >> int hfi_session_create(struct venus_inst *inst, const struct hfi_inst_ops
> >> *ops)
> >> {
> >> struct venus_core *core = inst->core;
> >> + int ret;
> >>
> >> if (!ops)
> >> return -EINVAL;
> >> @@ -183,12 +184,22 @@ int hfi_session_create(struct venus_inst *inst,
> >> const struct hfi_inst_ops *ops)
> >> init_completion(>done);
> >> inst->ops = ops;
> >>
> >> - mutex_lock(>lock);
> >> - list_add_tail(>list, >instances);
> >> - atomic_inc(>insts_count);
> >> + ret = mutex_lock_interruptible(>lock);
> >> + if (ret)
> >> + return ret;
> >
> > Why do we change to mutex_lock_interruptible() here? This makes this
>
> Because mutex_lock_interruptible is preferable in kernel docs, but I
> agree that changing mutex_lock with mutex_lock_interruptible should be
> subject of another lock related patches. I will drop this in next patch
> version.
>
> > function return an error even though we could obtain the lock just by
> > trying a bit harder.
>
> I didn't get that. The behavior of mutex_lock_interruptible is that same
> as mutex_lock, i.e. the it will sleep to acquire the lock. The
> difference is that the sleep could be interrupted by a signal. You might
> think about mutex_trylock?
Unless that mutex can be held by someone else for a rather long time
(i.e. to the point where we may want to give priority to signals when
userspace opens the device, since that's where hfi_session_create() is
called), I am not convinced this change is necessary? It may confuse
userspace into thinking there was a serious error while there is none.
Granted, userspace should manage this case, and from what I can see
this code is correct, but I'm not sure we would gain anything by
adding this extra complexity.
>
> >
> >> +
> >> + ret = atomic_read(>insts_count);
> >> + if (ret + 1 > core->max_sessions_supported) {
> >> + ret = -EAGAIN;
> >> + } else {
> >> + atomic_inc(>insts_count);
> >> + list_add_tail(>list, >instances);
> >> + ret = 0;
> >> + }
> >> +
> >> mutex_unlock(>lock);
> >>
> >> - return 0;
> >> + return ret;
> >> }
> >> EXPORT_SYMBOL_GPL(hfi_session_create);
> >>
> >> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c
> >> b/drivers/media/platform/qcom/venus/hfi_parser.c
> >> index 363ee2a65453..52898633a8e6 100644
> >> --- a/drivers/media/platform/qcom/venus/hfi_parser.c
> >> +++ b/drivers/media/platform/qcom/venus/hfi_parser.c
> >> @@ -276,6 +276,9 @@ u32 hfi_parser(struct venus_core *core, struct
> >> venus_inst *inst, void *buf,
> >> words_count--;
> >> }
> >>
> >> + if (!core->max_sessions_supported)
> >> + core->max_sessions_supported = MAX_SESSIONS;
> >> +
> >> parser_fini(inst, codecs, domain);
> >>
> >> return HFI_ERR_NONE;
> >> --
> >> 2.17.1
> >>
>
> --
> regards,
> Stan
Re: [PATCH v4 0/2] media: mtk-vcodec: fix builds when remoteproc is disabled
On Tue, Nov 24, 2020 at 5:39 PM Sakari Ailus wrote: > > On Tue, Nov 24, 2020 at 08:56:00AM +0900, Alexandre Courbot wrote: > > Gentle ping about this - we are already well into the 5.10 cycle so we > > don't have much time left if we want to merge this build breakage > > fix... > > Thanks for the ping. > > For the set: > > Acked-by: Sakari Ailus Ah, just saw these patches included in Mauro's latest pull request. Seems all is good, thanks! :)
Re: [PATCH 3/3] media: hfi_venus: Request interrupt for sync cmds
On Fri, Nov 20, 2020 at 9:12 AM Stanimir Varbanov
wrote:
>
> From: Vikash Garodia
>
> For synchronous commands, update the message queue variable.
> This would inform video firmware to raise interrupt on host
> CPU whenever there is a response for such commands.
>
> Signed-off-by: Vikash Garodia
> Signed-off-by: Stanimir Varbanov
> ---
> drivers/media/platform/qcom/venus/hfi_venus.c | 74 ++-
> 1 file changed, 41 insertions(+), 33 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
> b/drivers/media/platform/qcom/venus/hfi_venus.c
> index 4be4a75ddcb6..b8fdb464ba9c 100644
> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
> @@ -372,7 +372,7 @@ static void venus_soft_int(struct venus_hfi_device *hdev)
> }
>
> static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev,
> -void *pkt)
> +void *pkt, bool sync)
> {
> struct device *dev = hdev->core->dev;
> struct hfi_pkt_hdr *cmd_packet;
> @@ -397,15 +397,23 @@ static int venus_iface_cmdq_write_nolock(struct
> venus_hfi_device *hdev,
> if (rx_req)
> venus_soft_int(hdev);
>
> + /* Inform video firmware to raise interrupt for synchronous commands
> */
> + queue = >queues[IFACEQ_MSG_IDX];
> + if (sync) {
> + queue->qhdr->rx_req = 1;
> + /* ensure rx_req is updated in memory */
> + wmb();
> + }
Wouldn't it be safer to do this before calling venus_soft_int()? I
don't know what the firmware is supposed to do with rx_req but
intuitively it looks like it should be set before we signal it.
> +
> return 0;
> }
>
> -static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt)
> +static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt,
> bool sync)
> {
> int ret;
>
> mutex_lock(>lock);
> - ret = venus_iface_cmdq_write_nolock(hdev, pkt);
> + ret = venus_iface_cmdq_write_nolock(hdev, pkt, sync);
> mutex_unlock(>lock);
>
> return ret;
> @@ -428,7 +436,7 @@ static int venus_hfi_core_set_resource(struct venus_core
> *core, u32 id,
> if (ret)
> return ret;
>
> - ret = venus_iface_cmdq_write(hdev, pkt);
> + ret = venus_iface_cmdq_write(hdev, pkt, false);
> if (ret)
> return ret;
>
> @@ -778,7 +786,7 @@ static int venus_sys_set_debug(struct venus_hfi_device
> *hdev, u32 debug)
>
> pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug);
>
> - ret = venus_iface_cmdq_write(hdev, pkt);
> + ret = venus_iface_cmdq_write(hdev, pkt, false);
> if (ret)
> return ret;
>
> @@ -795,7 +803,7 @@ static int venus_sys_set_coverage(struct venus_hfi_device
> *hdev, u32 mode)
>
> pkt_sys_coverage_config(pkt, mode);
>
> - ret = venus_iface_cmdq_write(hdev, pkt);
> + ret = venus_iface_cmdq_write(hdev, pkt, false);
> if (ret)
> return ret;
>
> @@ -816,7 +824,7 @@ static int venus_sys_set_idle_message(struct
> venus_hfi_device *hdev,
>
> pkt_sys_idle_indicator(pkt, enable);
>
> - ret = venus_iface_cmdq_write(hdev, pkt);
> + ret = venus_iface_cmdq_write(hdev, pkt, false);
> if (ret)
> return ret;
>
> @@ -834,7 +842,7 @@ static int venus_sys_set_power_control(struct
> venus_hfi_device *hdev,
>
> pkt_sys_power_control(pkt, enable);
>
> - ret = venus_iface_cmdq_write(hdev, pkt);
> + ret = venus_iface_cmdq_write(hdev, pkt, false);
> if (ret)
> return ret;
>
> @@ -885,14 +893,14 @@ static int venus_sys_set_default_properties(struct
> venus_hfi_device *hdev)
> return ret;
> }
>
> -static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type)
> +static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type, bool
> sync)
> {
> struct venus_hfi_device *hdev = to_hfi_priv(inst->core);
> struct hfi_session_pkt pkt;
>
> pkt_session_cmd(, pkt_type, inst);
>
> - return venus_iface_cmdq_write(hdev, );
> + return venus_iface_cmdq_write(hdev, , sync);
> }
>
> static void venus_flush_debug_queue(struct venus_hfi_device *hdev)
> @@ -922,7 +930,7 @@ static int venus_prepare_power_collapse(struct
> venus_hfi_device *hdev,
>
> pkt_sys_pc_prep();
>
> - ret = venus_iface_cmdq_write(hdev, );
> + ret = venus_iface_cmdq_write(hdev, , false);
> if (ret)
> return ret;
>
> @@ -1064,13 +1072,13 @@ static int venus_core_init(struct venus_core *core)
>
> venus_set_state(hdev, VENUS_STATE_INIT);
>
> - ret = venus_iface_cmdq_write(hdev, );
> + ret = venus_iface_cmdq_write(hdev, , false);
> if (ret)
> return ret;
>
> pkt_sys_image_version(_pkt);
>
> -
Re: [PATCH 2/3] venus: Limit HFI sessions to the maximum supported
On Fri, Nov 20, 2020 at 9:12 AM Stanimir Varbanov
wrote:
>
> Currently we rely on firmware to return error when we reach the maximum
> supported number of sessions. But this errors are happened at reqbuf
> time which is a bit later. The more reasonable way looks like is to
> return the error on driver open.
>
> To achieve that modify hfi_session_create to return error when we reach
> maximum count of sessions and thus refuse open.
>
> Signed-off-by: Stanimir Varbanov
> ---
> drivers/media/platform/qcom/venus/core.h | 1 +
> drivers/media/platform/qcom/venus/hfi.c | 19 +++
> .../media/platform/qcom/venus/hfi_parser.c| 3 +++
> 3 files changed, 19 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
> index db0e6738281e..3a477fcdd3a8 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -96,6 +96,7 @@ struct venus_format {
> #define MAX_CAP_ENTRIES32
> #define MAX_ALLOC_MODE_ENTRIES 16
> #define MAX_CODEC_NUM 32
> +#define MAX_SESSIONS 16
>
> struct raw_formats {
> u32 buftype;
> diff --git a/drivers/media/platform/qcom/venus/hfi.c
> b/drivers/media/platform/qcom/venus/hfi.c
> index 638ed5cfe05e..8420be6d3991 100644
> --- a/drivers/media/platform/qcom/venus/hfi.c
> +++ b/drivers/media/platform/qcom/venus/hfi.c
> @@ -175,6 +175,7 @@ static int wait_session_msg(struct venus_inst *inst)
> int hfi_session_create(struct venus_inst *inst, const struct hfi_inst_ops
> *ops)
> {
> struct venus_core *core = inst->core;
> + int ret;
>
> if (!ops)
> return -EINVAL;
> @@ -183,12 +184,22 @@ int hfi_session_create(struct venus_inst *inst, const
> struct hfi_inst_ops *ops)
> init_completion(>done);
> inst->ops = ops;
>
> - mutex_lock(>lock);
> - list_add_tail(>list, >instances);
> - atomic_inc(>insts_count);
> + ret = mutex_lock_interruptible(>lock);
> + if (ret)
> + return ret;
Why do we change to mutex_lock_interruptible() here? This makes this
function return an error even though we could obtain the lock just by
trying a bit harder.
> +
> + ret = atomic_read(>insts_count);
> + if (ret + 1 > core->max_sessions_supported) {
> + ret = -EAGAIN;
> + } else {
> + atomic_inc(>insts_count);
> + list_add_tail(>list, >instances);
> + ret = 0;
> + }
> +
> mutex_unlock(>lock);
>
> - return 0;
> + return ret;
> }
> EXPORT_SYMBOL_GPL(hfi_session_create);
>
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c
> b/drivers/media/platform/qcom/venus/hfi_parser.c
> index 363ee2a65453..52898633a8e6 100644
> --- a/drivers/media/platform/qcom/venus/hfi_parser.c
> +++ b/drivers/media/platform/qcom/venus/hfi_parser.c
> @@ -276,6 +276,9 @@ u32 hfi_parser(struct venus_core *core, struct venus_inst
> *inst, void *buf,
> words_count--;
> }
>
> + if (!core->max_sessions_supported)
> + core->max_sessions_supported = MAX_SESSIONS;
> +
> parser_fini(inst, codecs, domain);
>
> return HFI_ERR_NONE;
> --
> 2.17.1
>
Re: [PATCH 1/3] venus: venc: Init the session only once in queue_setup
Hi Stan,
On Fri, Nov 20, 2020 at 9:12 AM Stanimir Varbanov
wrote:
>
> Init the hfi session only once in queue_setup and also cover that
> with inst->lock.
>
> Signed-off-by: Stanimir Varbanov
> ---
> drivers/media/platform/qcom/venus/venc.c | 98 ++--
> 1 file changed, 73 insertions(+), 25 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/venc.c
> b/drivers/media/platform/qcom/venus/venc.c
> index 4ecf78e30b59..3a2e449663d8 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -725,8 +725,10 @@ static int venc_init_session(struct venus_inst *inst)
> int ret;
>
> ret = hfi_session_init(inst, inst->fmt_cap->pixfmt);
> - if (ret)
> - return ret;
> + if (ret == -EINVAL)
> + return 0;
Why is it safe to ignore EINVAL here?
> + else if (ret)
> + goto deinit;
>
> ret = venus_helper_set_input_resolution(inst, inst->width,
> inst->height);
> @@ -762,17 +764,13 @@ static int venc_out_num_buffers(struct venus_inst
> *inst, unsigned int *num)
> struct hfi_buffer_requirements bufreq;
> int ret;
>
> - ret = venc_init_session(inst);
> + ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, );
> if (ret)
> return ret;
>
> - ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, );
> -
> *num = bufreq.count_actual;
>
> - hfi_session_deinit(inst);
> -
> - return ret;
> + return 0;
> }
>
> static int venc_queue_setup(struct vb2_queue *q,
> @@ -781,7 +779,7 @@ static int venc_queue_setup(struct vb2_queue *q,
> {
> struct venus_inst *inst = vb2_get_drv_priv(q);
> unsigned int num, min = 4;
> - int ret = 0;
> + int ret;
>
> if (*num_planes) {
> if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> @@ -803,6 +801,17 @@ static int venc_queue_setup(struct vb2_queue *q,
> return 0;
> }
>
> + ret = mutex_lock_interruptible(>lock);
> + if (ret)
> + return ret;
> +
> + ret = venc_init_session(inst);
> +
> + mutex_unlock(>lock);
> +
> + if (ret)
> + return ret;
> +
> switch (q->type) {
> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> *num_planes = inst->fmt_out->num_planes;
> @@ -838,6 +847,54 @@ static int venc_queue_setup(struct vb2_queue *q,
> return ret;
> }
>
> +static int venc_buf_init(struct vb2_buffer *vb)
> +{
> + struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> +
> + inst->buf_count++;
> +
> + return venus_helper_vb2_buf_init(vb);
> +}
> +
> +static void venc_release_session(struct venus_inst *inst)
> +{
> + int ret, abort = 0;
> +
> + mutex_lock(>lock);
> +
> + ret = hfi_session_deinit(inst);
> + abort = (ret && ret != -EINVAL) ? 1 : 0;
Here as well, I think a comment is warranted to explain why we can
ignore EINVAL.
> +
> + if (inst->session_error)
> + abort = 1;
> +
> + if (abort)
> + hfi_session_abort(inst);
> +
> + mutex_unlock(>lock);
> +
> + venus_pm_load_scale(inst);
> + INIT_LIST_HEAD(>registeredbufs);
> + venus_pm_release_core(inst);
> +}
> +
> +static void venc_buf_cleanup(struct vb2_buffer *vb)
> +{
> + struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct venus_buffer *buf = to_venus_buffer(vbuf);
> +
> + mutex_lock(>lock);
> + if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> + if (!list_empty(>registeredbufs))
> + list_del_init(>reg_list);
> + mutex_unlock(>lock);
> +
> + inst->buf_count--;
> + if (!inst->buf_count)
> + venc_release_session(inst);
We are calling venc_init_session() during the queue setup but
venc_release_session() when the last buffer is cleaned up. For
symmetry, wouldn't it make sense to call venc_init_session() when the
first buffer is initialized by venc_buf_init()? Otherwise we can
potentially have a scenario where the queue is set up, but no buffer
is ever created, leading to the session never being released.
> +}
> +
> static int venc_verify_conf(struct venus_inst *inst)
> {
> enum hfi_version ver = inst->core->res->hfi_version;
> @@ -888,38 +945,28 @@ static int venc_start_streaming(struct vb2_queue *q,
> unsigned int count)
> inst->sequence_cap = 0;
> inst->sequence_out = 0;
>
> - ret = venc_init_session(inst);
> - if (ret)
> - goto bufs_done;
> -
> ret = venus_pm_acquire_core(inst);
> if (ret)
> - goto deinit_sess;
> -
> - ret = venc_set_properties(inst);
> - if (ret)
> - goto deinit_sess;
> +
Re: [PATCH 2/2] media: mtk-vpu: dump VPU status when IPI times out
On Thu, Oct 29, 2020 at 10:17 AM Irui Wang wrote: > > when IPI time out, dump VPU status to get more debug information > > Signed-off-by: Irui Wang Reviewed-by: Alexandre Courbot
Re: [PATCH 1/2] media: mtk-vpu: VPU should be in idle state before system is suspended
On Thu, Oct 29, 2020 at 10:17 AM Irui Wang wrote: > > VPU should be in idle state before system is suspended > or it will work abnormally like VPU program counter not > in a correct address or VPU reset > > Signed-off-by: Irui Wang FWIW, Reviewed-by: Alexandre Courbot
Re: [PATCH v4 0/2] media: mtk-vcodec: fix builds when remoteproc is disabled
Gentle ping about this - we are already well into the 5.10 cycle so we don't have much time left if we want to merge this build breakage fix... On Tue, Oct 13, 2020 at 9:44 PM Alexandre Courbot wrote: > > No functional changes since v3, but it does the job at fixing the build > breakage. :) Please kindly take a look. > > Changes since v3: > * Removed obsolete Acked-bys > * Fixed indentation in Kconfig file > > Changes since v2: > * Use the FOO || !FOO magic suggested by Hans to ensure a built-in > module does not try to link against symbols in a module, > * Added a patch to split the VPU and SCP ops into their own source files > and make the optional build cleaner, > * Control the build of firmware implementations using two new transparent > Kconfig symbols. > > Changes since v1: > * Added Acked-by from Randy. > * Fixed typo in Kconfig description. > > Alexandre Courbot (2): > media: mtk-vcodec: move firmware implementations into their own files > media: mtk-vcodec: fix build breakage when one of VPU or SCP is > enabled > > drivers/media/platform/Kconfig| 28 ++- > drivers/media/platform/mtk-vcodec/Makefile| 10 +- > .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 2 +- > .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +- > .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 174 +- > .../media/platform/mtk-vcodec/mtk_vcodec_fw.h | 7 +- > .../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 52 ++ > .../platform/mtk-vcodec/mtk_vcodec_fw_scp.c | 73 > .../platform/mtk-vcodec/mtk_vcodec_fw_vpu.c | 109 +++ > 9 files changed, 277 insertions(+), 180 deletions(-) > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_scp.c > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_vpu.c > > -- > 2.29.0.rc1.297.gfa9743e501-goog >
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Tue, Nov 3, 2020 at 6:48 PM Hans Verkuil wrote: > There is one other place where this needs to be tested: testEvents() in > v4l2-test-controls.cpp: currently this only tests select(), but there > should be a second epoll test here as well that just tests EPOLLPRI. > > This would catch drivers that do not stream (i.e. no EPOLLIN/OUT) but > that do have controls (so support EPOLLPRI). IIUC this part should not require fixing - EPOLLPRI is behaving properly, the bug is only with EPOLLIN/EPOLLOUT.
[PATCH v2 0/2] media: always call poll_wait() on queues
do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This behavior causes a bug with the current poll implementations of vb2 and mem2mem, which only call poll_wait() if a queue-related (EPOLLIN or EPOLLOUT) event if present: if there is none during the first call (e.g. because userspace only wanted to listen to EPOLLPRI), then EPOLLIN and EPOLLOUT will never be signaled, event if they are requested later. This can be fixed by making the call to poll_wait() unconditional, thus making sure it will also be invoked during the first call. The issue has been discussed in more detail on https://www.spinics.net/lists/linux-media/msg179618.html. Alexandre Courbot (2): media: videobuf2: always call poll_wait() on queues media: v4l2-mem2mem: always call poll_wait() on queues drivers/media/common/videobuf2/videobuf2-core.c | 11 +-- drivers/media/v4l2-core/v4l2-mem2mem.c | 15 --- 2 files changed, 21 insertions(+), 5 deletions(-) -- 2.29.2
[PATCH v2 1/2] media: videobuf2: always call poll_wait() on queues
do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for vb2 which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll. Fix this by making the call to poll_wait() happen first thing and unconditionally in vb2_core_poll(). Signed-off-by: Alexandre Courbot --- drivers/media/common/videobuf2/videobuf2-core.c | 11 +-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c index 4eab6d81cce1..ef06f90f5c6b 100644 --- a/drivers/media/common/videobuf2/videobuf2-core.c +++ b/drivers/media/common/videobuf2/videobuf2-core.c @@ -2363,13 +2363,20 @@ __poll_t vb2_core_poll(struct vb2_queue *q, struct file *file, struct vb2_buffer *vb = NULL; unsigned long flags; + /* +* poll_wait() MUST be called on the first invocation on all the +* potential queues of interest, even if we are not interested in their +* events during this first call. Failure to do so will result in +* queue's events to be ignored because the poll_table won't be capable +* of adding new wait queues thereafter. +*/ + poll_wait(file, >done_wq, wait); + if (!q->is_output && !(req_events & (EPOLLIN | EPOLLRDNORM))) return 0; if (q->is_output && !(req_events & (EPOLLOUT | EPOLLWRNORM))) return 0; - poll_wait(file, >done_wq, wait); - /* * Start file I/O emulator only if streaming API has not been used yet. */ -- 2.29.2
[PATCH v2 2/2] media: v4l2-mem2mem: always call poll_wait() on queues
do_poll()/do_select() seem to set the _qproc member of poll_table to
NULL the first time they are called on a given table, making subsequent
calls of poll_wait() on that table no-ops. This is a problem for mem2mem
which calls poll_wait() on the V4L2 queues' waitqueues only when a
queue-related event is requested, which may not necessarily be the case
during the first poll.
For instance, a stateful decoder is typically only interested in
EPOLLPRI events when it starts, and will switch to listening to both
EPOLLPRI and EPOLLIN after receiving the initial resolution change event
and configuring the CAPTURE queue. However by the time that switch
happens and v4l2_m2m_poll_for_data() is called for the first time,
poll_wait() has become a no-op and the V4L2 queues waitqueues thus
cannot be registered.
Fix this by moving the registration to v4l2_m2m_poll() and do it whether
or not one of the queue-related events are requested.
Signed-off-by: Alexandre Courbot
---
drivers/media/v4l2-core/v4l2-mem2mem.c | 15 ---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index b221b4e438a1..e7f4bf5bc8dd 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -887,9 +887,6 @@ static __poll_t v4l2_m2m_poll_for_data(struct file *file,
src_q = v4l2_m2m_get_src_vq(m2m_ctx);
dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
- poll_wait(file, _q->done_wq, wait);
- poll_wait(file, _q->done_wq, wait);
-
/*
* There has to be at least one buffer queued on each queued_list, which
* means either in driver already or waiting for driver to claim it
@@ -922,9 +919,21 @@ __poll_t v4l2_m2m_poll(struct file *file, struct
v4l2_m2m_ctx *m2m_ctx,
struct poll_table_struct *wait)
{
struct video_device *vfd = video_devdata(file);
+ struct vb2_queue *src_q = v4l2_m2m_get_src_vq(m2m_ctx);
+ struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
__poll_t req_events = poll_requested_events(wait);
__poll_t rc = 0;
+ /*
+* poll_wait() MUST be called on the first invocation on all the
+* potential queues of interest, even if we are not interested in their
+* events during this first call. Failure to do so will result in
+* queue's events to be ignored because the poll_table won't be capable
+* of adding new wait queues thereafter.
+*/
+ poll_wait(file, _q->done_wq, wait);
+ poll_wait(file, _q->done_wq, wait);
+
if (req_events & (EPOLLOUT | EPOLLWRNORM | EPOLLIN | EPOLLRDNORM))
rc = v4l2_m2m_poll_for_data(file, m2m_ctx, wait);
--
2.29.2
Re: [PATCH] Documentation: gpio: fix typo and unclear legacy API section
On Sun, Nov 22, 2020 at 6:25 PM Alexandre Courbot wrote: > > The "Interacting With the Legacy GPIO Subsystem" of the documentation > was unclear at best, and even included a sentence that seems to say the > opposite of what it should say about the lifetime of the return value of > the conversion functions. > > Try to clarify things a bit and hopefully make that section more > readable. > > Signed-off-by: Alexandre Courbot Realized after sending this should also have a Reported-by: Andy Shevchenko Apologies for the omission Andy! > --- > Documentation/driver-api/gpio/consumer.rst | 18 ++ > 1 file changed, 10 insertions(+), 8 deletions(-) > > diff --git a/Documentation/driver-api/gpio/consumer.rst > b/Documentation/driver-api/gpio/consumer.rst > index 423492d125b9..173e4c7b037d 100644 > --- a/Documentation/driver-api/gpio/consumer.rst > +++ b/Documentation/driver-api/gpio/consumer.rst > @@ -440,18 +440,20 @@ For details refer to > Documentation/firmware-guide/acpi/gpio-properties.rst > > Interacting With the Legacy GPIO Subsystem > == > -Many kernel subsystems still handle GPIOs using the legacy integer-based > -interface. Although it is strongly encouraged to upgrade them to the safer > -descriptor-based API, the following two functions allow you to convert a GPIO > -descriptor into the GPIO integer namespace and vice-versa:: > +Many kernel subsystems and drivers still handle GPIOs using the legacy > +integer-based interface. It is strongly recommended to update these to the > new > +gpiod interface. For cases where both interfaces need to be used, the > following > +two functions allow to convert a GPIO descriptor into the GPIO integer > namespace > +and vice-versa:: > > int desc_to_gpio(const struct gpio_desc *desc) > struct gpio_desc *gpio_to_desc(unsigned gpio) > > -The GPIO number returned by desc_to_gpio() can be safely used as long as the > -GPIO descriptor has not been freed. All the same, a GPIO number passed to > -gpio_to_desc() must have been properly acquired, and usage of the returned > GPIO > -descriptor is only possible after the GPIO number has been released. > +The GPIO number returned by desc_to_gpio() can safely be used as a parameter > of > +the gpio\_*() functions for as long as the GPIO descriptor `desc` is not > freed. > +All the same, a GPIO number passed to gpio_to_desc() must first be properly > +acquired using e.g. gpio_request_one(), and the returned GPIO descriptor is > only > +considered valid until that GPIO number is released using gpio_free(). > > Freeing a GPIO obtained by one API with the other API is forbidden and an > unchecked error. > -- > 2.29.2 >
[PATCH] Documentation: gpio: fix typo and unclear legacy API section
The "Interacting With the Legacy GPIO Subsystem" of the documentation was unclear at best, and even included a sentence that seems to say the opposite of what it should say about the lifetime of the return value of the conversion functions. Try to clarify things a bit and hopefully make that section more readable. Signed-off-by: Alexandre Courbot --- Documentation/driver-api/gpio/consumer.rst | 18 ++ 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/Documentation/driver-api/gpio/consumer.rst b/Documentation/driver-api/gpio/consumer.rst index 423492d125b9..173e4c7b037d 100644 --- a/Documentation/driver-api/gpio/consumer.rst +++ b/Documentation/driver-api/gpio/consumer.rst @@ -440,18 +440,20 @@ For details refer to Documentation/firmware-guide/acpi/gpio-properties.rst Interacting With the Legacy GPIO Subsystem == -Many kernel subsystems still handle GPIOs using the legacy integer-based -interface. Although it is strongly encouraged to upgrade them to the safer -descriptor-based API, the following two functions allow you to convert a GPIO -descriptor into the GPIO integer namespace and vice-versa:: +Many kernel subsystems and drivers still handle GPIOs using the legacy +integer-based interface. It is strongly recommended to update these to the new +gpiod interface. For cases where both interfaces need to be used, the following +two functions allow to convert a GPIO descriptor into the GPIO integer namespace +and vice-versa:: int desc_to_gpio(const struct gpio_desc *desc) struct gpio_desc *gpio_to_desc(unsigned gpio) -The GPIO number returned by desc_to_gpio() can be safely used as long as the -GPIO descriptor has not been freed. All the same, a GPIO number passed to -gpio_to_desc() must have been properly acquired, and usage of the returned GPIO -descriptor is only possible after the GPIO number has been released. +The GPIO number returned by desc_to_gpio() can safely be used as a parameter of +the gpio\_*() functions for as long as the GPIO descriptor `desc` is not freed. +All the same, a GPIO number passed to gpio_to_desc() must first be properly +acquired using e.g. gpio_request_one(), and the returned GPIO descriptor is only +considered valid until that GPIO number is released using gpio_free(). Freeing a GPIO obtained by one API with the other API is forbidden and an unchecked error. -- 2.29.2
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Wed, Nov 11, 2020 at 9:47 PM Hans Verkuil wrote: > > On 11/11/2020 13:41, Alexandre Courbot wrote: > > On Thu, Nov 5, 2020 at 11:05 PM Alexandre Courbot wrote: > >> > >> On Thu, Nov 5, 2020 at 10:12 PM Hans Verkuil > >> wrote: > >>> > >>> On 05/11/2020 13:52, Alexandre Courbot wrote: > >>>> On Thu, Nov 5, 2020 at 9:36 PM Hans Verkuil > >>>> wrote: > >>>>> > >>>>> On 05/11/2020 13:21, Alexandre Courbot wrote: > >>>>>> On Tue, Nov 3, 2020 at 6:48 PM Hans Verkuil > >>>>>> wrote: > >>>>>>> > >>>>>>> On 03/11/2020 09:51, Alexandre Courbot wrote: > >>>>>>>> Hi Hans, > >>>>>>>> > >>>>>>>> On Sat, Oct 31, 2020 at 12:09 AM Hans Verkuil > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> On 22/10/2020 14:24, Alexandre Courbot wrote: > >>>>>>>>>> do_poll()/do_select() seem to set the _qproc member of poll_table > >>>>>>>>>> to > >>>>>>>>>> NULL the first time they are called on a given table, making > >>>>>>>>>> subsequent > >>>>>>>>>> calls of poll_wait() on that table no-ops. This is a problem for > >>>>>>>>>> mem2mem > >>>>>>>>>> which calls poll_wait() on the V4L2 queues' waitqueues only when a > >>>>>>>>>> queue-related event is requested, which may not necessarily be the > >>>>>>>>>> case > >>>>>>>>>> during the first poll. > >>>>>>>>>> > >>>>>>>>>> For instance, a stateful decoder is typically only interested in > >>>>>>>>>> EPOLLPRI events when it starts, and will switch to listening to > >>>>>>>>>> both > >>>>>>>>>> EPOLLPRI and EPOLLIN after receiving the initial resolution change > >>>>>>>>>> event > >>>>>>>>>> and configuring the CAPTURE queue. However by the time that switch > >>>>>>>>>> happens and v4l2_m2m_poll_for_data() is called for the first time, > >>>>>>>>>> poll_wait() has become a no-op and the V4L2 queues waitqueues thus > >>>>>>>>>> cannot be registered. > >>>>>>>>>> > >>>>>>>>>> Fix this by moving the registration to v4l2_m2m_poll() and do it > >>>>>>>>>> whether > >>>>>>>>>> or not one of the queue-related events are requested. > >>>>>>>>> > >>>>>>>>> This looks good, but would it be possible to add a test for this to > >>>>>>>>> v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp) > >>>>>>>>> > >>>>>>>>> If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then > >>>>>>>>> calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does > >>>>>>>>> there > >>>>>>>>> have to be an epoll_wait call in between? > >>>>>>>> > >>>>>>>> Even without an epoll_wait() in between the behavior is visible. > >>>>>>>> v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD > >>>>>>>> and this will trigger the bug. > >>>>>>>> > >>>>>>>>> Another reason for adding this test is that I wonder if regular > >>>>>>>>> capture > >>>>>>>>> or output V4L2 devices don't have the same issue. > >>>>>>>>> > >>>>>>>>> It's a very subtle bug and so adding a test for this to > >>>>>>>>> v4l2-compliance > >>>>>>>>> would be very useful. > >>>>>>>> > >>>>>>>> I fully agree, this is very counter-intuitive since what basically > >>>>>>>> happens is that the kernel's poll_wait() function becomes a no-op > >>>>>>>> after the poll()
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Thu, Nov 5, 2020 at 11:05 PM Alexandre Courbot wrote: > > On Thu, Nov 5, 2020 at 10:12 PM Hans Verkuil wrote: > > > > On 05/11/2020 13:52, Alexandre Courbot wrote: > > > On Thu, Nov 5, 2020 at 9:36 PM Hans Verkuil > > > wrote: > > >> > > >> On 05/11/2020 13:21, Alexandre Courbot wrote: > > >>> On Tue, Nov 3, 2020 at 6:48 PM Hans Verkuil > > >>> wrote: > > >>>> > > >>>> On 03/11/2020 09:51, Alexandre Courbot wrote: > > >>>>> Hi Hans, > > >>>>> > > >>>>> On Sat, Oct 31, 2020 at 12:09 AM Hans Verkuil > > >>>>> wrote: > > >>>>>> > > >>>>>> On 22/10/2020 14:24, Alexandre Courbot wrote: > > >>>>>>> do_poll()/do_select() seem to set the _qproc member of poll_table to > > >>>>>>> NULL the first time they are called on a given table, making > > >>>>>>> subsequent > > >>>>>>> calls of poll_wait() on that table no-ops. This is a problem for > > >>>>>>> mem2mem > > >>>>>>> which calls poll_wait() on the V4L2 queues' waitqueues only when a > > >>>>>>> queue-related event is requested, which may not necessarily be the > > >>>>>>> case > > >>>>>>> during the first poll. > > >>>>>>> > > >>>>>>> For instance, a stateful decoder is typically only interested in > > >>>>>>> EPOLLPRI events when it starts, and will switch to listening to both > > >>>>>>> EPOLLPRI and EPOLLIN after receiving the initial resolution change > > >>>>>>> event > > >>>>>>> and configuring the CAPTURE queue. However by the time that switch > > >>>>>>> happens and v4l2_m2m_poll_for_data() is called for the first time, > > >>>>>>> poll_wait() has become a no-op and the V4L2 queues waitqueues thus > > >>>>>>> cannot be registered. > > >>>>>>> > > >>>>>>> Fix this by moving the registration to v4l2_m2m_poll() and do it > > >>>>>>> whether > > >>>>>>> or not one of the queue-related events are requested. > > >>>>>> > > >>>>>> This looks good, but would it be possible to add a test for this to > > >>>>>> v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp) > > >>>>>> > > >>>>>> If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then > > >>>>>> calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does > > >>>>>> there > > >>>>>> have to be an epoll_wait call in between? > > >>>>> > > >>>>> Even without an epoll_wait() in between the behavior is visible. > > >>>>> v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD > > >>>>> and this will trigger the bug. > > >>>>> > > >>>>>> Another reason for adding this test is that I wonder if regular > > >>>>>> capture > > >>>>>> or output V4L2 devices don't have the same issue. > > >>>>>> > > >>>>>> It's a very subtle bug and so adding a test for this to > > >>>>>> v4l2-compliance > > >>>>>> would be very useful. > > >>>>> > > >>>>> I fully agree, this is very counter-intuitive since what basically > > >>>>> happens is that the kernel's poll_wait() function becomes a no-op > > >>>>> after the poll() hook of a driver is called for the first time. There > > >>>>> is no way one can expect this behavior just from browsing the code so > > >>>>> this is likely to affect other drivers. > > >>>>> > > >>>>> As for the test itself, we can easily reproduce the conditions for > > >>>>> failure in v4l2-test-buffers.cpp's captureBufs() function, but doing > > >>>>> so will make the streaming tests fail without being specific about the > > >>>>> cause. Or maybe we should add another pollmode to specifically test > > >>>
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Thu, Nov 5, 2020 at 10:12 PM Hans Verkuil wrote: > > On 05/11/2020 13:52, Alexandre Courbot wrote: > > On Thu, Nov 5, 2020 at 9:36 PM Hans Verkuil > > wrote: > >> > >> On 05/11/2020 13:21, Alexandre Courbot wrote: > >>> On Tue, Nov 3, 2020 at 6:48 PM Hans Verkuil > >>> wrote: > >>>> > >>>> On 03/11/2020 09:51, Alexandre Courbot wrote: > >>>>> Hi Hans, > >>>>> > >>>>> On Sat, Oct 31, 2020 at 12:09 AM Hans Verkuil > >>>>> wrote: > >>>>>> > >>>>>> On 22/10/2020 14:24, Alexandre Courbot wrote: > >>>>>>> do_poll()/do_select() seem to set the _qproc member of poll_table to > >>>>>>> NULL the first time they are called on a given table, making > >>>>>>> subsequent > >>>>>>> calls of poll_wait() on that table no-ops. This is a problem for > >>>>>>> mem2mem > >>>>>>> which calls poll_wait() on the V4L2 queues' waitqueues only when a > >>>>>>> queue-related event is requested, which may not necessarily be the > >>>>>>> case > >>>>>>> during the first poll. > >>>>>>> > >>>>>>> For instance, a stateful decoder is typically only interested in > >>>>>>> EPOLLPRI events when it starts, and will switch to listening to both > >>>>>>> EPOLLPRI and EPOLLIN after receiving the initial resolution change > >>>>>>> event > >>>>>>> and configuring the CAPTURE queue. However by the time that switch > >>>>>>> happens and v4l2_m2m_poll_for_data() is called for the first time, > >>>>>>> poll_wait() has become a no-op and the V4L2 queues waitqueues thus > >>>>>>> cannot be registered. > >>>>>>> > >>>>>>> Fix this by moving the registration to v4l2_m2m_poll() and do it > >>>>>>> whether > >>>>>>> or not one of the queue-related events are requested. > >>>>>> > >>>>>> This looks good, but would it be possible to add a test for this to > >>>>>> v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp) > >>>>>> > >>>>>> If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then > >>>>>> calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there > >>>>>> have to be an epoll_wait call in between? > >>>>> > >>>>> Even without an epoll_wait() in between the behavior is visible. > >>>>> v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD > >>>>> and this will trigger the bug. > >>>>> > >>>>>> Another reason for adding this test is that I wonder if regular capture > >>>>>> or output V4L2 devices don't have the same issue. > >>>>>> > >>>>>> It's a very subtle bug and so adding a test for this to v4l2-compliance > >>>>>> would be very useful. > >>>>> > >>>>> I fully agree, this is very counter-intuitive since what basically > >>>>> happens is that the kernel's poll_wait() function becomes a no-op > >>>>> after the poll() hook of a driver is called for the first time. There > >>>>> is no way one can expect this behavior just from browsing the code so > >>>>> this is likely to affect other drivers. > >>>>> > >>>>> As for the test itself, we can easily reproduce the conditions for > >>>>> failure in v4l2-test-buffers.cpp's captureBufs() function, but doing > >>>>> so will make the streaming tests fail without being specific about the > >>>>> cause. Or maybe we should add another pollmode to specifically test > >>>>> epoll in this setup? Can I get your thoughts? > >>>> > >>>> No, just keep it as part of the poll test. Just add comments at the place > >>>> where it fails describing this error. > >>>> > >>>> After all, it *is* a poll() bug, so it is only fair that it is tested as > >>>> part of the epoll test. > >>>> > >>>> Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it > >>>> again > >>>> with the actual value that you need? If that triggers this issue as well, > >>>> then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if > >>>> ev.events is 0, but perhaps EPOLLERR would work instead of 0). > >>> > >>> Yup, actually the following is enough to make v4l2-compliance -s fail > >>> with vicodec: > >> > >> Does it also fail with vivid? I am curious to know whether this issue is > >> m2m specific or a more general problem. > > > > It does fail actually! And that made me notice that vb2_poll() uses > > the same pattern as v4l2_m2m_poll() (probably because the latter is > > inspired by the former?) and needs to be fixed similarly. I will send > > another patch to fix vb2_poll() as well, thanks for pointing it out! > > I was afraid of that. > > Testing epoll for control events would be interesting as well. The > vivid radio device is an example of a device that has controls, but > does not do streaming (so is not using vb2). > > But from what I can see v4l2_ctrl_poll() does the right thing, so this > should be fine. Indeed, it unconditionally calls poll_wait() with all the wait queues that may wake us up (that is, only one), so there is no problem there.
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Thu, Nov 5, 2020 at 9:36 PM Hans Verkuil wrote: > > On 05/11/2020 13:21, Alexandre Courbot wrote: > > On Tue, Nov 3, 2020 at 6:48 PM Hans Verkuil > > wrote: > >> > >> On 03/11/2020 09:51, Alexandre Courbot wrote: > >>> Hi Hans, > >>> > >>> On Sat, Oct 31, 2020 at 12:09 AM Hans Verkuil > >>> wrote: > >>>> > >>>> On 22/10/2020 14:24, Alexandre Courbot wrote: > >>>>> do_poll()/do_select() seem to set the _qproc member of poll_table to > >>>>> NULL the first time they are called on a given table, making subsequent > >>>>> calls of poll_wait() on that table no-ops. This is a problem for mem2mem > >>>>> which calls poll_wait() on the V4L2 queues' waitqueues only when a > >>>>> queue-related event is requested, which may not necessarily be the case > >>>>> during the first poll. > >>>>> > >>>>> For instance, a stateful decoder is typically only interested in > >>>>> EPOLLPRI events when it starts, and will switch to listening to both > >>>>> EPOLLPRI and EPOLLIN after receiving the initial resolution change event > >>>>> and configuring the CAPTURE queue. However by the time that switch > >>>>> happens and v4l2_m2m_poll_for_data() is called for the first time, > >>>>> poll_wait() has become a no-op and the V4L2 queues waitqueues thus > >>>>> cannot be registered. > >>>>> > >>>>> Fix this by moving the registration to v4l2_m2m_poll() and do it whether > >>>>> or not one of the queue-related events are requested. > >>>> > >>>> This looks good, but would it be possible to add a test for this to > >>>> v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp) > >>>> > >>>> If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then > >>>> calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there > >>>> have to be an epoll_wait call in between? > >>> > >>> Even without an epoll_wait() in between the behavior is visible. > >>> v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD > >>> and this will trigger the bug. > >>> > >>>> Another reason for adding this test is that I wonder if regular capture > >>>> or output V4L2 devices don't have the same issue. > >>>> > >>>> It's a very subtle bug and so adding a test for this to v4l2-compliance > >>>> would be very useful. > >>> > >>> I fully agree, this is very counter-intuitive since what basically > >>> happens is that the kernel's poll_wait() function becomes a no-op > >>> after the poll() hook of a driver is called for the first time. There > >>> is no way one can expect this behavior just from browsing the code so > >>> this is likely to affect other drivers. > >>> > >>> As for the test itself, we can easily reproduce the conditions for > >>> failure in v4l2-test-buffers.cpp's captureBufs() function, but doing > >>> so will make the streaming tests fail without being specific about the > >>> cause. Or maybe we should add another pollmode to specifically test > >>> epoll in this setup? Can I get your thoughts? > >> > >> No, just keep it as part of the poll test. Just add comments at the place > >> where it fails describing this error. > >> > >> After all, it *is* a poll() bug, so it is only fair that it is tested as > >> part of the epoll test. > >> > >> Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again > >> with the actual value that you need? If that triggers this issue as well, > >> then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if > >> ev.events is 0, but perhaps EPOLLERR would work instead of 0). > > > > Yup, actually the following is enough to make v4l2-compliance -s fail > > with vicodec: > > Does it also fail with vivid? I am curious to know whether this issue is > m2m specific or a more general problem. It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Tue, Nov 3, 2020 at 6:48 PM Hans Verkuil wrote: > > On 03/11/2020 09:51, Alexandre Courbot wrote: > > Hi Hans, > > > > On Sat, Oct 31, 2020 at 12:09 AM Hans Verkuil > > wrote: > >> > >> On 22/10/2020 14:24, Alexandre Courbot wrote: > >>> do_poll()/do_select() seem to set the _qproc member of poll_table to > >>> NULL the first time they are called on a given table, making subsequent > >>> calls of poll_wait() on that table no-ops. This is a problem for mem2mem > >>> which calls poll_wait() on the V4L2 queues' waitqueues only when a > >>> queue-related event is requested, which may not necessarily be the case > >>> during the first poll. > >>> > >>> For instance, a stateful decoder is typically only interested in > >>> EPOLLPRI events when it starts, and will switch to listening to both > >>> EPOLLPRI and EPOLLIN after receiving the initial resolution change event > >>> and configuring the CAPTURE queue. However by the time that switch > >>> happens and v4l2_m2m_poll_for_data() is called for the first time, > >>> poll_wait() has become a no-op and the V4L2 queues waitqueues thus > >>> cannot be registered. > >>> > >>> Fix this by moving the registration to v4l2_m2m_poll() and do it whether > >>> or not one of the queue-related events are requested. > >> > >> This looks good, but would it be possible to add a test for this to > >> v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp) > >> > >> If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then > >> calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there > >> have to be an epoll_wait call in between? > > > > Even without an epoll_wait() in between the behavior is visible. > > v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD > > and this will trigger the bug. > > > >> Another reason for adding this test is that I wonder if regular capture > >> or output V4L2 devices don't have the same issue. > >> > >> It's a very subtle bug and so adding a test for this to v4l2-compliance > >> would be very useful. > > > > I fully agree, this is very counter-intuitive since what basically > > happens is that the kernel's poll_wait() function becomes a no-op > > after the poll() hook of a driver is called for the first time. There > > is no way one can expect this behavior just from browsing the code so > > this is likely to affect other drivers. > > > > As for the test itself, we can easily reproduce the conditions for > > failure in v4l2-test-buffers.cpp's captureBufs() function, but doing > > so will make the streaming tests fail without being specific about the > > cause. Or maybe we should add another pollmode to specifically test > > epoll in this setup? Can I get your thoughts? > > No, just keep it as part of the poll test. Just add comments at the place > where it fails describing this error. > > After all, it *is* a poll() bug, so it is only fair that it is tested as > part of the epoll test. > > Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again > with the actual value that you need? If that triggers this issue as well, > then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if > ev.events is 0, but perhaps EPOLLERR would work instead of 0). Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec: diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp index 8000db23..b63326cd 100644 --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp @@ -903,6 +903,10 @@ static int captureBufs(struct node *node, struct node *node_m2m_cap, const cv4l_ epollfd = epoll_create1(0); fail_on_test(epollfd < 0); + + ev.events = 0; + fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_ADD, node->g_fd(), )); + if (node->is_m2m) ev.events = EPOLLIN | EPOLLOUT | EPOLLPRI; else if (v4l_type_is_output(q.g_type())) @@ -910,7 +914,7 @@ static int captureBufs(struct node *node, struct node *node_m2m_cap, const cv4l_ else ev.events = EPOLLIN; ev.data.fd = node->g_fd(); - fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_ADD, node->g_fd(), )); + fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_MOD, node->g_fd(), )); } if (pollmode) > > The epoll_wait() will fail when
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
Hi Hans, On Sat, Oct 31, 2020 at 12:09 AM Hans Verkuil wrote: > > On 22/10/2020 14:24, Alexandre Courbot wrote: > > do_poll()/do_select() seem to set the _qproc member of poll_table to > > NULL the first time they are called on a given table, making subsequent > > calls of poll_wait() on that table no-ops. This is a problem for mem2mem > > which calls poll_wait() on the V4L2 queues' waitqueues only when a > > queue-related event is requested, which may not necessarily be the case > > during the first poll. > > > > For instance, a stateful decoder is typically only interested in > > EPOLLPRI events when it starts, and will switch to listening to both > > EPOLLPRI and EPOLLIN after receiving the initial resolution change event > > and configuring the CAPTURE queue. However by the time that switch > > happens and v4l2_m2m_poll_for_data() is called for the first time, > > poll_wait() has become a no-op and the V4L2 queues waitqueues thus > > cannot be registered. > > > > Fix this by moving the registration to v4l2_m2m_poll() and do it whether > > or not one of the queue-related events are requested. > > This looks good, but would it be possible to add a test for this to > v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp) > > If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then > calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there > have to be an epoll_wait call in between? Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug. > Another reason for adding this test is that I wonder if regular capture > or output V4L2 devices don't have the same issue. > > It's a very subtle bug and so adding a test for this to v4l2-compliance > would be very useful. I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers. As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts? Cheers, Alex.
Re: ERROR: modpost: "scp_get" undefined!
On Tue, Nov 3, 2020 at 12:48 PM kernel test robot wrote: > > Hi Yunfei, > > FYI, the error/warning still remains. > > tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > master > head: b7cbaf59f62f8ab8f157698f9e31642bff525bd0 > commit: c7244811b1c951dca812079d16b17cb241882a80 media: mtk-vcodec: add SCP > firmware ops > date: 5 weeks ago > config: sh-allmodconfig (attached as .config) > compiler: sh4-linux-gcc (GCC) 9.3.0 > reproduce (this is a W=1 build): > wget > https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O > ~/bin/make.cross > chmod +x ~/bin/make.cross > # > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c7244811b1c951dca812079d16b17cb241882a80 > git remote add linus > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git > git fetch --no-tags linus master > git checkout c7244811b1c951dca812079d16b17cb241882a80 > # save the attached .config to linux build tree > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=sh > > If you fix the issue, kindly add following tag as appropriate > Reported-by: kernel test robot > > All errors (new ones prefixed by >>, old ones prefixed by <<): > > ERROR: modpost: "scp_get_venc_hw_capa" > [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > ERROR: modpost: "scp_ipi_send" > [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > ERROR: modpost: "scp_put" > [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > >> ERROR: modpost: "scp_get" > >> [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > >> ERROR: modpost: "scp_get_vdec_hw_capa" > >> [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > >> ERROR: modpost: "scp_ipi_register" > >> [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > >> ERROR: modpost: "scp_mapping_dm_addr" > >> [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > >> ERROR: modpost: "scp_get_rproc" > >> [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > ERROR: modpost: "rproc_boot" > [drivers/media/platform/mtk-vcodec/mtk-vcodec-common.ko] undefined! > ERROR: modpost: "__delay" [drivers/net/phy/mdio-cavium.ko] undefined! This should be taken care of by https://lkml.org/lkml/2020/10/13/425.
Re: drivers/remoteproc/mtk_scp.c:645:34: warning: unused variable 'mtk_scp_of_match'
On Mon, Nov 2, 2020 at 9:09 AM kernel test robot wrote:
>
> Hi Alexandre,
>
> First bad commit (maybe != root cause):
>
> tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> master
> head: 3cea11cd5e3b00d91caf0b4730194039b45c5891
> commit: cbd2dca74926c0e4610c40923cc786b732c9e8ef remoteproc: scp: add
> COMPILE_TEST dependency
> date: 5 weeks ago
> config: x86_64-randconfig-a005-20201102 (attached as .config)
> compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project
> 235dfcf70abca65dba5d80f1a42d1485bab8980c)
> reproduce (this is a W=1 build):
> wget
> https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
> ~/bin/make.cross
> chmod +x ~/bin/make.cross
> # install x86_64 cross compiling tool for clang build
> # apt-get install binutils-x86-64-linux-gnu
> #
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cbd2dca74926c0e4610c40923cc786b732c9e8ef
> git remote add linus
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> git fetch --no-tags linus master
> git checkout cbd2dca74926c0e4610c40923cc786b732c9e8ef
> # save the attached .config to linux build tree
> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot
>
> All warnings (new ones prefixed by >>):
>
> >> drivers/remoteproc/mtk_scp.c:645:34: warning: unused variable
> >> 'mtk_scp_of_match' [-Wunused-const-variable]
>static const struct of_device_id mtk_scp_of_match[] = {
> ^
>1 warning generated.
>
> vim +/mtk_scp_of_match +645 drivers/remoteproc/mtk_scp.c
This happens when COMPILE_TEST is set but not OF_CONFIG. Sent a fix
for this: https://lkml.org/lkml/2020/11/2/102
[PATCH] remoteproc/mtk_scp: surround DT device IDs with CONFIG_OF
Now that this driver can be compiled with COMPILE_TEST, we have no
guarantee that CONFIG_OF will also be defined. When that happens, a
warning about mtk_scp_of_match being defined but unused will be reported
so make sure this variable is only defined if of_match_ptr() actually
uses it.
Fixes: cbd2dca74926c0e4610c40923cc786b732c9e8ef remoteproc: scp: add
COMPILE_TEST dependency
Reported-by: kernel test robot
Signed-off-by: Alexandre Courbot
---
drivers/remoteproc/mtk_scp.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/remoteproc/mtk_scp.c b/drivers/remoteproc/mtk_scp.c
index 577cbd5d421e..f74f22d4d1ff 100644
--- a/drivers/remoteproc/mtk_scp.c
+++ b/drivers/remoteproc/mtk_scp.c
@@ -772,12 +772,14 @@ static const struct mtk_scp_of_data mt8192_of_data = {
.host_to_scp_int_bit = MT8192_HOST_IPC_INT_BIT,
};
+#if defined(CONFIG_OF)
static const struct of_device_id mtk_scp_of_match[] = {
{ .compatible = "mediatek,mt8183-scp", .data = _of_data },
{ .compatible = "mediatek,mt8192-scp", .data = _of_data },
{},
};
MODULE_DEVICE_TABLE(of, mtk_scp_of_match);
+#endif
static struct platform_driver mtk_scp_driver = {
.probe = scp_probe,
--
2.29.1.341.ge80a0c044ae-goog
Re: [PATCH v4 2/2] media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled
On Tue, Oct 13, 2020 at 9:44 PM Alexandre Courbot wrote:
>
> The addition of MT8183 support added a dependency on the SCP remoteproc
> module. However the initial patch used the "select" Kconfig directive,
> which may result in the SCP module to not be compiled if remoteproc was
> disabled. In such a case, mtk-vcodec would try to link against
> non-existent SCP symbols. "select" was clearly misused here as explained
> in kconfig-language.txt.
>
> Replace this by a "depends" directive on at least one of the VPU and
> SCP modules, to allow the driver to be compiled as long as one of these
> is enabled, and adapt the code to support this new scenario.
>
> Also adapt the Kconfig text to explain the extra requirements for MT8173
> and MT8183.
>
> Reported-by: Sakari Ailus
> Signed-off-by: Alexandre Courbot
Fixes: bf1d556ad4e0 ("media: mtk-vcodec: abstract firmware interface")
Re: [PATCH v4 1/2] media: mtk-vcodec: move firmware implementations into their own files
On Tue, Oct 13, 2020 at 9:44 PM Alexandre Courbot wrote:
>
> mtk-vcodec supports two kinds of firmware, VPU and SCP. Both were
> supported from the same source files, but this is clearly unclean and
> makes it more difficult to disable support for one or the other.
>
> Move these implementations into their own file, after adding the
> necessary private interfaces.
>
> Signed-off-by: Alexandre Courbot
This should also have included a
Fixes: bf1d556ad4e0 ("media: mtk-vcodec: abstract firmware interface")
Sorry for the omission.
Re: [PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
On Thu, Oct 22, 2020 at 9:24 PM Alexandre Courbot wrote:
>
> do_poll()/do_select() seem to set the _qproc member of poll_table to
> NULL the first time they are called on a given table, making subsequent
> calls of poll_wait() on that table no-ops. This is a problem for mem2mem
> which calls poll_wait() on the V4L2 queues' waitqueues only when a
> queue-related event is requested, which may not necessarily be the case
> during the first poll.
>
> For instance, a stateful decoder is typically only interested in
> EPOLLPRI events when it starts, and will switch to listening to both
> EPOLLPRI and EPOLLIN after receiving the initial resolution change event
> and configuring the CAPTURE queue. However by the time that switch
> happens and v4l2_m2m_poll_for_data() is called for the first time,
> poll_wait() has become a no-op and the V4L2 queues waitqueues thus
> cannot be registered.
>
> Fix this by moving the registration to v4l2_m2m_poll() and do it whether
> or not one of the queue-related events are requested.
>
> Signed-off-by: Alexandre Courbot
> ---
> I seem to be hitting all the polling corner cases recently! ^_^; This
> time I was wondering why epoll_wait() never returned after I received
> the first resolution change event from the vicodec stateful decoder.
> This is why - please take a look!
>
> drivers/media/v4l2-core/v4l2-mem2mem.c | 8 +---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
> b/drivers/media/v4l2-core/v4l2-mem2mem.c
> index b221b4e438a1..65476ef2879f 100644
> --- a/drivers/media/v4l2-core/v4l2-mem2mem.c
> +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
> @@ -887,9 +887,6 @@ static __poll_t v4l2_m2m_poll_for_data(struct file *file,
> src_q = v4l2_m2m_get_src_vq(m2m_ctx);
> dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
>
> - poll_wait(file, _q->done_wq, wait);
> - poll_wait(file, _q->done_wq, wait);
> -
> /*
> * There has to be at least one buffer queued on each queued_list,
> which
> * means either in driver already or waiting for driver to claim it
> @@ -922,9 +919,14 @@ __poll_t v4l2_m2m_poll(struct file *file, struct
> v4l2_m2m_ctx *m2m_ctx,
>struct poll_table_struct *wait)
> {
> struct video_device *vfd = video_devdata(file);
> + struct vb2_queue *src_q = v4l2_m2m_get_src_vq(m2m_ctx);
> + struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
> __poll_t req_events = poll_requested_events(wait);
> __poll_t rc = 0;
>
> + poll_wait(file, _q->done_wq, wait);
> + poll_wait(file, _q->done_wq, wait);
This should probably include a comment to not move this back to
v4l2_m2m_poll_for_data(). I'll add one and send a v2 unless someone
points out that the premise of this patch is a bad idea to begin with.
> +
> if (req_events & (EPOLLOUT | EPOLLWRNORM | EPOLLIN | EPOLLRDNORM))
> rc = v4l2_m2m_poll_for_data(file, m2m_ctx, wait);
>
> --
> 2.29.0
>
[PATCH] media: v4l2-mem2mem: always call poll_wait() on queues
do_poll()/do_select() seem to set the _qproc member of poll_table to
NULL the first time they are called on a given table, making subsequent
calls of poll_wait() on that table no-ops. This is a problem for mem2mem
which calls poll_wait() on the V4L2 queues' waitqueues only when a
queue-related event is requested, which may not necessarily be the case
during the first poll.
For instance, a stateful decoder is typically only interested in
EPOLLPRI events when it starts, and will switch to listening to both
EPOLLPRI and EPOLLIN after receiving the initial resolution change event
and configuring the CAPTURE queue. However by the time that switch
happens and v4l2_m2m_poll_for_data() is called for the first time,
poll_wait() has become a no-op and the V4L2 queues waitqueues thus
cannot be registered.
Fix this by moving the registration to v4l2_m2m_poll() and do it whether
or not one of the queue-related events are requested.
Signed-off-by: Alexandre Courbot
---
I seem to be hitting all the polling corner cases recently! ^_^; This
time I was wondering why epoll_wait() never returned after I received
the first resolution change event from the vicodec stateful decoder.
This is why - please take a look!
drivers/media/v4l2-core/v4l2-mem2mem.c | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index b221b4e438a1..65476ef2879f 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -887,9 +887,6 @@ static __poll_t v4l2_m2m_poll_for_data(struct file *file,
src_q = v4l2_m2m_get_src_vq(m2m_ctx);
dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
- poll_wait(file, _q->done_wq, wait);
- poll_wait(file, _q->done_wq, wait);
-
/*
* There has to be at least one buffer queued on each queued_list, which
* means either in driver already or waiting for driver to claim it
@@ -922,9 +919,14 @@ __poll_t v4l2_m2m_poll(struct file *file, struct
v4l2_m2m_ctx *m2m_ctx,
struct poll_table_struct *wait)
{
struct video_device *vfd = video_devdata(file);
+ struct vb2_queue *src_q = v4l2_m2m_get_src_vq(m2m_ctx);
+ struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
__poll_t req_events = poll_requested_events(wait);
__poll_t rc = 0;
+ poll_wait(file, _q->done_wq, wait);
+ poll_wait(file, _q->done_wq, wait);
+
if (req_events & (EPOLLOUT | EPOLLWRNORM | EPOLLIN | EPOLLRDNORM))
rc = v4l2_m2m_poll_for_data(file, m2m_ctx, wait);
--
2.29.0
[PATCH v2] venus: vdec: return parsed crop information from stream
Per the stateful codec specification, VIDIOC_G_SELECTION with a target
of V4L2_SEL_TGT_COMPOSE is supposed to return the crop area of capture
buffers containing the decoded frame. Until now the driver did not get
that information from the firmware and just returned the dimensions of
CAPTURE buffers.
The firmware unfortunately does not always provide the crop information
from the stream ; also make sure to detect when that happens and
fallback to providing the coded size in these cases.
Signed-off-by: Alexandre Courbot
---
Changes since v1:
* Fall back to the previous behavior of returning the coded size if the
firmware does not report any crop information.
drivers/media/platform/qcom/venus/core.h | 1 +
drivers/media/platform/qcom/venus/vdec.c | 32
2 files changed, 28 insertions(+), 5 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/core.h
b/drivers/media/platform/qcom/venus/core.h
index 7b79a33dc9d6..3bc129a4f817 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -361,6 +361,7 @@ struct venus_inst {
unsigned int streamon_cap, streamon_out;
u32 width;
u32 height;
+ struct v4l2_rect crop;
u32 out_width;
u32 out_height;
u32 colorspace;
diff --git a/drivers/media/platform/qcom/venus/vdec.c
b/drivers/media/platform/qcom/venus/vdec.c
index ea13170a6a2c..8488411204c3 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -325,6 +325,10 @@ static int vdec_s_fmt(struct file *file, void *fh, struct
v4l2_format *f)
inst->width = format.fmt.pix_mp.width;
inst->height = format.fmt.pix_mp.height;
+ inst->crop.top = 0;
+ inst->crop.left = 0;
+ inst->crop.width = inst->width;
+ inst->crop.height = inst->height;
if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
inst->fmt_out = fmt;
@@ -343,6 +347,9 @@ vdec_g_selection(struct file *file, void *fh, struct
v4l2_selection *s)
s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
return -EINVAL;
+ s->r.top = 0;
+ s->r.left = 0;
+
switch (s->target) {
case V4L2_SEL_TGT_CROP_BOUNDS:
case V4L2_SEL_TGT_CROP_DEFAULT:
@@ -363,16 +370,12 @@ vdec_g_selection(struct file *file, void *fh, struct
v4l2_selection *s)
case V4L2_SEL_TGT_COMPOSE:
if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
return -EINVAL;
- s->r.width = inst->out_width;
- s->r.height = inst->out_height;
+ s->r = inst->crop;
break;
default:
return -EINVAL;
}
- s->r.top = 0;
- s->r.left = 0;
-
return 0;
}
@@ -1309,6 +1312,21 @@ static void vdec_event_change(struct venus_inst *inst,
inst->width = format.fmt.pix_mp.width;
inst->height = format.fmt.pix_mp.height;
+ /*
+* Some versions of the firmware do not report crop information for
+* all codecs. For these cases, set the crop to the coded resolution.
+*/
+ if (ev_data->input_crop.width > 0 && ev_data->input_crop.height > 0) {
+ inst->crop.left = ev_data->input_crop.left;
+ inst->crop.top = ev_data->input_crop.top;
+ inst->crop.width = ev_data->input_crop.width;
+ inst->crop.height = ev_data->input_crop.height;
+ } else {
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = ev_data->width;
+ inst->crop.height = ev_data->height;
+ }
inst->out_width = ev_data->width;
inst->out_height = ev_data->height;
@@ -1412,6 +1430,10 @@ static void vdec_inst_init(struct venus_inst *inst)
inst->fmt_cap = _formats[0];
inst->width = frame_width_min(inst);
inst->height = ALIGN(frame_height_min(inst), 32);
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = inst->width;
+ inst->crop.height = inst->height;
inst->out_width = frame_width_min(inst);
inst->out_height = frame_height_min(inst);
inst->fps = 30;
--
2.29.0.rc1.297.gfa9743e501-goog
Re: [PATCH] venus: venc: add handling for VIDIOC_ENCODER_CMD
Hi Dikshita,
On Mon, Oct 19, 2020 at 11:29 PM Dikshita Agarwal
wrote:
>
> Add handling for below commands in encoder:
> 1. V4L2_ENC_CMD_STOP
> 2. V4L2_ENC_CMD_START
I suspect this can be implemented more easily (and more safely) using
the m2m encoder helpers introduced recently. Please see this commit
for details:
https://github.com/torvalds/linux/commit/2b48e113866a6735de3a99531183afb6217c2a60
By making use of this you can probably get rid of venus_enc_state
entirely. Also this generic implementation should take care of corner
cases that this patch does not address (like streaming off while a
drain is in progress).
>
> Signed-off-by: Dikshita Agarwal
> ---
> drivers/media/platform/qcom/venus/core.h | 9 +
> drivers/media/platform/qcom/venus/venc.c | 64
> +++-
> 2 files changed, 72 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
> index e30eeaf..5c46936 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -276,6 +276,14 @@ enum venus_dec_state {
> VENUS_DEC_STATE_DRC = 7,
> };
>
> +enum venus_enc_state {
> + VENUS_ENC_STATE_DEINIT = 0,
> + VENUS_ENC_STATE_INIT= 1,
> + VENUS_ENC_STATE_ENCODING= 2,
> + VENUS_ENC_STATE_STOPPED = 3,
> + VENUS_ENC_STATE_DRAIN = 4,
> +};
> +
> struct venus_ts_metadata {
> bool used;
> u64 ts_ns;
> @@ -367,6 +375,7 @@ struct venus_inst {
> u8 quantization;
> u8 xfer_func;
> enum venus_dec_state codec_state;
> + enum venus_enc_state enc_state;
> wait_queue_head_t reconf_wait;
> unsigned int subscriptions;
> int buf_count;
> diff --git a/drivers/media/platform/qcom/venus/venc.c
> b/drivers/media/platform/qcom/venus/venc.c
> index f7fb6e3..c6143b0 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -498,6 +498,46 @@ static int venc_enum_frameintervals(struct file *file,
> void *fh,
> return 0;
> }
>
> +static int
> +venc_encoder_cmd(struct file *file, void *fh, struct v4l2_encoder_cmd *cmd)
> +{
> + struct venus_inst *inst = to_inst(file);
> + struct hfi_frame_data fdata = {0};
> + int ret = 0;
> +
> + ret = v4l2_m2m_ioctl_try_encoder_cmd(file, fh, cmd);
> + if (ret)
> + return ret;
> +
> + mutex_lock(>lock);
> +
> + if (cmd->cmd == V4L2_ENC_CMD_STOP &&
> + inst->enc_state == VENUS_ENC_STATE_ENCODING) {
> + /*
> +* Implement V4L2_ENC_CMD_STOP by enqueue an empty buffer on
> +* encoder input to signal EOS.
> +*/
> + if (!(inst->streamon_out && inst->streamon_cap))
> + goto unlock;
> +
> + fdata.buffer_type = HFI_BUFFER_INPUT;
> + fdata.flags |= HFI_BUFFERFLAG_EOS;
> + fdata.device_addr = 0xdeadb000;
> +
> + ret = hfi_session_process_buf(inst, );
> +
> + inst->enc_state = VENUS_ENC_STATE_DRAIN;
> + } else if (cmd->cmd == V4L2_ENC_CMD_START &&
> + inst->enc_state == VENUS_ENC_STATE_STOPPED) {
> +
> vb2_clear_last_buffer_dequeued(>fh.m2m_ctx->cap_q_ctx.q);
> + inst->enc_state = VENUS_ENC_STATE_ENCODING;
> + }
> +
> +unlock:
> + mutex_unlock(>lock);
> + return ret;
> +}
> +
> static const struct v4l2_ioctl_ops venc_ioctl_ops = {
> .vidioc_querycap = venc_querycap,
> .vidioc_enum_fmt_vid_cap = venc_enum_fmt,
> @@ -525,6 +565,7 @@ static int venc_enum_frameintervals(struct file *file,
> void *fh,
> .vidioc_enum_frameintervals = venc_enum_frameintervals,
> .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> + .vidioc_encoder_cmd = venc_encoder_cmd,
> };
>
> static int venc_set_properties(struct venus_inst *inst)
> @@ -884,6 +925,8 @@ static int venc_start_streaming(struct vb2_queue *q,
> unsigned int count)
> if (ret)
> goto deinit_sess;
>
> + inst->enc_state = VENUS_ENC_STATE_ENCODING;
> +
> mutex_unlock(>lock);
>
> return 0;
> @@ -903,8 +946,19 @@ static int venc_start_streaming(struct vb2_queue *q,
> unsigned int count)
> static void venc_vb2_buf_queue(struct vb2_buffer *vb)
> {
> struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>
> mutex_lock(>lock);
> +
> + if (inst->enc_state == VENUS_ENC_STATE_STOPPED) {
> + vbuf->sequence = inst->sequence_cap++;
> + vbuf->field = V4L2_FIELD_NONE;
> + vb2_set_plane_payload(vb, 0, 0);
> + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
> +
Re: [PATCH] venus: vdec: return parsed crop information from stream
On Mon, Oct 19, 2020 at 7:19 AM Fritz Koenig wrote:
>
> It looks like only h.264 streams are populating the event.input_crop
> struct when receiving the HFI_INDEX_EXTRADATA_INPUT_CROP message in
> event_seq_changed(). vp8/vp9 streams end up with the struct filled
> with 0.
Indeed. :( I guess we can fallback to the previous behavior of using
the coded resolution as visible rect when the reported visible rect's
area is 0. That will preserve the previous behavior until the firmware
starts reporting this information for all encoded streams.
>
> On Fri, Oct 9, 2020 at 1:45 AM Alexandre Courbot
> wrote:
> >
> > Per the stateful codec specification, VIDIOC_G_SELECTION with a target
> > of V4L2_SEL_TGT_COMPOSE is supposed to return the crop area of capture
> > buffers containing the decoded frame. Until now the driver did not get
> > that information from the firmware and just returned the dimensions of
> > CAPTURE buffers.
> >
> > Signed-off-by: Alexandre Courbot
> > ---
> > drivers/media/platform/qcom/venus/core.h | 1 +
> > drivers/media/platform/qcom/venus/vdec.c | 21 -
> > 2 files changed, 17 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/media/platform/qcom/venus/core.h
> > b/drivers/media/platform/qcom/venus/core.h
> > index 7b79a33dc9d6..3bc129a4f817 100644
> > --- a/drivers/media/platform/qcom/venus/core.h
> > +++ b/drivers/media/platform/qcom/venus/core.h
> > @@ -361,6 +361,7 @@ struct venus_inst {
> > unsigned int streamon_cap, streamon_out;
> > u32 width;
> > u32 height;
> > + struct v4l2_rect crop;
> > u32 out_width;
> > u32 out_height;
> > u32 colorspace;
> > diff --git a/drivers/media/platform/qcom/venus/vdec.c
> > b/drivers/media/platform/qcom/venus/vdec.c
> > index ea13170a6a2c..ee74346f0cae 100644
> > --- a/drivers/media/platform/qcom/venus/vdec.c
> > +++ b/drivers/media/platform/qcom/venus/vdec.c
> > @@ -325,6 +325,10 @@ static int vdec_s_fmt(struct file *file, void *fh,
> > struct v4l2_format *f)
> >
> > inst->width = format.fmt.pix_mp.width;
> > inst->height = format.fmt.pix_mp.height;
> > + inst->crop.top = 0;
> > + inst->crop.left = 0;
> > + inst->crop.width = inst->width;
> > + inst->crop.height = inst->height;
> >
> > if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> > inst->fmt_out = fmt;
> > @@ -343,6 +347,9 @@ vdec_g_selection(struct file *file, void *fh, struct
> > v4l2_selection *s)
> > s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
> > return -EINVAL;
> >
> > + s->r.top = 0;
> > + s->r.left = 0;
> > +
> > switch (s->target) {
> > case V4L2_SEL_TGT_CROP_BOUNDS:
> > case V4L2_SEL_TGT_CROP_DEFAULT:
> > @@ -363,16 +370,12 @@ vdec_g_selection(struct file *file, void *fh, struct
> > v4l2_selection *s)
> > case V4L2_SEL_TGT_COMPOSE:
> > if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> > return -EINVAL;
> > - s->r.width = inst->out_width;
> > - s->r.height = inst->out_height;
> > + s->r = inst->crop;
> > break;
> > default:
> > return -EINVAL;
> > }
> >
> > - s->r.top = 0;
> > - s->r.left = 0;
> > -
> > return 0;
> > }
> >
> > @@ -1309,6 +1312,10 @@ static void vdec_event_change(struct venus_inst
> > *inst,
> >
> > inst->width = format.fmt.pix_mp.width;
> > inst->height = format.fmt.pix_mp.height;
> > + inst->crop.left = ev_data->input_crop.left;
> > + inst->crop.top = ev_data->input_crop.top;
> > + inst->crop.width = ev_data->input_crop.width;
> > + inst->crop.height = ev_data->input_crop.height;
> >
> > inst->out_width = ev_data->width;
> > inst->out_height = ev_data->height;
> > @@ -1412,6 +1419,10 @@ static void vdec_inst_init(struct venus_inst *inst)
> > inst->fmt_cap = _formats[0];
> > inst->width = frame_width_min(inst);
> > inst->height = ALIGN(frame_height_min(inst), 32);
> > + inst->crop.left = 0;
> > + inst->crop.top = 0;
> > + inst->crop.width = inst->width;
> > + inst->crop.height = inst->height;
> > inst->out_width = frame_width_min(inst);
> > inst->out_height = frame_height_min(inst);
> > inst->fps = 30;
> > --
> > 2.28.0.1011.ga647a8990f-goog
> >
[PATCH v4 0/2] media: mtk-vcodec: fix builds when remoteproc is disabled
No functional changes since v3, but it does the job at fixing the build breakage. :) Please kindly take a look. Changes since v3: * Removed obsolete Acked-bys * Fixed indentation in Kconfig file Changes since v2: * Use the FOO || !FOO magic suggested by Hans to ensure a built-in module does not try to link against symbols in a module, * Added a patch to split the VPU and SCP ops into their own source files and make the optional build cleaner, * Control the build of firmware implementations using two new transparent Kconfig symbols. Changes since v1: * Added Acked-by from Randy. * Fixed typo in Kconfig description. Alexandre Courbot (2): media: mtk-vcodec: move firmware implementations into their own files media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled drivers/media/platform/Kconfig| 28 ++- drivers/media/platform/mtk-vcodec/Makefile| 10 +- .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 2 +- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +- .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 174 +- .../media/platform/mtk-vcodec/mtk_vcodec_fw.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 52 ++ .../platform/mtk-vcodec/mtk_vcodec_fw_scp.c | 73 .../platform/mtk-vcodec/mtk_vcodec_fw_vpu.c | 109 +++ 9 files changed, 277 insertions(+), 180 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_scp.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_vpu.c -- 2.29.0.rc1.297.gfa9743e501-goog
[PATCH v4 2/2] media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled
The addition of MT8183 support added a dependency on the SCP remoteproc
module. However the initial patch used the "select" Kconfig directive,
which may result in the SCP module to not be compiled if remoteproc was
disabled. In such a case, mtk-vcodec would try to link against
non-existent SCP symbols. "select" was clearly misused here as explained
in kconfig-language.txt.
Replace this by a "depends" directive on at least one of the VPU and
SCP modules, to allow the driver to be compiled as long as one of these
is enabled, and adapt the code to support this new scenario.
Also adapt the Kconfig text to explain the extra requirements for MT8173
and MT8183.
Reported-by: Sakari Ailus
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/Kconfig| 28 ++-
drivers/media/platform/mtk-vcodec/Makefile| 10 +--
.../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 18
3 files changed, 47 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index a3cb104956d5..7e152bbb4fa6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -253,17 +253,31 @@ config VIDEO_MEDIATEK_VCODEC
depends on MTK_IOMMU || COMPILE_TEST
depends on VIDEO_DEV && VIDEO_V4L2
depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on VIDEO_MEDIATEK_VPU || MTK_SCP
+ # The two following lines ensure we have the same state ("m" or "y") as
+ # our dependencies, to avoid missing symbols during link.
+ depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
+ depends on MTK_SCP || !MTK_SCP
select VIDEOBUF2_DMA_CONTIG
select V4L2_MEM2MEM_DEV
- select VIDEO_MEDIATEK_VPU
- select MTK_SCP
+ select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
+ select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
help
- Mediatek video codec driver provides HW capability to
- encode and decode in a range of video formats
- This driver rely on VPU driver to communicate with VPU.
+ Mediatek video codec driver provides HW capability to
+ encode and decode in a range of video formats on MT8173
+ and MT8183.
+
+ Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
+ also be selected. Support for MT8183 depends on MTK_SCP.
+
+ To compile this driver as modules, choose M here: the
+ modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
+
+config VIDEO_MEDIATEK_VCODEC_VPU
+ bool
- To compile this driver as modules, choose M here: the
- modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
+config VIDEO_MEDIATEK_VCODEC_SCP
+ bool
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 6e1ea3a9f052..4618d43dbbc8 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -25,5 +25,11 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o \
mtk_vcodec_fw.o \
- mtk_vcodec_fw_vpu.o \
- mtk_vcodec_fw_scp.o
+
+ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU),)
+mtk-vcodec-common-y += mtk_vcodec_fw_vpu.o
+endif
+
+ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP),)
+mtk-vcodec-common-y += mtk_vcodec_fw_scp.o
+endif
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
index 51f1694a7c7d..b41e66185cec 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
@@ -27,8 +27,26 @@ struct mtk_vcodec_fw_ops {
void (*release)(struct mtk_vcodec_fw *fw);
};
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU)
struct mtk_vcodec_fw *mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
enum mtk_vcodec_fw_use fw_use);
+#else
+static inline struct mtk_vcodec_fw *
+mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
+ enum mtk_vcodec_fw_use fw_use)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif /* CONFIG_VIDEO_MEDIATEK_VCODEC_VPU */
+
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP)
struct mtk_vcodec_fw *mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev);
+#else
+static inline struct mtk_vcodec_fw *
+mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif /* CONFIG_VIDEO_MEDIATEK_VCODEC_SCP */
#endif /* _MTK_VCODEC_FW_PRIV_H_ */
--
2.29.0.rc1.297.gfa9743e501-goog
[PATCH v4 1/2] media: mtk-vcodec: move firmware implementations into their own files
mtk-vcodec supports two kinds of firmware, VPU and SCP. Both were
supported from the same source files, but this is clearly unclean and
makes it more difficult to disable support for one or the other.
Move these implementations into their own file, after adding the
necessary private interfaces.
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/Makefile| 4 +-
.../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 2 +-
.../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +-
.../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 174 +-
.../media/platform/mtk-vcodec/mtk_vcodec_fw.h | 7 +-
.../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 34
.../platform/mtk-vcodec/mtk_vcodec_fw_scp.c | 73
.../platform/mtk-vcodec/mtk_vcodec_fw_vpu.c | 109 +++
8 files changed, 232 insertions(+), 173 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_scp.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index f679c6e1a3e9..6e1ea3a9f052 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -24,4 +24,6 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o \
- mtk_vcodec_fw.o
+ mtk_vcodec_fw.o \
+ mtk_vcodec_fw_vpu.o \
+ mtk_vcodec_fw_scp.o
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index d14bc208ea5e..145686d2c219 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -241,7 +241,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
dma_set_max_seg_size(>dev, DMA_BIT_MASK(32));
- dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
+ dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, DECODER);
if (IS_ERR(dev->fw_handler))
return PTR_ERR(dev->fw_handler);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index dcfa2c2d4def..3be8a04c4c67 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -293,7 +293,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
dma_set_max_seg_size(>dev, DMA_BIT_MASK(32));
- dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
+ dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, ENCODER);
if (IS_ERR(dev->fw_handler))
return PTR_ERR(dev->fw_handler);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
index 6c2a2568d844..94b39ae5c2e1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -1,193 +1,29 @@
// SPDX-License-Identifier: GPL-2.0
#include "mtk_vcodec_fw.h"
+#include "mtk_vcodec_fw_priv.h"
#include "mtk_vcodec_util.h"
#include "mtk_vcodec_drv.h"
-struct mtk_vcodec_fw_ops {
- int (*load_firmware)(struct mtk_vcodec_fw *fw);
- unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
- unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
- void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
- int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler, const char *name,
void *priv);
- int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
- unsigned int len, unsigned int wait);
-};
-
-struct mtk_vcodec_fw {
- enum mtk_vcodec_fw_type type;
- const struct mtk_vcodec_fw_ops *ops;
- struct platform_device *pdev;
- struct mtk_scp *scp;
-};
-
-static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
-{
- return vpu_load_firmware(fw->pdev);
-}
-
-static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
-{
- return vpu_get_vdec_hw_capa(fw->pdev);
-}
-
-static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
-{
- return vpu_get_venc_hw_capa(fw->pdev);
-}
-
-static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
- u32 dtcm_dmem_addr)
-{
- return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
-}
-
-static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler,
-
Re: [PATCH v3 2/2] media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled
Hi Mauro,
On Mon, Oct 12, 2020 at 4:43 PM Mauro Carvalho Chehab
wrote:
>
> Em Mon, 12 Oct 2020 14:35:57 +0900
> Alexandre Courbot escreveu:
>
> > The addition of MT8183 support added a dependency on the SCP remoteproc
> > module. However the initial patch used the "select" Kconfig directive,
> > which may result in the SCP module to not be compiled if remoteproc was
> > disabled. In such a case, mtk-vcodec would try to link against
> > non-existent SCP symbols. "select" was clearly misused here as explained
> > in kconfig-language.txt.
> >
> > Replace this by a "depends" directive on at least one of the VPU and
> > SCP modules, to allow the driver to be compiled as long as one of these
> > is enabled, and adapt the code to support this new scenario.
> >
> > Also adapt the Kconfig text to explain the extra requirements for MT8173
> > and MT8183.
> >
> > Reported-by: Sakari Ailus
> > Signed-off-by: Alexandre Courbot
> > Acked-by: Randy Dunlap # build-tested
> > Acked-by: Sakari Ailus
> > ---
> > drivers/media/platform/Kconfig| 22 +++
> > drivers/media/platform/mtk-vcodec/Makefile| 10 +++--
> > .../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 18 +++
> > 3 files changed, 44 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index a3cb104956d5..457b6c39ddc0 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -253,18 +253,32 @@ config VIDEO_MEDIATEK_VCODEC
> > depends on MTK_IOMMU || COMPILE_TEST
> > depends on VIDEO_DEV && VIDEO_V4L2
> > depends on ARCH_MEDIATEK || COMPILE_TEST
> > + depends on VIDEO_MEDIATEK_VPU || MTK_SCP
> > + # The two following lines ensure we have the same state ("m" or "y")
> > as
> > + # our dependencies, to avoid missing symbols during link.
> > + depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
> > + depends on MTK_SCP || !MTK_SCP
> > select VIDEOBUF2_DMA_CONTIG
> > select V4L2_MEM2MEM_DEV
> > - select VIDEO_MEDIATEK_VPU
> > - select MTK_SCP
> > + select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
> > + select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
> > help
> > Mediatek video codec driver provides HW capability to
> > - encode and decode in a range of video formats
> > - This driver rely on VPU driver to communicate with VPU.
> > + encode and decode in a range of video formats on MT8173
> > + and MT8183.
> > +
> > + Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
> > + also be selected. Support for MT8183 depends on MTK_SCP.
> >
> > To compile this driver as modules, choose M here: the
> > modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
>
> Just my 2 cents here, and to complement my last e-mail, the helper message
> here IMO is a lot more confusing than if you do this, instead:
>
> config VIDEO_MEDIATEK_CODEC
> depends on VIDEO_MEDIATEK_VPU_SCP || VIDEO_MEDIATEK_VPU
> default y
>
> config VIDEO_MEDIATEK_VPU
> depends on VIDEO_DEV && VIDEO_V4L2
> depends on ARCH_MEDIATEK || COMPILE_TEST
> tristate "Enable Mediatek Video Processor Unit for MT8173"
> help
> Select this option to enable Mediatek VPU on MT8173.
>
> config VIDEO_MEDIATEK_VPU_SCP
> depends on VIDEO_DEV && VIDEO_V4L2
> depends on ARCH_MEDIATEK || COMPILE_TEST
> tristate "Enable Mediatek Video Processor Unit for MT8183"
> help
> Select this option to enable Mediatek VPU on MT8183.
>
> To be clear, from my side, I can live with either one of the alternatives,
> but, IMHO, the above is a lot clearer for anyone wanting to use
> VPU, as, if MTK_SCP is disabled, the MT8183 Kconfig prompt will
> disappear.
So I have experimented a bit and it turns out that even after adding
these new entries and using the "default" option that you suggested,
one still needs to perform the
depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
depends on MTK_SCP || !MTK_SCP
magic to prevent the VCODEC module from being built-in while having
the SCP and/or VPU support as modules (which would trigger the same
link error as before).
Also from experiment, the additional menu entries
Re: [PATCH v3 2/2] media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled
On Tue, Oct 13, 2020 at 12:00 AM Randy Dunlap wrote:
>
> On 10/11/20 10:35 PM, Alexandre Courbot wrote:
> > The addition of MT8183 support added a dependency on the SCP remoteproc
> > module. However the initial patch used the "select" Kconfig directive,
> > which may result in the SCP module to not be compiled if remoteproc was
> > disabled. In such a case, mtk-vcodec would try to link against
> > non-existent SCP symbols. "select" was clearly misused here as explained
> > in kconfig-language.txt.
> >
> > Replace this by a "depends" directive on at least one of the VPU and
> > SCP modules, to allow the driver to be compiled as long as one of these
> > is enabled, and adapt the code to support this new scenario.
> >
> > Also adapt the Kconfig text to explain the extra requirements for MT8173
> > and MT8183.
> >
> > Reported-by: Sakari Ailus
> > Signed-off-by: Alexandre Courbot
> > Acked-by: Randy Dunlap # build-tested
>
> That Ack applied to v2. I have not tested nor acked this version of the patch.
Sorry about that - I was careless and left it in the log.
>
> > Acked-by: Sakari Ailus
> > ---
> > drivers/media/platform/Kconfig| 22 +++
> > drivers/media/platform/mtk-vcodec/Makefile| 10 +++--
> > .../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 18 +++
> > 3 files changed, 44 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index a3cb104956d5..457b6c39ddc0 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -253,18 +253,32 @@ config VIDEO_MEDIATEK_VCODEC
> > depends on MTK_IOMMU || COMPILE_TEST
> > depends on VIDEO_DEV && VIDEO_V4L2
> > depends on ARCH_MEDIATEK || COMPILE_TEST
> > + depends on VIDEO_MEDIATEK_VPU || MTK_SCP
> > + # The two following lines ensure we have the same state ("m" or "y")
> > as
> > + # our dependencies, to avoid missing symbols during link.
> > + depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
> > + depends on MTK_SCP || !MTK_SCP
> > select VIDEOBUF2_DMA_CONTIG
> > select V4L2_MEM2MEM_DEV
> > - select VIDEO_MEDIATEK_VPU
> > - select MTK_SCP
> > + select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
> > + select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
> > help
> > Mediatek video codec driver provides HW capability to
> > - encode and decode in a range of video formats
> > - This driver rely on VPU driver to communicate with VPU.
> > + encode and decode in a range of video formats on MT8173
> > + and MT8183.
> > +
> > + Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
> > + also be selected. Support for MT8183 depends on MTK_SCP.
> >
> > To compile this driver as modules, choose M here: the
> > modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
> >
> > +config VIDEO_MEDIATEK_VCODEC_VPU
> > + bool
> > +
> > +config VIDEO_MEDIATEK_VCODEC_SCP
> > + bool
> > +
> > config VIDEO_MEM2MEM_DEINTERLACE
> > tristate "Deinterlace support"
> > depends on VIDEO_DEV && VIDEO_V4L2
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile
> > b/drivers/media/platform/mtk-vcodec/Makefile
> > index 6e1ea3a9f052..4618d43dbbc8 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -25,5 +25,11 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > mtk-vcodec-common-y := mtk_vcodec_intr.o \
> > mtk_vcodec_util.o \
> > mtk_vcodec_fw.o \
> > - mtk_vcodec_fw_vpu.o \
> > - mtk_vcodec_fw_scp.o
> > +
> > +ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU),)
> > +mtk-vcodec-common-y += mtk_vcodec_fw_vpu.o
> > +endif
> > +
> > +ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP),)
> > +mtk-vcodec-common-y += mtk_vcodec_fw_scp.o
> > +endif
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
> > b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
> > index 51f1694a7c7d..b41e66185cec 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
> > @@ -27,8 +27,26 @@ struct mtk_vcodec_fw_ops {
> > void (*release)(struct m
[PATCH v3 1/2] media: mtk-vcodec: move firmware implementations into their own files
mtk-vcodec supports two kinds of firmware, VPU and SCP. Both were
supported from the same source files, but this is clearly unclean and
makes it more difficult to disable support for one or the other.
Move these implementations into their own file, after adding the
necessary private interfaces.
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/mtk-vcodec/Makefile| 4 +-
.../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 2 +-
.../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +-
.../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 174 +-
.../media/platform/mtk-vcodec/mtk_vcodec_fw.h | 7 +-
.../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 34
.../platform/mtk-vcodec/mtk_vcodec_fw_scp.c | 73
.../platform/mtk-vcodec/mtk_vcodec_fw_vpu.c | 109 +++
8 files changed, 232 insertions(+), 173 deletions(-)
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_scp.c
create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_vpu.c
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index f679c6e1a3e9..6e1ea3a9f052 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -24,4 +24,6 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o \
- mtk_vcodec_fw.o
+ mtk_vcodec_fw.o \
+ mtk_vcodec_fw_vpu.o \
+ mtk_vcodec_fw_scp.o
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index d14bc208ea5e..145686d2c219 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -241,7 +241,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
dma_set_max_seg_size(>dev, DMA_BIT_MASK(32));
- dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
+ dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, DECODER);
if (IS_ERR(dev->fw_handler))
return PTR_ERR(dev->fw_handler);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index dcfa2c2d4def..3be8a04c4c67 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -293,7 +293,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
}
dma_set_max_seg_size(>dev, DMA_BIT_MASK(32));
- dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
+ dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, ENCODER);
if (IS_ERR(dev->fw_handler))
return PTR_ERR(dev->fw_handler);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
index 6c2a2568d844..94b39ae5c2e1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -1,193 +1,29 @@
// SPDX-License-Identifier: GPL-2.0
#include "mtk_vcodec_fw.h"
+#include "mtk_vcodec_fw_priv.h"
#include "mtk_vcodec_util.h"
#include "mtk_vcodec_drv.h"
-struct mtk_vcodec_fw_ops {
- int (*load_firmware)(struct mtk_vcodec_fw *fw);
- unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
- unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
- void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
- int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler, const char *name,
void *priv);
- int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
- unsigned int len, unsigned int wait);
-};
-
-struct mtk_vcodec_fw {
- enum mtk_vcodec_fw_type type;
- const struct mtk_vcodec_fw_ops *ops;
- struct platform_device *pdev;
- struct mtk_scp *scp;
-};
-
-static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
-{
- return vpu_load_firmware(fw->pdev);
-}
-
-static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
-{
- return vpu_get_vdec_hw_capa(fw->pdev);
-}
-
-static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
-{
- return vpu_get_venc_hw_capa(fw->pdev);
-}
-
-static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
- u32 dtcm_dmem_addr)
-{
- return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
-}
-
-static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
- mtk_vcodec_ipi_handler handler,
-
[PATCH v3 0/2] media: mtk-vcodec: fix builds when remoteproc is disabled
Thanks everyone for the feedback on v2. This version has grown a little bit, but most of the added lines is just code moving around to new files. All in all this certainly makes the driver a little bit cleaner. Tested on both MT8173 and MT8183 and confirmed that the decoder was working on both. Changes since v2: * Use the FOO || !FOO magic suggested by Hans to ensure a built-in module does not try to link against symbols in a module, * Added a patch to split the VPU and SCP ops into their own source files and make the optional build cleaner, * Control the build of firmware implementations using two new transparent Kconfig symbols. Changes since v1: * Added Acked-by from Randy. * Fixed typo in Kconfig description. Alexandre Courbot (2): media: mtk-vcodec: move firmware implementations into their own files media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled drivers/media/platform/Kconfig| 22 ++- drivers/media/platform/mtk-vcodec/Makefile| 10 +- .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 2 +- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +- .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 174 +- .../media/platform/mtk-vcodec/mtk_vcodec_fw.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 52 ++ .../platform/mtk-vcodec/mtk_vcodec_fw_scp.c | 73 .../platform/mtk-vcodec/mtk_vcodec_fw_vpu.c | 109 +++ 9 files changed, 274 insertions(+), 177 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_scp.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_vpu.c -- 2.28.0.1011.ga647a8990f-goog
[PATCH v3 2/2] media: mtk-vcodec: fix build breakage when one of VPU or SCP is enabled
The addition of MT8183 support added a dependency on the SCP remoteproc
module. However the initial patch used the "select" Kconfig directive,
which may result in the SCP module to not be compiled if remoteproc was
disabled. In such a case, mtk-vcodec would try to link against
non-existent SCP symbols. "select" was clearly misused here as explained
in kconfig-language.txt.
Replace this by a "depends" directive on at least one of the VPU and
SCP modules, to allow the driver to be compiled as long as one of these
is enabled, and adapt the code to support this new scenario.
Also adapt the Kconfig text to explain the extra requirements for MT8173
and MT8183.
Reported-by: Sakari Ailus
Signed-off-by: Alexandre Courbot
Acked-by: Randy Dunlap # build-tested
Acked-by: Sakari Ailus
---
drivers/media/platform/Kconfig| 22 +++
drivers/media/platform/mtk-vcodec/Makefile| 10 +++--
.../platform/mtk-vcodec/mtk_vcodec_fw_priv.h | 18 +++
3 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index a3cb104956d5..457b6c39ddc0 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -253,18 +253,32 @@ config VIDEO_MEDIATEK_VCODEC
depends on MTK_IOMMU || COMPILE_TEST
depends on VIDEO_DEV && VIDEO_V4L2
depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on VIDEO_MEDIATEK_VPU || MTK_SCP
+ # The two following lines ensure we have the same state ("m" or "y") as
+ # our dependencies, to avoid missing symbols during link.
+ depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU
+ depends on MTK_SCP || !MTK_SCP
select VIDEOBUF2_DMA_CONTIG
select V4L2_MEM2MEM_DEV
- select VIDEO_MEDIATEK_VPU
- select MTK_SCP
+ select VIDEO_MEDIATEK_VCODEC_VPU if VIDEO_MEDIATEK_VPU
+ select VIDEO_MEDIATEK_VCODEC_SCP if MTK_SCP
help
Mediatek video codec driver provides HW capability to
- encode and decode in a range of video formats
- This driver rely on VPU driver to communicate with VPU.
+ encode and decode in a range of video formats on MT8173
+ and MT8183.
+
+ Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
+ also be selected. Support for MT8183 depends on MTK_SCP.
To compile this driver as modules, choose M here: the
modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
+config VIDEO_MEDIATEK_VCODEC_VPU
+ bool
+
+config VIDEO_MEDIATEK_VCODEC_SCP
+ bool
+
config VIDEO_MEM2MEM_DEINTERLACE
tristate "Deinterlace support"
depends on VIDEO_DEV && VIDEO_V4L2
diff --git a/drivers/media/platform/mtk-vcodec/Makefile
b/drivers/media/platform/mtk-vcodec/Makefile
index 6e1ea3a9f052..4618d43dbbc8 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -25,5 +25,11 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
mtk-vcodec-common-y := mtk_vcodec_intr.o \
mtk_vcodec_util.o \
mtk_vcodec_fw.o \
- mtk_vcodec_fw_vpu.o \
- mtk_vcodec_fw_scp.o
+
+ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU),)
+mtk-vcodec-common-y += mtk_vcodec_fw_vpu.o
+endif
+
+ifneq ($(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP),)
+mtk-vcodec-common-y += mtk_vcodec_fw_scp.o
+endif
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
index 51f1694a7c7d..b41e66185cec 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw_priv.h
@@ -27,8 +27,26 @@ struct mtk_vcodec_fw_ops {
void (*release)(struct mtk_vcodec_fw *fw);
};
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VCODEC_VPU)
struct mtk_vcodec_fw *mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
enum mtk_vcodec_fw_use fw_use);
+#else
+static inline struct mtk_vcodec_fw *
+mtk_vcodec_fw_vpu_init(struct mtk_vcodec_dev *dev,
+ enum mtk_vcodec_fw_use fw_use)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif /* CONFIG_VIDEO_MEDIATEK_VCODEC_VPU */
+
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VCODEC_SCP)
struct mtk_vcodec_fw *mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev);
+#else
+static inline struct mtk_vcodec_fw *
+mtk_vcodec_fw_scp_init(struct mtk_vcodec_dev *dev)
+{
+ return ERR_PTR(-ENODEV);
+}
+#endif /* CONFIG_VIDEO_MEDIATEK_VCODEC_SCP */
#endif /* _MTK_VCODEC_FW_PRIV_H_ */
--
2.28.0.1011.ga647a8990f-goog
Re: [PATCH v2] media: mtk-vcodec: fix builds when remoteproc is disabled
Hi Mauro, On Fri, Oct 9, 2020 at 3:34 PM Mauro Carvalho Chehab wrote: > > Em Fri, 9 Oct 2020 13:30:06 +0900 > Alexandre Courbot escreveu: > > > On Fri, Oct 9, 2020 at 1:13 AM Hans Verkuil > > wrote: > > > > >>> If VIDEO_MEDIATEK_VPU=y and MTK_SCP=m, then VIDEO_MEDIATEK_VCODEC can > > > >>> be configured > > > >>> to y, and then it won't be able to find the scp_ functions. > > > >>> > > > >>> To be honest, I'm not sure how to solve this. > > > >> > > > >> Found it. Add this: > > > >> > > > >> depends on MTK_SCP || !MTK_SCP > > > >> depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU > > > >> > > > >> Ugly as hell, but it appears to be the correct incantation for this. > > While the above does the job, I'm wondering if the better wouldn't > be to have this spit into 3 config dependencies. E. g. something like: > > config VIDEO_MEDIATEK_CODEC > depends on VIDEO_MEDIATEK_VPU_SCP || VIDEO_MEDIATEK_VPU > > config VIDEO_MEDIATEK_VPU > depends on VIDEO_DEV && VIDEO_V4L2 > depends on ARCH_MEDIATEK || COMPILE_TEST > tristate "support for Mediatek Video Processor Unit without SCP" > help > ... > > config VIDEO_MEDIATEK_VPU_SCP > depends on VIDEO_DEV && VIDEO_V4L2 > depends on ARCH_MEDIATEK || COMPILE_TEST > tristate "support for Mediatek Video Processor Unit with SCP" > help > ... Doing so would introduce two extra choices to enable the driver, so I'm a bit concerned this may be a bit confusing? Also I have experimented with this, and it appears that VIDEO_MEDIATEK_CODEC won't be automatically enabled if one of the new options is selected. So this means that after setting e.g. VIDEO_MEDIATEK_VPU_SCP, one still needs to manually enable VIDEO_MEDIATEK_CODEC otherwise the driver won't be compiled at all. > > And split the board-specific data for each variant on separate files, > doing something like this at the Makefile: > > obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ >mtk-vcodec-enc.o \ >mtk-vcodec-common.o > > ifneq ($(VIDEO_MEDIATEK_VPU_SCP),) > obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-fw-scp.o > endif > > ifneq ($(VIDEO_MEDIATEK_VPU),) > obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-fw-vpu.o > endif > > This will avoid the ugly ifdefs in the middle of mtk_vcodec_fw.c, > and the ugly "depends on FOO || !FOO" usage. > > It should also be simpler to add future variants of it in the > future, if needed. Indeed, the split makes sense regardless of the selection mechanism adopted. I will try to do it in the next revision.
Re: [PATCH 1/3] venus: vdec: Fix non reliable setting of LAST flag
On Tue, Sep 29, 2020 at 1:44 AM Stanimir Varbanov
wrote:
>
> In real use of dynamic-resolution-change it is observed that the
> LAST buffer flag (which marks the last decoded buffer with the
> resolution before the resolution-change event) is not reliably set.
>
> Fix this by set the LAST buffer flag on next queued capture buffer
> after the resolution-change event.
>
> Signed-off-by: Stanimir Varbanov
> ---
> drivers/media/platform/qcom/venus/core.h| 5 +-
> drivers/media/platform/qcom/venus/helpers.c | 6 +++
> drivers/media/platform/qcom/venus/vdec.c| 52 -
> 3 files changed, 38 insertions(+), 25 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
> index 7b79a33dc9d6..e30eeaf0ada9 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -274,7 +274,6 @@ enum venus_dec_state {
> VENUS_DEC_STATE_DRAIN = 5,
> VENUS_DEC_STATE_DECODING= 6,
> VENUS_DEC_STATE_DRC = 7,
> - VENUS_DEC_STATE_DRC_FLUSH_DONE = 8,
> };
>
> struct venus_ts_metadata {
> @@ -339,7 +338,7 @@ struct venus_ts_metadata {
> * @priv: a private for HFI operations callbacks
> * @session_type: the type of the session (decoder or encoder)
> * @hprop: a union used as a holder by get property
> - * @last_buf: last capture buffer for dynamic-resoluton-change
> + * @next_buf_last: a flag to mark next queued capture buffer as last
> */
> struct venus_inst {
> struct list_head list;
> @@ -401,7 +400,7 @@ struct venus_inst {
> union hfi_get_property hprop;
> unsigned int core_acquired: 1;
> unsigned int bit_depth;
> - struct vb2_buffer *last_buf;
> + bool next_buf_last;
> };
>
> #define IS_V1(core)((core)->res->hfi_version == HFI_VERSION_1XX)
> diff --git a/drivers/media/platform/qcom/venus/helpers.c
> b/drivers/media/platform/qcom/venus/helpers.c
> index 50439eb1ffea..5ca3920237c5 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -1347,6 +1347,12 @@ void venus_helper_vb2_buf_queue(struct vb2_buffer *vb)
>
> v4l2_m2m_buf_queue(m2m_ctx, vbuf);
>
> + /* Skip processing queued capture buffers after LAST flag */
> + if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
> + V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type) &&
> + inst->codec_state == VENUS_DEC_STATE_DRC)
> + goto unlock;
> +
> cache_payload(inst, vb);
>
> if (inst->session_type == VIDC_SESSION_TYPE_ENC &&
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
> index ea13170a6a2c..c11bdf3ca21b 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -914,10 +914,6 @@ static int vdec_start_capture(struct venus_inst *inst)
> return 0;
>
> reconfigure:
> - ret = hfi_session_flush(inst, HFI_FLUSH_OUTPUT, true);
> - if (ret)
> - return ret;
> -
> ret = vdec_output_conf(inst);
> if (ret)
> return ret;
> @@ -954,6 +950,7 @@ static int vdec_start_capture(struct venus_inst *inst)
> inst->streamon_cap = 1;
> inst->sequence_cap = 0;
> inst->reconfig = false;
> + inst->next_buf_last = false;
>
> return 0;
>
> @@ -985,6 +982,7 @@ static int vdec_start_output(struct venus_inst *inst)
> venus_helper_init_instance(inst);
> inst->sequence_out = 0;
> inst->reconfig = false;
> + inst->next_buf_last = false;
>
> ret = vdec_set_properties(inst);
> if (ret)
> @@ -1078,9 +1076,7 @@ static int vdec_stop_capture(struct venus_inst *inst)
> inst->codec_state = VENUS_DEC_STATE_STOPPED;
> break;
> case VENUS_DEC_STATE_DRC:
> - WARN_ON(1);
> - fallthrough;
> - case VENUS_DEC_STATE_DRC_FLUSH_DONE:
> + ret = hfi_session_flush(inst, HFI_FLUSH_OUTPUT, true);
> inst->codec_state = VENUS_DEC_STATE_CAPTURE_SETUP;
> venus_helper_free_dpb_bufs(inst);
> break;
> @@ -1204,9 +1200,28 @@ static void vdec_buf_cleanup(struct vb2_buffer *vb)
> static void vdec_vb2_buf_queue(struct vb2_buffer *vb)
> {
> struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + static const struct v4l2_event eos = { .type = V4L2_EVENT_EOS };
>
> vdec_pm_get_put(inst);
>
> + mutex_lock(>lock);
> +
> + if (inst->next_buf_last && V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type)
> &&
> + inst->codec_state == VENUS_DEC_STATE_DRC) {
> + vbuf->flags |= V4L2_BUF_FLAG_LAST;
> + vbuf->sequence = inst->sequence_cap++;
> +
Re: [PATCH 1/3] venus: vdec: Fix non reliable setting of LAST flag
On Tue, Sep 29, 2020 at 1:44 AM Stanimir Varbanov
wrote:
>
> In real use of dynamic-resolution-change it is observed that the
> LAST buffer flag (which marks the last decoded buffer with the
> resolution before the resolution-change event) is not reliably set.
>
> Fix this by set the LAST buffer flag on next queued capture buffer
> after the resolution-change event.
>
> Signed-off-by: Stanimir Varbanov
> ---
> drivers/media/platform/qcom/venus/core.h| 5 +-
> drivers/media/platform/qcom/venus/helpers.c | 6 +++
> drivers/media/platform/qcom/venus/vdec.c| 52 -
> 3 files changed, 38 insertions(+), 25 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
> index 7b79a33dc9d6..e30eeaf0ada9 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -274,7 +274,6 @@ enum venus_dec_state {
> VENUS_DEC_STATE_DRAIN = 5,
> VENUS_DEC_STATE_DECODING= 6,
> VENUS_DEC_STATE_DRC = 7,
> - VENUS_DEC_STATE_DRC_FLUSH_DONE = 8,
> };
>
> struct venus_ts_metadata {
> @@ -339,7 +338,7 @@ struct venus_ts_metadata {
> * @priv: a private for HFI operations callbacks
> * @session_type: the type of the session (decoder or encoder)
> * @hprop: a union used as a holder by get property
> - * @last_buf: last capture buffer for dynamic-resoluton-change
> + * @next_buf_last: a flag to mark next queued capture buffer as last
> */
> struct venus_inst {
> struct list_head list;
> @@ -401,7 +400,7 @@ struct venus_inst {
> union hfi_get_property hprop;
> unsigned int core_acquired: 1;
> unsigned int bit_depth;
> - struct vb2_buffer *last_buf;
> + bool next_buf_last;
> };
>
> #define IS_V1(core)((core)->res->hfi_version == HFI_VERSION_1XX)
> diff --git a/drivers/media/platform/qcom/venus/helpers.c
> b/drivers/media/platform/qcom/venus/helpers.c
> index 50439eb1ffea..5ca3920237c5 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -1347,6 +1347,12 @@ void venus_helper_vb2_buf_queue(struct vb2_buffer *vb)
>
> v4l2_m2m_buf_queue(m2m_ctx, vbuf);
>
> + /* Skip processing queued capture buffers after LAST flag */
> + if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
> + V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type) &&
> + inst->codec_state == VENUS_DEC_STATE_DRC)
> + goto unlock;
> +
> cache_payload(inst, vb);
>
> if (inst->session_type == VIDC_SESSION_TYPE_ENC &&
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
> index ea13170a6a2c..c11bdf3ca21b 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -914,10 +914,6 @@ static int vdec_start_capture(struct venus_inst *inst)
> return 0;
>
> reconfigure:
> - ret = hfi_session_flush(inst, HFI_FLUSH_OUTPUT, true);
> - if (ret)
> - return ret;
> -
> ret = vdec_output_conf(inst);
> if (ret)
> return ret;
> @@ -954,6 +950,7 @@ static int vdec_start_capture(struct venus_inst *inst)
> inst->streamon_cap = 1;
> inst->sequence_cap = 0;
> inst->reconfig = false;
> + inst->next_buf_last = false;
Is this needed? Whether a resolution change occurs should only be
dependent on what the OUTPUT queue receives, so even if the CAPTURE
queue is stopped and resumed for some reason, pending resolution
change events and their associated LAST buffer should still be
emitted. With this statement we are taking the risk of sending a
change resolution event without a corresponding LAST buffer (so the
following LAST buffer might be misinterpreted).
>
> return 0;
>
> @@ -985,6 +982,7 @@ static int vdec_start_output(struct venus_inst *inst)
> venus_helper_init_instance(inst);
> inst->sequence_out = 0;
> inst->reconfig = false;
> + inst->next_buf_last = false;
This one I understand better - if the client seeks, it should probably
check for pending events before resuming.
>
> ret = vdec_set_properties(inst);
> if (ret)
> @@ -1078,9 +1076,7 @@ static int vdec_stop_capture(struct venus_inst *inst)
> inst->codec_state = VENUS_DEC_STATE_STOPPED;
> break;
> case VENUS_DEC_STATE_DRC:
> - WARN_ON(1);
> - fallthrough;
> - case VENUS_DEC_STATE_DRC_FLUSH_DONE:
> + ret = hfi_session_flush(inst, HFI_FLUSH_OUTPUT, true);
> inst->codec_state = VENUS_DEC_STATE_CAPTURE_SETUP;
> venus_helper_free_dpb_bufs(inst);
> break;
> @@ -1204,9 +1200,28 @@ static void vdec_buf_cleanup(struct vb2_buffer *vb)
> static void vdec_vb2_buf_queue(struct vb2_buffer
[PATCH] venus: vdec: return parsed crop information from stream
Per the stateful codec specification, VIDIOC_G_SELECTION with a target
of V4L2_SEL_TGT_COMPOSE is supposed to return the crop area of capture
buffers containing the decoded frame. Until now the driver did not get
that information from the firmware and just returned the dimensions of
CAPTURE buffers.
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/qcom/venus/core.h | 1 +
drivers/media/platform/qcom/venus/vdec.c | 21 -
2 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/core.h
b/drivers/media/platform/qcom/venus/core.h
index 7b79a33dc9d6..3bc129a4f817 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -361,6 +361,7 @@ struct venus_inst {
unsigned int streamon_cap, streamon_out;
u32 width;
u32 height;
+ struct v4l2_rect crop;
u32 out_width;
u32 out_height;
u32 colorspace;
diff --git a/drivers/media/platform/qcom/venus/vdec.c
b/drivers/media/platform/qcom/venus/vdec.c
index ea13170a6a2c..ee74346f0cae 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -325,6 +325,10 @@ static int vdec_s_fmt(struct file *file, void *fh, struct
v4l2_format *f)
inst->width = format.fmt.pix_mp.width;
inst->height = format.fmt.pix_mp.height;
+ inst->crop.top = 0;
+ inst->crop.left = 0;
+ inst->crop.width = inst->width;
+ inst->crop.height = inst->height;
if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
inst->fmt_out = fmt;
@@ -343,6 +347,9 @@ vdec_g_selection(struct file *file, void *fh, struct
v4l2_selection *s)
s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
return -EINVAL;
+ s->r.top = 0;
+ s->r.left = 0;
+
switch (s->target) {
case V4L2_SEL_TGT_CROP_BOUNDS:
case V4L2_SEL_TGT_CROP_DEFAULT:
@@ -363,16 +370,12 @@ vdec_g_selection(struct file *file, void *fh, struct
v4l2_selection *s)
case V4L2_SEL_TGT_COMPOSE:
if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
return -EINVAL;
- s->r.width = inst->out_width;
- s->r.height = inst->out_height;
+ s->r = inst->crop;
break;
default:
return -EINVAL;
}
- s->r.top = 0;
- s->r.left = 0;
-
return 0;
}
@@ -1309,6 +1312,10 @@ static void vdec_event_change(struct venus_inst *inst,
inst->width = format.fmt.pix_mp.width;
inst->height = format.fmt.pix_mp.height;
+ inst->crop.left = ev_data->input_crop.left;
+ inst->crop.top = ev_data->input_crop.top;
+ inst->crop.width = ev_data->input_crop.width;
+ inst->crop.height = ev_data->input_crop.height;
inst->out_width = ev_data->width;
inst->out_height = ev_data->height;
@@ -1412,6 +1419,10 @@ static void vdec_inst_init(struct venus_inst *inst)
inst->fmt_cap = _formats[0];
inst->width = frame_width_min(inst);
inst->height = ALIGN(frame_height_min(inst), 32);
+ inst->crop.left = 0;
+ inst->crop.top = 0;
+ inst->crop.width = inst->width;
+ inst->crop.height = inst->height;
inst->out_width = frame_width_min(inst);
inst->out_height = frame_height_min(inst);
inst->fps = 30;
--
2.28.0.1011.ga647a8990f-goog
Re: [PATCH v2] media: mtk-vcodec: fix builds when remoteproc is disabled
On Fri, Oct 9, 2020 at 1:13 AM Hans Verkuil wrote: > > On 08/10/2020 16:02, Alexandre Courbot wrote: > > Hi Hans, thanks for taking the time to look at this! > > > > On Thu, Oct 8, 2020 at 10:12 PM Hans Verkuil > > wrote: > >> > >> On 08/10/2020 15:07, Hans Verkuil wrote: > >>> Hi Alexandre, > >>> > >>> On 04/10/2020 14:22, Alexandre Courbot wrote: > >>>> The addition of MT8183 support added a dependency on the SCP remoteproc > >>>> module. However the initial patch used the "select" Kconfig directive, > >>>> which may result in the SCP module to not be compiled if remoteproc was > >>>> disabled. In such a case, mtk-vcodec would try to link against > >>>> non-existent SCP symbols. "select" was clearly misused here as explained > >>>> in kconfig-language.txt. > >>>> > >>>> Replace this by a "depends" directive on at least one of the VPU and > >>>> SCP modules, to allow the driver to be compiled as long as one of these > >>>> is enabled, and adapt the code to support this new scenario. > >>>> > >>>> Also adapt the Kconfig text to explain the extra requirements for MT8173 > >>>> and MT8183. > >>>> > >>>> Reported-by: Sakari Ailus > >>>> Signed-off-by: Alexandre Courbot > >>>> Acked-by: Randy Dunlap # build-tested > >>>> --- > >>>> drivers/media/platform/Kconfig| 10 +-- > >>>> .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 72 --- > >>>> 2 files changed, 54 insertions(+), 28 deletions(-) > >>>> > >>>> diff --git a/drivers/media/platform/Kconfig > >>>> b/drivers/media/platform/Kconfig > >>>> index a3cb104956d5..98eb62e49ec2 100644 > >>>> --- a/drivers/media/platform/Kconfig > >>>> +++ b/drivers/media/platform/Kconfig > >>>> @@ -253,14 +253,16 @@ config VIDEO_MEDIATEK_VCODEC > >>>> depends on MTK_IOMMU || COMPILE_TEST > >>>> depends on VIDEO_DEV && VIDEO_V4L2 > >>>> depends on ARCH_MEDIATEK || COMPILE_TEST > >>>> +depends on VIDEO_MEDIATEK_VPU || MTK_SCP > >>> > >>> Close, but no cigar. > >>> > >>> If VIDEO_MEDIATEK_VPU=y and MTK_SCP=m, then VIDEO_MEDIATEK_VCODEC can be > >>> configured > >>> to y, and then it won't be able to find the scp_ functions. > >>> > >>> To be honest, I'm not sure how to solve this. > >> > >> Found it. Add this: > >> > >> depends on MTK_SCP || !MTK_SCP > >> depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU > >> > >> Ugly as hell, but it appears to be the correct incantation for this. > > > > But doesn't it mean that the driver can be compiled if !MTK_SCP and > > !VIDEO_MEDIATEK_VPU? That's the one case we want to avoid. > > No, because you still have: > > depends on VIDEO_MEDIATEK_VPU || MTK_SCP > > So at least one of these must be set. > > Just try it :-) Aha, I misread your message and thought you suggested replacing the dependencies with these two lines. In this case it would certainly work! Thanks for the suggestion, I'll send a v3 soon. > > Regards, > > Hans
Re: [PATCH v2] media: mtk-vcodec: fix builds when remoteproc is disabled
Hi Hans, thanks for taking the time to look at this! On Thu, Oct 8, 2020 at 10:12 PM Hans Verkuil wrote: > > On 08/10/2020 15:07, Hans Verkuil wrote: > > Hi Alexandre, > > > > On 04/10/2020 14:22, Alexandre Courbot wrote: > >> The addition of MT8183 support added a dependency on the SCP remoteproc > >> module. However the initial patch used the "select" Kconfig directive, > >> which may result in the SCP module to not be compiled if remoteproc was > >> disabled. In such a case, mtk-vcodec would try to link against > >> non-existent SCP symbols. "select" was clearly misused here as explained > >> in kconfig-language.txt. > >> > >> Replace this by a "depends" directive on at least one of the VPU and > >> SCP modules, to allow the driver to be compiled as long as one of these > >> is enabled, and adapt the code to support this new scenario. > >> > >> Also adapt the Kconfig text to explain the extra requirements for MT8173 > >> and MT8183. > >> > >> Reported-by: Sakari Ailus > >> Signed-off-by: Alexandre Courbot > >> Acked-by: Randy Dunlap # build-tested > >> --- > >> drivers/media/platform/Kconfig| 10 +-- > >> .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 72 --- > >> 2 files changed, 54 insertions(+), 28 deletions(-) > >> > >> diff --git a/drivers/media/platform/Kconfig > >> b/drivers/media/platform/Kconfig > >> index a3cb104956d5..98eb62e49ec2 100644 > >> --- a/drivers/media/platform/Kconfig > >> +++ b/drivers/media/platform/Kconfig > >> @@ -253,14 +253,16 @@ config VIDEO_MEDIATEK_VCODEC > >> depends on MTK_IOMMU || COMPILE_TEST > >> depends on VIDEO_DEV && VIDEO_V4L2 > >> depends on ARCH_MEDIATEK || COMPILE_TEST > >> +depends on VIDEO_MEDIATEK_VPU || MTK_SCP > > > > Close, but no cigar. > > > > If VIDEO_MEDIATEK_VPU=y and MTK_SCP=m, then VIDEO_MEDIATEK_VCODEC can be > > configured > > to y, and then it won't be able to find the scp_ functions. > > > > To be honest, I'm not sure how to solve this. > > Found it. Add this: > > depends on MTK_SCP || !MTK_SCP > depends on VIDEO_MEDIATEK_VPU || !VIDEO_MEDIATEK_VPU > > Ugly as hell, but it appears to be the correct incantation for this. But doesn't it mean that the driver can be compiled if !MTK_SCP and !VIDEO_MEDIATEK_VPU? That's the one case we want to avoid.
Re: [PATCH v2] media: mtk-vcodec: fix builds when remoteproc is disabled
On Mon, Oct 5, 2020 at 5:49 PM Sakari Ailus wrote: > > Hi Alexandre, > > On Sun, Oct 04, 2020 at 09:22:34PM +0900, Alexandre Courbot wrote: > > The addition of MT8183 support added a dependency on the SCP remoteproc > > module. However the initial patch used the "select" Kconfig directive, > > which may result in the SCP module to not be compiled if remoteproc was > > disabled. In such a case, mtk-vcodec would try to link against > > non-existent SCP symbols. "select" was clearly misused here as explained > > in kconfig-language.txt. > > > > Replace this by a "depends" directive on at least one of the VPU and > > SCP modules, to allow the driver to be compiled as long as one of these > > is enabled, and adapt the code to support this new scenario. > > > > Also adapt the Kconfig text to explain the extra requirements for MT8173 > > and MT8183. > > > > Reported-by: Sakari Ailus > > Signed-off-by: Alexandre Courbot > > Acked-by: Randy Dunlap # build-tested > > Thanks for the patch! > > Acked-by: Sakari Ailus Thanks! > > I wonder if this driver suffers from similar object lifetime management > issues than V4L2 and MC do, albeit not related to either. Say, what happens > if you unbind the other device while mtk-vcodec is in use? That's a question that maybe the driver maintainers can answer, but from my experience during development I have been able to unload one of the two mtk-vcodec-* modules while keeping the other one active.
Re: [PATCH v2] media: mtk-vcodec: fix builds when remoteproc is disabled
On Sun, Oct 4, 2020 at 9:22 PM Alexandre Courbot wrote:
>
> The addition of MT8183 support added a dependency on the SCP remoteproc
> module. However the initial patch used the "select" Kconfig directive,
> which may result in the SCP module to not be compiled if remoteproc was
> disabled. In such a case, mtk-vcodec would try to link against
> non-existent SCP symbols. "select" was clearly misused here as explained
> in kconfig-language.txt.
>
> Replace this by a "depends" directive on at least one of the VPU and
> SCP modules, to allow the driver to be compiled as long as one of these
> is enabled, and adapt the code to support this new scenario.
>
> Also adapt the Kconfig text to explain the extra requirements for MT8173
> and MT8183.
>
> Reported-by: Sakari Ailus
> Signed-off-by: Alexandre Courbot
> Acked-by: Randy Dunlap # build-tested
Forgot to send the changelog, so here it is:
Changes since v1:
* Added Acked-by from Randy.
* Fixed typo in Kconfig description.
> ---
> drivers/media/platform/Kconfig| 10 +--
> .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 72 ---
> 2 files changed, 54 insertions(+), 28 deletions(-)
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index a3cb104956d5..98eb62e49ec2 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -253,14 +253,16 @@ config VIDEO_MEDIATEK_VCODEC
> depends on MTK_IOMMU || COMPILE_TEST
> depends on VIDEO_DEV && VIDEO_V4L2
> depends on ARCH_MEDIATEK || COMPILE_TEST
> + depends on VIDEO_MEDIATEK_VPU || MTK_SCP
> select VIDEOBUF2_DMA_CONTIG
> select V4L2_MEM2MEM_DEV
> - select VIDEO_MEDIATEK_VPU
> - select MTK_SCP
> help
> Mediatek video codec driver provides HW capability to
> - encode and decode in a range of video formats
> - This driver rely on VPU driver to communicate with VPU.
> + encode and decode in a range of video formats on MT8173
> + and MT8183.
> +
> + Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
> + also be selected. Support for MT8183 depends on MTK_SCP.
>
> To compile this driver as modules, choose M here: the
> modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> index 6c2a2568d844..23a80027a8fb 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> @@ -13,6 +13,7 @@ struct mtk_vcodec_fw_ops {
> mtk_vcodec_ipi_handler handler, const char *name,
> void *priv);
> int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> unsigned int len, unsigned int wait);
> + void (*release)(struct mtk_vcodec_fw *fw);
> };
>
> struct mtk_vcodec_fw {
> @@ -22,6 +23,8 @@ struct mtk_vcodec_fw {
> struct mtk_scp *scp;
> };
>
> +#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VPU)
> +
> static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> {
> return vpu_load_firmware(fw->pdev);
> @@ -64,6 +67,27 @@ static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw
> *fw, int id, void *buf,
> return vpu_ipi_send(fw->pdev, id, buf, len);
> }
>
> +static void mtk_vcodec_vpu_release(struct mtk_vcodec_fw *fw)
> +{
> + put_device(>pdev->dev);
> +}
> +
> +static void mtk_vcodec_vpu_reset_handler(void *priv)
> +{
> + struct mtk_vcodec_dev *dev = priv;
> + struct mtk_vcodec_ctx *ctx;
> +
> + mtk_v4l2_err("Watchdog timeout!!");
> +
> + mutex_lock(>dev_mutex);
> + list_for_each_entry(ctx, >ctx_list, list) {
> + ctx->state = MTK_STATE_ABORT;
> + mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> + ctx->id);
> + }
> + mutex_unlock(>dev_mutex);
> +}
> +
> static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> .load_firmware = mtk_vcodec_vpu_load_firmware,
> .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> @@ -71,8 +95,13 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg =
> {
> .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> .ipi_send = mtk_vcodec_vpu_ipi_send,
> + .release = mtk_vcodec_vpu_release,
> };
>
> +#endif /* IS_ENABLED(CONFIG_VIDEO_MEDIATEK_
[PATCH v2] media: mtk-vcodec: fix builds when remoteproc is disabled
The addition of MT8183 support added a dependency on the SCP remoteproc
module. However the initial patch used the "select" Kconfig directive,
which may result in the SCP module to not be compiled if remoteproc was
disabled. In such a case, mtk-vcodec would try to link against
non-existent SCP symbols. "select" was clearly misused here as explained
in kconfig-language.txt.
Replace this by a "depends" directive on at least one of the VPU and
SCP modules, to allow the driver to be compiled as long as one of these
is enabled, and adapt the code to support this new scenario.
Also adapt the Kconfig text to explain the extra requirements for MT8173
and MT8183.
Reported-by: Sakari Ailus
Signed-off-by: Alexandre Courbot
Acked-by: Randy Dunlap # build-tested
---
drivers/media/platform/Kconfig| 10 +--
.../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 72 ---
2 files changed, 54 insertions(+), 28 deletions(-)
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index a3cb104956d5..98eb62e49ec2 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -253,14 +253,16 @@ config VIDEO_MEDIATEK_VCODEC
depends on MTK_IOMMU || COMPILE_TEST
depends on VIDEO_DEV && VIDEO_V4L2
depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on VIDEO_MEDIATEK_VPU || MTK_SCP
select VIDEOBUF2_DMA_CONTIG
select V4L2_MEM2MEM_DEV
- select VIDEO_MEDIATEK_VPU
- select MTK_SCP
help
Mediatek video codec driver provides HW capability to
- encode and decode in a range of video formats
- This driver rely on VPU driver to communicate with VPU.
+ encode and decode in a range of video formats on MT8173
+ and MT8183.
+
+ Note that support for MT8173 requires VIDEO_MEDIATEK_VPU to
+ also be selected. Support for MT8183 depends on MTK_SCP.
To compile this driver as modules, choose M here: the
modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
index 6c2a2568d844..23a80027a8fb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -13,6 +13,7 @@ struct mtk_vcodec_fw_ops {
mtk_vcodec_ipi_handler handler, const char *name,
void *priv);
int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
unsigned int len, unsigned int wait);
+ void (*release)(struct mtk_vcodec_fw *fw);
};
struct mtk_vcodec_fw {
@@ -22,6 +23,8 @@ struct mtk_vcodec_fw {
struct mtk_scp *scp;
};
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VPU)
+
static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
{
return vpu_load_firmware(fw->pdev);
@@ -64,6 +67,27 @@ static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw,
int id, void *buf,
return vpu_ipi_send(fw->pdev, id, buf, len);
}
+static void mtk_vcodec_vpu_release(struct mtk_vcodec_fw *fw)
+{
+ put_device(>pdev->dev);
+}
+
+static void mtk_vcodec_vpu_reset_handler(void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+
+ mtk_v4l2_err("Watchdog timeout!!");
+
+ mutex_lock(>dev_mutex);
+ list_for_each_entry(ctx, >ctx_list, list) {
+ ctx->state = MTK_STATE_ABORT;
+ mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
+ ctx->id);
+ }
+ mutex_unlock(>dev_mutex);
+}
+
static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
.load_firmware = mtk_vcodec_vpu_load_firmware,
.get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
@@ -71,8 +95,13 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
.map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
.ipi_register = mtk_vcodec_vpu_set_ipi_register,
.ipi_send = mtk_vcodec_vpu_ipi_send,
+ .release = mtk_vcodec_vpu_release,
};
+#endif /* IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VPU) */
+
+#if IS_ENABLED(CONFIG_MTK_SCP)
+
static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
{
return rproc_boot(scp_get_rproc(fw->scp));
@@ -107,6 +136,11 @@ static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw
*fw, int id, void *buf,
return scp_ipi_send(fw->scp, id, buf, len, wait);
}
+static void mtk_vcodec_scp_release(struct mtk_vcodec_fw *fw)
+{
+ scp_put(fw->scp);
+}
+
static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
.load_firmware = mtk_vcodec_scp_load_firmware,
.get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
@@ -114,23 +148,10 @@ static const struct mtk_vcodec_fw_ops
mtk_vcodec_rproc_msg = {
.map_dm_addr = mtk_vcod
Re: [PATCH] media: mtk-vcodec: fix builds when remoteproc is disabled
On Sun, Oct 4, 2020 at 1:50 AM Randy Dunlap wrote: > > On 10/3/20 6:09 AM, Alexandre Courbot wrote: > > The addition of MT8183 support added a dependency on the SCP remoteproc > > module. However the initial patch used the "select" Kconfig directive, > > which may result in the SCP module to not be compiled if remoteproc was > > disabled. In such a case, mtk-vcodec would try to link against > > non-existent SCP symbols. "select" was clearly misused here as explained > > in kconfig-language.txt. > > > > Replace this by a "depends" directive on at least one of the VPU and > > SCP modules, to allow the driver to be compiled as long as one of these > > is enabled, and adapt the code to support this new scenario. > > > > Also adapt the Kconfig text to explain the extra requirements for MT8173 > > and MT8183. > > > > Reported-by: Sakari Ailus > > Signed-off-by: Alexandre Courbot > > I was seeing this also, so I checked this patch. WFM. > > Acked-by: Randy Dunlap # build-tested Thanks for checking! I will send a v2 with your Acked-by and fix. > > See below. > > > --- > > drivers/media/platform/Kconfig| 11 +-- > > .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 72 --- > > 2 files changed, 55 insertions(+), 28 deletions(-) > > > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig > > index a3cb104956d5..e559d9c529b6 100644 > > --- a/drivers/media/platform/Kconfig > > +++ b/drivers/media/platform/Kconfig > > @@ -253,14 +253,17 @@ config VIDEO_MEDIATEK_VCODEC > > depends on MTK_IOMMU || COMPILE_TEST > > depends on VIDEO_DEV && VIDEO_V4L2 > > depends on ARCH_MEDIATEK || COMPILE_TEST > > + depends on VIDEO_MEDIATEK_VPU || MTK_SCP > > select VIDEOBUF2_DMA_CONTIG > > select V4L2_MEM2MEM_DEV > > - select VIDEO_MEDIATEK_VPU > > - select MTK_SCP > > help > > Mediatek video codec driver provides HW capability to > > - encode and decode in a range of video formats > > - This driver rely on VPU driver to communicate with VPU. > > + encode and decode in a range of video formats on MT8173 > > + and MT8183. > > + > > + Note that support for support for MT8173 requires > > Drop one of "support for" above. (or "for support" ;) > > > + VIDEO_MEDIATEK_VPU to also be selected. Support for > > + MT8183 depends on MTK_SCP. > > > > To compile this driver as modules, choose M here: the > > modules will be called mtk-vcodec-dec and mtk-vcodec-enc. > > > thanks. > -- > ~Randy
[PATCH] media: mtk-vcodec: fix builds when remoteproc is disabled
The addition of MT8183 support added a dependency on the SCP remoteproc
module. However the initial patch used the "select" Kconfig directive,
which may result in the SCP module to not be compiled if remoteproc was
disabled. In such a case, mtk-vcodec would try to link against
non-existent SCP symbols. "select" was clearly misused here as explained
in kconfig-language.txt.
Replace this by a "depends" directive on at least one of the VPU and
SCP modules, to allow the driver to be compiled as long as one of these
is enabled, and adapt the code to support this new scenario.
Also adapt the Kconfig text to explain the extra requirements for MT8173
and MT8183.
Reported-by: Sakari Ailus
Signed-off-by: Alexandre Courbot
---
drivers/media/platform/Kconfig| 11 +--
.../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 72 ---
2 files changed, 55 insertions(+), 28 deletions(-)
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index a3cb104956d5..e559d9c529b6 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -253,14 +253,17 @@ config VIDEO_MEDIATEK_VCODEC
depends on MTK_IOMMU || COMPILE_TEST
depends on VIDEO_DEV && VIDEO_V4L2
depends on ARCH_MEDIATEK || COMPILE_TEST
+ depends on VIDEO_MEDIATEK_VPU || MTK_SCP
select VIDEOBUF2_DMA_CONTIG
select V4L2_MEM2MEM_DEV
- select VIDEO_MEDIATEK_VPU
- select MTK_SCP
help
Mediatek video codec driver provides HW capability to
- encode and decode in a range of video formats
- This driver rely on VPU driver to communicate with VPU.
+ encode and decode in a range of video formats on MT8173
+ and MT8183.
+
+ Note that support for support for MT8173 requires
+ VIDEO_MEDIATEK_VPU to also be selected. Support for
+ MT8183 depends on MTK_SCP.
To compile this driver as modules, choose M here: the
modules will be called mtk-vcodec-dec and mtk-vcodec-enc.
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
index 6c2a2568d844..23a80027a8fb 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -13,6 +13,7 @@ struct mtk_vcodec_fw_ops {
mtk_vcodec_ipi_handler handler, const char *name,
void *priv);
int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
unsigned int len, unsigned int wait);
+ void (*release)(struct mtk_vcodec_fw *fw);
};
struct mtk_vcodec_fw {
@@ -22,6 +23,8 @@ struct mtk_vcodec_fw {
struct mtk_scp *scp;
};
+#if IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VPU)
+
static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
{
return vpu_load_firmware(fw->pdev);
@@ -64,6 +67,27 @@ static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw,
int id, void *buf,
return vpu_ipi_send(fw->pdev, id, buf, len);
}
+static void mtk_vcodec_vpu_release(struct mtk_vcodec_fw *fw)
+{
+ put_device(>pdev->dev);
+}
+
+static void mtk_vcodec_vpu_reset_handler(void *priv)
+{
+ struct mtk_vcodec_dev *dev = priv;
+ struct mtk_vcodec_ctx *ctx;
+
+ mtk_v4l2_err("Watchdog timeout!!");
+
+ mutex_lock(>dev_mutex);
+ list_for_each_entry(ctx, >ctx_list, list) {
+ ctx->state = MTK_STATE_ABORT;
+ mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
+ ctx->id);
+ }
+ mutex_unlock(>dev_mutex);
+}
+
static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
.load_firmware = mtk_vcodec_vpu_load_firmware,
.get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
@@ -71,8 +95,13 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
.map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
.ipi_register = mtk_vcodec_vpu_set_ipi_register,
.ipi_send = mtk_vcodec_vpu_ipi_send,
+ .release = mtk_vcodec_vpu_release,
};
+#endif /* IS_ENABLED(CONFIG_VIDEO_MEDIATEK_VPU) */
+
+#if IS_ENABLED(CONFIG_MTK_SCP)
+
static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
{
return rproc_boot(scp_get_rproc(fw->scp));
@@ -107,6 +136,11 @@ static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw
*fw, int id, void *buf,
return scp_ipi_send(fw->scp, id, buf, len, wait);
}
+static void mtk_vcodec_scp_release(struct mtk_vcodec_fw *fw)
+{
+ scp_put(fw->scp);
+}
+
static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
.load_firmware = mtk_vcodec_scp_load_firmware,
.get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
@@ -114,23 +148,10 @@ static const struct mtk_vcodec_fw_ops
mtk_vcodec_rproc_msg = {
.map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
Re: [PATCH RESEND RESEND] remoteproc: scp: add COMPILE_TEST dependency
On Tue, Sep 15, 2020 at 12:25 PM Bjorn Andersson wrote: > > On Tue 15 Sep 01:29 UTC 2020, Alexandre Courbot wrote: > > > This will improve this driver's build coverage. > > > > Reported-by: Ezequiel Garcia > > Signed-off-by: Alexandre Courbot > > --- > > Hi remoteproc maintainers, > > > > Second resend as I got no reaction for almost 1 month on this one-liner. > > Sorry about that. I fell behind on my inbox and have missed your > previous attempts. > > This has now been applied. No worries, thanks for the quick response. Mauro, the patch is applied on https://git.kernel.org/pub/scm/linux/kernel/git/andersson/remoteproc.git/commit/?id=5185e3a9dc2d68bb52e3e12400428aa060b87733, will it work for you to merge this into the media tree and apply the pull request on top? > > Regards, > Bjorn > > > Pretty please? > > > > As explained in > > https://www.spinics.net/lists/linux-media/msg175991.html, we need this > > patch in order to merge a driver series in the media tree. If that > > looks ok to you, can we pull it in the media tree along with the series > > that depends on it? > > > > drivers/remoteproc/Kconfig | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig > > index c6659dfea7c7..d1fcada71017 100644 > > --- a/drivers/remoteproc/Kconfig > > +++ b/drivers/remoteproc/Kconfig > > @@ -43,7 +43,7 @@ config INGENIC_VPU_RPROC > > > > config MTK_SCP > > tristate "Mediatek SCP support" > > - depends on ARCH_MEDIATEK > > + depends on ARCH_MEDIATEK || COMPILE_TEST > > select RPMSG_MTK_SCP > > help > > Say y here to support Mediatek's System Companion Processor (SCP) > > via > > -- > > 2.28.0.526.ge36021eeef-goog > >
[PATCH RESEND RESEND] remoteproc: scp: add COMPILE_TEST dependency
This will improve this driver's build coverage. Reported-by: Ezequiel Garcia Signed-off-by: Alexandre Courbot --- Hi remoteproc maintainers, Second resend as I got no reaction for almost 1 month on this one-liner. Pretty please? As explained in https://www.spinics.net/lists/linux-media/msg175991.html, we need this patch in order to merge a driver series in the media tree. If that looks ok to you, can we pull it in the media tree along with the series that depends on it? drivers/remoteproc/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig index c6659dfea7c7..d1fcada71017 100644 --- a/drivers/remoteproc/Kconfig +++ b/drivers/remoteproc/Kconfig @@ -43,7 +43,7 @@ config INGENIC_VPU_RPROC config MTK_SCP tristate "Mediatek SCP support" - depends on ARCH_MEDIATEK + depends on ARCH_MEDIATEK || COMPILE_TEST select RPMSG_MTK_SCP help Say y here to support Mediatek's System Companion Processor (SCP) via -- 2.28.0.526.ge36021eeef-goog
Re: [PATCH v4 2/3] media: uapi: Add VP9 stateless decoder controls
On Thu, Sep 10, 2020 at 3:04 PM Alexandre Courbot wrote: > > Hi Ezequiel, sorry for the late review! > > On Tue, May 19, 2020 at 2:40 AM Ezequiel Garcia > wrote: > > > > From: Boris Brezillon > > > > Add the VP9 stateless decoder controls plus the documentation that goes > > with it. > > > > Signed-off-by: Boris Brezillon > > Signed-off-by: Ezequiel Garcia > > --- > > .../userspace-api/media/v4l/biblio.rst| 10 + > > .../media/v4l/ext-ctrls-codec.rst | 550 ++ > > drivers/media/v4l2-core/v4l2-ctrls.c | 239 > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > > include/media/v4l2-ctrls.h| 1 + > > include/media/vp9-ctrls.h | 485 +++ Also another thing I missed: shouldn't the new pointer controls be added to union v4l2_ctrl_ptr in v4l2-ctrls.h?
Re: [PATCH v4 2/3] media: uapi: Add VP9 stateless decoder controls
Hi Ezequiel, sorry for the late review!
On Tue, May 19, 2020 at 2:40 AM Ezequiel Garcia wrote:
>
> From: Boris Brezillon
>
> Add the VP9 stateless decoder controls plus the documentation that goes
> with it.
>
> Signed-off-by: Boris Brezillon
> Signed-off-by: Ezequiel Garcia
> ---
> .../userspace-api/media/v4l/biblio.rst| 10 +
> .../media/v4l/ext-ctrls-codec.rst | 550 ++
> drivers/media/v4l2-core/v4l2-ctrls.c | 239
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/media/v4l2-ctrls.h| 1 +
> include/media/vp9-ctrls.h | 485 +++
> 6 files changed, 1286 insertions(+)
> create mode 100644 include/media/vp9-ctrls.h
>
> diff --git a/Documentation/userspace-api/media/v4l/biblio.rst
> b/Documentation/userspace-api/media/v4l/biblio.rst
> index 3c9634173e82..e09102e572fd 100644
> --- a/Documentation/userspace-api/media/v4l/biblio.rst
> +++ b/Documentation/userspace-api/media/v4l/biblio.rst
> @@ -414,3 +414,13 @@ VP8
> :title: RFC 6386: "VP8 Data Format and Decoding Guide"
>
> :author:J. Bankoski et al.
> +
> +.. _vp9:
> +
> +VP9
> +===
> +
> +
> +:title: VP9 Bitstream & Decoding Process Specification
> +
> +:author:Adrian Grange (Google), Peter de Rivaz (Argon Design), Jonathan
> Hunt (Argon Design)
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index d0d506a444b1..5c5f7dd868da 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -2668,6 +2668,556 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>- ``padding[3]``
>- Applications and drivers must set this to zero.
>
> +.. _v4l2-mpeg-vp9:
> +
> +``V4L2_CID_MPEG_VIDEO_VP9_FRAME_CONTEXT(0..3) (struct)``
> +Stores VP9 probabilities attached to a specific frame context. The VP9
> +specification allows using a maximum of 4 contexts. Each frame being
> +decoded refers to one of those context. See section '7.1.2 Refresh
> +probs semantics' section of :ref:`vp9` for more details about these
> +contexts.
> +
> +This control is bi-directional:
> +
> +* all 4 contexts must be initialized by userspace just after the
> + stream is started and before the first decoding request is submitted.
> +* the referenced context might be read by the kernel when a decoding
> + request is submitted, and will be updated after the decoder is done
> + decoding the frame if the `V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX` flag
> + is set.
> +* contexts will be read back by user space before each decoding request
> + to retrieve the updated probabilities.
> +* userspace will re-initialize the context to their default values when
> + a reset context is required.
Just to make sure I understand this part correctly, it means that if
frame A and B use the same context, and frame A has
V4L2_VP9_FRAME_FLAG_REFRESH_FRAME_CTX set, then user-space must wait
for frame A to get dequeued and read back this control from the
completed request before it can submit frame B?
> +
> +.. note::
> +
> + This compound control is not yet part of the public kernel API and
> + it is expected to change.
> +
> +.. c:type:: v4l2_ctrl_vp9_frame_ctx
> +
> +.. cssclass:: longtable
> +
> +.. tabularcolumns:: |p{5.8cm}|p{4.8cm}|p{6.6cm}|
> +
> +.. flat-table:: struct v4l2_ctrl_vp9_frame_ctx
> +:header-rows: 0
> +:stub-columns: 0
> +:widths: 1 1 2
> +
> +* - struct :c:type:`v4l2_vp9_probabilities`
> + - ``probs``
> + - Structure with VP9 probabilities attached to the context.
> +
> +.. c:type:: v4l2_vp9_probabilities
> +
> +.. cssclass:: longtable
> +
> +.. tabularcolumns:: |p{1.5cm}|p{6.3cm}|p{9.4cm}|
> +
> +.. flat-table:: struct v4l2_vp9_probabilities
> +:header-rows: 0
> +:stub-columns: 0
> +:widths: 1 1 2
> +
> +* - __u8
> + - ``tx8[2][1]``
> + - TX 8x8 probabilities.
> +* - __u8
> + - ``tx16[2][2]``
> + - TX 16x16 probabilities.
> +* - __u8
> + - ``tx32[2][3]``
> + - TX 32x32 probabilities.
> +* - __u8
> + - ``coef[4][2][2][6][6][3]``
> + - Coefficient probabilities.
> +* - __u8
> + - ``skip[3]``
> + - Skip probabilities.
> +* - __u8
> + - ``inter_mode[7][3]``
> + - Inter prediction mode probabilities.
> +* - __u8
> + - ``interp_filter[4][2]``
> + - Interpolation filter probabilities.
> +* - __u8
> + - ``is_inter[4]``
> + - Is inter-block probabilities.
> +* - __u8
> + - ``comp_mode[5]``
> + - Compound prediction mode probabilities.
> +* - __u8
> + - ``single_ref[5][2]``
> + - Single reference probabilities.
> +* - __u8
> + - ``comp_mode[5]``
> + - Compound reference probabilities.
> +
