Re: [PATCH v9 0/8] Add MT8173 Video Encoder Driver and VPU Driver
Hi Hans,
On Fri, 2016-04-29 at 13:14 +0200, Hans Verkuil wrote:
> Hi Tiffany,
>
> On 04/26/2016 10:58 AM, Tiffany Lin wrote:
> > ==
> > Introduction
> > ==
> >
> > The purpose of this series is to add the driver for video codec hw embedded
> > in the Mediatek's MT8173 SoCs.
> > Mediatek Video Codec is able to handle video encoding of in a range of
> > formats.
> >
> > This patch series also include VPU driver. Mediatek Video Codec driver rely
> > on VPU driver to load,
> > communicate with VPU.
> >
> > Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI
> > both have been merged in v4.6-rc1.
> >
> > ==
> > Device interface
> > ==
> >
> > In principle the driver bases on v4l2 memory-to-memory framework:
> > it provides a single video node and each opened file handle gets its own
> > private context with separate
> > buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source
> > buffers, i.e. raw video
> > frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
> >
> > ==
> > VPU (Video Processor Unit)
> > ==
> > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> > It is able to handle video decoding/encoding in a range of formats.
> > The driver provides with VPU firmware download, memory management and the
> > communication interface between CPU and VPU.
> > For VPU initialization, it will create virtual memory for CPU access and
> > physical address for VPU hw device access.
> > When a decode/encode instance opens a device node, vpu driver will download
> > vpu firmware to the device.
> > A decode/encode instant will decode/encode a frame using VPU interface to
> > interrupt vpu to handle decoding/encoding jobs.
> >
> > Please have a look at the code and comments will be very much appreciated.
>
> I build this using my daily build setup and got a bunch of errors and warnings
> from both the compiler (when this driver is compile-tested on a 32 bit arch)
> and from sparse/smatch.
>
> Can you fix these?
Our driver only support for ARM/ARM64 architecture.
I will add
MTK_IOMMU [=y] && (ARM || ARM64) to Kconfig for VIDEO_MEDIATEK_VCODEC.
And we will fix compiler warning.
best regards,
Tiffany
>
> linux-git-i686: WARNINGS
>
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c: In
> function 'load_requested_vpu':
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:498:117:
> warning: format '%lx' expects argument of type 'long unsigned int', but
> argument 4 has type 'size_t {aka unsigned int}' [-Wformat=]
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:498:117:
> warning: format '%lx' expects argument of type 'long unsigned int', but
> argument 5 has type 'size_t {aka unsigned int}' [-Wformat=]
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:501:410:
> warning: format '%lx' expects argument of type 'long unsigned int', but
> argument 4 has type 'size_t {aka unsigned int}' [-Wformat=]
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc_vpu_if.c:
> In function 'vpu_enc_ipi_handler':
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc_vpu_if.c:40:30:
> warning: cast to pointer from integer of different size
> [-Wint-to-pointer-cast]
> struct venc_vpu_inst *vpu = (struct venc_vpu_inst *)msg->venc_inst;
> ^
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:
> In function 'mtk_venc_worker':
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:1046:43:
> warning: format '%lx' expects argument of type 'long unsigned int', but
> argument 7 has type 'size_t {aka unsigned int}' [-Wformat=]
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:1046:43:
> warning: format '%lx' expects argument of type 'long unsigned int', but
> argument 10 has type 'size_t {aka unsigned int}' [-Wformat=]
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:1046:43:
> warning: format '%lx' expects argument of type 'long unsigned int', but
> argument 13 has type 'size_t {aka unsigned int}' [-Wformat=]
>
> Tip: use %zu or %zx for size_t.
>
> sparse: ERRORS
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c:261:24:
> warning: incorrect type in assignment (different base types)
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c:476:23:
> warning: symbol 'get_vp8_enc_comm_if' was not declared. Should it be static?
> /home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:237:6:
> warning: symbol 'vpu_clock_disable' was not declared. Should it be static?
>
Re: [PATCH v9 0/8] Add MT8173 Video Encoder Driver and VPU Driver
Hi Tiffany,
On 04/26/2016 10:58 AM, Tiffany Lin wrote:
> ==
> Introduction
> ==
>
> The purpose of this series is to add the driver for video codec hw embedded
> in the Mediatek's MT8173 SoCs.
> Mediatek Video Codec is able to handle video encoding of in a range of
> formats.
>
> This patch series also include VPU driver. Mediatek Video Codec driver rely
> on VPU driver to load,
> communicate with VPU.
>
> Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both
> have been merged in v4.6-rc1.
>
> ==
> Device interface
> ==
>
> In principle the driver bases on v4l2 memory-to-memory framework:
> it provides a single video node and each opened file handle gets its own
> private context with separate
> buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source
> buffers, i.e. raw video
> frames) and CAPTURE (for destination buffers, i.e. encoded video frames).
>
> ==
> VPU (Video Processor Unit)
> ==
> The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs.
> It is able to handle video decoding/encoding in a range of formats.
> The driver provides with VPU firmware download, memory management and the
> communication interface between CPU and VPU.
> For VPU initialization, it will create virtual memory for CPU access and
> physical address for VPU hw device access.
> When a decode/encode instance opens a device node, vpu driver will download
> vpu firmware to the device.
> A decode/encode instant will decode/encode a frame using VPU interface to
> interrupt vpu to handle decoding/encoding jobs.
>
> Please have a look at the code and comments will be very much appreciated.
I build this using my daily build setup and got a bunch of errors and warnings
from both the compiler (when this driver is compile-tested on a 32 bit arch)
and from sparse/smatch.
Can you fix these?
linux-git-i686: WARNINGS
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c: In
function 'load_requested_vpu':
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:498:117:
warning: format '%lx' expects argument of type 'long unsigned int', but
argument 4 has type 'size_t {aka unsigned int}' [-Wformat=]
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:498:117:
warning: format '%lx' expects argument of type 'long unsigned int', but
argument 5 has type 'size_t {aka unsigned int}' [-Wformat=]
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:501:410:
warning: format '%lx' expects argument of type 'long unsigned int', but
argument 4 has type 'size_t {aka unsigned int}' [-Wformat=]
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc_vpu_if.c:
In function 'vpu_enc_ipi_handler':
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc_vpu_if.c:40:30:
warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
struct venc_vpu_inst *vpu = (struct venc_vpu_inst *)msg->venc_inst;
^
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:
In function 'mtk_venc_worker':
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:1046:43:
warning: format '%lx' expects argument of type 'long unsigned int', but
argument 7 has type 'size_t {aka unsigned int}' [-Wformat=]
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:1046:43:
warning: format '%lx' expects argument of type 'long unsigned int', but
argument 10 has type 'size_t {aka unsigned int}' [-Wformat=]
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c:1046:43:
warning: format '%lx' expects argument of type 'long unsigned int', but
argument 13 has type 'size_t {aka unsigned int}' [-Wformat=]
Tip: use %zu or %zx for size_t.
sparse: ERRORS
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c:261:24:
warning: incorrect type in assignment (different base types)
/home/hans/work/build/media-git/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c:476:23:
warning: symbol 'get_vp8_enc_comm_if' was not declared. Should it be static?
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:237:6:
warning: symbol 'vpu_clock_disable' was not declared. Should it be static?
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:250:5:
warning: symbol 'vpu_clock_enable' was not declared. Should it be static?
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:436:54:
warning: incorrect type in return expression (different address spaces)
/home/hans/work/build/media-git/drivers/media/platform/mtk-vpu/mtk_vpu.c:504:14:
warning: incorrect type in assignment (different address spaces)
[PATCH v9 0/8] Add MT8173 Video Encoder Driver and VPU Driver
== Introduction == The purpose of this series is to add the driver for video codec hw embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to handle video encoding of in a range of formats. This patch series also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU. Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI both have been merged in v4.6-rc1. == Device interface == In principle the driver bases on v4l2 memory-to-memory framework: it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames). == VPU (Video Processor Unit) == The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs. It is able to handle video decoding/encoding in a range of formats. The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU. For VPU initialization, it will create virtual memory for CPU access and physical address for VPU hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device. A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs. Please have a look at the code and comments will be very much appreciated. Change in v9: 1. Rename idx in mtk_vcodec_ctx to id and curr_max_idx in mtk_vcodec_dev to id_counter. 2. Refine fops_vcodec_open VPU part Merge Julia Lawall's fixes[1][2] to "[PATCH v9 2/8] [media] VPU: mediatek: support Mediatek VPU" [1][PATCH] VPU: mediatek: fix simple_open.cocci warnings [2][PATCH] VPU: mediatek: fix platform_no_drv_owner.cocci warnings Change in v8: 1. Refine indentation 2. Refine colorspace information process vidioc_try_fmt_vid_out_mplane 3. Remove instance_mask in mtk_vcodec_dev, use curr_max_idx for instance index 4. Use kzalloc to allocate ctx 5. Refine fops_vcodec_open VPU Part 1. Refine vpu_load_firmware Change in v7: 1. Rebase against the master branch of git://linuxtv.org/media_tree.git 2. Add ycbcr_enc, quantization and xfer_func in try_fmt, g_fmt, s_fmt 3. Merge h264_enc and vp8_enc to venc directory Change in v6: 1. Add synchronization access protect between irq handler and work thread 2. Add DMA_ATTR_ALLOC_SINGLE_PAGES support 3. S_FMT will return coded_width, coded_height, so user space could allocate correct size memory that HW required 4. merge h264/vp8 enc ap and md32 ipi msg 5. separate h264/vp8 enc gop_size and intra_period handle 6. remove sizeimage relative code in work buffer function 7. Refine makefile to build as an module 8. Code clean up VPU Part 1. export symbols for building VPU as an module 2. change function from "wait_event_interruptible_timeout" to "wait_event_timeout" since CPU needs to wait for ACK from VPU even if it was interrupted by a signal v4l2-compliance test output: localhost Encode # ./v4l2-compliance -d /dev/video1 Driver Info: Driver name : mtk-vcodec-enc Card type : platform:mt8173 Bus info : platform:mt8173 Driver version: 4.4.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Compliance test for device /dev/video1 (not using libv4l2): Required ioctls: test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second video open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not
