Re: [RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
On 11/17/2015 01:54 PM, Tiffany Lin wrote: > == > Introduction > == > > The purpose of this RFC is to discuss the driver for a hw video codec > embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to > handle video encoding of in a range of formats. > > This RFC also include VPU driver. Mediatek Video Codec driver rely on > VPU driver to load, communicate with VPU. > > Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI. > MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start > discussion about the driver earlier so it could be merged sooner. The > driver posted here is the initial version, so I suppose it will require > more work. I plan on reviewing this patch series (at least the non-dt parts). It's busy, though, and I don't know exactly when I get the chance. But just so you know that someone will be reviewing it. Regards, Hans > > [1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html > > == > Device interface > == > > In principle the driver bases on memory-to-memory framework: > it provides a single video node and each opened file handle gets its own > private context with separate buffer queues. Each context consist of 2 > buffer queues: OUTPUT (for source buffers, i.e. raw video frames) > and CAPTURE (for destination buffers, i.e. encoded video frames). > > The process of encoding video data from stream is a bit more complicated > than typical memory-to-memory processing. We base on memory-to-memory > framework and add the complicated part in our vb2 and v4l2 callback > functionss. So we can base on well done m2m memory-to-memory framework, > reduce duplicate code and make our driver code simple. > > == > VPU (Video Processor Unit) > == > The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs. > It is able to handle video decoding/encoding of in a range of formats. > The driver provides with VPU firmware download, memory management and > the communication interface between CPU and VPU. > For VPU initialization, it will create virtual memory for CPU access and > IOMMU address for vcodec hw device access. When a decode/encode instance > opens a device node, vpu driver will download vpu firmware to the device. > A decode/encode instant will decode/encode a frame using VPU > interface to interrupt vpu to handle decoding/encoding jobs. > > Please have a look at the code and comments will be very much appreciated. > > Andrew-CT Chen (3): > dt-bindings: Add a binding for Mediatek Video Processor Unit > arm64: dts: mediatek: Add node for Mediatek Video Processor Unit > media: platform: mtk-vpu: Support Mediatek VPU > > Daniel Hsiao (1): > media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver > > Tiffany Lin (4): > dt-bindings: Add a binding for Mediatek Video Encoder > arm64: dts: mediatek: Add Video Encoder for MT8173 > media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver > media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver > > .../devicetree/bindings/media/mediatek-vcodec.txt | 58 + > .../devicetree/bindings/media/mediatek-vpu.txt | 27 + > arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 + > drivers/media/platform/Kconfig | 19 + > drivers/media/platform/Makefile|5 + > drivers/media/platform/mtk-vcodec/Kconfig |5 + > drivers/media/platform/mtk-vcodec/Makefile | 12 + > drivers/media/platform/mtk-vcodec/common/Makefile | 12 + > .../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++ > .../media/platform/mtk-vcodec/h264_enc/Makefile|9 + > .../platform/mtk-vcodec/h264_enc/venc_h264_if.c| 529 ++ > .../platform/mtk-vcodec/h264_enc/venc_h264_if.h| 53 + > .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 > .../platform/mtk-vcodec/include/venc_drv_base.h| 68 + > .../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++ > .../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++ > drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 + > drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 > > drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 + > .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++ > .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++ > .../media/platform/mtk-vcodec/mtk_vcodec_intr.c| 110 ++ > .../media/platform/mtk-vcodec/mtk_vcodec_intr.h| 30 + > drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 + > .../media/platform/mtk-vcodec/mtk_vcodec_util.c| 106 ++ > .../media/platform/mtk-vcodec/mtk_vcodec_util.h| 66 + > drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |9 + > .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 > .../p
[RESEND RFC/PATCH 0/8] Add MT8173 Video Encoder Driver and VPU Driver
== Introduction == The purpose of this RFC is to discuss the driver for a hw video codec embedded in the Mediatek's MT8173 SoCs. Mediatek Video Codec is able to handle video encoding of in a range of formats. This RFC also include VPU driver. Mediatek Video Codec driver rely on VPU driver to load, communicate with VPU. Internally the driver uses videobuf2 framework and MTK IOMMU and MTK SMI. MTK IOMMU and MTK SMI have not yet been merged, but we wanted to start discussion about the driver earlier so it could be merged sooner. The driver posted here is the initial version, so I suppose it will require more work. [1]http://lists.infradead.org/pipermail/linux-mediatek/2015-October/002525.html == Device interface == In principle the driver bases on memory-to-memory framework: it provides a single video node and each opened file handle gets its own private context with separate buffer queues. Each context consist of 2 buffer queues: OUTPUT (for source buffers, i.e. raw video frames) and CAPTURE (for destination buffers, i.e. encoded video frames). The process of encoding video data from stream is a bit more complicated than typical memory-to-memory processing. We base on memory-to-memory framework and add the complicated part in our vb2 and v4l2 callback functionss. So we can base on well done m2m memory-to-memory framework, reduce duplicate code and make our driver code simple. == VPU (Video Processor Unit) == The VPU driver for hw video codec embedded in Mediatek's MT8173 SOCs. It is able to handle video decoding/encoding of in a range of formats. The driver provides with VPU firmware download, memory management and the communication interface between CPU and VPU. For VPU initialization, it will create virtual memory for CPU access and IOMMU address for vcodec hw device access. When a decode/encode instance opens a device node, vpu driver will download vpu firmware to the device. A decode/encode instant will decode/encode a frame using VPU interface to interrupt vpu to handle decoding/encoding jobs. Please have a look at the code and comments will be very much appreciated. Andrew-CT Chen (3): dt-bindings: Add a binding for Mediatek Video Processor Unit arm64: dts: mediatek: Add node for Mediatek Video Processor Unit media: platform: mtk-vpu: Support Mediatek VPU Daniel Hsiao (1): media: platform: mtk-vcodec: Add Mediatek VP8 Video Encoder Driver Tiffany Lin (4): dt-bindings: Add a binding for Mediatek Video Encoder arm64: dts: mediatek: Add Video Encoder for MT8173 media: platform: mtk-vcodec: Add Mediatek V4L2 Video Encoder Driver media: platform: mtk-vcodec: Add Mediatek H264 Video Encoder Driver .../devicetree/bindings/media/mediatek-vcodec.txt | 58 + .../devicetree/bindings/media/mediatek-vpu.txt | 27 + arch/arm64/boot/dts/mediatek/mt8173.dtsi | 58 + drivers/media/platform/Kconfig | 19 + drivers/media/platform/Makefile|5 + drivers/media/platform/mtk-vcodec/Kconfig |5 + drivers/media/platform/mtk-vcodec/Makefile | 12 + drivers/media/platform/mtk-vcodec/common/Makefile | 12 + .../media/platform/mtk-vcodec/common/venc_drv_if.c | 159 ++ .../media/platform/mtk-vcodec/h264_enc/Makefile|9 + .../platform/mtk-vcodec/h264_enc/venc_h264_if.c| 529 ++ .../platform/mtk-vcodec/h264_enc/venc_h264_if.h| 53 + .../platform/mtk-vcodec/h264_enc/venc_h264_vpu.c | 341 .../platform/mtk-vcodec/include/venc_drv_base.h| 68 + .../platform/mtk-vcodec/include/venc_drv_if.h | 187 +++ .../platform/mtk-vcodec/include/venc_ipi_msg.h | 212 +++ drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h | 441 + drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 1773 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h | 28 + .../media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 535 ++ .../media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 122 ++ .../media/platform/mtk-vcodec/mtk_vcodec_intr.c| 110 ++ .../media/platform/mtk-vcodec/mtk_vcodec_intr.h| 30 + drivers/media/platform/mtk-vcodec/mtk_vcodec_pm.h | 26 + .../media/platform/mtk-vcodec/mtk_vcodec_util.c| 106 ++ .../media/platform/mtk-vcodec/mtk_vcodec_util.h| 66 + drivers/media/platform/mtk-vcodec/vp8_enc/Makefile |9 + .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.c | 371 .../platform/mtk-vcodec/vp8_enc/venc_vp8_if.h | 48 + .../platform/mtk-vcodec/vp8_enc/venc_vp8_vpu.c | 245 +++ drivers/media/platform/mtk-vpu/Makefile|1 + .../platform/mtk-vpu/h264_enc/venc_h264_vpu.h | 127 ++ .../media/platform/mtk-vpu/include/venc_ipi_msg.h | 212 +++ drivers/media/platform/mtk-vpu/mtk_vpu_core.c | 823 + drivers/media/platform/mtk-vpu/mtk_vpu_core.h | 161 ++ .../media/pl
