Re: [PATCH v3 1/9] staging: video: rockchip: add v4l2 decoder
On 4/12/19 12:12 AM, Nicolas Dufresne wrote: Le jeudi 11 avril 2019 à 09:29 +0800, Randy Li a écrit : We agreed with Maxime and Ezequiel that there will be two distinct format, V4L2_PIX_FMT_H264_SLICE_RAW and V4L2_PIX_FMT_H264_SLICE_ANNEXB. And user-pace will take care of providing the right information. I have no idea on what does the SLICE_RAW mean here. If you skip the NAL headers, SLICE_RAW is what is left. This is exactly what is passed in VAAPI, and what makes Rockchip HW difficult to support through this abstraction apparently. The rationale is that all the information that was in the rest of the bitstream is already passed in C structures. If this isn't the case, let me know, since it would be a bad name indeed. Yes, it is called as AVC1 as usual. SLICE_RAW doesn't indicate anything, it can be AVCCC or something else. You must follow a standard as there is. Paul asked us why not always required ANNEX-B with start code as a super set. This is not very efficient, since it means that whenever you have a stream from an ISOMP4 media, you endup having to convert the AVC headers into ANNEX-B start code, and you loose the optimization the AVC header provides (AVC format is basically just a NAL type and it's size, so you can navigate through the stream with very little reads). The But if a frame is consisted with more than 1 slice, it won't work, although it is not supported by current V4L2 framework. Anyway the start code is necessary in Annex B of the H.264 SPEC or hardware can't handle more than 1 nal unit for any purpose. Adding a 24bits or 32bits header won't cost much CPU time, much less than bitstream reconstruction. Anyway you won't not only meet this problem with H.264 but also H.265. other thing is the memory alignment the require needs. I suspect that I am sure bitstream doesn't required any memory alignment. if the HW require SLICE_RAW, you cannot just pass a random pointer offset. So you'd need another buffer the userspace data to. Hidden I think you can, hardware usually has a register to let it skip a few bits of the input stream. copies in drivers are quite bad, it's better to leave it to userspace when we can in my opinion. Now that I'm saying that out-loud, I wonder how the "one v4l2 buffer per frame" restriction can be respected for SLICE_RAW. The receiver would have no way to know where each slices are. Isn't there a big flaw in this restriction ? Or did I miss-understood the format, and SLICE_RAW is a bad name ? Annex B describe a byte stream format but it doesn't mean it should ship slice data with SPS or PPS SLICE means it won't pass SPS or PPS in-band. Only slice NALs and per slice auxiliary data if there is any. Do Rockchip needs the original SPS/PPS bitstream ? If so, all of them ? or just the activated one ? And when ? Yes, I though you know that, I talked with you before. I though that is why ezequiel want this. But that SPS/PPS is not 100% compatible with ITU document. The problem of bootlin demo is it follows Annex B but withou a start code We'll fix that if we decide to use this for testing. It was all early and unspecified. SLICE_ANNEXB will have a start code of course, as I Besides I found its SPS/PPS doesn't match the original file, I didn't make this clearly in the previous email. don't see how you can call this ANNEXB otherwise. And I'm pretty sure the reason Rockchip need it is to be able to split the slices (we need to keep in mind that multiple slices per frame is commonly used in video conference streaming). The format documentation should be really clear about what it is, and if another HW with another format comes that does not match, it should define and document a new format. cheers, Nicolas
Re: [PATCH v3 0/9] [WIP]: rockchip mpp for v4l2 video deocder
Sent from my iPad > On Apr 11, 2019, at 12:01 AM, Nicolas Dufresne wrote: > >> Le mercredi 10 avril 2019 à 20:42 +0800, ayaka a écrit : >> From: Randy 'ayaka' Li >> >> Although I really hate the bitstream construction in kernel and I think >> many people realise its problems, I still take the advise from ndufresne to >> release this version. This should be released in a early week but >> I was sick that time. >> >> After reviewed the documents from Rockchip and I have confirmed that with >> some rockchip staff, those documents are not update to date. So you may >> find some part is different comparing to official document. >> >> The v4l2-request-test from bootlin won't work. Its slice data doesn't >> have a start code which making it not a complete nal unit. And I found >> its slice header information may not be correct. Even comparing to the >> Big buck bunny's origin files, neither the slice data nor sequence >> information matches. > > The un-ordered reflist table generated by these tests or the v4l2 VAAPI > driver are known to be broken. They have never been tested. Ezequiel > and I are going to validate against MVPP userspace if this un-ordered > list effectively what we think it is, and will try and get some fixes > in at least one selected userspace. Otherwise it's not possible to test > easily. I have no idea on what the un-ordered means, hardware would only care about the POC of a reference frame which would match its list0(P or B) and list1(B slice). I would suggest to remove decode_param of AVC later, as its slice_param has contains enough information to decode a slice and its reference. > >> >> I extracted a slice data from Rockchip mpp to verify my driver, it work >> fine, you can find it on my github. I only verified the I slice now, >> I have not verified P or B slice. Hopefully it would work. > > If you have some information to share it would be nice. I'm sure we can > cooperate on this if you are willing to. > >> >> I have the same problem with v4l2-request-test on HEVC as well so even >> this version shipped with HEVC bitstream construction, I didn't know >> whether it would work. >> >> I need some time to prepare the userspace tool or it is really hard for >> HEVC to write slice info manually. >> >> Changlog >> v3: add AVC support for rkvdec >> v2: add MPEG-2 support for vdpu2 >> v1: inital version >> >> Randy Li (7): >> staging: video: rockchip: add v4l2 decoder >> rockchip: mpp: rkvdec: rbsp >> [WIP]: rockchip: mpp: HEVC decoder ctrl data >> [TEST]: rockchip: mpp: support qtable >> [TEST]: rockchip: mpp: vdpu2: move qtable to input buffer >> arm64: dts: rockchip: boost clocks for rk3328 >> arm64: dts: rockchip: add video codec for rk3328 >> >> ayaka (2): >> [WIP]: rockchip: mpp: H.264 decoder ctrl data >> [TEST]: rkvdec: spspps address alignment >> >> arch/arm64/boot/dts/rockchip/rk3328-evb.dts | 32 + >> .../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 + >> arch/arm64/boot/dts/rockchip/rk3328.dtsi | 112 +- >> drivers/staging/Kconfig |2 + >> drivers/staging/Makefile |1 + >> drivers/staging/rockchip-mpp/Kconfig | 33 + >> drivers/staging/rockchip-mpp/Makefile | 12 + >> drivers/staging/rockchip-mpp/mpp_debug.h | 87 + >> drivers/staging/rockchip-mpp/mpp_dev_common.c | 1390 +++ >> drivers/staging/rockchip-mpp/mpp_dev_common.h | 215 + >> drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 924 + >> drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 607 +++ >> drivers/staging/rockchip-mpp/mpp_service.c| 197 + >> drivers/staging/rockchip-mpp/mpp_service.h| 38 + >> .../staging/rockchip-mpp/rkvdec/avc-data.c| 239 ++ >> .../staging/rockchip-mpp/rkvdec/avc-data.h| 40 + >> drivers/staging/rockchip-mpp/rkvdec/avc.c | 259 ++ >> drivers/staging/rockchip-mpp/rkvdec/cabac.h | 3614 + >> drivers/staging/rockchip-mpp/rkvdec/hal.h | 70 + >> .../staging/rockchip-mpp/rkvdec/hevc-data.c | 208 + >> .../staging/rockchip-mpp/rkvdec/hevc-data.h | 27 + >> drivers/staging/rockchip-mpp/rkvdec/hevc.c| 169 + >> drivers/staging/rockchip-mpp/rkvdec/rbsp.c| 96 + >> drivers/staging/rockchip-mpp/rkvdec/rbsp.h| 30 + >> drivers/staging/rockchip-mpp/rkvdec/regs.h| 377 ++ >> drivers/staging/rockchip-mpp/vdpu2/avc.c | 165 + >> drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 + >>
Re: [PATCH v3 0/9] [WIP]: rockchip mpp for v4l2 video deocder
Sent from my iPad > On Apr 11, 2019, at 12:01 AM, Nicolas Dufresne wrote: > >> Le mercredi 10 avril 2019 à 20:42 +0800, ayaka a écrit : >> From: Randy 'ayaka' Li >> >> Although I really hate the bitstream construction in kernel and I think >> many people realise its problems, I still take the advise from ndufresne to >> release this version. This should be released in a early week but >> I was sick that time. >> >> After reviewed the documents from Rockchip and I have confirmed that with >> some rockchip staff, those documents are not update to date. So you may >> find some part is different comparing to official document. >> >> The v4l2-request-test from bootlin won't work. Its slice data doesn't >> have a start code which making it not a complete nal unit. And I found >> its slice header information may not be correct. Even comparing to the >> Big buck bunny's origin files, neither the slice data nor sequence >> information matches. > > The un-ordered reflist table generated by these tests or the v4l2 VAAPI > driver are known to be broken. They have never been tested. Ezequiel > and I are going to validate against MVPP userspace if this un-ordered > list effectively what we think it is, and will try and get some fixes > in at least one selected userspace. Otherwise it's not possible to test > easily. > >> >> I extracted a slice data from Rockchip mpp to verify my driver, it work >> fine, you can find it on my github. I only verified the I slice now, >> I have not verified P or B slice. Hopefully it would work. > > If you have some information to share it would be nice. I'm sure we can > cooperate on this if you are willing to. > >> >> I have the same problem with v4l2-request-test on HEVC as well so even >> this version shipped with HEVC bitstream construction, I didn't know >> whether it would work. >> >> I need some time to prepare the userspace tool or it is really hard for >> HEVC to write slice info manually. >> >> Changlog >> v3: add AVC support for rkvdec >> v2: add MPEG-2 support for vdpu2 >> v1: inital version >> >> Randy Li (7): >> staging: video: rockchip: add v4l2 decoder >> rockchip: mpp: rkvdec: rbsp >> [WIP]: rockchip: mpp: HEVC decoder ctrl data >> [TEST]: rockchip: mpp: support qtable >> [TEST]: rockchip: mpp: vdpu2: move qtable to input buffer >> arm64: dts: rockchip: boost clocks for rk3328 >> arm64: dts: rockchip: add video codec for rk3328 >> >> ayaka (2): >> [WIP]: rockchip: mpp: H.264 decoder ctrl data >> [TEST]: rkvdec: spspps address alignment >> >> arch/arm64/boot/dts/rockchip/rk3328-evb.dts | 32 + >> .../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 + >> arch/arm64/boot/dts/rockchip/rk3328.dtsi | 112 +- >> drivers/staging/Kconfig |2 + >> drivers/staging/Makefile |1 + >> drivers/staging/rockchip-mpp/Kconfig | 33 + >> drivers/staging/rockchip-mpp/Makefile | 12 + >> drivers/staging/rockchip-mpp/mpp_debug.h | 87 + >> drivers/staging/rockchip-mpp/mpp_dev_common.c | 1390 +++ >> drivers/staging/rockchip-mpp/mpp_dev_common.h | 215 + >> drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 924 + >> drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 607 +++ >> drivers/staging/rockchip-mpp/mpp_service.c| 197 + >> drivers/staging/rockchip-mpp/mpp_service.h| 38 + >> .../staging/rockchip-mpp/rkvdec/avc-data.c| 239 ++ >> .../staging/rockchip-mpp/rkvdec/avc-data.h| 40 + >> drivers/staging/rockchip-mpp/rkvdec/avc.c | 259 ++ >> drivers/staging/rockchip-mpp/rkvdec/cabac.h | 3614 + >> drivers/staging/rockchip-mpp/rkvdec/hal.h | 70 + >> .../staging/rockchip-mpp/rkvdec/hevc-data.c | 208 + >> .../staging/rockchip-mpp/rkvdec/hevc-data.h | 27 + >> drivers/staging/rockchip-mpp/rkvdec/hevc.c| 169 + >> drivers/staging/rockchip-mpp/rkvdec/rbsp.c| 96 + >> drivers/staging/rockchip-mpp/rkvdec/rbsp.h| 30 + >> drivers/staging/rockchip-mpp/rkvdec/regs.h| 377 ++ >> drivers/staging/rockchip-mpp/vdpu2/avc.c | 165 + >> drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 + >> drivers/staging/rockchip-mpp/vdpu2/mpeg2.c| 277 ++ >> drivers/staging/rockchip-mpp/vdpu2/regs.h | 670 +++ >> include/uapi/video/rk_vpu_service.h | 101 + >> 30 files changed, 10072 insertions(+), 4 deletions(-) >> create mode 100644 drivers/staging/rockchip-mpp/Kconf
[PATCH v3 9/9] arm64: dts: rockchip: add video codec for rk3328
From: Randy Li
Having problem with vepu2
The sclk_vdec_core for RKVDEC is in gpll at vendor kernel.
Signed-off-by: Randy Li
---
arch/arm64/boot/dts/rockchip/rk3328-evb.dts | 32 ++
.../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 ++
arch/arm64/boot/dts/rockchip/rk3328.dtsi | 108 +-
3 files changed, 170 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
b/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
index 8302d86d35c4..c89714f79f93 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3328-evb.dts
@@ -213,6 +213,18 @@
};
};
+&rkvdec {
+ status = "okay";
+};
+
+&rkvdec_mmu {
+ status = "okay";
+};
+
+&rkvdec_srv {
+ status = "okay";
+};
+
&sdio {
bus-width = <4>;
cap-sd-highspeed;
@@ -269,3 +281,23 @@
&usb_host0_ohci {
status = "okay";
};
+
+&vdpu {
+ status = "okay";
+};
+
+&vop {
+ status = "okay";
+};
+
+&vop_mmu {
+ status = "okay";
+};
+
+&vpu_mmu {
+ status = "okay";
+};
+
+&vpu_service{
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
b/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
index 2157a528276b..520e444806cc 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3328-rock64.dts
@@ -256,6 +256,14 @@
};
};
+&h265e {
+ status = "okay";
+};
+
+&h265e_mmu {
+ status = "okay";
+};
+
&i2s1 {
status = "okay";
@@ -300,6 +308,18 @@
};
};
+&rkvdec {
+ status = "okay";
+};
+
+&rkvdec_mmu {
+ status = "okay";
+};
+
+&rkvdec_srv {
+ status = "okay";
+};
+
&sdmmc {
bus-width = <4>;
cap-mmc-highspeed;
@@ -370,6 +390,10 @@
status = "okay";
};
+&vdpu {
+ status = "okay";
+};
+
&vop {
status = "okay";
};
@@ -377,3 +401,11 @@
&vop_mmu {
status = "okay";
};
+
+&vpu_mmu {
+ status = "okay";
+};
+
+&vpu_service{
+ status = "okay";
+};
diff --git a/arch/arm64/boot/dts/rockchip/rk3328.dtsi
b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
index 55a72abed6e7..9b3f8d22b60a 100644
--- a/arch/arm64/boot/dts/rockchip/rk3328.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk3328.dtsi
@@ -573,6 +573,27 @@
resets = <&cru SRST_GPU_A>;
};
+ venc_srv: venc-srv {
+ compatible = "rockchip,mpp-service";
+ status = "disabled";
+ };
+
+ h265e: h265e@ff33 {
+ compatible = "rockchip,hevc-encoder-v1";
+ reg = <0x0 0xff33 0 0x200>;
+ interrupts = ;
+ clocks = <&cru ACLK_H265>, <&cru PCLK_H265>,
+ <&cru SCLK_VENC_CORE>, <&cru SCLK_VENC_DSP>,
+ <&cru ACLK_RKVENC>, <&cru ACLK_AXISRAM>;
+ clock-names = "aclk_h265", "pclk_h265", "clk_core",
+ "clk_dsp", "aclk_venc", "aclk_axi2sram";
+ iommus = <&h265e_mmu>;
+ rockchip,srv = <&venc_srv>;
+ syscon = <&grf 0x040c 0x8000800 0x8>;
+ power-domains = <&power RK3328_PD_HEVC>;
+ status = "disabled";
+ };
+
h265e_mmu: iommu@ff330200 {
compatible = "rockchip,iommu";
reg = <0x0 0xff330200 0 0x100>;
@@ -584,6 +605,25 @@
status = "disabled";
};
+ vepu: vepu@ff34 {
+ compatible = "rockchip,vpu-encoder-v2";
+ reg = <0x0 0xff34 0x0 0x400>;
+ interrupts = ;
+ clocks = <&cru ACLK_H264>, <&cru HCLK_H264>,
+ <&cru SCLK_VENC_CORE>;
+ clock-names = "aclk_vcodec", "hclk_vcodec",
+ "clk_core";
+ resets = <&cru SRST_RKVENC_H264_A>,
+ <&cru SRST_RKVENC_H264_H>;
+ reset-names = "video_a", "video_h";
+ iommus = <&vepu_mmu>;
+ rockchip,srv = <&venc_srv>;
+ syscon = <&grf 0x040c 0x8000800 0x8>;
+ power-domains = <&power RK3328_PD_HEVC>;
+ status = "disabled";
+ };
+
+
vepu_mmu: iommu@ff340800 {
compatible = "rockchip,iommu";
reg = <0x0 0xff340800 0x0 0x40>;
@@ -595,6 +635,42 @@
status = "disabled";
};
+
+ vpu_service: vdpu-srv {
+ compatible = "rockchip,mpp-service";
+ status = "disabled";
+ };
+
+ vdpu: vpu-decoder@ff35 {
+ compatible = "rockchip,vpu-decoder-v2";
+ reg = <0x0 0xff350400 0x0 0x400>;
+ interrupts = ;
+ interrupt-names = "irq_dec";
+ resets = <&cru SRST_VCODEC_A>, <&cru SRST_VCODEC_H>;
+ reset-names = "video_a", "video_h";
+ clocks = <&cru ACLK_VPU>, <&cru HCLK_VPU>;
+ clock-names = "acl
[PATCH v3 8/9] arm64: dts: rockchip: boost clocks for rk3328
From: Randy Li Or VOP won't work well. Signed-off-by: Randy Li --- arch/arm64/boot/dts/rockchip/rk3328.dtsi | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/boot/dts/rockchip/rk3328.dtsi b/arch/arm64/boot/dts/rockchip/rk3328.dtsi index 84f14b132e8f..55a72abed6e7 100644 --- a/arch/arm64/boot/dts/rockchip/rk3328.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk3328.dtsi @@ -738,8 +738,8 @@ <0>, <2400>, <2400>, <2400>, <1500>, <1500>, - <1>, <1>, - <1>, <1>, + <3>, <1>, + <4>, <1>, <5000>, <1>, <1>, <1>, <5000>, <5000>, -- 2.20.1
[PATCH v3 6/9] [TEST]: rockchip: mpp: vdpu2: move qtable to input buffer
From: Randy Li
I want the memory region !!!
It can save more time if those data are prepared in userspace.
Signed-off-by: Randy Li
---
drivers/staging/rockchip-mpp/mpp_dev_common.c | 3 +--
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 ---
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 3 +++
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c| 20 +++
4 files changed, 16 insertions(+), 13 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.c
b/drivers/staging/rockchip-mpp/mpp_dev_common.c
index 21816ad8a43b..97c4d897f168 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.c
@@ -1217,8 +1217,7 @@ static int rockchip_mpp_queue_init(void *priv, struct
vb2_queue *src_vq,
src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
src_vq->drv_priv = session;
src_vq->mem_ops = &vb2_dma_contig_memops;
- src_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES |
- DMA_ATTR_NO_KERNEL_MAPPING;
+ src_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
src_vq->min_buffers_needed = 1;
src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h
b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 33d7725be67b..36770af53a95 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,9 +100,6 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
-
- dma_addr_t qtable_addr;
- void *qtable_vaddr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
index 1be73ab9c2be..92d68b962fe1 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
@@ -176,6 +176,9 @@ static int rkvdpu_s_fmt_vid_out_mplane(struct file *filp,
void *priv,
if (sizes >= SZ_16M)
return -EINVAL;
+ /* For those slice header data */
+ pix_mp->plane_fmt[pix_mp->num_planes - 1].sizeimage += SZ_1M;
+
if (vdpu_setup_ctrls(session, pix_mp->pixelformat))
return -EINVAL;
diff --git a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
index 837ee4a4a000..c12d1a8ef2da 100644
--- a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
+++ b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
@@ -113,7 +113,10 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void
*regs,
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
const struct v4l2_mpeg2_picture *picture;
+ const struct v4l2_ctrl_mpeg2_quantization *quantization;
struct vdpu2_regs *p_regs = regs;
+ void *qtable = NULL;
+ size_t stream_len = 0;
params = rockchip_mpp_get_cur_ctrl(session,
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
@@ -122,6 +125,8 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void
*regs,
sequence = ¶ms->sequence;
picture = ¶ms->picture;
+ quantization = rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
init_hw_cfg(p_regs);
@@ -198,7 +203,13 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void
*regs,
p_regs->sw64.rlc_vlc_base =
vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
p_regs->sw122.strm_start_bit = params->data_bit_offset;
- p_regs->sw51.stream_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+ stream_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+ p_regs->sw51.stream_len = stream_len;
+
+ qtable = vb2_plane_vaddr(&src_buf->vb2_buf, 0) + ALIGN(stream_len, 8);
+ mpeg2_dec_copy_qtable(qtable, quantization);
+p_regs->sw61.qtable_base = p_regs->sw64.rlc_vlc_base
+ + ALIGN(stream_len, 8);
return 0;
}
@@ -207,7 +218,6 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
- const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_picture *picture;
struct vb2_v4l2_buffer *dst_buf;
dma_addr_t cur_addr, fwd_addr, bwd_addr;
@@ -220,9 +230,6 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
picture = ¶ms->picture;
sequence = ¶ms->sequence;
- quantization = rockchip_mpp_get_cur_ctrl(session,
- V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
-
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
cur_addr = vb2_dma_contig_
[PATCH v3 5/9] [TEST]: rockchip: mpp: support qtable
From: Randy Li
I don't care, I don't want to store buffers for a session.
I just want to use it to verify the FFmpeg.
---
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 +++
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 5 -
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c| 13 -
3 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h
b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 36770af53a95..33d7725be67b 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,6 +100,9 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
+
+ dma_addr_t qtable_addr;
+ void *qtable_vaddr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
index dbd9f334562e..1be73ab9c2be 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
@@ -246,6 +246,9 @@ static int rkvdpu_open(struct file *filp)
if (IS_ERR_OR_NULL(session))
return PTR_ERR(session);
+ session->qtable_vaddr = dmam_alloc_coherent(mpp_dev->dev, 64 * 4,
+ &session->qtable_addr,
+ GFP_KERNEL);
filp->private_data = &session->fh;
pm_runtime_get_sync(mpp_dev->dev);
@@ -529,7 +532,7 @@ static int rockchip_mpp_rkvdpu_probe(struct platform_device
*pdev)
ret = mpp_dev_register_node(mpp_dev, mpp_dev->variant->node_name,
NULL, &rkvdpu_ioctl_ops);
if (ret)
- dev_err(dev, "register char device failed: %d\n", ret);
+ dev_err(dev, "register v4l2/media device failed: %d\n", ret);
memcpy(mpp_dev->fmt_out, fmt_out_templ, sizeof(fmt_out_templ));
memcpy(mpp_dev->fmt_cap, fmt_cap_templ, sizeof(fmt_cap_templ));
diff --git a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
index d32958c4cb20..837ee4a4a000 100644
--- a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
+++ b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
@@ -52,7 +52,7 @@ static const u8 intra_default_q_matrix[64] = {
static void mpeg2_dec_copy_qtable(u8 * qtable, const struct
v4l2_ctrl_mpeg2_quantization
*ctrl)
{
- int i, n;
+ int i;
if (!qtable || !ctrl)
return;
@@ -111,16 +111,12 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session,
void *regs,
struct vb2_v4l2_buffer *src_buf)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
- const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_sequence *sequence;
const struct v4l2_mpeg2_picture *picture;
struct vdpu2_regs *p_regs = regs;
params = rockchip_mpp_get_cur_ctrl(session,
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
- quantization = rockchip_mpp_get_cur_ctrl(session,
-
V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
-
if (!params)
return -EINVAL;
@@ -211,6 +207,7 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
+ const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_picture *picture;
struct vb2_v4l2_buffer *dst_buf;
dma_addr_t cur_addr, fwd_addr, bwd_addr;
@@ -223,6 +220,9 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
picture = ¶ms->picture;
sequence = ¶ms->sequence;
+ quantization = rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
+
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
cur_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
@@ -266,5 +266,8 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
p_regs->sw135.refer3_base = cur_addr >> 2;
}
+ mpeg2_dec_copy_qtable(session->qtable_vaddr, quantization);
+p_regs->sw61.qtable_base = session->qtable_addr;
+
return 0;
}
--
2.20.1
[PATCH v3 4/9] [WIP]: rockchip: mpp: H.264 decoder ctrl data
I really don't want to do this.
Signed-off-by: Randy Li
Signed-off-by: ayaka
---
drivers/staging/rockchip-mpp/Makefile | 2 +-
drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 8 +-
.../staging/rockchip-mpp/rkvdec/avc-data.c| 239 ++
.../staging/rockchip-mpp/rkvdec/avc-data.h| 40 +++
drivers/staging/rockchip-mpp/rkvdec/avc.c | 71 +-
drivers/staging/rockchip-mpp/vdpu2/avc.c | 165
6 files changed, 514 insertions(+), 11 deletions(-)
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.h
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/avc.c
diff --git a/drivers/staging/rockchip-mpp/Makefile
b/drivers/staging/rockchip-mpp/Makefile
index 8da33fa5142d..e2c2bf297812 100644
--- a/drivers/staging/rockchip-mpp/Makefile
+++ b/drivers/staging/rockchip-mpp/Makefile
@@ -2,7 +2,7 @@
rk-mpp-service-objs := mpp_service.o
rk-mpp-device-objs := mpp_dev_common.o
rk-mpp-vdec-objs := mpp_dev_rkvdec.o
-rk-mpp-vdec-objs += rkvdec/avc.o
+rk-mpp-vdec-objs += rkvdec/avc.o rkvdec/avc-data.o
rk-mpp-vdec-objs += rkvdec/hevc.o rkvdec/hevc-data.o rkvdec/rbsp.o
rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o vdpu2/mpeg2.o
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
index 756821dbf829..97abfdfc344f 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
@@ -284,13 +284,15 @@ static int rkvdec_s_fmt_vid_cap_mplane(struct file *filp,
void *priv,
pix_mp->plane_fmt[1].bytesperline * ALIGN(pix_mp->height,
8);
#else
- /* TODO: HEVC only request the height is aligned with 8 */
+ /*
+* TODO: H.264 would use 16 alignment while the resolution is
under HD,
+* HEVC only request the height is aligned with 8
+*/
pix_mp->plane_fmt[0].sizeimage =
pix_mp->plane_fmt[0].bytesperline * ALIGN(pix_mp->height,
- 16);
+ 8);
/* Additional space for motion vector */
pix_mp->plane_fmt[0].sizeimage *= 2;
- pix_mp->plane_fmt[0].sizeimage += SZ_4M;
pix_mp->plane_fmt[1].sizeimage = SZ_2M;
#endif
pix_mp->num_planes = 2;
diff --git a/drivers/staging/rockchip-mpp/rkvdec/avc-data.c
b/drivers/staging/rockchip-mpp/rkvdec/avc-data.c
new file mode 100644
index ..57172528f988
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/avc-data.c
@@ -0,0 +1,239 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li,
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "avc-data.h"
+
+static const u32 zig_zag_4x4[16] = {
+ 0, 1, 4, 8, 5, 2, 3, 6, 9, 12, 13, 10, 7, 11, 14, 15
+};
+
+static const u32 zig_zag_8x8[64] = {
+ 0, 1, 8, 16, 9, 2, 3, 10, 17, 24, 32, 25, 18, 11, 4, 5,
+ 12, 19, 26, 33, 40, 48, 41, 34, 27, 20, 13, 6, 7, 14, 21, 28,
+ 35, 42, 49, 56, 57, 50, 43, 36, 29, 22, 15, 23, 30, 37, 44, 51,
+ 58, 59, 52, 45, 38, 31, 39, 46, 53, 60, 61, 54, 47, 55, 62, 63
+};
+
+static void fill_is_long_term(struct rbsp *rbsp, const struct
v4l2_ctrl_h264_decode_param
+ *decode_param)
+{
+ u16 is_long_term = 0;
+ u8 i;
+
+ for (i = 0; i < 16; i++)
+ if (decode_param->dpb[i].
+ flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
+ is_long_term |= (1 << i);
+
+ rbsp_write_bits(rbsp, 16, is_long_term);
+}
+
+/* in zig-zag order */
+void rkvdec_avc_update_scaling_list(u8 * buf, const struct
+ v4l2_ctrl_h264_scaling_matrix
+ *scaling)
+{
+ u8 i, j;
+
+ for (i = 0; i < 6; i++)
+ for (j = 0; j < 16; j++)
+ buf[zig_zag_4x4[j] + (i << 4)] =
+ scaling->scaling_list_4x4[i][j];
+
+ for (i = 0; i < 2; i++)
+ for (j = 0; j < 64; j++)
+ buf[zig_zag_8x8[j] + (i << 6)] =
+ scaling->scaling_list_8x8[i][j];
+}
+
+int rkvdec_avc_write_sps(struct rbsp
[PATCH v3 7/9] [TEST]: rkvdec: spspps address alignment
I found the offset for cpu access is not equal to the DMA
opeartion.
Signed-off-by: ayaka
---
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 +++
drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 3 +++
drivers/staging/rockchip-mpp/rkvdec/avc.c | 14 +++---
3 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h
b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 36770af53a95..6718bcccde1f 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,6 +100,9 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
+
+ void *aux_vaddr;
+ dma_addr_t aux_addr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
index 97abfdfc344f..cc1fa9737bc1 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c
@@ -382,6 +382,9 @@ static int rkvdec_open(struct file *filp)
filp->private_data = &session->fh;
pm_runtime_get_sync(mpp_dev->dev);
+ session->aux_vaddr = dmam_alloc_coherent(mpp_dev->dev, SZ_1M,
+&session->aux_addr,
GFP_KERNEL);
+
mpp_debug_leave();
return 0;
}
diff --git a/drivers/staging/rockchip-mpp/rkvdec/avc.c
b/drivers/staging/rockchip-mpp/rkvdec/avc.c
index 1cb5b2208bfa..8266a990dca6 100644
--- a/drivers/staging/rockchip-mpp/rkvdec/avc.c
+++ b/drivers/staging/rockchip-mpp/rkvdec/avc.c
@@ -26,8 +26,8 @@
#include "regs.h"
#include "avc-data.h"
-static void generate_input_data(struct rkvdec_regs *p_regs,
- struct vb2_v4l2_buffer *src_buf,
+static void generate_input_data(struct mpp_session *session,
+ struct rkvdec_regs *p_regs,
const struct v4l2_ctrl_h264_sps *sps,
const struct v4l2_ctrl_h264_pps *pps,
const struct v4l2_ctrl_h264_scaling_matrix
@@ -44,8 +44,8 @@ static void generate_input_data(struct rkvdec_regs *p_regs,
stream_len = slice_param->size + 64;
- r_data = vb2_plane_vaddr(&src_buf->vb2_buf, 0);
- r_scaling_offs = ALIGN(stream_len, 16);
+ r_data = session->aux_vaddr;
+ r_scaling_offs = 0;
r_data += r_scaling_offs;
if (pps->flags & V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT) {
@@ -62,7 +62,7 @@ static void generate_input_data(struct rkvdec_regs *p_regs,
rkvdec_avc_write_sps(&rbsp, sps);
rkvdec_avc_write_pps(&rbsp, pps);
rkvdec_avc_write_pps_tail(&rbsp, scaling_addr, decode_param);
- p_regs->sw_pps_base = p_regs->sw_strm_rlc_base + r_sps_offs;
+ p_regs->sw_pps_base = session->aux_addr + r_sps_offs;
for (i = 1; i < 256; i++)
memset(r_data + r_sps_offs + i * 32, 0, 32);
@@ -72,7 +72,7 @@ static void generate_input_data(struct rkvdec_regs *p_regs,
r_rps_offs = ALIGN(r_rps_offs, 16);
rbsp_init(&rbsp, r_data + r_rps_offs, SZ_2M - r_rps_offs, 0);
rkvdec_avc_write_rps(&rbsp, sps, slice_param, decode_param);
- p_regs->sw_rps_base = p_regs->sw_strm_rlc_base + r_rps_offs;
+ p_regs->sw_rps_base = session->aux_addr + r_rps_offs;
}
static void init_hw_cfg(struct rkvdec_regs *p_regs)
@@ -238,7 +238,7 @@ int rkvdec_avc_gen_reg(struct mpp_session *session, void
*regs,
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
rkvdec_avc_gen_ref(p_regs, dst_buf, decode_param);
- generate_input_data(p_regs, src_buf, sps, pps, scaling, slice_param,
+ generate_input_data(session, p_regs, sps, pps, scaling, slice_param,
decode_param);
return 0;
--
2.20.1
[PATCH v3 3/9] [WIP]: rockchip: mpp: HEVC decoder ctrl data
From: Randy Li
Not done yet, not enough data.
Signed-off-by: Randy Li
---
drivers/staging/rockchip-mpp/Makefile | 4 +-
.../staging/rockchip-mpp/rkvdec/hevc-data.c | 208 ++
.../staging/rockchip-mpp/rkvdec/hevc-data.h | 27 +++
drivers/staging/rockchip-mpp/rkvdec/hevc.c| 2 +
4 files changed, 240 insertions(+), 1 deletion(-)
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.h
diff --git a/drivers/staging/rockchip-mpp/Makefile
b/drivers/staging/rockchip-mpp/Makefile
index 9722b0059563..8da33fa5142d 100644
--- a/drivers/staging/rockchip-mpp/Makefile
+++ b/drivers/staging/rockchip-mpp/Makefile
@@ -1,7 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
rk-mpp-service-objs := mpp_service.o
rk-mpp-device-objs := mpp_dev_common.o
-rk-mpp-vdec-objs := mpp_dev_rkvdec.o rkvdec/hevc.o rkvdec/avc.o
+rk-mpp-vdec-objs := mpp_dev_rkvdec.o
+rk-mpp-vdec-objs += rkvdec/avc.o
+rk-mpp-vdec-objs += rkvdec/hevc.o rkvdec/hevc-data.o rkvdec/rbsp.o
rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o vdpu2/mpeg2.o
obj-$(CONFIG_ROCKCHIP_MPP_SERVICE) += rk-mpp-service.o
diff --git a/drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
b/drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
new file mode 100644
index ..26694a2f46c5
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/hevc-data.c
@@ -0,0 +1,208 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2019 Randy Li,
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "hevc-data.h"
+
+/* 7.3.2.2.1 General sequence parameter set RBSP syntax */
+int rkvdec_hevc_write_sps(struct rbsp *rbsp,
+ const struct v4l2_ctrl_hevc_sps *sps)
+{
+ /* TODO: sps_video_parameter_set_id */
+ rbsp_write_bits(rbsp, 4, 0);
+ /* TODO: sps_seq_parameter_set_id */
+ rbsp_write_bits(rbsp, 4, 0);
+ /* chroma_format_idc */
+ rbsp_write_bits(rbsp, 2, sps->chroma_format_idc);
+ rbsp_write_bits(rbsp, 13, sps->pic_width_in_luma_samples);
+ rbsp_write_bits(rbsp, 13, sps->pic_height_in_luma_samples);
+ /* bit_depth_luma */
+ rbsp_write_bits(rbsp, 4, sps->bit_depth_luma_minus8 + 8);
+ rbsp_write_bits(rbsp, 4, sps->bit_depth_chroma_minus8 + 8);
+ /* log2_max_pic_order_cnt_lsb */
+ rbsp_write_bits(rbsp, 5, sps->log2_max_pic_order_cnt_lsb_minus4 + 4);
+ /* FIXME it is 3 bits in document */
+ rbsp_write_bits(rbsp, 2, sps->log2_diff_max_min_luma_coding_block_size);
+ /* log2_min_luma_coding_block_size */
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_min_luma_coding_block_size_minus3 + 3);
+ /* log2_min_transform_block_size */
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_min_luma_transform_block_size_minus2 + 2);
+ rbsp_write_bits(rbsp, 2,
+ sps->log2_diff_max_min_luma_transform_block_size);
+ rbsp_write_bits(rbsp, 3, sps->max_transform_hierarchy_depth_inter);
+ rbsp_write_bits(rbsp, 3, sps->max_transform_hierarchy_depth_intra);
+
+ rbsp_write_flag(rbsp, sps->scaling_list_enabled_flag);
+ rbsp_write_flag(rbsp, sps->amp_enabled_flag);
+ rbsp_write_flag(rbsp, sps->sample_adaptive_offset_enabled_flag);
+ rbsp_write_flag(rbsp, sps->pcm_enabled_flag);
+
+ /* pcm_sample_bit_depth_luma */
+ rbsp_write_bits(rbsp, 4, sps->pcm_sample_bit_depth_luma_minus1 + 1);
+ /* pcm_sample_bit_depth_chroma */
+ rbsp_write_bits(rbsp, 4, sps->pcm_sample_bit_depth_chroma_minus1 + 1);
+ rbsp_write_flag(rbsp, sps->pcm_loop_filter_disabled_flag);
+
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_diff_max_min_pcm_luma_coding_block_size);
+ /* log2_min_pcm_luma_coding_block_size */
+ rbsp_write_bits(rbsp, 3,
+ sps->log2_min_pcm_luma_coding_block_size_minus3 + 3);
+ rbsp_write_bits(rbsp, 7, sps->num_short_term_ref_pic_sets);
+ rbsp_write_flag(rbsp, sps->long_term_ref_pics_present_flag);
+ rbsp_write_bits(rbsp, 6, sps->num_long_term_ref_pics_sps);
+ rbsp_write_flag(rbsp, sps->sps_temporal_mvp_enabled_flag);
+ rbsp_write_flag(rbsp, sps->strong_intra_smoothing_enabled_flag);
+ /* Above is 100 bits total */
+#if 0
+ /* transform_skip_rotation_enabled_flag to
intra_smoothing_disabled_flag */
+ rbsp_write_bits(rbsp, 7, 0);
+ /* sps_max_dec_pic_buffering_minus1 */
+ rbsp_write_bits(rbsp, 4, sps->sps_max_dec_pic_buffering_mi
[PATCH v3 2/9] rockchip: mpp: rkvdec: rbsp
From: Randy Li
It is a bit writer.
Signed-off-by: Randy Li
---
drivers/staging/rockchip-mpp/rkvdec/rbsp.c | 96 ++
drivers/staging/rockchip-mpp/rkvdec/rbsp.h | 30 +++
2 files changed, 126 insertions(+)
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.c
create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.h
diff --git a/drivers/staging/rockchip-mpp/rkvdec/rbsp.c
b/drivers/staging/rockchip-mpp/rkvdec/rbsp.c
new file mode 100644
index ..3fbc50e9bca1
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/rbsp.c
@@ -0,0 +1,96 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2012 Vista Silicon S.L.
+ * Copyright (C) 2019 Randy Li
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include
+#include
+#include
+
+#include "rbsp.h"
+
+int rbsp_init(struct rbsp *rbsp, void *buf, int size, int bit_pos)
+{
+ if (!buf)
+ return -EINVAL;
+ if (DIV_ROUND_UP(bit_pos, 32) >= size)
+ return -EINVAL;
+
+ rbsp->buf = buf;
+ rbsp->size = size >> 2;
+ rbsp->pos = bit_pos;
+
+ return 0;
+}
+
+static inline int rbsp_read_bit(struct rbsp *rbsp)
+{
+ int shift = rbsp->pos % 32;
+ int ofs = rbsp->pos++ / 32;
+
+ if (ofs >= rbsp->size)
+ return -EINVAL;
+
+ return (rbsp->buf[ofs] >> shift) & 1;
+}
+
+static inline int rbsp_write_bit(struct rbsp *rbsp, int bit)
+{
+ int shift = rbsp->pos % 32;
+ int ofs = rbsp->pos++ / 32;
+
+ if (ofs >= rbsp->size)
+ return -EINVAL;
+
+ rbsp->buf[ofs] &= ~(1 << shift);
+ rbsp->buf[ofs] |= bit << shift;
+
+ return 0;
+}
+
+int rbsp_write_bits(struct rbsp *rbsp, int num, int value)
+{
+ int shift = rbsp->pos % 32;
+ int ofs = rbsp->pos / 32;
+
+ if (ofs >= rbsp->size)
+ return -EINVAL;
+
+ if (num + shift >= 32) {
+ u32 lbits = 32 - shift;
+ u32 hbits = num + shift - 32;
+
+ rbsp->buf[ofs] &= ~(((1 << lbits) - 1) << shift);
+ rbsp->buf[ofs] |= value << shift;
+
+ value >>= (32 - shift);
+ rbsp->buf[ofs + 1] &= ~(((1 << hbits) - 1));
+ rbsp->buf[ofs + 1] |= value;
+ } else {
+ rbsp->buf[ofs] &= ~(((1 << num) - 1) << shift);
+ rbsp->buf[ofs] |= value << shift;
+ }
+
+ rbsp->pos += num;
+
+ return 0;
+}
+
+int rbsp_write_flag(struct rbsp *rbsp, int value)
+{
+ if (value)
+ return rbsp_write_bit(rbsp, BIT(0));
+ else
+ return rbsp_write_bit(rbsp, 0);
+}
diff --git a/drivers/staging/rockchip-mpp/rkvdec/rbsp.h
b/drivers/staging/rockchip-mpp/rkvdec/rbsp.h
new file mode 100644
index ..d87c582bfd41
--- /dev/null
+++ b/drivers/staging/rockchip-mpp/rkvdec/rbsp.h
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2012 Vista Silicon S.L.
+ * Copyright (C) 2019 Randy Li
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#ifndef _RBSP_H_
+#define _RBSP_H_
+
+struct rbsp {
+ u32 *buf;
+ int size;
+ int pos;
+};
+
+int rbsp_init(struct rbsp *rbsp, void *buf, int size, int bit_pos);
+int rbsp_write_flag(struct rbsp *rbsp, int bit);
+int rbsp_write_bits(struct rbsp *rbsp, int num, int value);
+
+#endif
--
2.20.1
[PATCH v3 0/9] [WIP]: rockchip mpp for v4l2 video deocder
From: Randy 'ayaka' Li Although I really hate the bitstream construction in kernel and I think many people realise its problems, I still take the advise from ndufresne to release this version. This should be released in a early week but I was sick that time. After reviewed the documents from Rockchip and I have confirmed that with some rockchip staff, those documents are not update to date. So you may find some part is different comparing to official document. The v4l2-request-test from bootlin won't work. Its slice data doesn't have a start code which making it not a complete nal unit. And I found its slice header information may not be correct. Even comparing to the Big buck bunny's origin files, neither the slice data nor sequence information matches. I extracted a slice data from Rockchip mpp to verify my driver, it work fine, you can find it on my github. I only verified the I slice now, I have not verified P or B slice. Hopefully it would work. I have the same problem with v4l2-request-test on HEVC as well so even this version shipped with HEVC bitstream construction, I didn't know whether it would work. I need some time to prepare the userspace tool or it is really hard for HEVC to write slice info manually. Changlog v3: add AVC support for rkvdec v2: add MPEG-2 support for vdpu2 v1: inital version Randy Li (7): staging: video: rockchip: add v4l2 decoder rockchip: mpp: rkvdec: rbsp [WIP]: rockchip: mpp: HEVC decoder ctrl data [TEST]: rockchip: mpp: support qtable [TEST]: rockchip: mpp: vdpu2: move qtable to input buffer arm64: dts: rockchip: boost clocks for rk3328 arm64: dts: rockchip: add video codec for rk3328 ayaka (2): [WIP]: rockchip: mpp: H.264 decoder ctrl data [TEST]: rkvdec: spspps address alignment arch/arm64/boot/dts/rockchip/rk3328-evb.dts | 32 + .../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 + arch/arm64/boot/dts/rockchip/rk3328.dtsi | 112 +- drivers/staging/Kconfig |2 + drivers/staging/Makefile |1 + drivers/staging/rockchip-mpp/Kconfig | 33 + drivers/staging/rockchip-mpp/Makefile | 12 + drivers/staging/rockchip-mpp/mpp_debug.h | 87 + drivers/staging/rockchip-mpp/mpp_dev_common.c | 1390 +++ drivers/staging/rockchip-mpp/mpp_dev_common.h | 215 + drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 924 + drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 607 +++ drivers/staging/rockchip-mpp/mpp_service.c| 197 + drivers/staging/rockchip-mpp/mpp_service.h| 38 + .../staging/rockchip-mpp/rkvdec/avc-data.c| 239 ++ .../staging/rockchip-mpp/rkvdec/avc-data.h| 40 + drivers/staging/rockchip-mpp/rkvdec/avc.c | 259 ++ drivers/staging/rockchip-mpp/rkvdec/cabac.h | 3614 + drivers/staging/rockchip-mpp/rkvdec/hal.h | 70 + .../staging/rockchip-mpp/rkvdec/hevc-data.c | 208 + .../staging/rockchip-mpp/rkvdec/hevc-data.h | 27 + drivers/staging/rockchip-mpp/rkvdec/hevc.c| 169 + drivers/staging/rockchip-mpp/rkvdec/rbsp.c| 96 + drivers/staging/rockchip-mpp/rkvdec/rbsp.h| 30 + drivers/staging/rockchip-mpp/rkvdec/regs.h| 377 ++ drivers/staging/rockchip-mpp/vdpu2/avc.c | 165 + drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 + drivers/staging/rockchip-mpp/vdpu2/mpeg2.c| 277 ++ drivers/staging/rockchip-mpp/vdpu2/regs.h | 670 +++ include/uapi/video/rk_vpu_service.h | 101 + 30 files changed, 10072 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/rockchip-mpp/Kconfig create mode 100644 drivers/staging/rockchip-mpp/Makefile create mode 100644 drivers/staging/rockchip-mpp/mpp_debug.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.c create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc-data.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/avc.c create mode 100644 drivers/staging/rockchip-mpp/rkvdec/cabac.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hal.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.c create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc-data.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc.c create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.c create mode 100644 drivers/staging/rockchip-mpp/rkvdec/rbsp.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/regs.h create mode 100644 drivers/staging/rockchip-mpp/vdpu2/avc.c create mode 100644 drivers/staging/rockchip-mpp/vdpu2/hal.h create mode 100644 driv
Re: media: rockchip: the memory layout of multiplanes buffer for DMA address
On 3/12/19 4:22 PM, Tomasz Figa wrote: On Thu, Feb 28, 2019 at 12:13 AM Ayaka wrote: Hello all I am writing the v4l2 decoder driver for rockchip. Although I hear the suggest from the Hans before, it is ok for decoder to use single plane buffer format, but I still decide to the multi planes format. There is not a register for vdpau1 and vdpau2 setting the chroma starting address in both pixel image output(it has one only applied for jpeg) and reference. And the second plane should follow the first plane. It sounds pretty fix for the single plane, but the single plane can’t describe offset of the second plane, which is not the result of bytes per line multiples the height. There are two different memory access steps in those rockchip device, 16bytes for vdpau1 and vdpau2, 64bytes for rkvdec and 128bytes for rkvdec with a high resolution. Although those access steps can be adjusted by the memory cache registers. So it is hard to describe the pixel format with the single plane formats or userspace would need to do more work. Why not just adjust the bytes per line to a multiple of 16/64/128 bytes? Most of the platforms allocate buffers this way for performance reasons anyway. I can't, it depends on the device design. Also I need extra empty line(128/256 bytes) in each lines for the rkvdec. Maybe the device for chrome only request an alignment with 16 bytes per line and at vertical line, it is ok for the H.264 under the 1920x1080 resolution, but above that resolution would suffer a poor performance as well UHD. I made a mistake in the previous mail, the current suggest alignment for rkvdec is 256 bytes and plus an extra line. Also it requests a different alignment way for those chroma subsample are not 4:2:0. And it is not easy to align with a width value while its bitdepth is large than 8 bits. Currently, I use the dma-contig allocator in my driver, it would allocate the second plane first, which results that the second plane has a lower address than first plane, which device would request the second plane follows the first plane. I increase the sizeimage of the first plane to solve this problem now and let device can continue to run, but I need a way to solve this problem. Is there a way to control how does dma-contig allocate a buffer? I have not figured out the internal flow of the videobuf2. I know how to allocate two planes in different memory region which the s5p-mfc does with two alloc_devs, but that is not what I want. Last time in FOSDEM, kwiboo and I talk some problems of the request API and stateless decoder, I say the a method to describe a buffer with many offsets as the buffer meta data would solve the most of problems we talked, but I have no idea on how to implement it. And I think a buffer meta describing a buffer with many offsets would solve this problem as well. Sent from my iPad P.S. Please fix your email client to properly wrap the lines around the ~75 column line. Blame the thuderbird. Best regards, Tomasz
Re: media: rockchip: the memory layout of multiplanes buffer for DMA address
Sent from my iPad > On Mar 1, 2019, at 12:21 AM, Nicolas Dufresne wrote: > > Le jeudi 28 février 2019 à 09:12 +0800, Ayaka a écrit : >>> On Feb 28, 2019, at 5:07 AM, Nicolas Dufresne wrote: >>> >>> Hi Ayaka, >>> >>>> Le mercredi 27 février 2019 à 23:13 +0800, Ayaka a écrit : >>>> Last time in FOSDEM, kwiboo and I talk some problems of the request >>>> API and stateless decoder, I say the a method to describe a buffer >>>> with many offsets as the buffer meta data would solve the most of >>>> problems we talked, but I have no idea on how to implement it. And I >>>> think a buffer meta describing a buffer with many offsets would solve >>>> this problem as well. >>> >>> for single allocation case, the only supported in-between plane padding >>> is an evenly distributed padding. This padding is communicated to >>> userspace through S_FMT, by extending the width and height. Userspace >>> then reads the display width/height through G_SELECTION API. >> It can solve the partly problem, it is hard to use only width and height to >> describe a buffer. For hevc and rkvdec decoder in 128bytes mode, it is >> aligned with 128 bytes plus 128bytes. Sometimes, the padding data may. just >> less than a component. In that case, only offset would solve this problem. >>> For anything else, the MPLANE structure with one of the multi-plane >>> format can "express" such buffer, though from userspace they are >>> exposed as two memory pointer or DMABuf FDs (which make importation >> The video output(VOP) supports two address for luma and chroma and the new >> generation of rkvdec supports that as well. But for the general situation, >> we should think we can only set one DMA address to the device. >>> complicated if the buffer should initially be within a single >>> allocation). I'll leave to kernel dev the task to explain what is >>> feasible there (e.g. sub-buffering, etc.) >> I think it can use the same fd but with a different data_offset in struct >> v4l2_plane(with a larger number of byteused in the second plane). > > I think Hans said there is problem (something against the spec) in > using data_offset that way. The GStreamer implementation assumes that, > but got told recently that this might not be correct. I'd like Hans to > comment, since I haven't understood the reason yet. > >> >> If I can find a solution to solve this problem, it would be hard to future >> develop on the stateless media device. Please help on this problem. > > I don't think this is specific to state less CODEC. While more complex, > when dealing with CMA, the kernel can do that math to figure-out if two > memory pointers are from the same allocation. It quite easy if you know > in advance the spacing. > I check only the s5p-mfc use CMA only in media. Also there are two register for the luma and chroma start address. Which is different to the current problem in rockchip platform. > Without being identical, the framework does similar things when trying > to import USERPTR in a CMA based V4L2 driver. > >>> Nicolas >>>
Re: media: rockchip: the memory layout of multiplanes buffer for DMA address
> On Feb 28, 2019, at 5:07 AM, Nicolas Dufresne wrote: > > Hi Ayaka, > >> Le mercredi 27 février 2019 à 23:13 +0800, Ayaka a écrit : >> Last time in FOSDEM, kwiboo and I talk some problems of the request >> API and stateless decoder, I say the a method to describe a buffer >> with many offsets as the buffer meta data would solve the most of >> problems we talked, but I have no idea on how to implement it. And I >> think a buffer meta describing a buffer with many offsets would solve >> this problem as well. > > for single allocation case, the only supported in-between plane padding > is an evenly distributed padding. This padding is communicated to > userspace through S_FMT, by extending the width and height. Userspace > then reads the display width/height through G_SELECTION API. It can solve the partly problem, it is hard to use only width and height to describe a buffer. For hevc and rkvdec decoder in 128bytes mode, it is aligned with 128 bytes plus 128bytes. Sometimes, the padding data may. just less than a component. In that case, only offset would solve this problem. > > For anything else, the MPLANE structure with one of the multi-plane > format can "express" such buffer, though from userspace they are > exposed as two memory pointer or DMABuf FDs (which make importation The video output(VOP) supports two address for luma and chroma and the new generation of rkvdec supports that as well. But for the general situation, we should think we can only set one DMA address to the device. > complicated if the buffer should initially be within a single > allocation). I'll leave to kernel dev the task to explain what is > feasible there (e.g. sub-buffering, etc.) I think it can use the same fd but with a different data_offset in struct v4l2_plane(with a larger number of byteused in the second plane). If I can find a solution to solve this problem, it would be hard to future develop on the stateless media device. Please help on this problem. > > Nicolas >
media: rockchip: the memory layout of multiplanes buffer for DMA address
Hello all I am writing the v4l2 decoder driver for rockchip. Although I hear the suggest from the Hans before, it is ok for decoder to use single plane buffer format, but I still decide to the multi planes format. There is not a register for vdpau1 and vdpau2 setting the chroma starting address in both pixel image output(it has one only applied for jpeg) and reference. And the second plane should follow the first plane. It sounds pretty fix for the single plane, but the single plane can’t describe offset of the second plane, which is not the result of bytes per line multiples the height. There are two different memory access steps in those rockchip device, 16bytes for vdpau1 and vdpau2, 64bytes for rkvdec and 128bytes for rkvdec with a high resolution. Although those access steps can be adjusted by the memory cache registers. So it is hard to describe the pixel format with the single plane formats or userspace would need to do more work. Currently, I use the dma-contig allocator in my driver, it would allocate the second plane first, which results that the second plane has a lower address than first plane, which device would request the second plane follows the first plane. I increase the sizeimage of the first plane to solve this problem now and let device can continue to run, but I need a way to solve this problem. Is there a way to control how does dma-contig allocate a buffer? I have not figured out the internal flow of the videobuf2. I know how to allocate two planes in different memory region which the s5p-mfc does with two alloc_devs, but that is not what I want. Last time in FOSDEM, kwiboo and I talk some problems of the request API and stateless decoder, I say the a method to describe a buffer with many offsets as the buffer meta data would solve the most of problems we talked, but I have no idea on how to implement it. And I think a buffer meta describing a buffer with many offsets would solve this problem as well. Sent from my iPad
Re: [PATCH 0/4] WIP: rockchip mpp for v4l2 video decoder
Sent from my iPad > On Jan 31, 2019, at 10:03 PM, Ezequiel Garcia wrote: > > Hey Ayaka! > >> On Thu, 2019-01-31 at 11:13 +0800, ayaka wrote: >> From: Randy 'ayaka' Li >> >> Hello >> Those patches are based on the previous vendor driver I post before, >> but it can apply without the previous one. >> I really want to make it work before FOSDEM and I didn't. And upcoming >> the lunar new year holiday would last two week. >> >> I have verified the v4l2 part but I meet a problem with power or >> clock, it would be a complex problem, I would handle to solve it after I >> am back. But I just tell my design on this kind dirver. >> > > This the branch I'm about to submit: > > http://git.infradead.org/users/ezequielg/linux/shortlog/refs/heads/vpu-mpeg-2-almost-ready-for-upstream > > And it has no power issues. Perhaps you can take a look inside, > you might be just missing a few pm_ statements? Perhaps a devicetree thing? > I don’t think it is a complex problem, just I waste too time on v4l2 part All the current clock frequency at rk3328 is too low for H.264 decoding you may find my previous mail. >> A few questions I think you may ask I would like to answer it here. >> > > I have another question: is this patchset meant to support for mpeg-2 > decoding only? I can't find any other codec code. > I have not. I would like to move my work on userspace interface later, as I said in IRC. MPEG-2 is just a simple codec I choose for demo(userspace and some not normal picture coded like fileds) and if there is a codec parser there, I can solve it easily. Adding support to new codec is not complex in this driver once the interface problem is solved. I just copy the public rockchip mpp userspace library into it. It is the other advantage use the register structure. But I do like people to merge the final version I submit, I would prove it why later. > Thanks, > Ezequiel > >> 1. Why it is based on the previous vendor driver ? >> The vendor driver I post to mail list is a clean version which having a >> efficient work flow and those platform dependency controls having been >> merged into it. >> >> For the stateless device, V4L2 is just another interface for userspace >> to translate it into registers. >> >> 2. Why use a structure to store register infomation not marco ? >> I really don't want to define many marcos for a device having more than >> a hundred of registers. And there are many fields in a registers. >> >> For the VDPU1/VDPU2 which would support more than 10 video codecs, these >> two devices would re-use many registers for many codecs, It would be >> hard to show the conflict relations between them in marco but more easy >> with C union and structure. >> >> BTW, I would prefer to write a number of registers into the device >> though AHB bus not just generate one then write one, you can save some >> times here. >> >> >> Besides the two previous answers. I really have a big problem with v4l2 >> design. Which would driver wait the work of the previous picture is >> done, leading a large gap time more idle of the device. >> >> I am ok the current face that v4l2 stateless driver is not stateless. >> But it would limit the ability of stateless decoder. It is more flexible >> to those videos having different resolution or orientation sequences. >> >> But I don't like the method to reconstruct the bitstream in driver, it >> is a bad idea to limit what data the device want. Those problem is >> mainly talking in the HEVC slice thread. And it was ironic for the VPx >> serial codec, which mixed compressed data and umcompress header together >> and twisted. Even for the ITU H serial codec, it would become a problem >> in SVC or Google Android CTS test. >> >> And thanks kwiboo, ezequielg and Paulk, I copy some v4l2 code from their >> code. >> >> Randy Li (1): >> staging: video: rockchip: add video codec >> >> ayaka (3): >> [WIP]: staging: video: rockchip: add v4l2 common >> [WIP] staging: video: rockchip: vdpu2 >> [TEST]: rockchip: mpp: support qtable >> >> drivers/staging/Kconfig |2 + >> drivers/staging/Makefile |1 + >> drivers/staging/rockchip-mpp/Kconfig | 54 + >> drivers/staging/rockchip-mpp/Makefile |8 + >> drivers/staging/rockchip-mpp/mpp_debug.h | 87 ++ >> drivers/staging/rockchip-mpp/mpp_dev_common.c | 1365 + >> drivers/staging/rockchip-mpp/mpp_dev_common.h | 215 +++ >>
[PATCH 3/4] [TEST]: rockchip: mpp: support qtable
Yes, the buffer won't be freed.
I don't want to store buffers for a session.
I just want to use it to verify the FFmpeg.
Signed-off-by: ayaka
---
drivers/staging/rockchip-mpp/mpp_dev_common.h | 3 ++
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 3 ++
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c| 42 +++
3 files changed, 40 insertions(+), 8 deletions(-)
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_common.h
b/drivers/staging/rockchip-mpp/mpp_dev_common.h
index 36770af53a95..33d7725be67b 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_common.h
+++ b/drivers/staging/rockchip-mpp/mpp_dev_common.h
@@ -100,6 +100,9 @@ struct mpp_session {
struct v4l2_ctrl_handler ctrl_handler;
/* TODO: FIXME: slower than helper function ? */
struct v4l2_ctrl **ctrls;
+
+ dma_addr_t qtable_addr;
+ void *qtable_vaddr;
};
/* The context for the a task */
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
index 03ed080bb35c..9e00501c3577 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
@@ -402,6 +402,9 @@ static int rkvdpu_open(struct file *filp)
return error;
}
+ session->qtable_vaddr = dmam_alloc_coherent(mpp_dev->dev, 64 * 4,
+ &session->qtable_addr,
+ GFP_KERNEL);
filp->private_data = &session->fh;
mpp_debug_leave();
diff --git a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
index a16f7629a811..416348630779 100644
--- a/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
+++ b/drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
@@ -27,6 +27,34 @@
#define DEC_LITTLE_ENDIAN (1)
+static const u8 zigzag[64] = {
+ 0, 1, 8, 16, 9, 2, 3, 10,
+ 17, 24, 32, 25, 18, 11, 4, 5,
+ 12, 19, 26, 33, 40, 48, 41, 34,
+ 27, 20, 13, 6, 7, 14, 21, 28,
+ 35, 42, 49, 56, 57, 50, 43, 36,
+ 29, 22, 15, 23, 30, 37, 44, 51,
+ 58, 59, 52, 45, 38, 31, 39, 46,
+ 53, 60, 61, 54, 47, 55, 62, 63
+};
+
+static void mpeg2_dec_copy_qtable(u8 *qtable,
+ const struct v4l2_ctrl_mpeg2_quantization *ctrl)
+{
+ int i, n;
+
+ if (!qtable || !ctrl)
+ return;
+
+ for (i = 0; i < 64; i++) {
+ n = zigzag[i];
+ qtable[n + 0] = ctrl->intra_quantiser_matrix[i];
+ qtable[n + 64] = ctrl->non_intra_quantiser_matrix[i];
+ qtable[n + 128] = ctrl->chroma_intra_quantiser_matrix[i];
+ qtable[n + 192] = ctrl->chroma_non_intra_quantiser_matrix[i];
+ }
+}
+
static void init_hw_cfg(struct vdpu2_regs *p_regs)
{
p_regs->sw54.dec_strm_wordsp = 1;
@@ -61,7 +89,6 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void
*regs,
struct vb2_v4l2_buffer *src_buf)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
- const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_sequence *sequence;
const struct v4l2_mpeg2_picture *picture;
struct sg_table *sgt;
@@ -69,9 +96,6 @@ int rkvdpu_mpeg2_gen_reg(struct mpp_session *session, void
*regs,
params = rockchip_mpp_get_cur_ctrl(session,
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS);
- quantization = rockchip_mpp_get_cur_ctrl(session,
- V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
-
if (!params)
return -EINVAL;
@@ -164,6 +188,7 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
{
const struct v4l2_ctrl_mpeg2_slice_params *params;
const struct v4l2_mpeg2_sequence *sequence;
+ const struct v4l2_ctrl_mpeg2_quantization *quantization;
const struct v4l2_mpeg2_picture *picture;
struct vb2_v4l2_buffer *dst_buf;
dma_addr_t cur_addr, fwd_addr, bwd_addr;
@@ -177,6 +202,9 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
picture = ¶ms->picture;
sequence = ¶ms->sequence;
+ quantization = rockchip_mpp_get_cur_ctrl(session,
+ V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION);
+
dst_buf = v4l2_m2m_next_dst_buf(session->fh.m2m_ctx);
sgt = vb2_dma_sg_plane_desc(&dst_buf->vb2_buf, 0);
@@ -219,9 +247,7 @@ int rkvdpu_mpeg2_prepare_buf(struct mpp_session *session,
void *regs)
p_regs->sw134.refer2_base = bwd_addr >> 2;
p_regs->sw135.refer3_base = bwd_addr >> 2;
-#if 0
-//ref & qtable config
-p_regs->sw61.qtable_base = mpp_buffer_get_fd(ctx->qp_table);
-#endif
+ mpeg2_dec_copy_qtable(session->qtable_vaddr, quantization);
+p_regs->sw61.qtable_base = session->qtable_addr;
return 0;
}
--
2.20.1
[PATCH 4/4] staging: video: rockchip: add video codec
From: Randy Li Signed-off-by: Randy Li --- drivers/staging/Kconfig | 2 ++ drivers/staging/Makefile | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/staging/Kconfig b/drivers/staging/Kconfig index e4f608815c05..81634dd0a283 100644 --- a/drivers/staging/Kconfig +++ b/drivers/staging/Kconfig @@ -122,4 +122,6 @@ source "drivers/staging/axis-fifo/Kconfig" source "drivers/staging/erofs/Kconfig" +source "drivers/staging/rockchip-mpp/Kconfig" + endif # STAGING diff --git a/drivers/staging/Makefile b/drivers/staging/Makefile index 5868631e8f1b..22499c68c21e 100644 --- a/drivers/staging/Makefile +++ b/drivers/staging/Makefile @@ -51,3 +51,4 @@ obj-$(CONFIG_SOC_MT7621) += mt7621-dts/ obj-$(CONFIG_STAGING_GASKET_FRAMEWORK) += gasket/ obj-$(CONFIG_XIL_AXIS_FIFO)+= axis-fifo/ obj-$(CONFIG_EROFS_FS) += erofs/ +obj-$(CONFIG_ROCKCHIP_MPP_SERVICE) += rockchip-mpp/ -- 2.20.1
[PATCH 0/4] WIP: rockchip mpp for v4l2 video decoder
From: Randy 'ayaka' Li Hello Those patches are based on the previous vendor driver I post before, but it can apply without the previous one. I really want to make it work before FOSDEM and I didn't. And upcoming the lunar new year holiday would last two week. I have verified the v4l2 part but I meet a problem with power or clock, it would be a complex problem, I would handle to solve it after I am back. But I just tell my design on this kind dirver. A few questions I think you may ask I would like to answer it here. 1. Why it is based on the previous vendor driver ? The vendor driver I post to mail list is a clean version which having a efficient work flow and those platform dependency controls having been merged into it. For the stateless device, V4L2 is just another interface for userspace to translate it into registers. 2. Why use a structure to store register infomation not marco ? I really don't want to define many marcos for a device having more than a hundred of registers. And there are many fields in a registers. For the VDPU1/VDPU2 which would support more than 10 video codecs, these two devices would re-use many registers for many codecs, It would be hard to show the conflict relations between them in marco but more easy with C union and structure. BTW, I would prefer to write a number of registers into the device though AHB bus not just generate one then write one, you can save some times here. Besides the two previous answers. I really have a big problem with v4l2 design. Which would driver wait the work of the previous picture is done, leading a large gap time more idle of the device. I am ok the current face that v4l2 stateless driver is not stateless. But it would limit the ability of stateless decoder. It is more flexible to those videos having different resolution or orientation sequences. But I don't like the method to reconstruct the bitstream in driver, it is a bad idea to limit what data the device want. Those problem is mainly talking in the HEVC slice thread. And it was ironic for the VPx serial codec, which mixed compressed data and umcompress header together and twisted. Even for the ITU H serial codec, it would become a problem in SVC or Google Android CTS test. And thanks kwiboo, ezequielg and Paulk, I copy some v4l2 code from their code. Randy Li (1): staging: video: rockchip: add video codec ayaka (3): [WIP]: staging: video: rockchip: add v4l2 common [WIP] staging: video: rockchip: vdpu2 [TEST]: rockchip: mpp: support qtable drivers/staging/Kconfig |2 + drivers/staging/Makefile |1 + drivers/staging/rockchip-mpp/Kconfig | 54 + drivers/staging/rockchip-mpp/Makefile |8 + drivers/staging/rockchip-mpp/mpp_debug.h | 87 ++ drivers/staging/rockchip-mpp/mpp_dev_common.c | 1365 + drivers/staging/rockchip-mpp/mpp_dev_common.h | 215 +++ drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 878 +++ drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 755 + drivers/staging/rockchip-mpp/mpp_service.c| 197 +++ drivers/staging/rockchip-mpp/mpp_service.h| 38 + drivers/staging/rockchip-mpp/rkvdec/hal.h | 53 + drivers/staging/rockchip-mpp/rkvdec/regs.h| 395 + drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 + drivers/staging/rockchip-mpp/vdpu2/mpeg2.c| 253 +++ drivers/staging/rockchip-mpp/vdpu2/regs.h | 699 + 16 files changed, 5052 insertions(+) create mode 100644 drivers/staging/rockchip-mpp/Kconfig create mode 100644 drivers/staging/rockchip-mpp/Makefile create mode 100644 drivers/staging/rockchip-mpp/mpp_debug.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hal.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/regs.h create mode 100644 drivers/staging/rockchip-mpp/vdpu2/hal.h create mode 100644 drivers/staging/rockchip-mpp/vdpu2/mpeg2.c create mode 100644 drivers/staging/rockchip-mpp/vdpu2/regs.h -- 2.20.1
[PATCH 2/4] [WIP] staging: video: rockchip: vdpu2
From: Randy 'ayaka' Li
It doesn't work yet, I am suffering unknow power or
clock problem, but the vendor driver I post to ML
would work.
I want to put the implementation of those v4l2 ioctl
which related to device in echo device's files, but
the current inheritance looks ugly.
TODO:
qp table
Signed-off-by: Randy Li
Signed-off-by: Randy Li
---
drivers/staging/rockchip-mpp/Makefile| 2 +-
drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 588 ++--
drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 ++
drivers/staging/rockchip-mpp/vdpu2/mpeg2.c | 227 ++
drivers/staging/rockchip-mpp/vdpu2/regs.h| 699 +++
5 files changed, 1361 insertions(+), 207 deletions(-)
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/hal.h
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/mpeg2.c
create mode 100644 drivers/staging/rockchip-mpp/vdpu2/regs.h
diff --git a/drivers/staging/rockchip-mpp/Makefile
b/drivers/staging/rockchip-mpp/Makefile
index 36d2958ea7f4..5aa0c596b706 100644
--- a/drivers/staging/rockchip-mpp/Makefile
+++ b/drivers/staging/rockchip-mpp/Makefile
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
rk-mpp-service-objs := mpp_service.o
rk-mpp-device-objs := mpp_dev_common.o
-rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o
+rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o vdpu2/mpeg2.o
obj-$(CONFIG_ROCKCHIP_MPP_SERVICE) += rk-mpp-service.o
obj-$(CONFIG_ROCKCHIP_MPP_DEVICE) += rk-mpp-device.o
diff --git a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
index 5789c8940543..03ed080bb35c 100644
--- a/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
+++ b/drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c
@@ -24,61 +24,18 @@
#include
#include
+#include
+#include
+#include
+#include
+
#include "mpp_debug.h"
#include "mpp_dev_common.h"
+#include "vdpu2/hal.h"
#define RKVDPU2_DRIVER_NAME"mpp_vdpu2"
#define RKVDPU2_NODE_NAME "vpu-service"
-/* The maximum registers number of all the version */
-#define ROCKCHIP_VDPU2_REG_NUM 159
-
-/* The first register of the decoder is Reg50(0x000c8) */
-#define RKVDPU2_REG_DEC_CTRL 0x0c8
-#define RKVDPU2_REG_DEC_CTRL_INDEX (50)
-
-#define RKVDPU2_REG_SYS_CTRL 0x0d4
-#define RKVDPU2_REG_SYS_CTRL_INDEX (53)
-#defineRKVDPU2_GET_FORMAT(x) ((x) & 0xf)
-#defineRKVDPU2_FMT_H264D 0
-#defineRKVDPU2_FMT_MPEG4D 1
-#defineRKVDPU2_FMT_H263D 2
-#defineRKVDPU2_FMT_JPEGD 3
-#defineRKVDPU2_FMT_VC1D4
-#defineRKVDPU2_FMT_MPEG2D 5
-#defineRKVDPU2_FMT_MPEG1D 6
-#defineRKVDPU2_FMT_VP6D7
-#defineRKVDPU2_FMT_RESERVED8
-#defineRKVDPU2_FMT_VP7D9
-#defineRKVDPU2_FMT_VP8D10
-#defineRKVDPU2_FMT_AVSD11
-
-#define RKVDPU2_REG_DEC_INT_EN 0x0dc
-#define RKVDPU2_REG_DEC_INT_EN_INDEX (55)
-#defineRKVDPU2_INT_TIMEOUT BIT(13)
-#defineRKVDPU2_INT_STRM_ERROR BIT(12)
-#defineRKVDPU2_INT_SLICE BIT(9)
-#defineRKVDPU2_INT_ASO_ERROR BIT(8)
-#defineRKVDPU2_INT_BUF_EMPTY BIT(6)
-#defineRKVDPU2_INT_BUS_ERROR BIT(5)
-#defineRKVDPU2_DEC_INT BIT(4)
-#defineRKVDPU2_DEC_IRQ_DIS BIT(1)
-#defineRKVDPU2_DEC_INT_RAW BIT(0)
-
-#define RKVDPU2_REG_DEC_DEV_CTRL 0x0e4
-#define RKVDPU2_REG_DEC_DEV_CTRL_INDEX (57)
-#defineRKVDPU2_DEC_CLOCK_GATE_EN BIT(4)
-#defineRKVDPU2_DEC_START BIT(0)
-
-#define RKVDPU2_REG59 0x0ec
-#define RKVDPU2_REG59_INDEX(59)
-
-#define RKVDPU2_REG_DIR_MV_BASE 0x0f8
-#define RKVDPU2_REG_DIR_MV_BASE_INDEX (62)
-
-#define RKVDPU2_REG_STREAM_RLC_BASE0x100
-#define RKVDPU2_REG_STREAM_RLC_BASE_INDEX (64)
-
#define to_rkvdpu_task(ctx)\
container_of(ctx, struct rkvdpu_task, mpp_task)
#define to_rkvdpu_dev(dev) \
@@ -102,184 +59,389 @@ struct rkvdpu_task {
u32 reg[ROCKCHIP_VDPU2_REG_NUM];
u32 idx;
- struct extra_info_for_iommu ext_inf;
u32 strm_base;
u32 irq_status;
};
-/*
- * file handle translate information
- */
-static const char trans_tbl_default[] = {
- 61, 62, 63, 64, 131, 134, 135, 148
+static struct rockchip_mpp_control vdpu_control
[PATCH 1/4] [WIP]: staging: video: rockchip: add v4l2 common
From: Randy 'ayaka' Li The current version is designed for multi-planes buffers. TODO: improve the interface and work flow of v4l2 finish a task before it would be dequeued Signed-off-by: Randy Li Signed-off-by: Randy Li --- drivers/staging/rockchip-mpp/Kconfig | 54 + drivers/staging/rockchip-mpp/Makefile |8 + drivers/staging/rockchip-mpp/mpp_debug.h | 87 ++ drivers/staging/rockchip-mpp/mpp_dev_common.c | 1365 + drivers/staging/rockchip-mpp/mpp_dev_common.h | 212 +++ drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 878 +++ drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 576 +++ drivers/staging/rockchip-mpp/mpp_service.c| 197 +++ drivers/staging/rockchip-mpp/mpp_service.h| 38 + drivers/staging/rockchip-mpp/rkvdec/hal.h | 53 + drivers/staging/rockchip-mpp/rkvdec/regs.h| 395 + 11 files changed, 3863 insertions(+) create mode 100644 drivers/staging/rockchip-mpp/Kconfig create mode 100644 drivers/staging/rockchip-mpp/Makefile create mode 100644 drivers/staging/rockchip-mpp/mpp_debug.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hal.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/regs.h diff --git a/drivers/staging/rockchip-mpp/Kconfig b/drivers/staging/rockchip-mpp/Kconfig new file mode 100644 index ..9d58cd1ab6e4 --- /dev/null +++ b/drivers/staging/rockchip-mpp/Kconfig @@ -0,0 +1,54 @@ +# SPDX-License-Identifier: GPL-2.0 +menu "ROCKCHIP_MPP" + depends on ARCH_ROCKCHIP + +config ROCKCHIP_MPP_SERVICE + tristate "mpp service scheduler" + default n + help + rockchip mpp service. + +config ROCKCHIP_MPP_DEVICE + tristate "mpp device framework" + depends on ROCKCHIP_MPP_SERVICE + select V4L2_MEM2MEM_DEV + select VIDEOBUF2_DMA_SG + default n + help + rockchip mpp device framework. + +config ROCKCHIP_MPP_VDEC_DEVICE + tristate "video decoder device driver" + depends on ROCKCHIP_MPP_DEVICE + default n + help + rockchip mpp video decoder and hevc decoder. + +config ROCKCHIP_MPP_VDPU1_DEVICE + tristate "VPU decoder v1 device driver" + depends on ROCKCHIP_MPP_DEVICE + default n + help + rockchip mpp vpu decoder v1. + +config ROCKCHIP_MPP_VEPU1_DEVICE + tristate "VPU encoder v1 device driver" + depends on ROCKCHIP_MPP_DEVICE + default n + help + rockchip mpp vpu encoder v1. + +config ROCKCHIP_MPP_VDPU2_DEVICE + tristate "VPU decoder v2 device driver" + depends on ROCKCHIP_MPP_DEVICE + default n + help + rockchip mpp vpu decoder v2. + +config ROCKCHIP_MPP_VEPU2_DEVICE + tristate "VPU encoder v2 device driver" + depends on ROCKCHIP_MPP_DEVICE + default n + help + rockchip mpp vpu encoder v2. +endmenu diff --git a/drivers/staging/rockchip-mpp/Makefile b/drivers/staging/rockchip-mpp/Makefile new file mode 100644 index ..36d2958ea7f4 --- /dev/null +++ b/drivers/staging/rockchip-mpp/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +rk-mpp-service-objs := mpp_service.o +rk-mpp-device-objs := mpp_dev_common.o +rk-mpp-vdpu2-objs := mpp_dev_vdpu2.o + +obj-$(CONFIG_ROCKCHIP_MPP_SERVICE) += rk-mpp-service.o +obj-$(CONFIG_ROCKCHIP_MPP_DEVICE) += rk-mpp-device.o +obj-$(CONFIG_ROCKCHIP_MPP_VDPU2_DEVICE) += rk-mpp-vdpu2.o diff --git a/drivers/staging/rockchip-mpp/mpp_debug.h b/drivers/staging/rockchip-mpp/mpp_debug.h new file mode 100644 index ..bd6c0e594da3 --- /dev/null +++ b/drivers/staging/rockchip-mpp/mpp_debug.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2016 - 2017 Fuzhou Rockchip Electronics Co., Ltd + * + * This software is licensed under the terms of the GNU General Public + * License version 2, as published by the Free Software Foundation, and + * may be copied, distributed, and modified under those terms. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + */ + +#ifndef _ROCKCHIP_MPP_DEBUG_H_ +#define _ROCKCHIP_MPP_DEBUG_H_ + +#include + +/* + * debug flag usage: + * +--+---+ + * | 8bit | 24bit| + * +--+---+ + * 0~23 bit is for different
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Sent from my iPad > On Jan 30, 2019, at 3:17 PM, Tomasz Figa wrote: > >> On Wed, Jan 30, 2019 at 3:28 PM Ayaka wrote: >> >> >> >> Sent from my iPad >> >>> On Jan 30, 2019, at 11:35 AM, Tomasz Figa wrote: >>> >>> On Wed, Jan 30, 2019 at 11:29 AM Alexandre Courbot >>> wrote: >>>> >>>>>> On Wed, Jan 30, 2019 at 6:41 AM Nicolas Dufresne >>>>>> wrote: >>>>>> >>>>>> Le mardi 29 janvier 2019 à 16:44 +0900, Alexandre Courbot a écrit : >>>>>> On Fri, Jan 25, 2019 at 10:04 PM Paul Kocialkowski >>>>>> wrote: >>>>>>> Hi, >>>>>>> >>>>>>>> On Thu, 2019-01-24 at 20:23 +0800, Ayaka wrote: >>>>>>>> Sent from my iPad >>>>>>>> >>>>>>>>> On Jan 24, 2019, at 6:27 PM, Paul Kocialkowski >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>>> On Thu, 2019-01-10 at 21:32 +0800, ayaka wrote: >>>>>>>>>> I forget a important thing, for the rkvdec and rk hevc decoder, it >>>>>>>>>> would >>>>>>>>>> requests cabac table, scaling list, picture parameter set and >>>>>>>>>> reference >>>>>>>>>> picture storing in one or various of DMA buffers. I am not talking >>>>>>>>>> about >>>>>>>>>> the data been parsed, the decoder would requests a raw data. >>>>>>>>>> >>>>>>>>>> For the pps and rps, it is possible to reuse the slice header, just >>>>>>>>>> let >>>>>>>>>> the decoder know the offset from the bitstream bufer, I would >>>>>>>>>> suggest to >>>>>>>>>> add three properties(with sps) for them. But I think we need a >>>>>>>>>> method to >>>>>>>>>> mark a OUTPUT side buffer for those aux data. >>>>>>>>> >>>>>>>>> I'm quite confused about the hardware implementation then. From what >>>>>>>>> you're saying, it seems that it takes the raw bitstream elements >>>>>>>>> rather >>>>>>>>> than parsed elements. Is it really a stateless implementation? >>>>>>>>> >>>>>>>>> The stateless implementation was designed with the idea that only the >>>>>>>>> raw slice data should be passed in bitstream form to the decoder. For >>>>>>>>> H.264, it seems that some decoders also need the slice header in raw >>>>>>>>> bitstream form (because they take the full slice NAL unit), see the >>>>>>>>> discussions in this thread: >>>>>>>>> media: docs-rst: Document m2m stateless video decoder interface >>>>>>>> >>>>>>>> Stateless just mean it won’t track the previous result, but I don’t >>>>>>>> think you can define what a date the hardware would need. Even you >>>>>>>> just build a dpb for the decoder, it is still stateless, but parsing >>>>>>>> less or more data from the bitstream doesn’t stop a decoder become a >>>>>>>> stateless decoder. >>>>>>> >>>>>>> Yes fair enough, the format in which the hardware decoder takes the >>>>>>> bitstream parameters does not make it stateless or stateful per-se. >>>>>>> It's just that stateless decoders should have no particular reason for >>>>>>> parsing the bitstream on their own since the hardware can be designed >>>>>>> with registers for each relevant bitstream element to configure the >>>>>>> decoding pipeline. That's how GPU-based decoder implementations are >>>>>>> implemented (VAAPI/VDPAU/NVDEC, etc). >>>>>>> >>>>>>> So the format we have agreed on so far for the stateless interface is >>>>>>> to pass parsed elements via v4l2 control structures. >>>>>>> >>>>>>> If the hardware can only work by parsing the bitstream its
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Sent from my iPad > On Jan 30, 2019, at 5:41 AM, Nicolas Dufresne wrote: > >> Le mardi 29 janvier 2019 à 16:44 +0900, Alexandre Courbot a écrit : >> On Fri, Jan 25, 2019 at 10:04 PM Paul Kocialkowski >> wrote: >>> Hi, >>> >>>> On Thu, 2019-01-24 at 20:23 +0800, Ayaka wrote: >>>> Sent from my iPad >>>> >>>>> On Jan 24, 2019, at 6:27 PM, Paul Kocialkowski >>>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>>> On Thu, 2019-01-10 at 21:32 +0800, ayaka wrote: >>>>>> I forget a important thing, for the rkvdec and rk hevc decoder, it would >>>>>> requests cabac table, scaling list, picture parameter set and reference >>>>>> picture storing in one or various of DMA buffers. I am not talking about >>>>>> the data been parsed, the decoder would requests a raw data. >>>>>> >>>>>> For the pps and rps, it is possible to reuse the slice header, just let >>>>>> the decoder know the offset from the bitstream bufer, I would suggest to >>>>>> add three properties(with sps) for them. But I think we need a method to >>>>>> mark a OUTPUT side buffer for those aux data. >>>>> >>>>> I'm quite confused about the hardware implementation then. From what >>>>> you're saying, it seems that it takes the raw bitstream elements rather >>>>> than parsed elements. Is it really a stateless implementation? >>>>> >>>>> The stateless implementation was designed with the idea that only the >>>>> raw slice data should be passed in bitstream form to the decoder. For >>>>> H.264, it seems that some decoders also need the slice header in raw >>>>> bitstream form (because they take the full slice NAL unit), see the >>>>> discussions in this thread: >>>>> media: docs-rst: Document m2m stateless video decoder interface >>>> >>>> Stateless just mean it won’t track the previous result, but I don’t >>>> think you can define what a date the hardware would need. Even you >>>> just build a dpb for the decoder, it is still stateless, but parsing >>>> less or more data from the bitstream doesn’t stop a decoder become a >>>> stateless decoder. >>> >>> Yes fair enough, the format in which the hardware decoder takes the >>> bitstream parameters does not make it stateless or stateful per-se. >>> It's just that stateless decoders should have no particular reason for >>> parsing the bitstream on their own since the hardware can be designed >>> with registers for each relevant bitstream element to configure the >>> decoding pipeline. That's how GPU-based decoder implementations are >>> implemented (VAAPI/VDPAU/NVDEC, etc). >>> >>> So the format we have agreed on so far for the stateless interface is >>> to pass parsed elements via v4l2 control structures. >>> >>> If the hardware can only work by parsing the bitstream itself, I'm not >>> sure what the best solution would be. Reconstructing the bitstream in >>> the kernel is a pretty bad option, but so is parsing in the kernel or >>> having the data both in parsed and raw forms. Do you see another >>> possibility? >> >> Is reconstructing the bitstream so bad? The v4l2 controls provide a >> generic interface to an encoded format which the driver needs to >> convert into a sequence that the hardware can understand. Typically >> this is done by populating hardware-specific structures. Can't we >> consider that in this specific instance, the hardware-specific >> structure just happens to be identical to the original bitstream >> format? > > At maximum allowed bitrate for let's say HEVC (940MB/s iirc), yes, it Lucky, most of hardware won’t be able to processing such a big buffer. General speaking, the register is 24bits for stream length in bytes. > would be really really bad. In GStreamer project we have discussed for > a while (but have never done anything about) adding the ability through > a bitmask to select which part of the stream need to be parsed, as > parsing itself was causing some overhead. Maybe similar thing applies, > though as per our new design, it's the fourcc that dictate the driver > behaviour, we'd need yet another fourcc for drivers that wants the full > bitstream (which seems odd if you have already parsed everything, I > think this need some clarification). > >> >> I agree that this is not strictly optimal for that particular >> hardware, but such is the cost of abstractions, and in this specific >> case I don't believe the cost would be particularly high?
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Sent from my iPad > On Jan 30, 2019, at 11:35 AM, Tomasz Figa wrote: > > On Wed, Jan 30, 2019 at 11:29 AM Alexandre Courbot > wrote: >> >>> On Wed, Jan 30, 2019 at 6:41 AM Nicolas Dufresne >>> wrote: >>> >>>> Le mardi 29 janvier 2019 à 16:44 +0900, Alexandre Courbot a écrit : >>>> On Fri, Jan 25, 2019 at 10:04 PM Paul Kocialkowski >>>> wrote: >>>>> Hi, >>>>> >>>>>> On Thu, 2019-01-24 at 20:23 +0800, Ayaka wrote: >>>>>> Sent from my iPad >>>>>> >>>>>>> On Jan 24, 2019, at 6:27 PM, Paul Kocialkowski >>>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>>> On Thu, 2019-01-10 at 21:32 +0800, ayaka wrote: >>>>>>>> I forget a important thing, for the rkvdec and rk hevc decoder, it >>>>>>>> would >>>>>>>> requests cabac table, scaling list, picture parameter set and reference >>>>>>>> picture storing in one or various of DMA buffers. I am not talking >>>>>>>> about >>>>>>>> the data been parsed, the decoder would requests a raw data. >>>>>>>> >>>>>>>> For the pps and rps, it is possible to reuse the slice header, just let >>>>>>>> the decoder know the offset from the bitstream bufer, I would suggest >>>>>>>> to >>>>>>>> add three properties(with sps) for them. But I think we need a method >>>>>>>> to >>>>>>>> mark a OUTPUT side buffer for those aux data. >>>>>>> >>>>>>> I'm quite confused about the hardware implementation then. From what >>>>>>> you're saying, it seems that it takes the raw bitstream elements rather >>>>>>> than parsed elements. Is it really a stateless implementation? >>>>>>> >>>>>>> The stateless implementation was designed with the idea that only the >>>>>>> raw slice data should be passed in bitstream form to the decoder. For >>>>>>> H.264, it seems that some decoders also need the slice header in raw >>>>>>> bitstream form (because they take the full slice NAL unit), see the >>>>>>> discussions in this thread: >>>>>>> media: docs-rst: Document m2m stateless video decoder interface >>>>>> >>>>>> Stateless just mean it won’t track the previous result, but I don’t >>>>>> think you can define what a date the hardware would need. Even you >>>>>> just build a dpb for the decoder, it is still stateless, but parsing >>>>>> less or more data from the bitstream doesn’t stop a decoder become a >>>>>> stateless decoder. >>>>> >>>>> Yes fair enough, the format in which the hardware decoder takes the >>>>> bitstream parameters does not make it stateless or stateful per-se. >>>>> It's just that stateless decoders should have no particular reason for >>>>> parsing the bitstream on their own since the hardware can be designed >>>>> with registers for each relevant bitstream element to configure the >>>>> decoding pipeline. That's how GPU-based decoder implementations are >>>>> implemented (VAAPI/VDPAU/NVDEC, etc). >>>>> >>>>> So the format we have agreed on so far for the stateless interface is >>>>> to pass parsed elements via v4l2 control structures. >>>>> >>>>> If the hardware can only work by parsing the bitstream itself, I'm not >>>>> sure what the best solution would be. Reconstructing the bitstream in >>>>> the kernel is a pretty bad option, but so is parsing in the kernel or >>>>> having the data both in parsed and raw forms. Do you see another >>>>> possibility? >>>> >>>> Is reconstructing the bitstream so bad? The v4l2 controls provide a >>>> generic interface to an encoded format which the driver needs to >>>> convert into a sequence that the hardware can understand. Typically >>>> this is done by populating hardware-specific structures. Can't we >>>> consider that in this specific instance, the hardware-specific >>>> structure just happens to be identical to the original bitstream
Re: [PATCH v2 1/2] media: uapi: Add H264 low-level decoder API compound controls.
Sent from my iPad
> On Jan 24, 2019, at 10:23 PM, Maxime Ripard wrote:
>
> Hi!
>
> On Sun, Jan 20, 2019 at 08:48:32PM +0800, ayaka wrote:
>>>>> +struct v4l2_ctrl_h264_scaling_matrix {
>>>>> +__u8 scaling_list_4x4[6][16];
>>>>> +__u8 scaling_list_8x8[6][64];
>>>>> +};
>>>> I wonder which decoder want this.
>>> I'm not sure I follow you, scaling lists are an important part of the
>>> decoding process, so all of them?
>>
>> Not actually, when the scaling list is in the sequence(a flag for it), we
>> need to tell the decoder a scaling table.
>
> Right, that's why the scaling list has a control of its own.
>
>> But the initial state of that table is known, so for some decoder,
>> it would have a internal table.
>
> That control is optional, so you can just ignore that setting in that
> case
>
>> And for some decoder, it wants in the Z order while the others
>> won't.
>
> We're designing a generic API here, so it doesn't matter. Some will
I know, I just wonder whether the other driver request it
> have to convert it internally in the drivers for the Z order, while
> others will be able to use it as is.
>
>>>>> +struct v4l2_ctrl_h264_slice_param {
>>>>> +/* Size in bytes, including header */
>>>>> +__u32 size;
>>>>> +/* Offset in bits to slice_data() from the beginning of this slice.
>>>>> */
>>>>> +__u32 header_bit_size;
>>>>> +
>>>>> +__u16 first_mb_in_slice;
>>>>> +__u8 slice_type;
>>>>> +__u8 pic_parameter_set_id;
>>>>> +__u8 colour_plane_id;
>>>>> +__u16 frame_num;
>>>>> +__u16 idr_pic_id;
>>>>> +__u16 pic_order_cnt_lsb;
>>>>> +__s32 delta_pic_order_cnt_bottom;
>>>>> +__s32 delta_pic_order_cnt0;
>>>>> +__s32 delta_pic_order_cnt1;
>>>>> +__u8 redundant_pic_cnt;
>>>>> +
>>>>> +struct v4l2_h264_pred_weight_table pred_weight_table;
>>>>> +/* Size in bits of dec_ref_pic_marking() syntax element. */
>>>>> +__u32 dec_ref_pic_marking_bit_size;
>>>>> +/* Size in bits of pic order count syntax. */
>>>>> +__u32 pic_order_cnt_bit_size;
>>>>> +
>>>>> +__u8 cabac_init_idc;
>>>>> +__s8 slice_qp_delta;
>>>>> +__s8 slice_qs_delta;
>>>>> +__u8 disable_deblocking_filter_idc;
>>>>> +__s8 slice_alpha_c0_offset_div2;
>>>>> +__s8 slice_beta_offset_div2;
>>>>> +__u32 slice_group_change_cycle;
>>>>> +
>>>>> +__u8 num_ref_idx_l0_active_minus1;
>>>>> +__u8 num_ref_idx_l1_active_minus1;
>>>>> +/* Entries on each list are indices
>>>>> + * into v4l2_ctrl_h264_decode_param.dpb[]. */
>>>>> +__u8 ref_pic_list0[32];
>>>>> +__u8 ref_pic_list1[32];
>>>>> +
>>>>> +__u8 flags;
>>>>> +};
>>>>> +
>>>> We need some addtional properties or the Rockchip won't work.
>>>> 1. u16 idr_pic_id for identifies IDR (instantaneous decoding refresh)
>>>> picture
>>> idr_pic_id is already there
>> Sorry for miss that.
>>>
>>>> 2. u16 ref_pic_mk_len for length of decoded reference picture marking bits
>>>> 3. u8 poc_length for length of picture order count field in stream
>>>>
>>>> The last two are used for the hardware to skip a part stream.
>>> I'm not sure what you mean here, those parameters are not in the
>>> bitstream, what do you want to use them for?
>>
>> Or Rockchip's decoder won't work. Their decoder can't find the data part
>> without skip some segments in slice data.
>>
>> I should say something more about the stateless decoder, it is hard to
>> define what a stateless decoder will do, some would like to parse more
>> information but some won't. You even have no idea on what it would
>> accelerate. OK, I should say for those ISO H serial codec, it would be more
>> simple but for those VPx serial, the decoders design is a mess.
>
> Can't you use header_bit_size in that case to skip over the the parts
> of the slice you don't care about and go to the data?
>
No, the decoder request extra size of those two segment of h.264
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Sent from my iPad
> On Jan 24, 2019, at 6:27 PM, Paul Kocialkowski
> wrote:
>
> Hi,
>
>> On Thu, 2019-01-10 at 21:32 +0800, ayaka wrote:
>> I forget a important thing, for the rkvdec and rk hevc decoder, it would
>> requests cabac table, scaling list, picture parameter set and reference
>> picture storing in one or various of DMA buffers. I am not talking about
>> the data been parsed, the decoder would requests a raw data.
>>
>> For the pps and rps, it is possible to reuse the slice header, just let
>> the decoder know the offset from the bitstream bufer, I would suggest to
>> add three properties(with sps) for them. But I think we need a method to
>> mark a OUTPUT side buffer for those aux data.
>
> I'm quite confused about the hardware implementation then. From what
> you're saying, it seems that it takes the raw bitstream elements rather
> than parsed elements. Is it really a stateless implementation?
>
> The stateless implementation was designed with the idea that only the
> raw slice data should be passed in bitstream form to the decoder. For
> H.264, it seems that some decoders also need the slice header in raw
> bitstream form (because they take the full slice NAL unit), see the
> discussions in this thread:
> media: docs-rst: Document m2m stateless video decoder interface
Stateless just mean it won’t track the previous result, but I don’t think you
can define what a date the hardware would need. Even you just build a dpb for
the decoder, it is still stateless, but parsing less or more data from the
bitstream doesn’t stop a decoder become a stateless decoder.
>
> Can you detail exactly what the rockchip decoder absolutely needs in
> raw bitstream format?
>
> Cheers,
>
> Paul
>
>>> On 1/8/19 6:00 PM, Ayaka wrote:
>>> Sent from my iPad
>>>
>>>> On Jan 8, 2019, at 4:38 PM, Paul Kocialkowski
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>> On Tue, 2019-01-08 at 09:16 +0800, Ayaka wrote:
>>>>>
>>>>> Sent from my iPad
>>>>>
>>>>>> On Jan 7, 2019, at 5:57 PM, Paul Kocialkowski
>>>>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>>> On Mon, 2019-01-07 at 11:49 +0800, Randy Li wrote:
>>>>>>>> On 12/12/18 8:51 PM, Paul Kocialkowski wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On Wed, 2018-12-05 at 21:59 +0100, Jernej Škrabec wrote:
>>>>>>>>
>>>>>>>>>> +
>>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_BEFORE0x01
>>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_AFTER0x02
>>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR0x03
>>>>>>>>>> +
>>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX16
>>>>>>>>>> +
>>>>>>>>>> +struct v4l2_hevc_dpb_entry {
>>>>>>>>>> +__u32buffer_tag;
>>>>>>>>>> +__u8rps;
>>>>>>>>>> +__u8field_pic;
>>>>>>>>>> +__u16pic_order_cnt[2];
>>>>>>>>>> +};
>>>>>>> Please add a property for reference index, if that rps is not used for
>>>>>>> this, some device would request that(not the rockchip one). And
>>>>>>> Rockchip's VDPU1 and VDPU2 for AVC would request a similar property.
>>>>>> What exactly is that reference index? Is it a bitstream element or
>>>>>> something deduced from the bitstream?
>>>>>>
>>>>> picture order count(POC) for HEVC and frame_num in AVC. I think it is
>>>>> the number used in list0(P slice and B slice) and list1(B slice).
>>>> The picture order count is already the last field of the DPB entry
>>>> structure. There is one for each field picture.
>>> As we are not sure whether there is a field coded slice or CTU, I would
>>> hold this part and else about the field.
>>>>>>> Adding another buffer_tag for referring the memory of the motion vectors
>>>>>>> for each frames. Or a better method is add a meta data to echo picture
>>>>>>> buffer, since the picture output is just the same as the original,
>>>>>>>
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Sent from my iPad
> On Jan 24, 2019, at 6:36 PM, Paul Kocialkowski
> wrote:
>
> Hi,
>
>> On Tue, 2019-01-08 at 18:00 +0800, Ayaka wrote:
>>
>> Sent from my iPad
>>
>>> On Jan 8, 2019, at 4:38 PM, Paul Kocialkowski
>>> wrote:
>>>
>>> Hi,
>>>
>>>> On Tue, 2019-01-08 at 09:16 +0800, Ayaka wrote:
>>>>
>>>> Sent from my iPad
>>>>
>>>>> On Jan 7, 2019, at 5:57 PM, Paul Kocialkowski
>>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>>> On Mon, 2019-01-07 at 11:49 +0800, Randy Li wrote:
>>>>>>> On 12/12/18 8:51 PM, Paul Kocialkowski wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Wed, 2018-12-05 at 21:59 +0100, Jernej Škrabec wrote:
>>>>>>>
>>>>>>>>> +
>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_BEFORE0x01
>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_AFTER0x02
>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR0x03
>>>>>>>>> +
>>>>>>>>> +#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX16
>>>>>>>>> +
>>>>>>>>> +struct v4l2_hevc_dpb_entry {
>>>>>>>>> +__u32buffer_tag;
>>>>>>>>> +__u8rps;
>>>>>>>>> +__u8field_pic;
>>>>>>>>> +__u16pic_order_cnt[2];
>>>>>>>>> +};
>>>>>>
>>>>>> Please add a property for reference index, if that rps is not used for
>>>>>> this, some device would request that(not the rockchip one). And
>>>>>> Rockchip's VDPU1 and VDPU2 for AVC would request a similar property.
>>>>>
>>>>> What exactly is that reference index? Is it a bitstream element or
>>>>> something deduced from the bitstream?
>>>>>
>>>> picture order count(POC) for HEVC and frame_num in AVC. I think it is
>>>> the number used in list0(P slice and B slice) and list1(B slice).
>>>
>>> The picture order count is already the last field of the DPB entry
>>> structure. There is one for each field picture.
>> As we are not sure whether there is a field coded slice or CTU, I
>> would hold this part and else about the field.
>
> I'm not sure what you meant here, sorry.
As we talked in IRC, I am not sure the field coded picture is supported in HEVC.
And I don’t why there would be two pic order cnt, a picture can only be used a
short term or a long term reference at one picture decoding
>
>>>>>> Adding another buffer_tag for referring the memory of the motion vectors
>>>>>> for each frames. Or a better method is add a meta data to echo picture
>>>>>> buffer, since the picture output is just the same as the original,
>>>>>> display won't care whether the motion vectors are written the button of
>>>>>> picture or somewhere else.
>>>>>
>>>>> The motion vectors are passed as part of the raw bitstream data, in the
>>>>> slices. Is there a case where the motion vectors are coded differently?
>>>> No, it is an additional cache for decoder, even FFmpeg having such
>>>> data, I think allwinner must output it into somewhere.
>>>
>>> Ah yes I see what you mean! This is handled internally by our driver
>>> and not exposed to userspace. I don't think it would be a good idea to
>>> expose this cache or request that userspace allocates it like a video
>>> buffer.
>>>
>> No, usually the driver should allocate, as the user space have no
>> idea on size of each devices.
>> But for advantage user, application can fix a broken picture with a
>> proper data or analysis a object motion from that.
>> So I would suggest attaching this information to a picture buffer as
>> a meta data.
>
> Right, the driver will allocate chunks of memory for the decoding
> metadata used by the hardware decoder.
>
> Well, I don't think V4L2 has any mechanism to expose this data for now
> and since it's very specific to the hardware implementation, I guess
> the interest in having that is generally pretty low.
>
> That's maybe something that could be added later if someone wants to
> work on it, but I think we are better off ke
Re: [PATCH v2 1/2] media: uapi: Add H264 low-level decoder API compound controls.
I am sorry I am a little busy for the lunar new year recently and the
H.264 syntax rules are little complex, I will try explain my ideas more
clear here.
On 1/17/19 7:01 PM, Maxime Ripard wrote:
Hi,
On Tue, Jan 08, 2019 at 05:52:28PM +0800, Randy 'ayaka' Li wrote:
+struct v4l2_ctrl_h264_scaling_matrix {
+ __u8 scaling_list_4x4[6][16];
+ __u8 scaling_list_8x8[6][64];
+};
I wonder which decoder want this.
I'm not sure I follow you, scaling lists are an important part of the
decoding process, so all of them?
Not actually, when the scaling list is in the sequence(a flag for it),
we need to tell the decoder a scaling table. But the initial state of
that table is known, so for some decoder, it would have a internal
table. And for some decoder, it wants in the Z order while the others won't.
+struct v4l2_ctrl_h264_slice_param {
+ /* Size in bytes, including header */
+ __u32 size;
+ /* Offset in bits to slice_data() from the beginning of this slice. */
+ __u32 header_bit_size;
+
+ __u16 first_mb_in_slice;
+ __u8 slice_type;
+ __u8 pic_parameter_set_id;
+ __u8 colour_plane_id;
+ __u16 frame_num;
+ __u16 idr_pic_id;
+ __u16 pic_order_cnt_lsb;
+ __s32 delta_pic_order_cnt_bottom;
+ __s32 delta_pic_order_cnt0;
+ __s32 delta_pic_order_cnt1;
+ __u8 redundant_pic_cnt;
+
+ struct v4l2_h264_pred_weight_table pred_weight_table;
+ /* Size in bits of dec_ref_pic_marking() syntax element. */
+ __u32 dec_ref_pic_marking_bit_size;
+ /* Size in bits of pic order count syntax. */
+ __u32 pic_order_cnt_bit_size;
+
+ __u8 cabac_init_idc;
+ __s8 slice_qp_delta;
+ __s8 slice_qs_delta;
+ __u8 disable_deblocking_filter_idc;
+ __s8 slice_alpha_c0_offset_div2;
+ __s8 slice_beta_offset_div2;
+ __u32 slice_group_change_cycle;
+
+ __u8 num_ref_idx_l0_active_minus1;
+ __u8 num_ref_idx_l1_active_minus1;
+ /* Entries on each list are indices
+* into v4l2_ctrl_h264_decode_param.dpb[]. */
+ __u8 ref_pic_list0[32];
+ __u8 ref_pic_list1[32];
+
+ __u8 flags;
+};
+
We need some addtional properties or the Rockchip won't work.
1. u16 idr_pic_id for identifies IDR (instantaneous decoding refresh)
picture
idr_pic_id is already there
Sorry for miss that.
2. u16 ref_pic_mk_len for length of decoded reference picture marking bits
3. u8 poc_length for length of picture order count field in stream
The last two are used for the hardware to skip a part stream.
I'm not sure what you mean here, those parameters are not in the
bitstream, what do you want to use them for?
Or Rockchip's decoder won't work. Their decoder can't find the data part
without skip some segments in slice data.
I should say something more about the stateless decoder, it is hard to
define what a stateless decoder will do, some would like to parse more
information but some won't. You even have no idea on what it would
accelerate. OK, I should say for those ISO H serial codec, it would be
more simple but for those VPx serial, the decoders design is a mess.
+#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01
+#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE0x02
+#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x04
+
+struct v4l2_h264_dpb_entry {
+ __u32 tag;
+ __u16 frame_num;
+ __u16 pic_num;
Although the long term reference would use picture order count
and short term for frame num, but only one of them is used
for a entry of a dpb.
Besides, for a frame picture frame_num = pic_num * 2,
and frame_num = pic_num * 2 + 1 for a filed.
I'm not sure what is your point?
I found I was wrong at the last email.
But stateless hardware decoder usually don't care about whether it is long
term or short term, as the real dpb updating or management work are not done
by the the driver or device and decoding job would only use the two list(or
one list for slice P) for reference pictures. So those flag for long term or
status can be removed as well.
And I agree above with my last mail, so I would suggest to keep a
property as index for both frame_num and pic_num, as only one of them
would be used for a picture decoding once time.
+ /* Note that field is indicated by v4l2_buffer.field */
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
+};
+
+struct v4l2_ctrl_h264_decode_param {
+ __u32 num_slices;
+ __u8 idr_pic_flag;
+ __u8 nal_ref_idc;
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 ref_pic_list_p0[32];
+ __u8 ref_pic_list_b0[32];
+ __u8 ref_pic_list_b1[32];
I would prefer to keep only two list, list0 and list 1.
I'm not even sure why this is needed in the first place anymore. It's
not part of the bit
Re: [PATCH v2 1/2] media: uapi: Add H264 low-level decoder API compound controls.
I forget a important thing, for the rkvdec and rk hevc decoder, it would
requests cabac table, scaling list, picture parameter set and reference
picture storing in one or various of DMA buffers. I am not talking about
the data been parsed, the decoder would requests a raw data.
For the pps and rps, it is possible to reuse the slice header, just let
the decoder know the offset from the bitstream bufer, I would suggest to
add three properties(with sps) for them. But I think we need a method to
mark a OUTPUT side buffer for those aux data.
On 1/9/19 1:01 AM, ayaka wrote:
On 1/8/19 5:52 PM, Randy 'ayaka' Li wrote:
On Thu, Nov 15, 2018 at 03:56:49PM +0100, Maxime Ripard wrote:
From: Pawel Osciak
Stateless video codecs will require both the H264 metadata and
slices in
order to be able to decode frames.
This introduces the definitions for a new pixel format for H264
slices that
have been parsed, as well as the structures used to pass the
metadata from
the userspace to the kernel.
Co-Developed-by: Maxime Ripard
Signed-off-by: Pawel Osciak
Signed-off-by: Guenter Roeck
Signed-off-by: Maxime Ripard
---
Documentation/media/uapi/v4l/biblio.rst | 9 +
.../media/uapi/v4l/extended-controls.rst | 364
++
.../media/uapi/v4l/pixfmt-compressed.rst | 20 +
.../media/uapi/v4l/vidioc-queryctrl.rst | 30 ++
.../media/videodev2.h.rst.exceptions | 5 +
drivers/media/v4l2-core/v4l2-ctrls.c | 42 ++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h | 10 +
include/uapi/linux/v4l2-controls.h | 166
include/uapi/linux/videodev2.h | 11 +
10 files changed, 658 insertions(+)
+#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01
+#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02
+#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x04
+
+struct v4l2_h264_dpb_entry {
+ __u32 tag;
+ __u16 frame_num;
+ __u16 pic_num;
Although the long term reference would use picture order count
and short term for frame num, but only one of them is used
for a entry of a dpb.
Besides, for a frame picture frame_num = pic_num * 2,
and frame_num = pic_num * 2 + 1 for a filed.
I mistook something before and something Herman told me is wrong, I
read the book explaining the ITU standard.
The index of a short term reference picture would be frame_num or POC
and LongTermPicNum for long term.
But stateless hardware decoder usually don't care about whether it is
long term or short term, as the real dpb updating or management work
are not done by the the driver or device and decoding job would only
use the two list(or one list for slice P) for reference pictures. So
those flag for long term or status can be removed as well.
Stateless decoder would care about just reference index of this
picture and maybe some extra property for the filed coded below.
Keeping a property here for the index of a picture is enough.
+ /* Note that field is indicated by v4l2_buffer.field */
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
+};
+
+struct v4l2_ctrl_h264_decode_param {
+ __u32 num_slices;
+ __u8 idr_pic_flag;
+ __u8 nal_ref_idc;
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 ref_pic_list_p0[32];
+ __u8 ref_pic_list_b0[32];
+ __u8 ref_pic_list_b1[32];
I would prefer to keep only two list, list0 and list 1.
Anyway P slice just use the list0 and B would use the both.
+ struct v4l2_h264_dpb_entry dpb[16];
+};
+
#endif
diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
index 173a94d2cbef..dd028e0bf306 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -643,6 +643,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
H264 with start codes */
#define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
H264 without start codes */
#define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
H264 MVC */
+#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
H264 parsed slices */
#define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /*
H263 */
#define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /*
MPEG-1 ES */
#define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /*
MPEG-2 ES */
@@ -1631,6 +1632,11 @@ struct v4l2_ext_control {
__u32 __user *p_u32;
struct v4l2_ctrl_mpeg2_slice_params __user
*p_mpeg2_slice_params;
struct v4l2_ctrl_mpeg2_q
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
I forget a important thing, for the rkvdec and rk hevc decoder, it would
requests cabac table, scaling list, picture parameter set and reference
picture storing in one or various of DMA buffers. I am not talking about
the data been parsed, the decoder would requests a raw data.
For the pps and rps, it is possible to reuse the slice header, just let
the decoder know the offset from the bitstream bufer, I would suggest to
add three properties(with sps) for them. But I think we need a method to
mark a OUTPUT side buffer for those aux data.
On 1/8/19 6:00 PM, Ayaka wrote:
Sent from my iPad
On Jan 8, 2019, at 4:38 PM, Paul Kocialkowski
wrote:
Hi,
On Tue, 2019-01-08 at 09:16 +0800, Ayaka wrote:
Sent from my iPad
On Jan 7, 2019, at 5:57 PM, Paul Kocialkowski
wrote:
Hi,
On Mon, 2019-01-07 at 11:49 +0800, Randy Li wrote:
On 12/12/18 8:51 PM, Paul Kocialkowski wrote:
Hi,
On Wed, 2018-12-05 at 21:59 +0100, Jernej Škrabec wrote:
+
+#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_BEFORE0x01
+#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_AFTER0x02
+#define V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR0x03
+
+#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX16
+
+struct v4l2_hevc_dpb_entry {
+__u32buffer_tag;
+__u8rps;
+__u8field_pic;
+__u16pic_order_cnt[2];
+};
Please add a property for reference index, if that rps is not used for
this, some device would request that(not the rockchip one). And
Rockchip's VDPU1 and VDPU2 for AVC would request a similar property.
What exactly is that reference index? Is it a bitstream element or
something deduced from the bitstream?
picture order count(POC) for HEVC and frame_num in AVC. I think it is
the number used in list0(P slice and B slice) and list1(B slice).
The picture order count is already the last field of the DPB entry
structure. There is one for each field picture.
As we are not sure whether there is a field coded slice or CTU, I would hold
this part and else about the field.
Adding another buffer_tag for referring the memory of the motion vectors
for each frames. Or a better method is add a meta data to echo picture
buffer, since the picture output is just the same as the original,
display won't care whether the motion vectors are written the button of
picture or somewhere else.
The motion vectors are passed as part of the raw bitstream data, in the
slices. Is there a case where the motion vectors are coded differently?
No, it is an additional cache for decoder, even FFmpeg having such
data, I think allwinner must output it into somewhere.
Ah yes I see what you mean! This is handled internally by our driver
and not exposed to userspace. I don't think it would be a good idea to
expose this cache or request that userspace allocates it like a video
buffer.
No, usually the driver should allocate, as the user space have no idea on size
of each devices.
But for advantage user, application can fix a broken picture with a proper data
or analysis a object motion from that.
So I would suggest attaching this information to a picture buffer as a meta
data.
+
+struct v4l2_hevc_pred_weight_table {
+__u8luma_log2_weight_denom;
+__s8delta_chroma_log2_weight_denom;
+
+__s8delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+__s8luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+__s8delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+__s8chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+
+__s8delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+__s8luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+__s8delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+__s8chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+};
+
Those properties I think are not necessary are applying for the
Rockchip's device, may not work for the others.
Yes, it's possible that some of the elements are not necessary for some
decoders. What we want is to cover all the elements that might be
required for a decoder.
I wonder whether allwinner need that, those sao flag usually ignored
by decoder in design. But more is better than less, it is hard to
extend a v4l2 structure in the future, maybe a new HEVC profile
would bring a new property, it is still too early for HEVC.
Yes this is used by our decoder. The idea is to have all the basic
bitstream elements in the structures (even if some decoders don't use
them all) and add others for extension as separate controls later.
+struct v4l2_ctrl_hevc_slice_params {
+__u32bit_size;
+__u32data_bit_offset;
+
+/* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
+__u8nal_unit_type;
+__u8nuh_temporal_id_plus1;
+
+/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
+__u8slice_type;
+__u8colour_plane_id;
+__u16slice_pic_order_cnt;
+
Re: P010 fourcc format support - Was: Re: Kernel error "Unknown pixelformat 0x00000000" occurs when I start capture video
Sent from my iPad
> On Jan 9, 2019, at 9:01 PM, Mauro Carvalho Chehab
> wrote:
>
> Em Wed, 9 Jan 2019 14:19:00 +0200
> Sakari Ailus escreveu:
>
>>> On Tue, Jan 08, 2019 at 04:49:16PM -0200, Mauro Carvalho Chehab wrote:
>>> Em Tue, 8 Jan 2019 21:11:41 +0500
>>> Mikhail Gavrilov escreveu:
>>>
>>>> On Tue, 8 Jan 2019 at 20:57, Laurent Pinchart
>>>> wrote:
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Your device exposes five formats: YUY2 (YUYV), YV12 (YVU420), NV12, P010
>>>>> and
>>>>> BGR3 (BGR24). They are all supported by V4L2 and the uvcvideo driver
>>>>> except
>>>>> for the P010 format. This would be easy to fix in the uvcvideo driver if
>>>>> it
>>>>> wasn't for the fact that the P010 format isn't support by V4L2. Adding
>>>>> support
>>>>> for it isn't difficult, but I don't have time to do this myself at the
>>>>> moment.
>>>>> Would you consider volunteering if I guide you ? :-)
>>>>>
>>>>
>>>> Sure, I'd be happy to help. What is required of me?
>>>
>>> It shouldn't be hard.
>>>
>>> First, you need to add the new format at include/uapi/linux/videodev2.h,
>>> like this one:
>>>
>>>#define V4L2_PIX_FMT_YUV420M v4l2_fourcc('Y', 'M', '1', '2') /* 12
>>> YUV420 planar */
>>>
>>> Please put it together with the other YUV formats.
>>>
>>> As the fourcc "P010" was not used on Linux yet, you could use it,
>>> e. g., something like:
>>>
>>>#define V4L2_PIX_FMT_YUV_P10 v4l2_fourcc('P', '0', '1', '0') /* 10
>>> YUV420 planar */
>>>
>>> You need then to document it. Each V4L2 format should have a description,
>>> like this:
>>>
>>>
>>> https://linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-yuv420m.html
>>>
>>> This is generated via a text file (using ReST syntax). On the above example,
>>> it is produced by this file:
>>>
>>>
>>> https://git.linuxtv.org/media_tree.git/tree/Documentation/media/uapi/v4l/pixfmt-yuv420m.rst
>>>
>>> Writing it would take a little more time, but, provided that you don't
>>> copy what's written from external docs, you could take a look at the
>>> Internet for the P010 descriptions, and use the pixfmt-yuy420m.rst file
>>> as the basis for a new pixfmt-p010.rst file.
>>
>> There is some work done on this but it's not finished; searching "P010" in
>> Patchwork yields this:
>>
>> https://patchwork.linuxtv.org/patch/39752/>
>
> Good point! I'm c/c the author of it.
>
> The actual patch for media is this one:
>
>https://patchwork.linuxtv.org/patch/39753/
>
> It sounds that the author didn't sent any version after that.
>
> The goal seemed to be to add P010 support at DRM for the rockchip driver.
>
> specifically with regards to patch 2/3, the issues seemed to be:
>
>- some naming issues with the multiplane format variants;
>- a typo: simliar -> similar;
>- a comment about the usage of 1/2 UTF code (½). Not sure if
> Sphinx will handle it well for both html and pdf outputs.
> It should, but better to double check.
>
> Ayaka,
>
> There is a UVC media device that supports P010 device. We're discussing
> about adding support for it on media. The full thread is at:
>
> https://lore.kernel.org/linux-media/20190109121900.hbrpttmxz3gao...@valkosipuli.retiisi.org.uk/T/#m8c395156ca0e898e4c8b1e2c6309d912bc414804
>
> We've seen that you tried to submit a patch series for DRM adding
> support for it at the rockship driver. What's the status of such
> series?
Rockchip would use another 10bit variant of NV12, which is merged as NV12LE40
at Gstreamer and I sent another patch for it, but I didn’t receive any feedback
from that.
>
> Thanks,
> Mauro
Re: [PATCH v2 1/2] media: uapi: Add H264 low-level decoder API compound controls.
On 1/8/19 5:52 PM, Randy 'ayaka' Li wrote:
On Thu, Nov 15, 2018 at 03:56:49PM +0100, Maxime Ripard wrote:
From: Pawel Osciak
Stateless video codecs will require both the H264 metadata and slices in
order to be able to decode frames.
This introduces the definitions for a new pixel format for H264 slices that
have been parsed, as well as the structures used to pass the metadata from
the userspace to the kernel.
Co-Developed-by: Maxime Ripard
Signed-off-by: Pawel Osciak
Signed-off-by: Guenter Roeck
Signed-off-by: Maxime Ripard
---
Documentation/media/uapi/v4l/biblio.rst | 9 +
.../media/uapi/v4l/extended-controls.rst | 364 ++
.../media/uapi/v4l/pixfmt-compressed.rst | 20 +
.../media/uapi/v4l/vidioc-queryctrl.rst | 30 ++
.../media/videodev2.h.rst.exceptions | 5 +
drivers/media/v4l2-core/v4l2-ctrls.c | 42 ++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h| 10 +
include/uapi/linux/v4l2-controls.h| 166
include/uapi/linux/videodev2.h| 11 +
10 files changed, 658 insertions(+)
+#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01
+#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE0x02
+#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x04
+
+struct v4l2_h264_dpb_entry {
+ __u32 tag;
+ __u16 frame_num;
+ __u16 pic_num;
Although the long term reference would use picture order count
and short term for frame num, but only one of them is used
for a entry of a dpb.
Besides, for a frame picture frame_num = pic_num * 2,
and frame_num = pic_num * 2 + 1 for a filed.
I mistook something before and something Herman told me is wrong, I read
the book explaining the ITU standard.
The index of a short term reference picture would be frame_num or POC
and LongTermPicNum for long term.
But stateless hardware decoder usually don't care about whether it is
long term or short term, as the real dpb updating or management work are
not done by the the driver or device and decoding job would only use the
two list(or one list for slice P) for reference pictures. So those flag
for long term or status can be removed as well.
Stateless decoder would care about just reference index of this picture
and maybe some extra property for the filed coded below. Keeping a
property here for the index of a picture is enough.
+ /* Note that field is indicated by v4l2_buffer.field */
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
+};
+
+struct v4l2_ctrl_h264_decode_param {
+ __u32 num_slices;
+ __u8 idr_pic_flag;
+ __u8 nal_ref_idc;
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 ref_pic_list_p0[32];
+ __u8 ref_pic_list_b0[32];
+ __u8 ref_pic_list_b1[32];
I would prefer to keep only two list, list0 and list 1.
Anyway P slice just use the list0 and B would use the both.
+ struct v4l2_h264_dpb_entry dpb[16];
+};
+
#endif
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 173a94d2cbef..dd028e0bf306 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -643,6 +643,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with
start codes */
#define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264
without start codes */
#define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
+#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed
slices */
#define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263
*/
#define V4L2_PIX_FMT_MPEG1v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES
*/
#define V4L2_PIX_FMT_MPEG2v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES
*/
@@ -1631,6 +1632,11 @@ struct v4l2_ext_control {
__u32 __user *p_u32;
struct v4l2_ctrl_mpeg2_slice_params __user
*p_mpeg2_slice_params;
struct v4l2_ctrl_mpeg2_quantization __user
*p_mpeg2_quantization;
+ struct v4l2_ctrl_h264_sps __user *p_h264_sps;
+ struct v4l2_ctrl_h264_pps __user *p_h264_pps;
+ struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
+ struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
+ struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
void __user *ptr;
};
} __attribute__ ((packed));
@@ -1678,6 +1684,11 @@ enum v4l2_ctrl_type {
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Sent from my iPad
> On Jan 8, 2019, at 4:38 PM, Paul Kocialkowski
> wrote:
>
> Hi,
>
>> On Tue, 2019-01-08 at 09:16 +0800, Ayaka wrote:
>>
>> Sent from my iPad
>>
>>> On Jan 7, 2019, at 5:57 PM, Paul Kocialkowski
>>> wrote:
>>>
>>> Hi,
>>>
>>>>> On Mon, 2019-01-07 at 11:49 +0800, Randy Li wrote:
>>>>> On 12/12/18 8:51 PM, Paul Kocialkowski wrote:
>>>>> Hi,
>>>>>
>>>>> On Wed, 2018-12-05 at 21:59 +0100, Jernej Škrabec wrote:
>>>>>
>>>>>>> +
>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_BEFORE0x01
>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_ST_CURR_AFTER0x02
>>>>>>> +#define V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR0x03
>>>>>>> +
>>>>>>> +#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX16
>>>>>>> +
>>>>>>> +struct v4l2_hevc_dpb_entry {
>>>>>>> +__u32buffer_tag;
>>>>>>> +__u8rps;
>>>>>>> +__u8field_pic;
>>>>>>> +__u16pic_order_cnt[2];
>>>>>>> +};
>>>>
>>>> Please add a property for reference index, if that rps is not used for
>>>> this, some device would request that(not the rockchip one). And
>>>> Rockchip's VDPU1 and VDPU2 for AVC would request a similar property.
>>>
>>> What exactly is that reference index? Is it a bitstream element or
>>> something deduced from the bitstream?
>>>
>> picture order count(POC) for HEVC and frame_num in AVC. I think it is
>> the number used in list0(P slice and B slice) and list1(B slice).
>
> The picture order count is already the last field of the DPB entry
> structure. There is one for each field picture.
As we are not sure whether there is a field coded slice or CTU, I would hold
this part and else about the field.
>
>>>> Adding another buffer_tag for referring the memory of the motion vectors
>>>> for each frames. Or a better method is add a meta data to echo picture
>>>> buffer, since the picture output is just the same as the original,
>>>> display won't care whether the motion vectors are written the button of
>>>> picture or somewhere else.
>>>
>>> The motion vectors are passed as part of the raw bitstream data, in the
>>> slices. Is there a case where the motion vectors are coded differently?
>> No, it is an additional cache for decoder, even FFmpeg having such
>> data, I think allwinner must output it into somewhere.
>
> Ah yes I see what you mean! This is handled internally by our driver
> and not exposed to userspace. I don't think it would be a good idea to
> expose this cache or request that userspace allocates it like a video
> buffer.
>
No, usually the driver should allocate, as the user space have no idea on size
of each devices.
But for advantage user, application can fix a broken picture with a proper data
or analysis a object motion from that.
So I would suggest attaching this information to a picture buffer as a meta
data.
>>>>>>> +
>>>>>>> +struct v4l2_hevc_pred_weight_table {
>>>>>>> +__u8luma_log2_weight_denom;
>>>>>>> +__s8delta_chroma_log2_weight_denom;
>>>>>>> +
>>>>>>> +__s8delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>>>>> +__s8luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>>>>> +__s8delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>>>>>>> +__s8chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>>>>>>> +
>>>>>>> +__s8delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>>>>> +__s8luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>>>>> +__s8delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>>>>>>> +__s8chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>>>>>>> +};
>>>>>>> +
>>>> Those properties I think are not necessary are applying for the
>>>> Rockchip's device, may not work for the others.
>>>
>>> Yes, it's possible that some of the elements are not necessary for some
>>> decoders. What we want is to cover all the elements that might be
>>> required for a decoder.
>> I
Re: [PATCH v2 1/2] media: uapi: Add H264 low-level decoder API compound controls.
On Thu, Nov 15, 2018 at 03:56:49PM +0100, Maxime Ripard wrote: > From: Pawel Osciak > > Stateless video codecs will require both the H264 metadata and slices in > order to be able to decode frames. > > This introduces the definitions for a new pixel format for H264 slices that > have been parsed, as well as the structures used to pass the metadata from > the userspace to the kernel. > > Co-Developed-by: Maxime Ripard > Signed-off-by: Pawel Osciak > Signed-off-by: Guenter Roeck > Signed-off-by: Maxime Ripard > --- > Documentation/media/uapi/v4l/biblio.rst | 9 + > .../media/uapi/v4l/extended-controls.rst | 364 ++ > .../media/uapi/v4l/pixfmt-compressed.rst | 20 + > .../media/uapi/v4l/vidioc-queryctrl.rst | 30 ++ > .../media/videodev2.h.rst.exceptions | 5 + > drivers/media/v4l2-core/v4l2-ctrls.c | 42 ++ > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > include/media/v4l2-ctrls.h| 10 + > include/uapi/linux/v4l2-controls.h| 166 > include/uapi/linux/videodev2.h| 11 + > 10 files changed, 658 insertions(+) > > diff --git a/Documentation/media/uapi/v4l/biblio.rst > b/Documentation/media/uapi/v4l/biblio.rst > index 386d6cf83e9c..73aeb7ce47d2 100644 > --- a/Documentation/media/uapi/v4l/biblio.rst > +++ b/Documentation/media/uapi/v4l/biblio.rst > @@ -115,6 +115,15 @@ ITU BT.1119 > > :author:International Telecommunication Union (http://www.itu.ch) > > +.. _h264: > + > +ITU H.264 > += > + > +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic > Audiovisual Services" > + > +:author:International Telecommunication Union (http://www.itu.ch) > + > .. _jfif: > > JFIF > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst > b/Documentation/media/uapi/v4l/extended-controls.rst > index 65a1d873196b..87c0d151577f 100644 > --- a/Documentation/media/uapi/v4l/extended-controls.rst > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > @@ -1674,6 +1674,370 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - > non-intra-coded frames, in zigzag scanning order. Only relevant for > non-4:2:0 YUV formats. > > +.. _v4l2-mpeg-h264: > + > +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)`` > +Specifies the sequence parameter set (as extracted from the > +bitstream) for the associated H264 slice data. This includes the > +necessary parameters for configuring a stateless hardware decoding > +pipeline for H264. The bitstream parameters are defined according > +to :ref:`h264`. Unless there's a specific comment, refer to the > +specification for the documentation of these fields. > + > +.. c:type:: v4l2_ctrl_h264_sps > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_sps > +:header-rows: 0 > +:stub-columns: 0 > +:widths: 1 1 2 > + > +* - __u8 > + - ``profile_idc`` > + - > +* - __u8 > + - ``constraint_set_flags`` > + - TODO > +* - __u8 > + - ``level_idc`` > + - > +* - __u8 > + - ``seq_parameter_set_id`` > + - > +* - __u8 > + - ``chroma_format_idc`` > + - > +* - __u8 > + - ``bit_depth_luma_minus8`` > + - > +* - __u8 > + - ``bit_depth_chroma_minus8`` > + - > +* - __u8 > + - ``log2_max_frame_num_minus4`` > + - > +* - __u8 > + - ``pic_order_cnt_type`` > + - > +* - __u8 > + - ``log2_max_pic_order_cnt_lsb_minus4`` > + - > +* - __u8 > + - ``max_num_ref_frames`` > + - > +* - __u8 > + - ``num_ref_frames_in_pic_order_cnt_cycle`` > + - > +* - __s32 > + - ``offset_for_ref_frame[255]`` > + - > +* - __s32 > + - ``offset_for_non_ref_pic`` > + - > +* - __s32 > + - ``offset_for_top_to_bottom_field`` > + - > +* - __u16 > + - ``pic_width_in_mbs_minus1`` > + - > +* - __u16 > + - ``pic_height_in_map_units_minus1`` > + - > +* - __u8 > + - ``flags`` > + - TODO > + > +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)`` > +Specifies the picture parameter set (as extracted from the > +bitstream) for the associated H264 slice data. This includes the > +necessary parameters for configuring a stateless hardware decoding > +pipeline for H264. The bitstream parameters are defined according > +to :ref:`h264`. Unless there's a specific comment, refer to the > +specification for the documentation of these fields. > + > +.. c:type:: v4l2_ctrl_h264_pps > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_pps > +:header-rows: 0 > +:stub-columns: 0 > +:widths: 1 1 2 > + > +* - __u8 > + - ``pic_parameter_set_id`` > + - > +* - __u8 > + - ``seq_parameter_set_id`` > + - > +* - __u8 > + - ``num_slice_groups_minus1`` > + - > +* - __u8 > + - ``num_
Re: [PATCH 4/4] arm64: dts: rockchip: add video codec for rk3399
Sent from my iPad > On Jan 8, 2019, at 2:33 PM, Tomasz Figa wrote: > >> On Mon, Jan 7, 2019 at 2:30 AM Ayaka wrote: >> >> Hello Ezequiel >> >> Sent from my iPad >> >>>> On Jan 7, 2019, at 1:21 AM, Ezequiel Garcia >>>> wrote: >>>> >>>> On Sun, 6 Jan 2019 at 13:16, Ayaka wrote: >>>> >>>> >>>> >>>> Sent from my iPad >>>> >>>>> On Jan 7, 2019, at 12:04 AM, Ezequiel Garcia >>>>> wrote: >>>>> >>>>> On Sun, 2019-01-06 at 23:05 +0800, Ayaka wrote: >>>>>>> On Jan 6, 2019, at 10:22 PM, Ezequiel Garcia >>>>>>> wrote: >>>>>>> >>>>>>> Hi Randy, >>>>>>> >>>>>>> Thanks a lot for this patches. They are really useful >>>>>>> to provide more insight into the VPU hardware. >>>>>>> >>>>>>> This change will make the vpu encoder and vpu decoder >>>>>>> completely independent, can they really work in parallel? >>>>>> As I said it depends on the platform, but with this patch, the user >>>>>> space would think they can work at the same time. >>>>> >>>>> >>>>> I think there is some confusion. >>>>> >>>>> The devicetree is one thing: it is a hardware representation, >>>>> a way to describe the hardware, for the kernel/bootloader to >>>>> parse. >>>>> >>>>> The userspace view will depend on the driver implementation. >>>>> >>>>> The current devicetree and driver (without your patches), >>>>> model the VPU as a single piece of hardware, exposing a decoder >>>>> and an encoder. >>>>> >>>>> The V4L driver will then create two video devices, i.e. /dev/videoX >>>>> and /dev/videoY. So userspace sees an independent view of the >>>>> devices. >>>>> >>>> I knew that, the problem is that the driver should not always create a >>>> decoder and encoder pair, they may not exist at some platforms, even some >>>> platforms doesn’t have a encoder. You may have a look on the rk3328 I post >>>> on the first email as example. >>> >>> That is correct. But that still doesn't tackle my question: is the >>> hardware able to run a decoding and an encoding job in parallel? >>> >> For rk3328, yes, you see I didn’t draw them in the same box. >>> If not, then it's wrong to describe them as independent entities. >>> >>>>> However, they are internally connected, and thus we can >>>>> easily avoid two jobs running in parallel. >>>>> >>>> That is what the mpp service did in my patches, handing the relationship >>>> between each devices. And it is not a easy work, maybe a 4k decoder would >>>> be blocked by another high frame rate encoding work or another decoder >>>> session. The vendor kernel have more worry about this, but not in this >>>> version. >>> >>> Right. That is one way to design it. Another way is having a single >>> devicetree node for the VPU encoder/decoder "complex". >> No, you can’t assume which one is in the combo group, it can be various. you >> see, in the rk3328, the vdpu is paired with an avs+ decoder. That is why I >> use a virtual device standing for scheduler. > > First of all, thanks for all the input. Having more understanding of > the hardware and shortcomings of the current V4L2 APIs is really > important to let us further evolve the API and make sure that it works > for further use cases. I replied the problems of the v4l2 request API in the other threads. I am waiting the feedback from those threads. > > As for the Device Tree itself, it doesn't always describe the hardware > in 100%. Also please note the merged device tree for the video codec won’t fix for most of the rockchip platform. > Most of the time it's just the necessary information to > choose and instantiate the right drivers and bind to the right > hardware resources. The information on which hardware instances on the > SoC can work independently can of course be described in DT (e.g. by > sub-nodes of a video-codec complex OR a set of phandles, e.g. > rockchip,shared-instances), but it's also perfectly fine to defer this > kind of knowledge to the drivers themselves. I wish there is a common mechanism for those device would share some resources. Although there is a multiple functions framework, but that is not I want. They are multiple functions but they are used at the same time not separately. > > Best regards, > Tomasz
Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
__u8collocated_ref_idx; >>>>> +__u8five_minus_max_num_merge_cand; >>>>> +__u8use_integer_mv_flag; >>>>> +__s8slice_qp_delta; >>>>> +__s8slice_cb_qp_offset; >>>>> +__s8slice_cr_qp_offset; >>>>> +__s8slice_act_y_qp_offset; >>>>> +__s8slice_act_cb_qp_offset; >>>>> +__s8slice_act_cr_qp_offset; >>>>> +__u8slice_deblocking_filter_disabled_flag; >>>>> +__s8slice_beta_offset_div2; >>>>> +__s8slice_tc_offset_div2; >>>>> +__u8slice_loop_filter_across_slices_enabled_flag; >>>>> + >>>>> +/* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */ >>>>> +__u8pic_struct; >> I think the decoder doesn't care about this, it is used for display. > > The purpose of this field is to indicate whether the current picture is > a progressive frame or an interlaced field picture, which is useful for > decoding. > > At least our decoder has a register field to indicate frame/top > field/bottom field, so we certainly need to keep the info around. > Looking at the spec and the ffmpeg implementation, it looks like this > flag of the bitstream is the usual way to report field coding. It depends whether the decoder cares about scan type or more, I wonder prefer general_interlaced_source_flag for just scan type, it would be better than reading another SEL. > > Cheers, > > Paul Randy “ayaka” LI > >>>>> + >>>>> +/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */ >>>>> +struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]; >>>>> +__u8num_active_dpb_entries; >>>>> +__u8ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]; >>>>> +__u8ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]; >>>>> + >>>>> +__u8num_rps_poc_st_curr_before; >>>>> +__u8num_rps_poc_st_curr_after; >>>>> +__u8num_rps_poc_lt_curr; >>>>> + >>>>> +/* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter >>>>> */ >>>>> +struct v4l2_hevc_pred_weight_table pred_weight_table; >>>>> +}; >>>>> + >>>>> #endif > -- > Paul Kocialkowski, Bootlin (formerly Free Electrons) > Embedded Linux and kernel engineering > https://bootlin.com >
Re: [PATCH 4/4] arm64: dts: rockchip: add video codec for rk3399
Hello Ezequiel Sent from my iPad > On Jan 7, 2019, at 1:21 AM, Ezequiel Garcia > wrote: > >> On Sun, 6 Jan 2019 at 13:16, Ayaka wrote: >> >> >> >> Sent from my iPad >> >>> On Jan 7, 2019, at 12:04 AM, Ezequiel Garcia wrote: >>> >>> On Sun, 2019-01-06 at 23:05 +0800, Ayaka wrote: >>>>> On Jan 6, 2019, at 10:22 PM, Ezequiel Garcia >>>>> wrote: >>>>> >>>>> Hi Randy, >>>>> >>>>> Thanks a lot for this patches. They are really useful >>>>> to provide more insight into the VPU hardware. >>>>> >>>>> This change will make the vpu encoder and vpu decoder >>>>> completely independent, can they really work in parallel? >>>> As I said it depends on the platform, but with this patch, the user space >>>> would think they can work at the same time. >>> >>> >>> I think there is some confusion. >>> >>> The devicetree is one thing: it is a hardware representation, >>> a way to describe the hardware, for the kernel/bootloader to >>> parse. >>> >>> The userspace view will depend on the driver implementation. >>> >>> The current devicetree and driver (without your patches), >>> model the VPU as a single piece of hardware, exposing a decoder >>> and an encoder. >>> >>> The V4L driver will then create two video devices, i.e. /dev/videoX >>> and /dev/videoY. So userspace sees an independent view of the >>> devices. >>> >> I knew that, the problem is that the driver should not always create a >> decoder and encoder pair, they may not exist at some platforms, even some >> platforms doesn’t have a encoder. You may have a look on the rk3328 I post >> on the first email as example. > > That is correct. But that still doesn't tackle my question: is the > hardware able to run a decoding and an encoding job in parallel? > For rk3328, yes, you see I didn’t draw them in the same box. > If not, then it's wrong to describe them as independent entities. > >>> However, they are internally connected, and thus we can >>> easily avoid two jobs running in parallel. >>> >> That is what the mpp service did in my patches, handing the relationship >> between each devices. And it is not a easy work, maybe a 4k decoder would be >> blocked by another high frame rate encoding work or another decoder session. >> The vendor kernel have more worry about this, but not in this version. > > Right. That is one way to design it. Another way is having a single > devicetree node for the VPU encoder/decoder "complex". No, you can’t assume which one is in the combo group, it can be various. you see, in the rk3328, the vdpu is paired with an avs+ decoder. That is why I use a virtual device standing for scheduler. > > Thanks for the input! > -- > Ezequiel García, VanguardiaSur > www.vanguardiasur.com.ar
Re: [PATCH 4/4] arm64: dts: rockchip: add video codec for rk3399
Sent from my iPad
> On Jan 7, 2019, at 12:04 AM, Ezequiel Garcia wrote:
>
> On Sun, 2019-01-06 at 23:05 +0800, Ayaka wrote:
>>> On Jan 6, 2019, at 10:22 PM, Ezequiel Garcia wrote:
>>>
>>> Hi Randy,
>>>
>>> Thanks a lot for this patches. They are really useful
>>> to provide more insight into the VPU hardware.
>>>
>>> This change will make the vpu encoder and vpu decoder
>>> completely independent, can they really work in parallel?
>> As I said it depends on the platform, but with this patch, the user space
>> would think they can work at the same time.
>
>
> I think there is some confusion.
>
> The devicetree is one thing: it is a hardware representation,
> a way to describe the hardware, for the kernel/bootloader to
> parse.
>
> The userspace view will depend on the driver implementation.
>
> The current devicetree and driver (without your patches),
> model the VPU as a single piece of hardware, exposing a decoder
> and an encoder.
>
> The V4L driver will then create two video devices, i.e. /dev/videoX
> and /dev/videoY. So userspace sees an independent view of the
> devices.
>
I knew that, the problem is that the driver should not always create a decoder
and encoder pair, they may not exist at some platforms, even some platforms
doesn’t have a encoder. You may have a look on the rk3328 I post on the first
email as example.
> However, they are internally connected, and thus we can
> easily avoid two jobs running in parallel.
>
That is what the mpp service did in my patches, handing the relationship
between each devices. And it is not a easy work, maybe a 4k decoder would be
blocked by another high frame rate encoding work or another decoder session.
The vendor kernel have more worry about this, but not in this version.
> So, in other words, if the VPU can issue a decoder and encoder
> job in parallel, then it's useful to model it as two independent
> devices. Otherwise, it's better not to.
>
> I hope this can clarify things a bit for you!
>
I would review the request API for those codecs structures and some basic
designs, people from LibreELEC told me a few noticed inform.
> PS: Too bad I won't be at FOSDEM to discuss this personally.
I am sure Paulk from bootlin would be there and some others guy related, I have
not cared about this for a little long time, there are more huge problem I
should point out now.
>
> Thanks,
> Ezequiel
>
>> BTW, not only the decoder, there is a post processor with the decoder, it
>> can be used as part of the decoder pipeline with only a macro block delay
>> or process the data from an external buffer.
>> I forget to write a note on what this driver doesn’t present. The real one
>> would have much complex scheduler system, but this one is just a queue.
>> More task management feature are not there.
>> Also the clock boosting is removing and the loading analysis, which are
>> useful for encoder, especially on the rv1108.
>>> Could you provide more details about what is
>>> shared between these devices?
>> No, if Rockchip doesn’t tell, my mouth is sealed.
>>> Thanks a lot!
>>>
>>>> On Sun, 2019-01-06 at 02:31 +0800, Randy Li wrote:
>>>> It offers an example how a full features video codec
>>>> should be configured.
>>>>
>>>> The original clocks assignment don't look good, if the clocks
>>>> lower than 300MHZ, most of decoing tasks would suffer from
>>>> timeout problem, 500MHZ is also a little high for RK3399
>>>> running in a stable state.
>>>>
>>>> Signed-off-by: Randy Li
>>>> ---
>>>> .../boot/dts/rockchip/rk3399-sapphire.dtsi| 29
>>>> arch/arm64/boot/dts/rockchip/rk3399.dtsi | 68 +--
>>>> 2 files changed, 90 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>>>> b/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>>>> index 946d3589575a..c3db878bae45 100644
>>>> --- a/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>>>> +++ b/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>>>> @@ -632,6 +632,35 @@
>>>> dr_mode = "host";
>>>> };
>>>>
>>>> +&rkvdec {
>>>> +status = "okay";
>>>> +};
>>>> +
>>>> +&rkvdec_srv {
>>>> +status = "okay";
>>>> +};
>>>> +
>>>> +&vdec_mmu {
>>>> +status =
Re: [PATCH 4/4] arm64: dts: rockchip: add video codec for rk3399
> On Jan 6, 2019, at 10:22 PM, Ezequiel Garcia wrote:
>
> Hi Randy,
>
> Thanks a lot for this patches. They are really useful
> to provide more insight into the VPU hardware.
>
> This change will make the vpu encoder and vpu decoder
> completely independent, can they really work in parallel?
As I said it depends on the platform, but with this patch, the user space would
think they can work at the same time. BTW, not only the decoder, there is a
post processor with the decoder, it can be used as part of the decoder pipeline
with only a macro block delay or process the data from an external buffer.
I forget to write a note on what this driver doesn’t present. The real one
would have much complex scheduler system, but this one is just a queue. More
task management feature are not there.
Also the clock boosting is removing and the loading analysis, which are useful
for encoder, especially on the rv1108.
> Could you provide more details about what is
> shared between these devices?
No, if Rockchip doesn’t tell, my mouth is sealed.
> Thanks a lot!
>
>> On Sun, 2019-01-06 at 02:31 +0800, Randy Li wrote:
>> It offers an example how a full features video codec
>> should be configured.
>>
>> The original clocks assignment don't look good, if the clocks
>> lower than 300MHZ, most of decoing tasks would suffer from
>> timeout problem, 500MHZ is also a little high for RK3399
>> running in a stable state.
>>
>> Signed-off-by: Randy Li
>> ---
>> .../boot/dts/rockchip/rk3399-sapphire.dtsi| 29
>> arch/arm64/boot/dts/rockchip/rk3399.dtsi | 68 +--
>> 2 files changed, 90 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> b/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> index 946d3589575a..c3db878bae45 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> @@ -632,6 +632,35 @@
>> dr_mode = "host";
>> };
>>
>> +&rkvdec {
>> +status = "okay";
>> +};
>> +
>> +&rkvdec_srv {
>> +status = "okay";
>> +};
>> +
>> +&vdec_mmu {
>> +status = "okay";
>> +};
>> +
>> +&vdpu {
>> +status = "okay";
>> +};
>> +
>> +&vepu {
>> +status = "okay";
>> +};
>> +
>> +&vpu_service {
>> +status = "okay";
>> +};
>> +
>> +&vpu_mmu {
>> +status = "okay";
>> +
>> +};
>> +
>> &vopb {
>> status = "okay";
>> };
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> index b22b2e40422b..5fa3247e7bf0 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> @@ -1242,16 +1242,39 @@
>> status = "disabled";
>> };
>>
>> -vpu: video-codec@ff65 {
>> -compatible = "rockchip,rk3399-vpu";
>> -reg = <0x0 0xff65 0x0 0x800>;
>> -interrupts = ,
>> - ;
>> -interrupt-names = "vepu", "vdpu";
>> +vpu_service: vpu-srv {
>> +compatible = "rockchip,mpp-service";
>> +status = "disabled";
>> +};
>> +
>> +vepu: vpu-encoder@ff65 {
>> +compatible = "rockchip,vpu-encoder-v2";
>> +reg = <0x0 0xff65 0x0 0x400>;
>> +interrupts = ;
>> +interrupt-names = "irq_enc";
>> clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
>> -clock-names = "aclk", "hclk";
>> +clock-names = "aclk_vcodec", "hclk_vcodec";
>> +resets = <&cru SRST_H_VCODEC>, <&cru SRST_A_VCODEC>;
>> +reset-names = "video_h", "video_a";
>> iommus = <&vpu_mmu>;
>> power-domains = <&power RK3399_PD_VCODEC>;
>> +rockchip,srv = <&vpu_service>;
>> +status = "disabled";
>> +};
>> +
>> +vdpu: vpu-decoder@ff650400 {
>> +compatible = "rockchip,vpu-decoder-v2";
>> +reg = <0x0 0xff650400 0x0 0x400>;
>> +interrupts = ;
>> +interrupt-names = "irq_dec";
>> +iommus = <&vpu_mmu>;
>> +clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
>> +clock-names = "aclk_vcodec", "hclk_vcodec";
>> +resets = <&cru SRST_H_VCODEC>, <&cru SRST_A_VCODEC>;
>> +reset-names = "video_h", "video_a";
>> +power-domains = <&power RK3399_PD_VCODEC>;
>> +rockchip,srv = <&vpu_service>;
>> +status = "disabled";
>> };
>>
>> vpu_mmu: iommu@ff650800 {
>> @@ -1261,11 +1284,42 @@
>> interrupt-names = "vpu_mmu";
>> clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
>> clock-names = "aclk", "iface";
>> +assigned-clocks = <&cru ACLK_VCODEC_PRE>;
>> +assigned-clock-parents = <&cru PLL_GPLL>;
>> #iommu-cells = <0>;
>> power-domains = <&power RK3399_PD_VCODEC>;
>> status = "disabled";
>> };
>>
>> +rkvdec_srv: rkvdec-srv {
>> +compatible = "rockchip,mpp-service";
>> +status = "disabled";
>> +};
>> +
>> +rkvdec: video-decoder@ff66 {
>> +compatible = "rockchip,video-decoder-v1";
>> +reg = <0x0 0xff
Re: [PATCH 4/4] arm64: dts: rockchip: add video codec for rk3399
> On Jan 6, 2019, at 10:22 PM, Ezequiel Garcia wrote:
>
> Hi Randy,
>
> Thanks a lot for this patches. They are really useful
> to provide more insight into the VPU hardware.
>
> This change will make the vpu encoder and vpu decoder
> completely independent, can they really work in parallel?
As I said it depends on the platform, but with this patch, the user space would
think they can work at the same time. BTW, not only the decoder, there is a
post processor with the decoder, it can be used as part of the decoder pipeline
with only a macro block delay or process the data from an external buffer.
I forget to write a note on what this driver doesn’t present. The real one
would have much complex scheduler system, but this one is just a queue. More
task management feature are not there.
Also the clock boosting is removing and the loading analysis, which are useful
for encoder, especially on the rv1108.
> Could you provide more details about what is
> shared between these devices?
No, if Rockchip doesn’t tell, my mouth is sealed.
> Thanks a lot!
>
>> On Sun, 2019-01-06 at 02:31 +0800, Randy Li wrote:
>> It offers an example how a full features video codec
>> should be configured.
>>
>> The original clocks assignment don't look good, if the clocks
>> lower than 300MHZ, most of decoing tasks would suffer from
>> timeout problem, 500MHZ is also a little high for RK3399
>> running in a stable state.
>>
>> Signed-off-by: Randy Li
>> ---
>> .../boot/dts/rockchip/rk3399-sapphire.dtsi| 29
>> arch/arm64/boot/dts/rockchip/rk3399.dtsi | 68 +--
>> 2 files changed, 90 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> b/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> index 946d3589575a..c3db878bae45 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3399-sapphire.dtsi
>> @@ -632,6 +632,35 @@
>> dr_mode = "host";
>> };
>>
>> +&rkvdec {
>> +status = "okay";
>> +};
>> +
>> +&rkvdec_srv {
>> +status = "okay";
>> +};
>> +
>> +&vdec_mmu {
>> +status = "okay";
>> +};
>> +
>> +&vdpu {
>> +status = "okay";
>> +};
>> +
>> +&vepu {
>> +status = "okay";
>> +};
>> +
>> +&vpu_service {
>> +status = "okay";
>> +};
>> +
>> +&vpu_mmu {
>> +status = "okay";
>> +
>> +};
>> +
>> &vopb {
>> status = "okay";
>> };
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> index b22b2e40422b..5fa3247e7bf0 100644
>> --- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi
>> @@ -1242,16 +1242,39 @@
>> status = "disabled";
>> };
>>
>> -vpu: video-codec@ff65 {
>> -compatible = "rockchip,rk3399-vpu";
>> -reg = <0x0 0xff65 0x0 0x800>;
>> -interrupts = ,
>> - ;
>> -interrupt-names = "vepu", "vdpu";
>> +vpu_service: vpu-srv {
>> +compatible = "rockchip,mpp-service";
>> +status = "disabled";
>> +};
>> +
>> +vepu: vpu-encoder@ff65 {
>> +compatible = "rockchip,vpu-encoder-v2";
>> +reg = <0x0 0xff65 0x0 0x400>;
>> +interrupts = ;
>> +interrupt-names = "irq_enc";
>> clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
>> -clock-names = "aclk", "hclk";
>> +clock-names = "aclk_vcodec", "hclk_vcodec";
>> +resets = <&cru SRST_H_VCODEC>, <&cru SRST_A_VCODEC>;
>> +reset-names = "video_h", "video_a";
>> iommus = <&vpu_mmu>;
>> power-domains = <&power RK3399_PD_VCODEC>;
>> +rockchip,srv = <&vpu_service>;
>> +status = "disabled";
>> +};
>> +
>> +vdpu: vpu-decoder@ff650400 {
>> +compatible = "rockchip,vpu-decoder-v2";
>> +reg = <0x0 0xff650400 0x0 0x400>;
>> +interrupts = ;
>> +interrupt-names = "irq_dec";
>> +iommus = <&vpu_mmu>;
>> +clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
>> +clock-names = "aclk_vcodec", "hclk_vcodec";
>> +resets = <&cru SRST_H_VCODEC>, <&cru SRST_A_VCODEC>;
>> +reset-names = "video_h", "video_a";
>> +power-domains = <&power RK3399_PD_VCODEC>;
>> +rockchip,srv = <&vpu_service>;
>> +status = "disabled";
>> };
>>
>> vpu_mmu: iommu@ff650800 {
>> @@ -1261,11 +1284,42 @@
>> interrupt-names = "vpu_mmu";
>> clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
>> clock-names = "aclk", "iface";
>> +assigned-clocks = <&cru ACLK_VCODEC_PRE>;
>> +assigned-clock-parents = <&cru PLL_GPLL>;
>> #iommu-cells = <0>;
>> power-domains = <&power RK3399_PD_VCODEC>;
>> status = "disabled";
>> };
>>
>> +rkvdec_srv: rkvdec-srv {
>> +compatible = "rockchip,mpp-service";
>> +status = "disabled";
>> +};
>> +
>> +rkvdec: video-decoder@ff66 {
>> +compatible = "rockchip,video-decoder-v1";
>> +
Re: [PATCH v6 1/3] [media] v4l: add parsed MPEG-2 support
On 07/09/2017 08:32 AM, Nicolas Dufresne wrote: Le samedi 08 juillet 2017 à 13:16 +0800, ayaka a écrit : On 07/08/2017 02:33 AM, Nicolas Dufresne wrote: Le samedi 08 juillet 2017 à 01:29 +0800, ayaka a écrit : On 07/04/2017 05:29 PM, Hugues FRUCHET wrote: Hi Randy, Thanks for review, and sorry for late reply, answers inline. BR, Hugues. On 06/11/2017 01:41 PM, ayaka wrote: On 04/28/2017 09:25 PM, Hugues Fruchet wrote: Add "parsed MPEG-2" pixel format & related controls needed by stateless video decoders. In order to decode the video bitstream chunk provided by user on output queue, stateless decoders require also some extra data resulting from this video bitstream chunk parsing. Those parsed extra data have to be set by user through control framework using the dedicated mpeg video extended controls introduced in this patchset. I have compared those v4l2 controls with the registers of the rockchip video IP. Most of them are met, but only lacks of sw_init_qp. In case of MPEG-1/2, this register seems forced to 1, please double check the on2 headers parsing library related to MPEG2. Nevertheless, I see this hardware register used with VP8/H264. Yes, it is forced to be 1. We can skip this field for MPEG1/2 Hence, no need to put this field on MPEG-2 interface, but should come with VP8/H264. Here is the full translation table of the registers of the rockchip video IP. q_scale_type sw_qscale_type concealment_motion_vectorssw_con_mv_e intra_dc_precision sw_intra_dc_prec intra_vlc_format sw_intra_vlc_tab frame_pred_frame_dct sw_frame_pred_dct alternate_scan sw_alt_scan_flag_e f_code sw_fcode_bwd_ver sw_fcode_bwd_hor sw_fcode_fwd_ver sw_fcode_fwd_hor full_pel_forward_vector sw_mv_accuracy_fwd full_pel_backward_vector sw_mv_accuracy_bwd I also saw you add two format for parsed MPEG-2/MPEG-1 format, I would not recommand to do that. We need to differentiate MPEG-1/MPEG-2, not all the fields are applicable depending on version. Usually the MPEG-2 decoder could support MPEG-1, as I know, the syntax of byte stream of them are the same. That is what google does, because for a few video format and some hardware, they just request a offsets from the original video byte stream. I don't understand your comment, perhaps have you some as a basis of discussion ? I mean V4L2-PIX-FMT-MPEG2-PARSED V4L2-PIX-FMT-MPEG1-PARSED I wonder whether you want use the new format to inform the userspace that this device is for stateless video decoder, as google defined something like V4L2_PIX_FMT_H264_SLICE. I think the driver registers some controls is enough for the userspace to detect whether it is a stateless device. Or it will increase the work of the userspace(I mean Gstreamer). Just a note that SLICE has nothing to do with PARSED here. You could have an H264 decoder that is stateless and support handling slices rather then full frames (e.g. V4L2_PIX_FMT_H264_SLICE_PARSED could be valid). Actually, they have the same meanings, the H264_SLICE is not a slice, it is an access unit in the rockchip vpu driver for Google. Let's make sure this never get into mainline unmodified. H264_SLICE should indicate that encoded buffer need to contains at least one slice. We already have a format that indicates that a complete AU must be passed. I do have active project going on where we really want to pass slice for low latency cases and I would really appreciate if that name can be used. I think the hardware could support multiple slices in an AU, one slice in an AU or slice itself. Well, but the in chrome os, it just means an parsed H264 stream. I would not worry to much about Gst, as we will likely use this device through the libv4l2 here, hence will only notice the "emulated" V4L2_PIX_FMT_MPEG2 and ignore the _PARSED variant. And without libv4l2, we'd just ignore this driver completely. I doubt we will implement per- device parsing inside Gst itself if it's already done in an external library for us. libv4l2 might need some fixing, but hopefully it's not beyond repair. As Gstreamer has merged the VA-API before, I would like to merge it into Gstreamer directly. Also for performance reason and my experience, the buffer management would be a big problem, we need to increase the another layer to v4l2 plugins then. When the parser is split from its caller, it would be hard to add a path for error handing or something else. I totally fail to understand your point here. Existing driver have a separate core that do hide all the parsing and error handling, yet it works relatively well this w
Re: [PATCH v6 1/3] [media] v4l: add parsed MPEG-2 support
On 07/08/2017 02:33 AM, Nicolas Dufresne wrote: Le samedi 08 juillet 2017 à 01:29 +0800, ayaka a écrit : On 07/04/2017 05:29 PM, Hugues FRUCHET wrote: Hi Randy, Thanks for review, and sorry for late reply, answers inline. BR, Hugues. On 06/11/2017 01:41 PM, ayaka wrote: On 04/28/2017 09:25 PM, Hugues Fruchet wrote: Add "parsed MPEG-2" pixel format & related controls needed by stateless video decoders. In order to decode the video bitstream chunk provided by user on output queue, stateless decoders require also some extra data resulting from this video bitstream chunk parsing. Those parsed extra data have to be set by user through control framework using the dedicated mpeg video extended controls introduced in this patchset. I have compared those v4l2 controls with the registers of the rockchip video IP. Most of them are met, but only lacks of sw_init_qp. In case of MPEG-1/2, this register seems forced to 1, please double check the on2 headers parsing library related to MPEG2. Nevertheless, I see this hardware register used with VP8/H264. Yes, it is forced to be 1. We can skip this field for MPEG1/2 Hence, no need to put this field on MPEG-2 interface, but should come with VP8/H264. Here is the full translation table of the registers of the rockchip video IP. q_scale_type sw_qscale_type concealment_motion_vectorssw_con_mv_e intra_dc_precision sw_intra_dc_prec intra_vlc_format sw_intra_vlc_tab frame_pred_frame_dct sw_frame_pred_dct alternate_scan sw_alt_scan_flag_e f_code sw_fcode_bwd_ver sw_fcode_bwd_hor sw_fcode_fwd_ver sw_fcode_fwd_hor full_pel_forward_vector sw_mv_accuracy_fwd full_pel_backward_vector sw_mv_accuracy_bwd I also saw you add two format for parsed MPEG-2/MPEG-1 format, I would not recommand to do that. We need to differentiate MPEG-1/MPEG-2, not all the fields are applicable depending on version. Usually the MPEG-2 decoder could support MPEG-1, as I know, the syntax of byte stream of them are the same. That is what google does, because for a few video format and some hardware, they just request a offsets from the original video byte stream. I don't understand your comment, perhaps have you some as a basis of discussion ? I mean V4L2-PIX-FMT-MPEG2-PARSED V4L2-PIX-FMT-MPEG1-PARSED I wonder whether you want use the new format to inform the userspace that this device is for stateless video decoder, as google defined something like V4L2_PIX_FMT_H264_SLICE. I think the driver registers some controls is enough for the userspace to detect whether it is a stateless device. Or it will increase the work of the userspace(I mean Gstreamer). Just a note that SLICE has nothing to do with PARSED here. You could have an H264 decoder that is stateless and support handling slices rather then full frames (e.g. V4L2_PIX_FMT_H264_SLICE_PARSED could be valid). Actually, they have the same meanings, the H264_SLICE is not a slice, it is an access unit in the rockchip vpu driver for Google. I would not worry to much about Gst, as we will likely use this device through the libv4l2 here, hence will only notice the "emulated" V4L2_PIX_FMT_MPEG2 and ignore the _PARSED variant. And without libv4l2, we'd just ignore this driver completely. I doubt we will implement per- device parsing inside Gst itself if it's already done in an external library for us. libv4l2 might need some fixing, but hopefully it's not beyond repair. As Gstreamer has merged the VA-API before, I would like to merge it into Gstreamer directly. Also for performance reason and my experience, the buffer management would be a big problem, we need to increase the another layer to v4l2 plugins then. When the parser is split from its caller, it would be hard to add a path for error handing or something else. P.S I have dropped my original plan about writing a new v4l2 driver for rockchip, but reuse the v4l2 logic from google and rewrite the hal part and rewrite what I need to save the time. It would comes soon Offset from the beginning of original video bitstream is supported within proposed interface, see v4l2_mpeg_video_mpeg2_pic_hd->offset field. Signed-off-by: Hugues Fruchet --- Documentation/media/uapi/v4l/extended-controls.rst | 363 + Documentation/media/uapi/v4l/pixfmt-013.rst| 10 + Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 38 ++- Documentation/media/videodev2.h.rst.exceptions | 6 + drivers/media/v4l2-core/v4l2-ctrls.c | 53 +++ drivers/media/v4l2-core/v4l2-ioctl.c
Re: [PATCH v6 1/3] [media] v4l: add parsed MPEG-2 support
On 07/04/2017 05:29 PM, Hugues FRUCHET wrote: Hi Randy, Thanks for review, and sorry for late reply, answers inline. BR, Hugues. On 06/11/2017 01:41 PM, ayaka wrote: On 04/28/2017 09:25 PM, Hugues Fruchet wrote: Add "parsed MPEG-2" pixel format & related controls needed by stateless video decoders. In order to decode the video bitstream chunk provided by user on output queue, stateless decoders require also some extra data resulting from this video bitstream chunk parsing. Those parsed extra data have to be set by user through control framework using the dedicated mpeg video extended controls introduced in this patchset. I have compared those v4l2 controls with the registers of the rockchip video IP. Most of them are met, but only lacks of sw_init_qp. In case of MPEG-1/2, this register seems forced to 1, please double check the on2 headers parsing library related to MPEG2. Nevertheless, I see this hardware register used with VP8/H264. Yes, it is forced to be 1. We can skip this field for MPEG1/2 Hence, no need to put this field on MPEG-2 interface, but should come with VP8/H264. Here is the full translation table of the registers of the rockchip video IP. q_scale_type sw_qscale_type concealment_motion_vectorssw_con_mv_e intra_dc_precision sw_intra_dc_prec intra_vlc_format sw_intra_vlc_tab frame_pred_frame_dct sw_frame_pred_dct alternate_scan sw_alt_scan_flag_e f_code sw_fcode_bwd_ver sw_fcode_bwd_hor sw_fcode_fwd_ver sw_fcode_fwd_hor full_pel_forward_vector sw_mv_accuracy_fwd full_pel_backward_vector sw_mv_accuracy_bwd I also saw you add two format for parsed MPEG-2/MPEG-1 format, I would not recommand to do that. We need to differentiate MPEG-1/MPEG-2, not all the fields are applicable depending on version. Usually the MPEG-2 decoder could support MPEG-1, as I know, the syntax of byte stream of them are the same. That is what google does, because for a few video format and some hardware, they just request a offsets from the original video byte stream. I don't understand your comment, perhaps have you some as a basis of discussion ? I mean V4L2-PIX-FMT-MPEG2-PARSED V4L2-PIX-FMT-MPEG1-PARSED I wonder whether you want use the new format to inform the userspace that this device is for stateless video decoder, as google defined something like V4L2_PIX_FMT_H264_SLICE. I think the driver registers some controls is enough for the userspace to detect whether it is a stateless device. Or it will increase the work of the userspace(I mean Gstreamer). Offset from the beginning of original video bitstream is supported within proposed interface, see v4l2_mpeg_video_mpeg2_pic_hd->offset field. Signed-off-by: Hugues Fruchet --- Documentation/media/uapi/v4l/extended-controls.rst | 363 + Documentation/media/uapi/v4l/pixfmt-013.rst| 10 + Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 38 ++- Documentation/media/videodev2.h.rst.exceptions | 6 + drivers/media/v4l2-core/v4l2-ctrls.c | 53 +++ drivers/media/v4l2-core/v4l2-ioctl.c | 2 + include/uapi/linux/v4l2-controls.h | 94 ++ include/uapi/linux/videodev2.h | 8 + 8 files changed, 572 insertions(+), 2 deletions(-) diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst index abb1057..b48eac9 100644 --- a/Documentation/media/uapi/v4l/extended-controls.rst +++ b/Documentation/media/uapi/v4l/extended-controls.rst @@ -1827,6 +1827,369 @@ enum v4l2_mpeg_cx2341x_video_median_filter_type - not insert, 1 = insert packets. +MPEG-2 Parsed Control Reference +- + +The MPEG-2 parsed decoding controls are needed by stateless video decoders. +Those decoders expose :ref:`Compressed formats ` :ref:`V4L2_PIX_FMT_MPEG1_PARSED` or :ref:`V4L2_PIX_FMT_MPEG2_PARSED`. +In order to decode the video bitstream chunk provided by user on output queue, +stateless decoders require also some extra data resulting from this video +bitstream chunk parsing. Those parsed extra data have to be set by user +through control framework using the mpeg video extended controls defined +in this section. Those controls have been defined based on MPEG-2 standard +ISO/IEC 13818-2, and so derive directly from the MPEG-2 video bitstream syntax +including how it is coded inside bitstream (enumeration values for ex.). + +MPEG-2 Parsed Control IDs +^^^ + +.. _mpeg2-parsed-control-id: + +.. c:type:: V4L2_CID_MPEG_VIDEO_MP
Fwd: Re: [PATCH v6 1/3] [media] v4l: add parsed MPEG-2 support
I don't know why it was sent in html format, I am sorry to re-send it again.
On 04/28/2017 09:25 PM, Hugues Fruchet wrote:
Add "parsed MPEG-2" pixel format & related controls
needed by stateless video decoders.
In order to decode the video bitstream chunk provided
by user on output queue, stateless decoders require
also some extra data resulting from this video bitstream
chunk parsing.
Those parsed extra data have to be set by user through
control framework using the dedicated mpeg video extended
controls introduced in this patchset.
I have compared those v4l2 controls with the registers of the rockchip
video IP.
Most of them are met, but only lacks of sw_init_qp.
Here is the full translation table of the registers of the rockchip
video IP.
q_scale_type sw_qscale_type
concealment_motion_vectorssw_con_mv_e
intra_dc_precision sw_intra_dc_prec
intra_vlc_format sw_intra_vlc_tab
frame_pred_frame_dct sw_frame_pred_dct
alternate_scan sw_alt_scan_flag_e
f_code sw_fcode_bwd_ver
sw_fcode_bwd_hor
sw_fcode_fwd_ver
sw_fcode_fwd_hor
full_pel_forward_vector sw_mv_accuracy_fwd
full_pel_backward_vector sw_mv_accuracy_bwd
I also saw you add two format for parsed MPEG-2/MPEG-1 format, I would
not recommand to do that.
That is what google does, because for a few video format and some
hardware, they just request a offsets from the original video byte stream.
Signed-off-by: Hugues Fruchet
---
Documentation/media/uapi/v4l/extended-controls.rst | 363 +
Documentation/media/uapi/v4l/pixfmt-013.rst| 10 +
Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 38 ++-
Documentation/media/videodev2.h.rst.exceptions | 6 +
drivers/media/v4l2-core/v4l2-ctrls.c | 53 +++
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/uapi/linux/v4l2-controls.h | 94 ++
include/uapi/linux/videodev2.h | 8 +
8 files changed, 572 insertions(+), 2 deletions(-)
diff --git a/Documentation/media/uapi/v4l/extended-controls.rst
b/Documentation/media/uapi/v4l/extended-controls.rst
index abb1057..b48eac9 100644
--- a/Documentation/media/uapi/v4l/extended-controls.rst
+++ b/Documentation/media/uapi/v4l/extended-controls.rst
@@ -1827,6 +1827,369 @@ enum v4l2_mpeg_cx2341x_video_median_filter_type -
not insert, 1 = insert packets.
+MPEG-2 Parsed Control Reference
+-
+
+The MPEG-2 parsed decoding controls are needed by stateless video decoders.
+Those decoders expose :ref:`Compressed formats `
:ref:`V4L2_PIX_FMT_MPEG1_PARSED` or
:ref:`V4L2_PIX_FMT_MPEG2_PARSED`.
+In order to decode the video bitstream chunk provided by user on output queue,
+stateless decoders require also some extra data resulting from this video
+bitstream chunk parsing. Those parsed extra data have to be set by user
+through control framework using the mpeg video extended controls defined
+in this section. Those controls have been defined based on MPEG-2 standard
+ISO/IEC 13818-2, and so derive directly from the MPEG-2 video bitstream syntax
+including how it is coded inside bitstream (enumeration values for ex.).
+
+MPEG-2 Parsed Control IDs
+^^^
+
+.. _mpeg2-parsed-control-id:
+
+.. c:type:: V4L2_CID_MPEG_VIDEO_MPEG2_SEQ_HDR
+(enum)
+
+.. tabularcolumns:: |p{4.0cm}|p{2.5cm}|p{11.0cm}|
+
+.. c:type:: v4l2_mpeg_video_mpeg2_seq_hdr
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_mpeg_video_mpeg2_seq_hdr
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u16
+ - ``width``
+ - Video width in pixels.
+* - __u16
+ - ``height``
+ - Video height in pixels.
+* - __u8
+ - ``aspect_ratio_info``
+ - Aspect ratio code as in the bitstream (1: 1:1 square pixels,
+2: 4:3 display, 3: 16:9 display, 4: 2.21:1 display)
+* - __u8
+ - ``framerate code``
+ - Framerate code as in the bitstream
+(1: 24000/1001.0 '23.976 fps, 2: 24.0, 3: 25.0,
+4: 3/1001.0 '29.97, 5: 30.0, 6: 50.0, 7: 6/1001.0,
+8: 60.0)
+* - __u16
+ - ``vbv_buffer_size``
+ - Video Buffering Verifier size, expressed in 16KBytes unit.
+* - __u32
+ - ``bitrate_value``
+ - Bitrate value as in the bitstream, expressed in 400bps unit
+* - __u16
+ - ``constrained_parameters_flag``
+ - Set to 1 if this bitstream uses constrained parameters.
+* - __u8
+ - ``load_intra_quantiser_matrix``
+ - If set to 1, ``intra_quantiser_matrix`` table is to be used for
+decoding.
+* - __u8
+ - ``load_non_intra_quantiser_matrix``
+ - If set to 1, ``non_intra_quantiser_matrix`` table is to be used for
+decoding.
+* - __u8
+ - ``intra_quantiser_matrix[64]``
+ - Intra quantization table, in zig-zag scan order.
+* - __u8
+ - ``non_intra_quantiser_matrix[64]``
+ - Non-intra quantization table, in zig
Re: [RFC] V4L2 unified low-level decoder API
On 06/08/2017 06:56 PM, Hans Verkuil wrote: Hi Alexandre, On 08/06/17 11:59, Alexandre Courbot wrote: On Thu, Jun 8, 2017 at 5:56 PM, Pawel Osciak wrote: Hi, On Fri, May 19, 2017 at 1:08 AM, Hugues FRUCHET wrote: Before merging this work Hans would like to have feedback from peers, in order to be sure that this is inline with other SoC vendors drivers expectations. Thomasz, Pawel, could you give your view regarding ChromeOS and Rockchip driver ? The drivers for Rockchip codecs are submitted to the public Chromium OS kernel and working on our RK-based platforms. We have also since added a VP9 API as That driver still lacks a number of feature comparing to the rockchip android driver, since google never requests them. Also the performance is not as good as the android one. That is why I start to write a new driver myself. Also the rockchip platform is very complex, that driver won't be work on all the SoCs without a large of modification(now only two chips are supported) well, which is also working on devices that support it. This gives us a set of H.264, VP8 and VP9 APIs on both kernel and userspace side (in the open source Chromium browser) that are working currently and can be used for further testing. We are interested in merging the API patches as well as these drivers upstream (they were posted on this list two years ago), however we've been blocked by the progress of request API, which is required for this. Alexandre Courbot is currently Well the request API looks fine for me, I just can't understand why it is not merged except those are a few functions have a reference problem stopping build v4l2 as a module. looking into creating a minimal version of the request API that would provide enough functionality for stateless codecs, and also plans to further work on re-submitting the particular codec API patches, once the request API is finalized. Hi everyone, It is a bit scary to start hacking on V4L with something as disruptive as the request API, so please bear with me as I will inevitably post code that will go from cutely clueless to downright offensive. Yeah, you went straight into the deep end of the pool :-) I am very, very pleased to see Google picking up this work. We need more core V4L2 developers, so welcome! Thankfully Pawel is not too far from my desk, and we (Pawel, Tomasz and myself) had a very fruitful in-person brainstorming session last week with Laurent, so this should limit the damage. In any case, I think that everybody agrees that this needs to be pushed forward. Chromium OS in particular has a big interest to see this feature landing upstream, so I will dedicate some cycles to this. Absolutely! From reading the meetings reports (e.g. https://www.spinics.net/lists/linux-media/msg106699.html) I understand that we want to support several use-cases with this and we already have several proposals with code. Chromium in a first time is interested in stateless codecs support, and this use-case also seems to be the simplest to implement, so we would like to start pushing in that direction first. This should give us a reasonably sized code base to rely upon and implement the other use-cases as we see clearer through this. I still need to study a bit the existing proposals and to clearly lay out the conclusions of our meeting with Laurent, but the general idea has not changed too much, except maybe that we thought it may be nice to make state management less explicit to userspace by default. I would be interested in knowing whether there are updated versions of the implementations mentioned in the meeting report above, and/or a merge of these work? Also, if someone is actively working on this at the moment, we will definitely want to sync on IRC or anywhere else. Laurent has been the last one working on this, but his code doesn't have the control handling :-( My latest (well, December 2015) tree with the control request code is here: https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=requests2 It's AFAIK a slightly newer version from what ChromeOS uses. Excited to work with you all! :) Looking forward to your code! Regards, Hans
Re: [PATCH v6 2/3] v4l: Add 10/16-bits per channel YUV pixel formats
On 04/18/2017 03:33 AM, Mauro Carvalho Chehab wrote:
Em Sun, 5 Mar 2017 18:00:32 +0800
Randy Li escreveu:
The formats added by this patch are:
V4L2_PIX_FMT_P010
V4L2_PIX_FMT_P010M
V4L2_PIX_FMT_P016
V4L2_PIX_FMT_P016M
Currently, none of driver uses those format.
Also a variant of V4L2_PIX_FMT_P010M pixel format is added.
The V4L2_PIX_FMT_P010CM is a compat variant of the V4L2_PIX_FMT_P010,
Some developers from Gstreamer think it should be renamed as CM10 for
the P010CM, I don't have much idea about that.
which uses the unused 6 bits to store the next pixel. And with
the alignment requirement of the hardware, it usually would be
some extra space left at the end of a stride.
You should check your patches with checkpatch... I'm getting
this:
WARNING: 'simliar' may be misspelled - perhaps 'similar'?
#61: FILE: Documentation/media/uapi/v4l/pixfmt-p010.rst:13:
+chroma samples as simliar to ``V4L2_PIX_FMT_NV12``
I am sorry about that
WARNING: 'simliar' may be misspelled - perhaps 'similar'?
#334: FILE: Documentation/media/uapi/v4l/pixfmt-p016.rst:13:
+chroma samples as simliar to ``V4L2_PIX_FMT_NV12``
Signed-off-by: Randy Li
---
Documentation/media/uapi/v4l/pixfmt-p010.rst | 126
Documentation/media/uapi/v4l/pixfmt-p010m.rst | 135 ++
Documentation/media/uapi/v4l/pixfmt-p016.rst | 125
Documentation/media/uapi/v4l/pixfmt-p016m.rst | 134 +
Documentation/media/uapi/v4l/yuv-formats.rst | 4 +
include/uapi/linux/videodev2.h| 5 +
6 files changed, 529 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010m.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016m.rst
diff --git a/Documentation/media/uapi/v4l/pixfmt-p010.rst
b/Documentation/media/uapi/v4l/pixfmt-p010.rst
new file mode 100644
index 000..59ed118
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-p010.rst
@@ -0,0 +1,126 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-P010:
+
+**
+V4L2_PIX_FMT_P010 ('P010')
+**
+
+
+V4L2_PIX_FMT_P010
+Formats with ½ horizontal and vertical chroma resolution. One luminance and
+one chrominance plane with alternating
+chroma samples as simliar to ``V4L2_PIX_FMT_NV12``
It is probably ok to use the UTF symbol for 1/2, but you should check
if both PDF and HTML outputs will be ok.
I see, I would upload a new version later
+
+
+Description
+===
+
+It is a two-plane versions of the YUV 4:2:0 format. The three
+components are separated into two sub-images or planes. The Y plane is
+first. The Y plane has 16 bits per pixel, but only 10 bits are used with the
+rest 6 bits set to zero. For ``V4L2_PIX_FMT_P010``, a combined CbCr plane
+immediately follows the Y plane in memory. The CbCr
+plane is the same width, in bytes, as the Y plane (and of the image),
+but is half as tall in pixels. Each CbCr pair belongs to four pixels.
+For example, Cb\ :sub:`0`/Cr\ :sub:`0` belongs to Y'\ :sub:`00`,
+Y'\ :sub:`01`, Y'\ :sub:`10`, Y'\ :sub:`11`.
+If the Y plane has pad bytes after each row, then the CbCr plane has as
+many pad bytes after its rows.
+
+**Byte Order.**
+Each cell is two bytes.
+
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+
+* - start + 0:
+ - Y'\ :sub:`00`
+ - Y'\ :sub:`01`
+ - Y'\ :sub:`02`
+ - Y'\ :sub:`03`
+* - start + 4:
+ - Y'\ :sub:`10`
+ - Y'\ :sub:`11`
+ - Y'\ :sub:`12`
+ - Y'\ :sub:`13`
+* - start + 8:
+ - Y'\ :sub:`20`
+ - Y'\ :sub:`21`
+ - Y'\ :sub:`22`
+ - Y'\ :sub:`23`
+* - start + 12:
+ - Y'\ :sub:`30`
+ - Y'\ :sub:`31`
+ - Y'\ :sub:`32`
+ - Y'\ :sub:`33`
+* - start + 16:
+ - Cb\ :sub:`00`
+ - Cr\ :sub:`00`
+ - Cb\ :sub:`01`
+ - Cr\ :sub:`01`
+* - start + 20:
+ - Cb\ :sub:`10`
+ - Cr\ :sub:`10`
+ - Cb\ :sub:`11`
+ - Cr\ :sub:`11`
+
+
+**Color Sample Location..**
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+
+* -
+ - 0
+ -
+ - 1
+ - 2
+ -
+ - 3
+* - 0
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 1
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+* - 2
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 3
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
diff --git a/Documentation/media/uapi/v4l/pixfmt-p010m.rst
b/Documentation/media/uapi/v4l/pixfmt-p010m.rst
new file mode 100644
index 000..6697d15
--- /dev/
Re: [PATCH v6 1/3] drm_fourcc: Add new P010, P016 video format
從我的 iPad 傳送
> Clint Taylor 於 2017年3月28日 上午6:49 寫道：
>
>> On 03/26/2017 09:05 PM, Ayaka wrote:
>>
>>
>> 從我的 iPad 傳送
>>
>>>> Ander Conselvan De Oliveira 於 2017年3月14日 下午9:53 寫道：
>>>>
>>>> On Tue, 2017-03-07 at 04:27 +0800, Ayaka wrote:
>>>>
>>>> 從我的 iPad 傳送
>>>>
>>>>>> Ville Syrjälä 於 2017年3月7日 上午2:34 寫道：
>>>>>>
>>>>>> On Tue, Mar 07, 2017 at 01:58:23AM +0800, Ayaka wrote:
>>>>>>
>>>>>>
>>>>>> 從我的 iPad 傳送
>>>>>>
>>>>>>>> Ville Syrjälä 於 2017年3月6日 下午9:06 寫道：
>>>>>>>>
>>>>>>>> On Sun, Mar 05, 2017 at 06:00:31PM +0800, Randy Li wrote:
>>>>>>>> P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
>>>>>>>> per channel video format.
>>>>>>>>
>>>>>>>> P016 is a planar 4:2:0 YUV with interleaved UV plane, 16 bits
>>>>>>>> per channel video format.
>>>>>>>>
>>>>>>>> V3: Added P012 and fixed cpp for P010
>>>>>>>> V4: format definition refined per review
>>>>>>>> V5: Format comment block for each new pixel format
>>>>>>>> V6: reversed Cb/Cr order in comments
>>>>>>>> v7: reversed Cb/Cr order in comments of header files, remove
>>>>>>>> the wrong part of commit message.
>>>>>>>
>>>>>>> What? Why? You just undid what Clint did in v6.
>>>>>>
>>>>>> He missed a file also keeping the wrong description of rockchip.
>>>>>
>>>>> I don't follow. Who missed what exactly?
>>>>
>>>> What he sent is v5, I increase the order number twice in the message, it
>>>> confuse me as well.
>>>> I think Clint forgot the include/uapi/drm/drm_fourcc.h .
>>>
>>> Clint did send a v6, and that updates "include/uapi/drm/drm_fourcc.h":
>>>
>>> https://patchwork.freedesktop.org/patch/141342/
>> Oh, yes but he still used Cr:Cb, but I think it should be Cb:Cr
>> since I think the V is after the U.
>
> From the MSDN fourcc website:
> "If the combined U-V array is addressed as an array of DWORDs, the least
> significant word (LSW) contains the U value and the most significant word
> (MSW) contains the V value. The stride of the combined U-V plane is equal to
> the stride of the Y plane. The U-V plane has half as many lines as the Y
> plane."
>
> The LSW contains U and the MSW contains V, hence the Cr:Cb in the comments of
> the V6 patch.
>
> -Clint
ok it is correct. I think I mistake it that is why my converter for gstreamer
doesn't work
>
>>>
>>>
>>> Ander
>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Cc: Daniel Stone
>>>>>>>> Cc: Ville Syrjälä
>>>>>>>>
>>>>>>>> Signed-off-by: Randy Li
>>>>>>>> Signed-off-by: Clint Taylor
>>>>>>>> ---
>>>>>>>> drivers/gpu/drm/drm_fourcc.c | 3 +++
>>>>>>>> include/uapi/drm/drm_fourcc.h | 21 +
>>>>>>>> 2 files changed, 24 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/drivers/gpu/drm/drm_fourcc.c
>>>>>>>> b/drivers/gpu/drm/drm_fourcc.c
>>>>>>>> index 90d2cc8..3e0fd58 100644
>>>>>>>> --- a/drivers/gpu/drm/drm_fourcc.c
>>>>>>>> +++ b/drivers/gpu/drm/drm_fourcc.c
>>>>>>>> @@ -165,6 +165,9 @@ const struct drm_format_info
>>>>>>>> *__drm_format_info(u32 format)
>>>>>>>> { .format = DRM_FORMAT_UYVY,.depth = 0, .num_planes = 1,
>>>>>>>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>>>>>>> { .format = DRM_FORMAT_VYUY,.depth = 0, .num_planes = 1,
>>>>>>>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>>>>>>> { .format = DRM_FORMAT_AYUV,.depth = 0, .num_planes = 1,
>>>>>>>> .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
>>>>>>>> +{ .format = DRM_FORMAT_P010,.depth
Re: [PATCH v6 1/3] drm_fourcc: Add new P010, P016 video format
從我的 iPad 傳送
> Ander Conselvan De Oliveira 於 2017年3月14日 下午9:53 寫道：
>
>> On Tue, 2017-03-07 at 04:27 +0800, Ayaka wrote:
>>
>> 從我的 iPad 傳送
>>
>>>> Ville Syrjälä 於 2017年3月7日 上午2:34 寫道：
>>>>
>>>> On Tue, Mar 07, 2017 at 01:58:23AM +0800, Ayaka wrote:
>>>>
>>>>
>>>> 從我的 iPad 傳送
>>>>
>>>>>> Ville Syrjälä 於 2017年3月6日 下午9:06 寫道：
>>>>>>
>>>>>> On Sun, Mar 05, 2017 at 06:00:31PM +0800, Randy Li wrote:
>>>>>> P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
>>>>>> per channel video format.
>>>>>>
>>>>>> P016 is a planar 4:2:0 YUV with interleaved UV plane, 16 bits
>>>>>> per channel video format.
>>>>>>
>>>>>> V3: Added P012 and fixed cpp for P010
>>>>>> V4: format definition refined per review
>>>>>> V5: Format comment block for each new pixel format
>>>>>> V6: reversed Cb/Cr order in comments
>>>>>> v7: reversed Cb/Cr order in comments of header files, remove
>>>>>> the wrong part of commit message.
>>>>>
>>>>> What? Why? You just undid what Clint did in v6.
>>>>
>>>> He missed a file also keeping the wrong description of rockchip.
>>>
>>> I don't follow. Who missed what exactly?
>>
>> What he sent is v5, I increase the order number twice in the message, it
>> confuse me as well.
>> I think Clint forgot the include/uapi/drm/drm_fourcc.h .
>
> Clint did send a v6, and that updates "include/uapi/drm/drm_fourcc.h":
>
> https://patchwork.freedesktop.org/patch/141342/
Oh, yes but he still used Cr:Cb, but I think it should be Cb:Cr
since I think the V is after the U.
>
>
> Ander
>
>>>
>>>
>>>>>
>>>>>>
>>>>>> Cc: Daniel Stone
>>>>>> Cc: Ville Syrjälä
>>>>>>
>>>>>> Signed-off-by: Randy Li
>>>>>> Signed-off-by: Clint Taylor
>>>>>> ---
>>>>>> drivers/gpu/drm/drm_fourcc.c | 3 +++
>>>>>> include/uapi/drm/drm_fourcc.h | 21 +
>>>>>> 2 files changed, 24 insertions(+)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
>>>>>> index 90d2cc8..3e0fd58 100644
>>>>>> --- a/drivers/gpu/drm/drm_fourcc.c
>>>>>> +++ b/drivers/gpu/drm/drm_fourcc.c
>>>>>> @@ -165,6 +165,9 @@ const struct drm_format_info *__drm_format_info(u32
>>>>>> format)
>>>>>> { .format = DRM_FORMAT_UYVY,.depth = 0, .num_planes = 1,
>>>>>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>>>>> { .format = DRM_FORMAT_VYUY,.depth = 0, .num_planes = 1,
>>>>>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>>>>> { .format = DRM_FORMAT_AYUV,.depth = 0, .num_planes = 1,
>>>>>> .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
>>>>>> +{ .format = DRM_FORMAT_P010,.depth = 0, .num_planes =
>>>>>> 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>>>>> +{ .format = DRM_FORMAT_P012,.depth = 0, .num_planes =
>>>>>> 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>>>>> +{ .format = DRM_FORMAT_P016,.depth = 0, .num_planes =
>>>>>> 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>>>>> };
>>>>>>
>>>>>> unsigned int i;
>>>>>> diff --git a/include/uapi/drm/drm_fourcc.h
>>>>>> b/include/uapi/drm/drm_fourcc.h
>>>>>> index ef20abb..306f979 100644
>>>>>> --- a/include/uapi/drm/drm_fourcc.h
>>>>>> +++ b/include/uapi/drm/drm_fourcc.h
>>>>>> @@ -128,6 +128,27 @@ extern "C" {
>>>>>> #define DRM_FORMAT_NV42fourcc_code('N', 'V', '4', '2') /*
>>>>>> non-subsampled Cb:Cr plane */
>>>>>>
>>>>>> /*
>>>>>> + * 2 plane YCbCr MSB aligned
>>>>>> + * index 0 = Y plane, [15:0] Y:x [10:6] little endian
>>>>>> + * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [10:6:10:6] little endian
>>>>>> + */
>>>>>> +#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
>>>>>> subsampled Cb:Cr plane 10 bits per channel */
>>>>>> +
>>>>>> +/*
>>>>>> + * 2 plane YCbCr MSB aligned
>>>>>> + * index 0 = Y plane, [15:0] Y:x [12:4] little endian
>>>>>> + * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [12:4:12:4] little endian
>>>>>> + */
>>>>>> +#define DRM_FORMAT_P012fourcc_code('P', '0', '1', '2') /* 2x2
>>>>>> subsampled Cb:Cr plane 12 bits per channel */
>>>>>> +
>>>>>> +/*
>>>>>> + * 2 plane YCbCr MSB aligned
>>>>>> + * index 0 = Y plane, [15:0] Y little endian
>>>>>> + * index 1 = Cb:Cr plane, [31:0] Cb:Cr [16:16] little endian
>>>>>> + */
>>>>>> +#define DRM_FORMAT_P016fourcc_code('P', '0', '1', '6') /* 2x2
>>>>>> subsampled Cb:Cr plane 16 bits per channel */
>>>>>> +
>>>>>> +/*
>>>>>> * 3 plane YCbCr
>>>>>> * index 0: Y plane, [7:0] Y
>>>>>> * index 1: Cb plane, [7:0] Cb
>>>>>> --
>>>>>> 2.7.4
>>>>>
>>>>> --
>>>>> Ville Syrjälä
>>>>> Intel OTC
>>>
>>> --
>>> Ville Syrjälä
>>> Intel OTC
>>
>> ___
>> dri-devel mailing list
>> dri-de...@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH v6 1/3] drm_fourcc: Add new P010, P016 video format
從我的 iPad 傳送
> Ville Syrjälä 於 2017年3月7日 上午2:34 寫道：
>
>> On Tue, Mar 07, 2017 at 01:58:23AM +0800, Ayaka wrote:
>>
>>
>> 從我的 iPad 傳送
>>
>>>> Ville Syrjälä 於 2017年3月6日 下午9:06 寫道：
>>>>
>>>> On Sun, Mar 05, 2017 at 06:00:31PM +0800, Randy Li wrote:
>>>> P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
>>>> per channel video format.
>>>>
>>>> P016 is a planar 4:2:0 YUV with interleaved UV plane, 16 bits
>>>> per channel video format.
>>>>
>>>> V3: Added P012 and fixed cpp for P010
>>>> V4: format definition refined per review
>>>> V5: Format comment block for each new pixel format
>>>> V6: reversed Cb/Cr order in comments
>>>> v7: reversed Cb/Cr order in comments of header files, remove
>>>> the wrong part of commit message.
>>>
>>> What? Why? You just undid what Clint did in v6.
>> He missed a file also keeping the wrong description of rockchip.
>
> I don't follow. Who missed what exactly?
What he sent is v5, I increase the order number twice in the message, it
confuse me as well.
I think Clint forgot the include/uapi/drm/drm_fourcc.h .
>
>
>>>
>>>>
>>>> Cc: Daniel Stone
>>>> Cc: Ville Syrjälä
>>>>
>>>> Signed-off-by: Randy Li
>>>> Signed-off-by: Clint Taylor
>>>> ---
>>>> drivers/gpu/drm/drm_fourcc.c | 3 +++
>>>> include/uapi/drm/drm_fourcc.h | 21 +
>>>> 2 files changed, 24 insertions(+)
>>>>
>>>> diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
>>>> index 90d2cc8..3e0fd58 100644
>>>> --- a/drivers/gpu/drm/drm_fourcc.c
>>>> +++ b/drivers/gpu/drm/drm_fourcc.c
>>>> @@ -165,6 +165,9 @@ const struct drm_format_info *__drm_format_info(u32
>>>> format)
>>>> { .format = DRM_FORMAT_UYVY,.depth = 0, .num_planes = 1,
>>>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>>> { .format = DRM_FORMAT_VYUY,.depth = 0, .num_planes = 1,
>>>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>>> { .format = DRM_FORMAT_AYUV,.depth = 0, .num_planes = 1,
>>>> .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
>>>> +{ .format = DRM_FORMAT_P010,.depth = 0, .num_planes = 2,
>>>> .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>>> +{ .format = DRM_FORMAT_P012,.depth = 0, .num_planes = 2,
>>>> .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>>> +{ .format = DRM_FORMAT_P016,.depth = 0, .num_planes = 2,
>>>> .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>>> };
>>>>
>>>> unsigned int i;
>>>> diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
>>>> index ef20abb..306f979 100644
>>>> --- a/include/uapi/drm/drm_fourcc.h
>>>> +++ b/include/uapi/drm/drm_fourcc.h
>>>> @@ -128,6 +128,27 @@ extern "C" {
>>>> #define DRM_FORMAT_NV42fourcc_code('N', 'V', '4', '2') /*
>>>> non-subsampled Cb:Cr plane */
>>>>
>>>> /*
>>>> + * 2 plane YCbCr MSB aligned
>>>> + * index 0 = Y plane, [15:0] Y:x [10:6] little endian
>>>> + * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [10:6:10:6] little endian
>>>> + */
>>>> +#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
>>>> subsampled Cb:Cr plane 10 bits per channel */
>>>> +
>>>> +/*
>>>> + * 2 plane YCbCr MSB aligned
>>>> + * index 0 = Y plane, [15:0] Y:x [12:4] little endian
>>>> + * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [12:4:12:4] little endian
>>>> + */
>>>> +#define DRM_FORMAT_P012fourcc_code('P', '0', '1', '2') /* 2x2
>>>> subsampled Cb:Cr plane 12 bits per channel */
>>>> +
>>>> +/*
>>>> + * 2 plane YCbCr MSB aligned
>>>> + * index 0 = Y plane, [15:0] Y little endian
>>>> + * index 1 = Cb:Cr plane, [31:0] Cb:Cr [16:16] little endian
>>>> + */
>>>> +#define DRM_FORMAT_P016fourcc_code('P', '0', '1', '6') /* 2x2
>>>> subsampled Cb:Cr plane 16 bits per channel */
>>>> +
>>>> +/*
>>>> * 3 plane YCbCr
>>>> * index 0: Y plane, [7:0] Y
>>>> * index 1: Cb plane, [7:0] Cb
>>>> --
>>>> 2.7.4
>>>
>>> --
>>> Ville Syrjälä
>>> Intel OTC
>
> --
> Ville Syrjälä
> Intel OTC
Re: [PATCH v6 1/3] drm_fourcc: Add new P010, P016 video format
從我的 iPad 傳送
> Ville Syrjälä 於 2017年3月6日 下午9:06 寫道：
>
>> On Sun, Mar 05, 2017 at 06:00:31PM +0800, Randy Li wrote:
>> P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
>> per channel video format.
>>
>> P016 is a planar 4:2:0 YUV with interleaved UV plane, 16 bits
>> per channel video format.
>>
>> V3: Added P012 and fixed cpp for P010
>> V4: format definition refined per review
>> V5: Format comment block for each new pixel format
>> V6: reversed Cb/Cr order in comments
>> v7: reversed Cb/Cr order in comments of header files, remove
>> the wrong part of commit message.
>
> What? Why? You just undid what Clint did in v6.
He missed a file also keeping the wrong description of rockchip.
>
>>
>> Cc: Daniel Stone
>> Cc: Ville Syrjälä
>>
>> Signed-off-by: Randy Li
>> Signed-off-by: Clint Taylor
>> ---
>> drivers/gpu/drm/drm_fourcc.c | 3 +++
>> include/uapi/drm/drm_fourcc.h | 21 +
>> 2 files changed, 24 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
>> index 90d2cc8..3e0fd58 100644
>> --- a/drivers/gpu/drm/drm_fourcc.c
>> +++ b/drivers/gpu/drm/drm_fourcc.c
>> @@ -165,6 +165,9 @@ const struct drm_format_info *__drm_format_info(u32
>> format)
>>{ .format = DRM_FORMAT_UYVY,.depth = 0, .num_planes = 1,
>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>{ .format = DRM_FORMAT_VYUY,.depth = 0, .num_planes = 1,
>> .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>{ .format = DRM_FORMAT_AYUV,.depth = 0, .num_planes = 1,
>> .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
>> +{ .format = DRM_FORMAT_P010,.depth = 0, .num_planes = 2,
>> .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>> +{ .format = DRM_FORMAT_P012,.depth = 0, .num_planes = 2,
>> .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>> +{ .format = DRM_FORMAT_P016,.depth = 0, .num_planes = 2,
>> .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
>>};
>>
>>unsigned int i;
>> diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
>> index ef20abb..306f979 100644
>> --- a/include/uapi/drm/drm_fourcc.h
>> +++ b/include/uapi/drm/drm_fourcc.h
>> @@ -128,6 +128,27 @@ extern "C" {
>> #define DRM_FORMAT_NV42fourcc_code('N', 'V', '4', '2') /*
>> non-subsampled Cb:Cr plane */
>>
>> /*
>> + * 2 plane YCbCr MSB aligned
>> + * index 0 = Y plane, [15:0] Y:x [10:6] little endian
>> + * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [10:6:10:6] little endian
>> + */
>> +#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
>> subsampled Cb:Cr plane 10 bits per channel */
>> +
>> +/*
>> + * 2 plane YCbCr MSB aligned
>> + * index 0 = Y plane, [15:0] Y:x [12:4] little endian
>> + * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [12:4:12:4] little endian
>> + */
>> +#define DRM_FORMAT_P012fourcc_code('P', '0', '1', '2') /* 2x2
>> subsampled Cb:Cr plane 12 bits per channel */
>> +
>> +/*
>> + * 2 plane YCbCr MSB aligned
>> + * index 0 = Y plane, [15:0] Y little endian
>> + * index 1 = Cb:Cr plane, [31:0] Cb:Cr [16:16] little endian
>> + */
>> +#define DRM_FORMAT_P016fourcc_code('P', '0', '1', '6') /* 2x2
>> subsampled Cb:Cr plane 16 bits per channel */
>> +
>> +/*
>> * 3 plane YCbCr
>> * index 0: Y plane, [7:0] Y
>> * index 1: Cb plane, [7:0] Cb
>> --
>> 2.7.4
>
> --
> Ville Syrjälä
> Intel OTC
Re: [PATCH v2 2/2] [media] v4l: Add 10/16-bits per channel YUV pixel formats
從我的 iPad 傳送 > Mauro Carvalho Chehab 於 2017年2月3日 下午10:04 寫道： > > Em Thu, 5 Jan 2017 20:27:17 +0200 > Sakari Ailus escreveu: > >> Hi Randy, >> >>> On Thu, Jan 05, 2017 at 11:22:26PM +0800, ayaka wrote: >>> >>> >>>> On 01/05/2017 06:30 PM, Sakari Ailus wrote: >>>> Hi Randy, >>>> >>>> Thanks for the update. >>>> >>>>> On Thu, Jan 05, 2017 at 12:29:11AM +0800, Randy Li wrote: >>>>> The formats added by this patch are: >>>>>V4L2_PIX_FMT_P010 >>>>>V4L2_PIX_FMT_P010M >>>>>V4L2_PIX_FMT_P016 >>>>>V4L2_PIX_FMT_P016M >>>>> Currently, none of driver uses those format, but some video device >>>>> has been confirmed with could as those format for video output. >>>>> The Rockchip's new decoder has supported those 10 bits format for >>>>> profile_10 HEVC/AVC video. >>>>> >>>>> Signed-off-by: Randy Li >>>>> >>>>> v4l2 >>>>> --- >>>>> Documentation/media/uapi/v4l/pixfmt-p010.rst | 86 >>>>> Documentation/media/uapi/v4l/pixfmt-p010m.rst | 94 ++ >>>>> Documentation/media/uapi/v4l/pixfmt-p016.rst | 126 >>>>> >>>>> Documentation/media/uapi/v4l/pixfmt-p016m.rst | 136 >>>>> ++ >>>> You need to include the formats in pixfmt.rst in order to compile the >>>> documentation. >>>> >>>> $ make htmldocs >>>> >>>> And you'll find it in Documentation/output/media/uapi/v4l/v4l2.html . >>>> >>>> In Debian you'll need to install sphinx-common and python3-sphinx-rtd-theme >>>> . >>> OK, I would fix them in new version. >>> The view of byte order for P010 serial is left empty, it is a little hard >>> for me to use flat-table to draw them. Is there possible to use something >>> like latex to do this job? >> >> Hmm. Not as far as I know. We recently switched from DocBook mostly due to >> ReST being more simple to use AFAIU. I think LaTeX output could be produced >> ReST, that might not be very helpful here though. > > No, you can't use LaTeX, as it won't be properly displayed on all output > formats. There are a few options to define tables in ReST, but we prefer > using flat-table because the other formats are harder to maintain at the > V4L2 uAPI documentation. > > Just one note about this series: it won't be merged upstream until > someone adds a driver needing those pixel formats. > I made a mistake, the pixel format I want is not P010, I would post a new patch for that. > Regards, > Mauro > > > Thanks, > Mauro -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Request API: stateless VPU: the buffer mechanism and DPB management
On 01/17/2017 10:59 PM, Nicolas Dufresne wrote: Le mardi 17 janvier 2017 à 20:46 +0800, herman.c...@rock-chips.com a écrit : If we move parser or part of DPB management mechanism into kernel we will face a issue as follows: One customer requires dpb management do a flush when stream occurs in order to keep output frame clean. While another one requires output frame with error to keep output frame smooth. And when only one field has a error one customer wants to do a simple field copy to recover. The driver should send all frames and simply mark the corrupted frames using V4L2_BUF_FLAG_ERROR. This way, the userspace can then make their own decision. It is also important to keep track and cleanup the buffers meta's (which are application specific). If the driver silently drops frame, it makes that management much harder. About flushing and draining operation, they are respectively signalled to the driver using STREAMOFF and CMD_STOP. These are some operation related to strategy rather then mechanism. I think it is not a good idea to bring such kind of flexible process to kernel driver. So here is the ultimate challenge that how to reasonably move the parser and flexible process which is encapsuled in firmware to a userspace - kernel stateless driver model. Moving the parsers in the kernel (on the main CPU) is not acceptable. No, what is not what I said. What I want to do is "The whole plan in userspace is just injecting a parsing operation and set those v4l2 control in kernel before enqueue a buffer into OUTPUT, which would keep the most compatible with those stateful video IP(those with a firmware). " This is too much of a security threat. Userspace should parse the data into structures, doing any validation required before end. My main question and that should have an impact decision, is if those structures can be made generic. PDB handling is not that trivial (my reference is VAAPI here, maybe they are doing it wrong) and with driver specific structures, we would have this code copy-pasted over and over. So with driver specific structures, it's probably better to keep all the parsing and reordering logic outside (hence together). The result to keep DPB and re-order inside the kernel is want to be compatible with the original buffer operation. Anyway the DPB structure could be common. That remains, that some driver will deal with reordering on the firmware side (even the if they don't parse), hence we need to take this into consideration. No sure what do you mean, I think all those driver with firmware would do that. regards, Nicolas Hello all: I have recently finish the learning of the H.264 codec and ready to write the driver. Although I have not get deep in syntax of H.264 but I think I just need to reuse and extended the VA-API H264 Parser from gstreamer. The whole plan in userspace is just injecting a parsing operation and set those v4l2 control in kernel before enqueue a buffer into OUTPUT, which would keep the most compatible with those stateful video IP(those with a firmware). But in order to do that, I can't avoid the management of DPB. I decided to moving the DPB management job from userspace in kernel. Also the video IP(On2 on rk3288 and the transition video IP on those future SoC than rk3288, rkv don't have this problem) would a special way to manage the DPB, which requests the same reference frame is storing in the same reference index in the runtime(actually it is its Motion Vector data appended in decoded YUV data would not be moved). I would suggest to keep those job in kernel, the userspace just to need update the list0 and list1 of DPB. DPB is self managed in kernel the userspace don't need to even dequeue the buffer from CAPTURE until the re-order is done. The kernel driver would also re-order the CAPTURE buffer into display order, and blocking the operation on CAPTURE until a buffer is ready to place in the very display order. If I don't do that, I have to get the buffer once it is decoded, and marking its result with the poc, I could only begin the processing of the next frame only after those thing are done. Which would effect the performance badly. That is what chromebook did(I hear that from the other staff, I didn't get invoke in chromium project yet). So I would suggest that doing the re-order job in kernel, and inform the the userspace the buffers are ready when the new I frame(key frame) is pushed into the video IP. Although moving those job into kernel would increase the loading, but I think it is worth to do that, but I don't know whether all those thought are correct and high performance(It is more important than API compatible especially for those 4K video). And I want to know more ideas about this topic. I would begin the writing the new driver after the coming culture new year vacation(I would go to the Europe), I wish we can decide the final mechanism before I begin this job. -- To unsubscribe from this list
Re: [PATCH v2 2/2] [media] v4l: Add 10/16-bits per channel YUV pixel formats
On 01/05/2017 06:30 PM, Sakari Ailus wrote: Hi Randy, Thanks for the update. On Thu, Jan 05, 2017 at 12:29:11AM +0800, Randy Li wrote: The formats added by this patch are: V4L2_PIX_FMT_P010 V4L2_PIX_FMT_P010M V4L2_PIX_FMT_P016 V4L2_PIX_FMT_P016M Currently, none of driver uses those format, but some video device has been confirmed with could as those format for video output. The Rockchip's new decoder has supported those 10 bits format for profile_10 HEVC/AVC video. Signed-off-by: Randy Li v4l2 --- Documentation/media/uapi/v4l/pixfmt-p010.rst | 86 Documentation/media/uapi/v4l/pixfmt-p010m.rst | 94 ++ Documentation/media/uapi/v4l/pixfmt-p016.rst | 126 Documentation/media/uapi/v4l/pixfmt-p016m.rst | 136 ++ You need to include the formats in pixfmt.rst in order to compile the documentation. $ make htmldocs And you'll find it in Documentation/output/media/uapi/v4l/v4l2.html . In Debian you'll need to install sphinx-common and python3-sphinx-rtd-theme . OK, I would fix them in new version. The view of byte order for P010 serial is left empty, it is a little hard for me to use flat-table to draw them. Is there possible to use something like latex to do this job? Regarding P010 and the rest --- I'm fine with that, considering also that NV12 was never a great name for a format... -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/2] drm_fourcc: Add new P010, P016 video format
從我的 iPad 傳送
> Daniel Stone 於 2017年1月5日 上午1:02 寫道：
>
> Hi Randy,
>
>> On 4 January 2017 at 16:29, Randy Li wrote:
>> index 90d2cc8..23c8e99 100644
>> --- a/drivers/gpu/drm/drm_fourcc.c
>> +++ b/drivers/gpu/drm/drm_fourcc.c
>> @@ -165,6 +165,9 @@ const struct drm_format_info *__drm_format_info(u32
>> format)
>>{ .format = DRM_FORMAT_UYVY,.depth = 0,
>> .num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>{ .format = DRM_FORMAT_VYUY,.depth = 0,
>> .num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
>>{ .format = DRM_FORMAT_AYUV,.depth = 0,
>> .num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
>> + /* FIXME a pixel in Y for P010 is 10 bits */
>> + { .format = DRM_FORMAT_P010,.depth = 0,
>> .num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
>
> It seems like P010 stores each Y component in a 16-bit value, with the
> bottom 6 bits ignored. So I think cpp here should be 2.
No, the rest bits are used to store the next pixel. The P010 is just a 10 bits
color depth pixel format.
>
> Cheers,
> Daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] drm_fourcc: Add new P010 video format
On 01/04/2017 11:56 PM, Ville Syrjälä wrote:
On Mon, Jan 02, 2017 at 04:50:03PM +0800, Randy Li wrote:
P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
per channel video format. Rockchip's vop support this
video format(little endian only) as the input video format.
Signed-off-by: Randy Li
---
include/uapi/drm/drm_fourcc.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 9e1bb7f..d2721da 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -119,6 +119,7 @@ extern "C" {
#define DRM_FORMAT_NV61 fourcc_code('N', 'V', '6', '1') /* 2x1
subsampled Cb:Cr plane */
#define DRM_FORMAT_NV24 fourcc_code('N', 'V', '2', '4') /*
non-subsampled Cr:Cb plane */
#define DRM_FORMAT_NV42 fourcc_code('N', 'V', '4', '2') /*
non-subsampled Cb:Cr plane */
+#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
subsampled Cr:Cb plane 10 bits per channel */
We could use a better description of the format here. IIRC there is
10bits of actual data contained in each 16bits. So there should be a
proper comment explaning in which way the bits are stored.
It is a little hard to describe P010, which leaves a problem cpp
information in the new patches.
Also I have no idea how to draw the byte-order table the rst document
for v4l2.
/*
* 3 plane YCbCr
--
2.7.4
___
dri-devel mailing list
dri-de...@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] [media] v4l: Add 10-bits per channel YUV pixel formats
On 01/02/2017 07:07 PM, Sakari Ailus wrote:
Hi,
On Mon, Jan 02, 2017 at 06:53:16PM +0800, ayaka wrote:
On 01/02/2017 05:10 PM, Sakari Ailus wrote:
Hi Randy,
Thanks for the patch.
On Mon, Jan 02, 2017 at 04:50:04PM +0800, Randy Li wrote:
The formats added by this patch are:
V4L2_PIX_FMT_P010
V4L2_PIX_FMT_P010M
Currently, none of driver uses those format, but some video device
has been confirmed with could as those format for video output.
The Rockchip's new decoder has supported those format for profile_10
HEVC/AVC video.
Signed-off-by: Randy Li
If the format resembles the existing formats but on a different bit depth,
it should be named in similar fashion.
Do you mean it would be better if it is called as NV12_10?
If it otherwise resembles NV12 but just has 10 bits per pixel, I think
NV12_10 is a good name for it.
The main reason I don't like it is that there is a various of software
having used the P010 for this kind of pixel format. It would leadi a
conflict between them(and I never saw it is used as NV12_10), as the
P010 is more common to be used.
I left a problem unsolved for P010 in v4l2, P010 have two variant,
little endian and big endian. Which could be easy identified in drm
driver(there is a flag for all the big endian pixel format).
Could you also add ReST documentation for the format, please?
I will.
The common requirement for merging patches that change interfaces has been
that there's a user for that change. It'll still help you to get this
The kernel used in rockchip has supported that format in drm driver, but
just we don't have a agreement about the pixel format. As the Gstreamer and
some others would call it with a P010_ prefix, but Mark(rockchip's drm
author) prefer the something like NV12_10, that is why I sent out those
patches, I want the upstream decided its final name.
Ack.
I think we haven't really tried to unify the format naming in the past
between the two subsystems. If existing conventions exist on both regarding
this format, then it's probably better to follow those.
Cc Laurent as well.
reviewed now so the interface that the future hopefully-in-mainline driver
provides will not change.
---
include/uapi/linux/videodev2.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 46e8a2e3..9e03f20 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -551,6 +551,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV61v4l2_fourcc('N', 'V', '6', '1') /* 16 Y/CrCb
4:2:2 */
#define V4L2_PIX_FMT_NV24v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr
4:4:4 */
#define V4L2_PIX_FMT_NV42v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb
4:4:4 */
+#define V4L2_PIX_FMT_P010v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr
4:2:0, 10 bits per channel */
/* two non contiguous planes - one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr
4:2:0 */
@@ -559,6 +560,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV61M v4l2_fourcc('N', 'M', '6', '1') /* 16 Y/CrCb
4:2:2 */
#define V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr
4:2:0 64x32 macroblocks */
#define V4L2_PIX_FMT_NV12MT_16X16 v4l2_fourcc('V', 'M', '1', '2') /* 12
Y/CbCr 4:2:0 16x16 macroblocks */
+#define V4L2_PIX_FMT_P010M v4l2_fourcc('P', 'M', '1', '0') /* 15 Y/CbCr
4:2:0, 10 bits per channel */
/* three planes - Y Cb, Cr */
#define V4L2_PIX_FMT_YUV410 v4l2_fourcc('Y', 'U', 'V', '9') /* 9 YUV 4:1:0
*/
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] [media] v4l: Add 10-bits per channel YUV pixel formats
On 01/02/2017 05:10 PM, Sakari Ailus wrote:
Hi Randy,
Thanks for the patch.
On Mon, Jan 02, 2017 at 04:50:04PM +0800, Randy Li wrote:
The formats added by this patch are:
V4L2_PIX_FMT_P010
V4L2_PIX_FMT_P010M
Currently, none of driver uses those format, but some video device
has been confirmed with could as those format for video output.
The Rockchip's new decoder has supported those format for profile_10
HEVC/AVC video.
Signed-off-by: Randy Li
If the format resembles the existing formats but on a different bit depth,
it should be named in similar fashion.
Do you mean it would be better if it is called as NV12_10?
Could you also add ReST documentation for the format, please?
I will.
The common requirement for merging patches that change interfaces has been
that there's a user for that change. It'll still help you to get this
The kernel used in rockchip has supported that format in drm driver, but
just we don't have a agreement about the pixel format. As the Gstreamer
and some others would call it with a P010_ prefix, but Mark(rockchip's
drm author) prefer the something like NV12_10, that is why I sent out
those patches, I want the upstream decided its final name.
reviewed now so the interface that the future hopefully-in-mainline driver
provides will not change.
---
include/uapi/linux/videodev2.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 46e8a2e3..9e03f20 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -551,6 +551,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV61v4l2_fourcc('N', 'V', '6', '1') /* 16 Y/CrCb
4:2:2 */
#define V4L2_PIX_FMT_NV24v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr
4:4:4 */
#define V4L2_PIX_FMT_NV42v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb
4:4:4 */
+#define V4L2_PIX_FMT_P010v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr
4:2:0, 10 bits per channel */
/* two non contiguous planes - one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr
4:2:0 */
@@ -559,6 +560,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV61M v4l2_fourcc('N', 'M', '6', '1') /* 16 Y/CrCb
4:2:2 */
#define V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr
4:2:0 64x32 macroblocks */
#define V4L2_PIX_FMT_NV12MT_16X16 v4l2_fourcc('V', 'M', '1', '2') /* 12
Y/CbCr 4:2:0 16x16 macroblocks */
+#define V4L2_PIX_FMT_P010M v4l2_fourcc('P', 'M', '1', '0') /* 15 Y/CbCr
4:2:0, 10 bits per channel */
/* three planes - Y Cb, Cr */
#define V4L2_PIX_FMT_YUV410 v4l2_fourcc('Y', 'U', 'V', '9') /* 9 YUV 4:1:0
*/
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: gstreamer: v4l2videodec plugin
在 2016年05月15日 15:41, Nicolas Dufresne 写道: Le vendredi 13 mai 2016 à 11:45 +0300, Stanimir Varbanov a écrit : One thing which bothers me is how the extra-controls property working, i.e. I failed to change the h264 profile for example with below construct: extra-controls="controls,h264_profile=4;" Yes, and profile should be negotiated with downstream in GStreamer. For common controls, like bitrate, it should be exposed as separate properties instead. Please try the new patches in https://github.com/hizukiayaka/gst-plugins-good And look at this commit https://github.com/hizukiayaka/gst-plugins-good/commit/92b99ba9322cf8a1039b877315b12bc9813e95cf NOTE: even you could set those extra controls and driver accepted. It doesn't mean it would work. I looked at the s5p-mfc driver, it just set it then leave it alone. I may fix this bug in the few weeks. Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] [media] s5p-mfc: remove unnecessary check in try_fmt
We don't need to request the sizeimage or num_planes
in try_fmt.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 9 -
1 file changed, 9 deletions(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index a66a9f9..2f76aba 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -1043,10 +1043,6 @@ static int vidioc_try_fmt(struct file *file, void *priv,
struct v4l2_format *f)
mfc_err("failed to try output format\n");
return -EINVAL;
}
- if (pix_fmt_mp->plane_fmt[0].sizeimage == 0) {
- mfc_err("must be set encoding output size\n");
- return -EINVAL;
- }
if ((dev->variant->version_bit & fmt->versions) == 0) {
mfc_err("Unsupported format by this MFC version.\n");
return -EINVAL;
@@ -1060,11 +1056,6 @@ static int vidioc_try_fmt(struct file *file, void *priv,
struct v4l2_format *f)
mfc_err("failed to try output format\n");
return -EINVAL;
}
-
- if (fmt->num_planes != pix_fmt_mp->num_planes) {
- mfc_err("failed to try output format\n");
- return -EINVAL;
- }
if ((dev->variant->version_bit & fmt->versions) == 0) {
mfc_err("Unsupported format by this MFC version.\n");
return -EINVAL;
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3] [media] s5p-mfc: fix a typo in s5p_mfc_dec
It is a cosmetic commit.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index f2d6376..391ed9c 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -573,7 +573,7 @@ static int vidioc_reqbufs(struct file *file, void *priv,
struct s5p_mfc_ctx *ctx = fh_to_ctx(priv);
if (reqbufs->memory != V4L2_MEMORY_MMAP) {
- mfc_err("Only V4L2_MEMORY_MAP is supported\n");
+ mfc_err("Only V4L2_MEMORY_MMAP is supported\n");
return -EINVAL;
}
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RESEND PATCH] [PATCH 0/3] A few patches make s5p-mfc work with Gstreamer
A user-space application will follow the V4l2 document, but I meet a problem when I test it with Gstreamer. I look at the V4l2 document, the driver check the external setting which the document not cover, so that is what the second patch do. The first patch comes from chromium project, but I didn't find it is merged, so I sent it again with new kernel API. But I have not test it with 4.6-rc6, as I meet sysmmu problem there. But all those patches have been confirmed at Linux 4.1.16. ayaka (3): [media] s5p-mfc: Add handling of buffer freeing reqbufs request [media] s5p-mfc: remove unnecessary check in try_fmt [media] s5p-mfc: fix a typo in s5p_mfc_dec drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 2 +- drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 12 +++- 2 files changed, 4 insertions(+), 10 deletions(-) -- 2.5.5 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3] [media] s5p-mfc: Add handling of buffer freeing reqbufs request
The encoder forget the work to call hardware to release its buffers.
This patch came from chromium project. I just change its code
style and make the API match with new kernel.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index 034b5c1..a66a9f9 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -1144,7 +1144,10 @@ static int vidioc_reqbufs(struct file *file, void *priv,
return -EINVAL;
if (reqbufs->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
if (reqbufs->count == 0) {
+ mfc_debug(2, "Freeing buffers\n");
ret = vb2_reqbufs(&ctx->vq_dst, reqbufs);
+ s5p_mfc_hw_call(dev->mfc_ops, release_codec_buffers,
+ ctx);
ctx->capture_state = QUEUE_FREE;
return ret;
}
--
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RESEND PATCH] [media] s5p-mfc: don't close instance after free OUTPUT buffers
Please also notice those patches (A few patches make s5p-mfc work with
Gstreamer)
I resent to mail recently.
Without them, the encoder won't work with Gstreamer either. I hope it
will be
merged as soon as possible.
在 2016年05月07日 06:11, Javier Martinez Canillas 写道:
From: ayaka
User-space applications can use the VIDIOC_REQBUFS ioctl to determine if a
memory mapped, user pointer or DMABUF based I/O is supported by the driver.
So a set of VIDIOC_REQBUFS ioctl calls will be made with count 0 and then
the real VIDIOC_REQBUFS call with count == n. But for count 0, the driver
not only frees the buffer but also closes the MFC instance and s5p_mfc_ctx
state is set to MFCINST_FREE.
The VIDIOC_REQBUFS handler for the output device checks if the s5p_mfc_ctx
state is set to MFCINST_INIT (which happens on an VIDIOC_S_FMT) and fails
otherwise. So after a VIDIOC_REQBUFS(n), future VIDIOC_REQBUFS(n) calls
will fails unless a VIDIOC_S_FMT ioctl calls happens before the reqbufs.
But applications may first set the format and then attempt to determine
the I/O methods supported by the driver (for example Gstramer does it) so
the state won't be set to MFCINST_INIT again and VIDIOC_REQBUFS will fail.
To avoid this issue, only free the buffers on VIDIOC_REQBUFS(0) but don't
close the MFC instance to allow future VIDIOC_REQBUFS(n) calls to succeed.
Signed-off-by: ayaka
[javier: Rewrote changelog to explain the problem more detailed]
Signed-off-by: Javier Martinez Canillas
---
Hello,
This is a resend of a patch posted by Ayaka some time ago [0].
Without $SUBJECT, trying to decode a video using Gstramer fails
on an Exynos5422 Odroid XU4 board with following error message:
$ gst-launch-1.0 filesrc location=test.mov ! qtdemux ! h264parse !
v4l2video0dec ! videoconvert ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
[ 3947.114756] vidioc_reqbufs:576: Only V4L2_MEMORY_MAP is supported
[ 3947.114771] vidioc_reqbufs:576: Only V4L2_MEMORY_MAP is supported
[ 3947.114903] reqbufs_output:484: Reqbufs called in an invalid state
[ 3947.114913] reqbufs_output:510: Failed allocating buffers for OUTPUT queue
ERROR: from element /GstPipeline:pipeline0/v4l2video0dec:v4l2video0dec0: Failed
to allocate required memory.
Additional debug info:
gstv4l2videodec.c(575): gst_v4l2_video_dec_handle_frame ():
/GstPipeline:pipeline0/v4l2video0dec:v4l2video0dec0:
Buffer pool activation failed
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
[0]: https://patchwork.linuxtv.org/patch/32794/
Best regards,
Javier
drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index f2d6376ce618..8b9467de2d6a 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -474,7 +474,6 @@ static int reqbufs_output(struct s5p_mfc_dev *dev, struct
s5p_mfc_ctx *ctx,
ret = vb2_reqbufs(&ctx->vq_src, reqbufs);
if (ret)
goto out;
- s5p_mfc_close_mfc_inst(dev, ctx);
ctx->src_bufs_cnt = 0;
ctx->output_state = QUEUE_FREE;
} else if (ctx->output_state == QUEUE_FREE) {
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] s5p-mfc: correct the formats info for encoder
The NV12M is supported by all the version of MFC, so it is better
to use it as default OUTPUT format.
MFC v5 doesn't support NV21, I have tested it, for the SEC doc
it is not supported either.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index d26b248..4ea3796 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -32,7 +32,7 @@
#include "s5p_mfc_intr.h"
#include "s5p_mfc_opr.h"
-#define DEF_SRC_FMT_ENCV4L2_PIX_FMT_NV12MT
+#define DEF_SRC_FMT_ENCV4L2_PIX_FMT_NV12M
#define DEF_DST_FMT_ENCV4L2_PIX_FMT_H264
static struct s5p_mfc_fmt formats[] = {
@@ -67,8 +67,7 @@ static struct s5p_mfc_fmt formats[] = {
.codec_mode = S5P_MFC_CODEC_NONE,
.type = MFC_FMT_RAW,
.num_planes = 2,
- .versions = MFC_V5_BIT | MFC_V6_BIT | MFC_V7_BIT |
- MFC_V8_BIT,
+ .versions = MFC_V6_BIT | MFC_V7_BIT | MFC_V8_BIT,
},
{
.name = "H264 Encoded Stream",
--
1.9.3
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] s5p-mfc: correct the formats info for encoder
I am very sorry to forget the Sign-off again and thank you for reviewing. ayaka (1): s5p-mfc: correct the formats info for encoder drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- 1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] s5p-mfc: correct the formats info for encoder
The NV12M is supported by all the version of MFC, so it is better
to use it as default OUTPUT format.
MFC v5 doesn't support NV21, I have tested it, for the SEC doc
it is not supported either.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index d26b248..4ea3796 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -32,7 +32,7 @@
#include "s5p_mfc_intr.h"
#include "s5p_mfc_opr.h"
-#define DEF_SRC_FMT_ENCV4L2_PIX_FMT_NV12MT
+#define DEF_SRC_FMT_ENCV4L2_PIX_FMT_NV12M
#define DEF_DST_FMT_ENCV4L2_PIX_FMT_H264
static struct s5p_mfc_fmt formats[] = {
@@ -67,8 +67,7 @@ static struct s5p_mfc_fmt formats[] = {
.codec_mode = S5P_MFC_CODEC_NONE,
.type = MFC_FMT_RAW,
.num_planes = 2,
- .versions = MFC_V5_BIT | MFC_V6_BIT | MFC_V7_BIT |
- MFC_V8_BIT,
+ .versions = MFC_V6_BIT | MFC_V7_BIT | MFC_V8_BIT,
},
{
.name = "H264 Encoded Stream",
--
1.9.3
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] s5p-mfc: correct the formats info for encoder
I am very sorry to forget the Sign-off again and thank you for reviewing. ayaka (1): s5p-mfc: correct the formats info for encoder drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- 1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] s5p-mfc: correct the formats info for encoder
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 2014/09/23 00:28, Kamil Debski wrote: > Hi Ayaka, > > Sorry for such a late reply - I just noticed this patch. > >> The NV12M is supported by all the version of MFC, so it is better >> to use it as default OUTPUT format. MFC v5 doesn't support NV21, >> I have tested it, for the SEC doc it is not supported either. > > A proper Sign-off is missing here. > Sorry to miss it again. > According to the documentation of MFC v5 I have non-tiled format > is supported. Which documentation were you looking at? > But the V4L2_PIX_FMT_NV12MT is only supported by MFC_V5_BIT from your code, V4L2_PIX_FMT_NV12M is supported by all the version. >> From my documentation: > ++ ENC_MAP_FOR_CUR0xC51C Memory structure setting > register of the current frame.R/W 0x > > Bits NameDescription Reset Value [31:2] RESERVED > Reserved0 [1:0] > ENC_MAP_FOR_CUR Memory structure of the current frame 0 : Linear > mode 3 : 64x32 tiled mode 0 ++ > In the page 2277. The same result. I think the V4L2_PIX_FMT_NV12MT is 64x32 Tiles mode, but what I remove for MFC v5 is V4L2_PIX_FMT_NV21M. > Best wishes, > - -- ayaka -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJUIFNLAAoJEPsGh4kgR4i5HcIP/2qwn7uIFq66qpSajXmtcLx3 3/wt8n26u6GhTMUnIJKZS07FGtv7qizUVqeY3WmfWQw3jLaUjZeVviVH08y3DrE8 +7Vjq2rxz57ou4bBtc4qIgTWB7z2yuVSpBOYUB94laItQ7KDap4EgLf89m4KaKTt 5nULR0byxXh+RuUOw80v0eP/TBz7SRfYZnulASV9QlGS6T3Xp6v4U6W8LbSbieR5 63PwPxYP7aDVb5R6qzaLIVXNuI53vn5VhrQ6JJUfKee5YSbkV/Ff6XK+7/P162Pn 5cVt06X+RUeZXHGqCroMNb9cdm+7JHOZL458NPn4NmTJnFcPNu6JzW9iLymHeHC8 iFmNhpDuHJBulKsW44lqKe1fHT22a5C/oJAI1ZS9c3yrH+TqHkfEkUJjglSRByzj ptTFFZVTCdiL5VwnDlfowR4ZzrkZuoWzHIn5cGeHogvbLbxCbtV67+IFpWlXfyJu rKnCI+DKYb5cjEiHm7kzGbAO04AfNMT79sNwrD+sPuvnaFyRiy2rjKv3ubnPFRVp 3agNRzAcCgmsW3K10P3ism4ceJUqeZtFvieCQrjiQdxj8EB7QAcgOhgn3K//zrQ1 mQP7xuVQcwpaRIOx/3jSlVWYFrkFs2+tmgS9oEn+v40gXOQXk8rML21gHvpDuCnf qJXx0UVYQRV7Bhgv8EFW =SNTL -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] fix enum_fmt for s5p-mfc
The patch remove the non-multiplanar api for enum_fmt in s5p-mfc. Now enum_fmt in the driver will only work with multiplanar. I think it is a bug, hverkuil think it is too. so I made this patch. Actually gstreamer doesn't care about it, it will merge the result from non-multiplanar and multiplanar. ayaka (1): media: fix enum_fmt for s5p-mfc drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 24 +++- drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 24 +++- 2 files changed, 6 insertions(+), 42 deletions(-) -- 1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] media: fix enum_fmt for s5p-mfc
As the s5p-mfc is a driver which use multiplanar api, so the
vidioc_enum_fmt_vid serial of ioctl should only for
multiplanar, non-multiplanar shouldn't be implemented at all.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 24 +++-
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 24 +++-
2 files changed, 6 insertions(+), 42 deletions(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index 4d93835..6611a7a 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -283,17 +283,13 @@ static int vidioc_querycap(struct file *file, void *priv,
/* Enumerate format */
static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
- bool mplane, bool out)
+ bool out)
{
struct s5p_mfc_dev *dev = video_drvdata(file);
struct s5p_mfc_fmt *fmt;
int i, j = 0;
for (i = 0; i < ARRAY_SIZE(formats); ++i) {
- if (mplane && formats[i].num_planes == 1)
- continue;
- else if (!mplane && formats[i].num_planes > 1)
- continue;
if (out && formats[i].type != MFC_FMT_DEC)
continue;
else if (!out && formats[i].type != MFC_FMT_RAW)
@@ -313,28 +309,16 @@ static int vidioc_enum_fmt(struct file *file, struct
v4l2_fmtdesc *f,
return 0;
}
-static int vidioc_enum_fmt_vid_cap(struct file *file, void *pirv,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, false);
-}
-
static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
struct v4l2_fmtdesc *f)
{
- return vidioc_enum_fmt(file, f, true, false);
-}
-
-static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, true);
+ return vidioc_enum_fmt(file, f, false);
}
static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *priv,
struct v4l2_fmtdesc *f)
{
- return vidioc_enum_fmt(file, f, true, true);
+ return vidioc_enum_fmt(file, f, true);
}
/* Get format */
@@ -878,9 +862,7 @@ static int vidioc_subscribe_event(struct v4l2_fh *fh,
/* v4l2_ioctl_ops */
static const struct v4l2_ioctl_ops s5p_mfc_dec_ioctl_ops = {
.vidioc_querycap = vidioc_querycap,
- .vidioc_enum_fmt_vid_cap = vidioc_enum_fmt_vid_cap,
.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
- .vidioc_enum_fmt_vid_out = vidioc_enum_fmt_vid_out,
.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
.vidioc_g_fmt_vid_cap_mplane = vidioc_g_fmt,
.vidioc_g_fmt_vid_out_mplane = vidioc_g_fmt,
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index 3abe468..4725a6f 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -953,17 +953,13 @@ static int vidioc_querycap(struct file *file, void *priv,
}
static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
- bool mplane, bool out)
+ bool out)
{
struct s5p_mfc_dev *dev = video_drvdata(file);
struct s5p_mfc_fmt *fmt;
int i, j = 0;
for (i = 0; i < ARRAY_SIZE(formats); ++i) {
- if (mplane && formats[i].num_planes == 1)
- continue;
- else if (!mplane && formats[i].num_planes > 1)
- continue;
if (out && formats[i].type != MFC_FMT_RAW)
continue;
else if (!out && formats[i].type != MFC_FMT_ENC)
@@ -983,28 +979,16 @@ static int vidioc_enum_fmt(struct file *file, struct
v4l2_fmtdesc *f,
return -EINVAL;
}
-static int vidioc_enum_fmt_vid_cap(struct file *file, void *pirv,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, false);
-}
-
static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
struct v4l2_fmtdesc *f)
{
- return vidioc_enum_fmt(file, f, true, false);
-}
-
-static int vidioc_enum_fmt_vid_out(struct file *file, void *prov,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, true);
+ return vidioc_enum_fmt(f
[PATCH] fix enum_fmt for s5p-mfc
The patch remove the non-multiplanar api for enum_fmt in s5p-mfc. Now enum_fmt in the driver will only work with multiplanar. I think it is a bug, hverkuil think it is too. so I made this patch. Actually gstreamer doesn't care about it, it will merge the result from non-multiplanar and multiplanar. ayaka (1): media: fix enum_fmt for s5p-mfc drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 24 +++- drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 24 +++- 2 files changed, 6 insertions(+), 42 deletions(-) -- 1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] media: fix enum_fmt for s5p-mfc
As the s5p-mfc is a driver which use multiplanar api, so the
vidioc_enum_fmt_vid serial of ioctl should only for
multiplanar, non-multiplanar shouldn't be implemented at all.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 24 +++-
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 24 +++-
2 files changed, 6 insertions(+), 42 deletions(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index 4d93835..6611a7a 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -283,17 +283,13 @@ static int vidioc_querycap(struct file *file, void *priv,
/* Enumerate format */
static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
- bool mplane, bool out)
+ bool out)
{
struct s5p_mfc_dev *dev = video_drvdata(file);
struct s5p_mfc_fmt *fmt;
int i, j = 0;
for (i = 0; i < ARRAY_SIZE(formats); ++i) {
- if (mplane && formats[i].num_planes == 1)
- continue;
- else if (!mplane && formats[i].num_planes > 1)
- continue;
if (out && formats[i].type != MFC_FMT_DEC)
continue;
else if (!out && formats[i].type != MFC_FMT_RAW)
@@ -313,28 +309,16 @@ static int vidioc_enum_fmt(struct file *file, struct
v4l2_fmtdesc *f,
return 0;
}
-static int vidioc_enum_fmt_vid_cap(struct file *file, void *pirv,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, false);
-}
-
static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
struct v4l2_fmtdesc *f)
{
- return vidioc_enum_fmt(file, f, true, false);
-}
-
-static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, true);
+ return vidioc_enum_fmt(file, f, false);
}
static int vidioc_enum_fmt_vid_out_mplane(struct file *file, void *priv,
struct v4l2_fmtdesc *f)
{
- return vidioc_enum_fmt(file, f, true, true);
+ return vidioc_enum_fmt(file, f, true);
}
/* Get format */
@@ -878,9 +862,7 @@ static int vidioc_subscribe_event(struct v4l2_fh *fh,
/* v4l2_ioctl_ops */
static const struct v4l2_ioctl_ops s5p_mfc_dec_ioctl_ops = {
.vidioc_querycap = vidioc_querycap,
- .vidioc_enum_fmt_vid_cap = vidioc_enum_fmt_vid_cap,
.vidioc_enum_fmt_vid_cap_mplane = vidioc_enum_fmt_vid_cap_mplane,
- .vidioc_enum_fmt_vid_out = vidioc_enum_fmt_vid_out,
.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out_mplane,
.vidioc_g_fmt_vid_cap_mplane = vidioc_g_fmt,
.vidioc_g_fmt_vid_out_mplane = vidioc_g_fmt,
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index 3abe468..4725a6f 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -953,17 +953,13 @@ static int vidioc_querycap(struct file *file, void *priv,
}
static int vidioc_enum_fmt(struct file *file, struct v4l2_fmtdesc *f,
- bool mplane, bool out)
+ bool out)
{
struct s5p_mfc_dev *dev = video_drvdata(file);
struct s5p_mfc_fmt *fmt;
int i, j = 0;
for (i = 0; i < ARRAY_SIZE(formats); ++i) {
- if (mplane && formats[i].num_planes == 1)
- continue;
- else if (!mplane && formats[i].num_planes > 1)
- continue;
if (out && formats[i].type != MFC_FMT_RAW)
continue;
else if (!out && formats[i].type != MFC_FMT_ENC)
@@ -983,28 +979,16 @@ static int vidioc_enum_fmt(struct file *file, struct
v4l2_fmtdesc *f,
return -EINVAL;
}
-static int vidioc_enum_fmt_vid_cap(struct file *file, void *pirv,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, false);
-}
-
static int vidioc_enum_fmt_vid_cap_mplane(struct file *file, void *pirv,
struct v4l2_fmtdesc *f)
{
- return vidioc_enum_fmt(file, f, true, false);
-}
-
-static int vidioc_enum_fmt_vid_out(struct file *file, void *prov,
- struct v4l2_fmtdesc *f)
-{
- return vidioc_enum_fmt(file, f, false, true);
+ return vidioc_enum_fmt(f
[PATCH] s5p-mfc: correct the formats info for encoder
The NV12M is supported by all the version of MFC, so it is better
to use it as default OUTPUT format.
MFC v5 doesn't support NV21, I have tested it, for the SEC doc
it is not supported either.
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index d26b248..4ea3796 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -32,7 +32,7 @@
#include "s5p_mfc_intr.h"
#include "s5p_mfc_opr.h"
-#define DEF_SRC_FMT_ENCV4L2_PIX_FMT_NV12MT
+#define DEF_SRC_FMT_ENCV4L2_PIX_FMT_NV12M
#define DEF_DST_FMT_ENCV4L2_PIX_FMT_H264
static struct s5p_mfc_fmt formats[] = {
@@ -67,8 +67,7 @@ static struct s5p_mfc_fmt formats[] = {
.codec_mode = S5P_MFC_CODEC_NONE,
.type = MFC_FMT_RAW,
.num_planes = 2,
- .versions = MFC_V5_BIT | MFC_V6_BIT | MFC_V7_BIT |
- MFC_V8_BIT,
+ .versions = MFC_V6_BIT | MFC_V7_BIT | MFC_V8_BIT,
},
{
.name = "H264 Encoded Stream",
--
1.9.3
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] correct formats info in s5p-mfc encoder
I have tested it in exynos 4412. I enable MFC and with 64MB buffer in echo bank. ayaka (1): s5p-mfc: correct the formats info for encoder drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- 1.9.3 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] s5p-mfc: encoder handles buffers freeing
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
于 2014年07月04日 12:13, Pawel Osciak 写道:
> Hi,
>
> On Fri, Jul 4, 2014 at 12:39 PM, ayaka mailto:ay...@soulik.info>> wrote:
>
> Add handling of buffer freeing reqbufs request to the encoder of
> s5p-mfc.
>
> Signed-off-by: ayaka mailto:ay...@soulik.info>>
>
>
> We've had a verified and tested fix for this in ChromeOS tree for some
time now, but haven't gotten to upstreaming it yet. It's a combination
of the below two diffs:
>
>
https://chromium-review.googlesource.com/#/c/65926/2/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
>
https://chromium-review.googlesource.com/#/c/66624/2/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
>
> Have you tested and verified your patch? Unless I'm looking at a
different base, it will not work, because:
Yes I have do a little test, but the patch was for gstreamer project, it
mainly used it on buffer allocation detecting, so I forget it again even
I have been told.
> - the call to reqbufs it too late and will not execute, because the check for
> state != QUEUE_FREE
above it will be true and you'll hit the early return;
> - it will also leak private buffer memory, as you can see in the above
patches, you also need to release_codec_buffers on the OUTPUT queue.
>
Please ignore my patch, I am so careless and making so much mistake and
I didn't know a project have done that. Thank you for telling me that.
> Thanks,
> Pawel
>
>
>
> ---
> drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 10 ++
> 1 file changed, 10 insertions(+)
>
> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> index d26b248..74fb80b 100644
> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> @@ -1166,6 +1166,11 @@ static int vidioc_reqbufs(struct file
*file, void *priv,
> mfc_err("error in vb2_reqbufs() for E(D)\n");
> return ret;
> }
> + if (reqbufs->count == 0) {
> + mfc_debug(2, "Freeing buffers\n");
> + ctx->capture_state = QUEUE_FREE;
> + return ret;
> + }
> ctx->capture_state = QUEUE_BUFS_REQUESTED;
>
> ret = s5p_mfc_hw_call(ctx->dev->mfc_ops,
> @@ -1200,6 +1205,11 @@ static int vidioc_reqbufs(struct file
*file, void *priv,
> mfc_err("error in vb2_reqbufs() for E(S)\n");
> return ret;
> }
> + if (reqbufs->count == 0) {
> + mfc_debug(2, "Freeing buffers\n");
> + ctx->output_state = QUEUE_FREE;
> + return ret;
> + }
> ctx->output_state = QUEUE_BUFS_REQUESTED;
> } else {
> mfc_err("invalid buf type\n");
> --
> 1.9.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-media" in
> the body of a message to majord...@vger.kernel.org
<mailto:majord...@vger.kernel.org>
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
-BEGIN PGP SIGNATURE-
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBAgAGBQJTti7PAAoJEPsGh4kgR4i5peYQAIZYX8rWsiPnG4SDo4DryffI
g8CnfNr6ndmgxbKiW4OLceSmFmWwmrJj+JQT5fMtO7psAhRL3VmsTZOAn5eMxkpc
z4+/etqF+vKCR/UJf6MeUN/Q4P4RitFulWNGgSfBP0DWoQcxTVdFFSfWBMAEKClI
+11IbN/5IEvRNcGfc9S4sZxwuTOyxrPCMT+X8LayO8zDBC/qMpoOjlFMH2buPCu7
SsvEMjk6AZtIl7oTSF7saPO+Y6C13vuJZHcjCLI7rZlJb5+JajorERNMTqnYZb9j
Qa9jeeTqjtsA42L3y+JUNh6TPjXlF0dZYFfMBQvvaKBNyvKbB+jXkujUJpLZY09F
6baVb/JQlkCehLhi7GUlhKtghgDxXBRBGmw0U7FMA6C4pDr8Z67O4ePdfM1MmPbV
FwBjDALWJuioWiSwZFzhU/lzAhypN5EFKNMKa+pS8SJuU9S1PwURP3VQaV8y7urR
NTsiHyC53JL5wYzfn52jQMcYOC06W+ABjlB2i4L2kmSxfNrexdWZbIH0VLTaHKqt
e1Wl1ZDLXYjE8UXfxbmIOLaSYOPRc5iAkXfB1PpTlETdIvc0znC/dV7xPY2Im6p3
Yc6FnDus6oSP4Lxo+NrIGs4TcqF2cTvHu7QPikFTRIr1bCD1NFTp3L8w1Jlk999q
72b9+Bg5FebefLBseNi1
=3KV/
-END PGP SIGNATURE-
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] s5p-mfc: encoder handles buffers freeing
Add handling of buffer freeing reqbufs request to the encoder of
s5p-mfc.
Signed-off-by: ayaka
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index d26b248..74fb80b 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -1166,6 +1166,11 @@ static int vidioc_reqbufs(struct file *file, void *priv,
mfc_err("error in vb2_reqbufs() for E(D)\n");
return ret;
}
+ if (reqbufs->count == 0) {
+ mfc_debug(2, "Freeing buffers\n");
+ ctx->capture_state = QUEUE_FREE;
+ return ret;
+ }
ctx->capture_state = QUEUE_BUFS_REQUESTED;
ret = s5p_mfc_hw_call(ctx->dev->mfc_ops,
@@ -1200,6 +1205,11 @@ static int vidioc_reqbufs(struct file *file, void *priv,
mfc_err("error in vb2_reqbufs() for E(S)\n");
return ret;
}
+ if (reqbufs->count == 0) {
+ mfc_debug(2, "Freeing buffers\n");
+ ctx->output_state = QUEUE_FREE;
+ return ret;
+ }
ctx->output_state = QUEUE_BUFS_REQUESTED;
} else {
mfc_err("invalid buf type\n");
--
1.9.3
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] s5p-mfc: encoder could free buffers
[PATCH] s5p-mfc: encoder could free buffers
The patch is necessary or the buffers could be freeed but
it would break the state of encoder in s5p-mfc. It is also need
by some application which would detect the buffer allocation
way, like gstreamer.
---
drivers/media/platform/s5p-mfc/s5p_mfc_enc.c | 8
1 file changed, 8 insertions(+)
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index d26b248..1a7518f 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -1166,6 +1166,10 @@ static int vidioc_reqbufs(struct file *file, void
*priv,
mfc_err("error in vb2_reqbufs() for E(D)\n");
return ret;
}
+ if (reqbufs->count == 0) {
+ mfc_debug(2, "Freeing buffers\n");
+ return ret;
+ }
ctx->capture_state = QUEUE_BUFS_REQUESTED;
ret = s5p_mfc_hw_call(ctx->dev->mfc_ops,
@@ -1200,6 +1204,10 @@ static int vidioc_reqbufs(struct file *file, void
*priv,
mfc_err("error in vb2_reqbufs() for E(S)\n");
return ret;
}
+ if (reqbufs->count == 0) {
+ mfc_debug(2, "Freeing buffers\n");
+ return ret;
+ }
ctx->output_state = QUEUE_BUFS_REQUESTED;
} else {
mfc_err("invalid buf type\n");
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
mfc: fixed the request buffer but failed in enqueue a dequeued OUTPUT(input)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 My kernel is 3.13.7, as I have reported before, mfc encoder can't request buffer, I increase the mfc size in dts file from 0x80 to 0x400, then it can request buffer now, but it can't enqueue the input(OUTPUT in v4l2 side) buffer which is just dequeued after stream has start. Here is the log of samsung demo, I don't get any information in dmesg root@kagami:~# ./mfc-encode -m /dev/video1 -c h264 mfc codec encoding example application Andrzej Hajda Copyright 2012 Samsung Electronics Co., Ltd. 50.370398146:args.c:parse_args:187: codec: H264 50.386230729:mfc.c:mfc_create:87: MFC device /dev/video1 opened with fd=3 50.402226771:v4l_dev.c:v4l_req_bufs:116: Succesfully requested 16 buffers for device 3:0 50.402356979:func_dev.c:func_req_bufs:42: Succesfully requested 16 buffers for device -1:1 50.402523896:func_dev.c:func_enq_buf:113: Enqueued buffer 0/16 to -1:1 50.402677896:func_dev.c:func_enq_buf:113: Enqueued buffer 1/16 to -1:1 50.402819771:func_dev.c:func_enq_buf:113: Enqueued buffer 2/16 to -1:1 50.403051729:func_dev.c:func_enq_buf:113: Enqueued buffer 3/16 to -1:1 50.403199646:func_dev.c:func_enq_buf:113: Enqueued buffer 4/16 to -1:1 50.403604562:func_dev.c:func_enq_buf:113: Enqueued buffer 5/16 to -1:1 50.403880187:func_dev.c:func_enq_buf:113: Enqueued buffer 6/16 to -1:1 50.404194396:func_dev.c:func_enq_buf:113: Enqueued buffer 7/16 to -1:1 50.404580146:func_dev.c:func_enq_buf:113: Enqueued buffer 8/16 to -1:1 50.404951021:func_dev.c:func_enq_buf:113: Enqueued buffer 9/16 to -1:1 50.405400896:func_dev.c:func_enq_buf:113: Enqueued buffer 10/16 to -1:1 50.405554729:func_dev.c:func_enq_buf:113: Enqueued buffer 11/16 to -1:1 50.405721271:func_dev.c:func_enq_buf:113: Enqueued buffer 12/16 to -1:1 50.405791896:func_dev.c:func_enq_buf:113: Enqueued buffer 13/16 to -1:1 50.405933604:func_dev.c:func_enq_buf:113: Enqueued buffer 14/16 to -1:1 50.406003646:func_dev.c:func_enq_buf:113: Enqueued buffer 15/16 to -1:1 50.473507729:v4l_dev.c:v4l_req_bufs:116: Succesfully requested 4 buffers for device 3:1 50.473621146:func_dev.c:func_req_bufs:42: Succesfully requested 4 buffers for device 4:0 50.473815312:v4l_dev.c:v4l_enq_buf:211: Enqueued buffer 0/4 to 3:1 50.474005771:v4l_dev.c:v4l_enq_buf:211: Enqueued buffer 1/4 to 3:1 50.474252854:v4l_dev.c:v4l_enq_buf:211: Enqueued buffer 2/4 to 3:1 50.474754854:v4l_dev.c:v4l_enq_buf:211: Enqueued buffer 3/4 to 3:1 State [enq cnt/max]: [Off 0 0/0|Rdy 16 0/250] [Off 0 0/0|Off 4 0/0] [Off 0 0/0|Off 0 0/0] State [enq cnt/max]: [Off 0 0/0|Rdy 16 0/250] [Off 0 0/0|Off 4 0/0] [Off 0 0/0|Off 0 0/0] 50.476985521:func_dev.c:func_deq_buf:79: Dequeued buffer 0/16 from - -1:1 ret=25344 v4l_dev.c:v4l_enq_buf:207: error: Error 22 enq buffer 0/16 to 3:0 50.477464521:io_dev.c:process_chain:165: pair 0:1 ret=-1 I have tried [Linaro-mm-sig] [PATCH][RFC] mm: Don't put CMA pages on per-cpu lists but it doesn't have any effect and if I don't modify the size in dts it will failed in request buffer either. In the program wroten by myself(I have tested work in manufacturer's 3.5 kernel), it can't dequeue the OUPUT at all neither the second frame in CAPTURE(with V4L2_CID_MPEG_VIDEO_HEADER_MODE in 0), of course, there is not any frame in OUTPUT, it is impossible to get the the second frame in CAPTURE. Thank you ayaka -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJTNrvoAAoJEPb4VsMIzTziUtwH/RBoyPSLUieV+fJ0+/KkvBxN WlkFATzPJD+AuiF0co2ggWrx0DnA1M8qFGHecNJW6mf+oL9+kz+2ASFBT+g224Pa q5m1ykAGGnfo1sVtcMOf+HcDxq2dYMG9AD8Cjo39oPZXxgeJLRl2Wd/uePIBIj/H IBGHX7I94XCAgU9OzMq8TviqwE9pyXVMNCa9DX0XBGYKuJDkeKVgXZXt/loGa1BZ IV74aS0jKN8iO7npWSgpTvWG4nSDtw7arDfPTR1N6CsqgJvY+z9K0sr3LFIzu/UW gPUW+eK08qyeemZr5W39akf56x/B+pua8npB+H3NKSxVysTgVvBZ5YieupEfXrE= =gquq -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
