Re: [PATCH 1/5] [media] rockchip/rga: v4l2 m2m support
On 06/27/2017 11:49 PM, Nicolas Dufresne wrote: Le mardi 27 juin 2017 à 23:11 +0800, Jacob Chen a écrit : Hi Nicolas. 2017-06-26 23:49 GMT+08:00 Nicolas Dufresne <nico...@ndufresne.ca>: Le lundi 26 juin 2017 à 22:51 +0800, Jacob Chen a écrit : Rockchip RGA is a separate 2D raster graphic acceleration unit. It accelerates 2D graphics operations, such as point/line drawing, image scaling, rotation, BitBLT, alpha blending and image blur/sharpness. The drvier is mostly based on s5p-g2d v4l2 m2m driver. And supports various operations from the rendering pipeline. - copy - fast solid color fill - rotation - flip - alpha blending The code in rga-hw.c is used to configure regs accroding to operations. The code in rga-buf.c is used to create private mmu table for RGA. The tables is stored in a list, and be removed when buffer is cleanup. Signed-off-by: Jacob Chen <jacob-c...@iotwrt.com> --- drivers/media/platform/Kconfig| 11 + drivers/media/platform/Makefile | 2 + drivers/media/platform/rockchip-rga/Makefile | 3 + drivers/media/platform/rockchip-rga/rga-buf.c | 176 + drivers/media/platform/rockchip-rga/rga-hw.c | 456 drivers/media/platform/rockchip-rga/rga-hw.h | 434 drivers/media/platform/rockchip-rga/rga.c | 979 ++ drivers/media/platform/rockchip-rga/rga.h | 133 8 files changed, 2194 insertions(+) create mode 100644 drivers/media/platform/rockchip-rga/Makefile create mode 100644 drivers/media/platform/rockchip-rga/rga-buf.c create mode 100644 drivers/media/platform/rockchip-rga/rga-hw.c create mode 100644 drivers/media/platform/rockchip-rga/rga-hw.h create mode 100644 drivers/media/platform/rockchip-rga/rga.c create mode 100644 drivers/media/platform/rockchip-rga/rga.h Could be nice to generalize. We could setup a control and fill the values base on porter duff operations, then drivers can implement a subset. Right now, there is no generic way for userspace to know if a driver is just doing copies with some transformations, or if it can actually do alpha blending hence used for composting streams. Note that I haven't looked at all possibilities, Freescale IMX.6 seems to have a similar driver, which has been wrapped in GStreamer with this proposed elements: https://bugzilla.gnome.org/show_bug.cgi?id=772766 Yeah, i also want it use a generic api. "porter duff operations" looks good, i will look at it. +#define V4L2_CID_RGA_ALHPA_REG0 (V4L2_CID_USER_BASE | 0x1002) +#define V4L2_CID_RGA_ALHPA_REG1 (V4L2_CID_USER_BASE | 0x1003) It's not obvious why there is two CID, and how this differ from existing V4L2_CID_ALPHA (the global alpha control). They are used to calculate factors for below formulas. dst alpha = Factor1 * src alpha + Factor2 * dst alpha dst color = Factor3 * src color + Factor4 * dst color I have no idea how to generalize it, and there is no upstream application need it, so i just simply exposed the reg. In my memory, it is is used for convert AYUV to ARGB. Then maybe it's better to just not expose it in the public API in the initial patch (nothing forces you to enable everything). The idea is that it can be added later as needed, taking the time to figure-out a new API or to figure-out how this matches anything that exist. + +/* Operation values */ +#define OP_COPY 0 +#define OP_SOLID_FILL 1 +#define OP_ALPHA_BLEND 2 + +struct rga_frame *rga_get_frame(struct rga_ctx *ctx, enum v4l2_buf_type type); + +/* RGA Buffers Manage Part */ +extern const struct vb2_ops rga_qops; +void *rga_buf_find_page(struct vb2_buffer *vb); +void rga_buf_clean(struct rga_ctx *ctx); + +/* RGA Hardware Part */ +void rga_write(struct rockchip_rga *rga, u32 reg, u32 value); +u32 rga_read(struct rockchip_rga *rga, u32 reg); +void rga_mod(struct rockchip_rga *rga, u32 reg, u32 val, u32 mask); +void rga_start(struct rockchip_rga *rga); +void rga_cmd_set(struct rga_ctx *ctx, void *src_mmu_pages, void *dst_mmu_pages); + +#endif -- Randy Li
Re: [RFC] V4L2 unified low-level decoder API
On 06/06/2017 03:59 PM, Hugues FRUCHET wrote:
Hi Randy,
Did you get a chance to review interface ?
Oh, I have had look a quick view on it yesterday. The video IP of that
platform doesn't come from the On2, right?
I would really appreciate your feedback in order that we move forward on
this topic and get at least one implementation merged.
I am a little busy recently, I will give you a feedback before the next
Monday.
btw, only the MPEG-2 is supported?
I am not very familiar with MPEG-2 standard, I am more familiar with
MPEG-4 PART 10 or HEVC. As the MPEG-2 is more simple, I may not meet any
problem to understand it.
Best regards,
Hugues.
On 05/19/2017 10:15 AM, Randy Li wrote:
On 05/19/2017 04:08 PM, Hugues FRUCHET wrote:
Hi all,
Here is the latest st-delta MPEG2 code:
[PATCH v6 0/3] Add support for MPEG-2 in DELTA video decoder
http://www.mail-archive.com/linux-media@vger.kernel.org/msg112067.html
[PATCH v2 0/3] Add a libv4l plugin for video bitstream parsing
http://www.mail-archive.com/linux-media@vger.kernel.org/msg112076.html
I would review it.
Before merging this work Hans would like to have feedback from peers, in
order to be sure that this is inline with other SoC vendors drivers
expectations.
Thomasz, Pawel, could you give your view regarding ChromeOS and Rockchip
driver ?
The work of the rockchip just re-start a few weeks age, I have just
finished the driver probing type as I decide to make a clean beginning.
The video IP of the rockchip is too complext with a different combine.
The pixel format will begin in JPEG then AVC. I am now more familiar
with those codec now.
Laurent, could you give your view regarding Renesas driver ?
I have also added in appendice [7] the materials presented by Laurent at
ELC 2017 in Portland to introduce stateless video codecs and V4L2
request API, thanks for this presentation Laurent.
Best regards,
Hugues.
On 02/07/2017 08:21 AM, Hugues FRUCHET wrote:
Hi,
Here is an update regarding MPEG-2 implementation based on ST video decoder:
* MPEG-2 API + DELTA kernel driver:
http://www.mail-archive.com/linux-media@vger.kernel.org/msg107405.html
* libv4l-codecparsers plugin including MPEG-2 back-end:
http://www.mail-archive.com/linux-media@vger.kernel.org/msg107812.html
Please note that this is implemented & functional using currently available
V4L2 control framework (no Request API), assuming that user side keeps unicity of
S_EXT_CTRL() / QBUF(OUTPUT) pair.
Request API will remove this constraint, but the point here is to define
control interface, as far as I have understood code, Request API will not
affect those control definitions.
Some updates inline thereafter regarding activities on this subject; me for
MPEG-2 on ST platform and Randy Li, Nicolas Dufresne, Ayaka for H264 on
Rockchip platform:
On 11/14/2016 10:55 AM, Hans Verkuil wrote:
On 10/27/2016 09:42 AM, Hugues FRUCHET wrote:
Hi,
This RFC aims to start discussions in order to define the codec
specific controls structures to fulfill the low-level decoder API
needed by non "Stream API" based decoders ("stateless" or "Frame API" based
decoders).
Let's refer to this as 'stateful' decoders and 'stateless' decoders.
This is the preferred terminology (and much more descriptive than
'Stream' vs 'Frame'). It's also not really a new API, although it does
rely on the Request API.
Several implementation exists now which runs on several SoC and
various software frameworks.
The idea is to find the communalities between all those
implementations and SoC to define a single unified interface in V4L2 includes.
Even if "Request API" is needed to pass those codec specific controls
from userspace down to kernel on a per-buffer basis, we can start
discussions and define the controls in parallel of its development.
We can even propose some implementations based on existing V4L2
control framework (which doesn't support "per-frame" basis) by
ensuring atomicity of sequence S_EXT_CTRL(header[i])/QBUF(stream[i]).
Constraint can then be relaxed when "Request API" is merged.
I would like to propose to work on a "per-codec" basis, having at
least
2 different SoC and 2 different frameworks to test and validate controls.
To do so, I have tried to identify some people that have worked on
this subject and have proposed some implementations, feel free to
correct me and enhance the list if needed:
* MPEG2/MPEG4
- Florent Revest for Allwinner A13 CedarX support [1] tested with
VLC -> libVA + sunxi-cedrus-drv-video -> V4L2
- Myself for STMicroelectronics Delta support [2] tested with
GStreamer V4L2 -> libv4l2 + libv4l-delta plugin -> V4L2
Available on ST platform with [2] & [2.1] patchset series.
* VP8
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [3] tested
with Chromium -> V4L2
- Jung Zhao for Rockchip RK3288 VPU support [4]
* H264
- Pawel Osciak for Rockchip RK3288, RK3399
Re: [RFC] V4L2 unified low-level decoder API
On 05/19/2017 04:08 PM, Hugues FRUCHET wrote:
Hi all,
Here is the latest st-delta MPEG2 code:
[PATCH v6 0/3] Add support for MPEG-2 in DELTA video decoder
http://www.mail-archive.com/linux-media@vger.kernel.org/msg112067.html
[PATCH v2 0/3] Add a libv4l plugin for video bitstream parsing
http://www.mail-archive.com/linux-media@vger.kernel.org/msg112076.html
I would review it.
Before merging this work Hans would like to have feedback from peers, in
order to be sure that this is inline with other SoC vendors drivers
expectations.
Thomasz, Pawel, could you give your view regarding ChromeOS and Rockchip
driver ?
The work of the rockchip just re-start a few weeks age, I have just
finished the driver probing type as I decide to make a clean beginning.
The video IP of the rockchip is too complext with a different combine.
The pixel format will begin in JPEG then AVC. I am now more familiar
with those codec now.
Laurent, could you give your view regarding Renesas driver ?
I have also added in appendice [7] the materials presented by Laurent at
ELC 2017 in Portland to introduce stateless video codecs and V4L2
request API, thanks for this presentation Laurent.
Best regards,
Hugues.
On 02/07/2017 08:21 AM, Hugues FRUCHET wrote:
Hi,
Here is an update regarding MPEG-2 implementation based on ST video decoder:
* MPEG-2 API + DELTA kernel driver:
http://www.mail-archive.com/linux-media@vger.kernel.org/msg107405.html
* libv4l-codecparsers plugin including MPEG-2 back-end:
http://www.mail-archive.com/linux-media@vger.kernel.org/msg107812.html
Please note that this is implemented & functional using currently available
V4L2 control framework (no Request API), assuming that user side keeps unicity of
S_EXT_CTRL() / QBUF(OUTPUT) pair.
Request API will remove this constraint, but the point here is to define
control interface, as far as I have understood code, Request API will not
affect those control definitions.
Some updates inline thereafter regarding activities on this subject; me for
MPEG-2 on ST platform and Randy Li, Nicolas Dufresne, Ayaka for H264 on
Rockchip platform:
On 11/14/2016 10:55 AM, Hans Verkuil wrote:
On 10/27/2016 09:42 AM, Hugues FRUCHET wrote:
Hi,
This RFC aims to start discussions in order to define the codec
specific controls structures to fulfill the low-level decoder API
needed by non "Stream API" based decoders ("stateless" or "Frame API" based
decoders).
Let's refer to this as 'stateful' decoders and 'stateless' decoders.
This is the preferred terminology (and much more descriptive than
'Stream' vs 'Frame'). It's also not really a new API, although it does
rely on the Request API.
Several implementation exists now which runs on several SoC and
various software frameworks.
The idea is to find the communalities between all those
implementations and SoC to define a single unified interface in V4L2 includes.
Even if "Request API" is needed to pass those codec specific controls
from userspace down to kernel on a per-buffer basis, we can start
discussions and define the controls in parallel of its development.
We can even propose some implementations based on existing V4L2
control framework (which doesn't support "per-frame" basis) by
ensuring atomicity of sequence S_EXT_CTRL(header[i])/QBUF(stream[i]).
Constraint can then be relaxed when "Request API" is merged.
I would like to propose to work on a "per-codec" basis, having at
least
2 different SoC and 2 different frameworks to test and validate controls.
To do so, I have tried to identify some people that have worked on
this subject and have proposed some implementations, feel free to
correct me and enhance the list if needed:
* MPEG2/MPEG4
- Florent Revest for Allwinner A13 CedarX support [1] tested with
VLC -> libVA + sunxi-cedrus-drv-video -> V4L2
- Myself for STMicroelectronics Delta support [2] tested with
GStreamer V4L2 -> libv4l2 + libv4l-delta plugin -> V4L2
Available on ST platform with [2] & [2.1] patchset series.
* VP8
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [3] tested
with Chromium -> V4L2
- Jung Zhao for Rockchip RK3288 VPU support [4]
* H264
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [5] tested
with Chromium -> V4L2
- Randy Li for Rockchip RK3288 VPU support [6] tested with VLC? ->
libVA + rockchip-va-driver -> V4L2 VLC? -> libVDPAU +
rockchip-va-driver -> V4L2
Tested with Gstreamer -> VA-API element -> Rockchip VA-API driver -> V4L2
https://github.com/rockchip-linux/libvdpau-rockchip
Study on-going for H264 userland/kernel partitioning in this thread:
Request API: stateless VPU: the buffer mechanism and DPB management:
http://www.mail-archive.com/linux-media@vger.kernel.org/msg107165.html
I can work to define MPEG2/MPEG4 controls and propose functional
implementations for those codecs, and will be glad to co-work with
you Flor
[PATCH v6 2/3] v4l: Add 10/16-bits per channel YUV pixel formats
The formats added by this patch are:
V4L2_PIX_FMT_P010
V4L2_PIX_FMT_P010M
V4L2_PIX_FMT_P016
V4L2_PIX_FMT_P016M
Currently, none of driver uses those format.
Also a variant of V4L2_PIX_FMT_P010M pixel format is added.
The V4L2_PIX_FMT_P010CM is a compat variant of the V4L2_PIX_FMT_P010,
which uses the unused 6 bits to store the next pixel. And with
the alignment requirement of the hardware, it usually would be
some extra space left at the end of a stride.
Signed-off-by: Randy Li <ay...@soulik.info>
---
Documentation/media/uapi/v4l/pixfmt-p010.rst | 126
Documentation/media/uapi/v4l/pixfmt-p010m.rst | 135 ++
Documentation/media/uapi/v4l/pixfmt-p016.rst | 125
Documentation/media/uapi/v4l/pixfmt-p016m.rst | 134 +
Documentation/media/uapi/v4l/yuv-formats.rst | 4 +
include/uapi/linux/videodev2.h| 5 +
6 files changed, 529 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010m.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016m.rst
diff --git a/Documentation/media/uapi/v4l/pixfmt-p010.rst
b/Documentation/media/uapi/v4l/pixfmt-p010.rst
new file mode 100644
index 000..59ed118
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-p010.rst
@@ -0,0 +1,126 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-P010:
+
+**
+V4L2_PIX_FMT_P010 ('P010')
+**
+
+
+V4L2_PIX_FMT_P010
+Formats with ½ horizontal and vertical chroma resolution. One luminance and
+one chrominance plane with alternating
+chroma samples as simliar to ``V4L2_PIX_FMT_NV12``
+
+
+Description
+===
+
+It is a two-plane versions of the YUV 4:2:0 format. The three
+components are separated into two sub-images or planes. The Y plane is
+first. The Y plane has 16 bits per pixel, but only 10 bits are used with the
+rest 6 bits set to zero. For ``V4L2_PIX_FMT_P010``, a combined CbCr plane
+immediately follows the Y plane in memory. The CbCr
+plane is the same width, in bytes, as the Y plane (and of the image),
+but is half as tall in pixels. Each CbCr pair belongs to four pixels.
+For example, Cb\ :sub:`0`/Cr\ :sub:`0` belongs to Y'\ :sub:`00`,
+Y'\ :sub:`01`, Y'\ :sub:`10`, Y'\ :sub:`11`.
+If the Y plane has pad bytes after each row, then the CbCr plane has as
+many pad bytes after its rows.
+
+**Byte Order.**
+Each cell is two bytes.
+
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+
+* - start + 0:
+ - Y'\ :sub:`00`
+ - Y'\ :sub:`01`
+ - Y'\ :sub:`02`
+ - Y'\ :sub:`03`
+* - start + 4:
+ - Y'\ :sub:`10`
+ - Y'\ :sub:`11`
+ - Y'\ :sub:`12`
+ - Y'\ :sub:`13`
+* - start + 8:
+ - Y'\ :sub:`20`
+ - Y'\ :sub:`21`
+ - Y'\ :sub:`22`
+ - Y'\ :sub:`23`
+* - start + 12:
+ - Y'\ :sub:`30`
+ - Y'\ :sub:`31`
+ - Y'\ :sub:`32`
+ - Y'\ :sub:`33`
+* - start + 16:
+ - Cb\ :sub:`00`
+ - Cr\ :sub:`00`
+ - Cb\ :sub:`01`
+ - Cr\ :sub:`01`
+* - start + 20:
+ - Cb\ :sub:`10`
+ - Cr\ :sub:`10`
+ - Cb\ :sub:`11`
+ - Cr\ :sub:`11`
+
+
+**Color Sample Location..**
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+
+* -
+ - 0
+ -
+ - 1
+ - 2
+ -
+ - 3
+* - 0
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 1
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+* - 2
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 3
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
diff --git a/Documentation/media/uapi/v4l/pixfmt-p010m.rst
b/Documentation/media/uapi/v4l/pixfmt-p010m.rst
new file mode 100644
index 000..6697d15
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-p010m.rst
@@ -0,0 +1,135 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-P010M:
+
+***
+V4L2_PIX_FMT_P010M ('PM10')
+***
+
+
+V4L2_PIX_FMT_P010M
+Variation of ``V4L2_PIX_FMT_P010`` with planes non contiguous in memory.
+
+
+Description
+===
+
+This is a multi-planar, two-plane version of the YUV 4:2:0 format. The
+three components are separated into two sub-images or planes.
+``V4L2_PIX_FMT_P010M`` differs from ``V4L2_PIX_FMT_P010`` in that the
+two planes are non-contiguous in memory, i.e. the chroma plane do not
+necessarily immediately follows the luma plane. The luminanc
[PATCH v6 3/3] drm/rockchip: Support 10 bits yuv format in vop
The rockchip use a special pixel format for those yuv pixel
format with 10 bits color depth.
Signed-off-by: Randy Li <ay...@soulik.info>
---
drivers/gpu/drm/rockchip/rockchip_drm_fb.c | 1 +
drivers/gpu/drm/rockchip/rockchip_drm_vop.c | 34 ++---
drivers/gpu/drm/rockchip/rockchip_drm_vop.h | 1 +
drivers/gpu/drm/rockchip/rockchip_vop_reg.c | 2 ++
include/uapi/drm/drm_fourcc.h | 11 ++
5 files changed, 46 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
b/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
index c9ccdf8..250fd29 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_fb.c
@@ -230,6 +230,7 @@ void rockchip_drm_mode_config_init(struct drm_device *dev)
*/
dev->mode_config.max_width = 4096;
dev->mode_config.max_height = 4096;
+ dev->mode_config.allow_fb_modifiers = true;
dev->mode_config.funcs = _drm_mode_config_funcs;
dev->mode_config.helper_private = _mode_config_helpers;
diff --git a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
index 76c79ac..45da270 100644
--- a/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
+++ b/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
@@ -232,6 +232,7 @@ static enum vop_data_format vop_convert_format(uint32_t
format)
case DRM_FORMAT_BGR565:
return VOP_FMT_RGB565;
case DRM_FORMAT_NV12:
+ case DRM_FORMAT_P010:
return VOP_FMT_YUV420SP;
case DRM_FORMAT_NV16:
return VOP_FMT_YUV422SP;
@@ -249,12 +250,28 @@ static bool is_yuv_support(uint32_t format)
case DRM_FORMAT_NV12:
case DRM_FORMAT_NV16:
case DRM_FORMAT_NV24:
+ case DRM_FORMAT_P010:
return true;
default:
return false;
}
}
+static bool is_support_fb(struct drm_framebuffer *fb)
+{
+ switch (fb->format->format) {
+ case DRM_FORMAT_NV12:
+ case DRM_FORMAT_NV16:
+ case DRM_FORMAT_NV24:
+ return true;
+ case DRM_FORMAT_P010:
+ if (fb->modifier == DRM_FORMAT_MOD_ROCKCHIP_10BITS)
+ return true;
+ default:
+ return false;
+ }
+
+}
static bool is_alpha_support(uint32_t format)
{
switch (format) {
@@ -680,7 +697,7 @@ static int vop_plane_atomic_check(struct drm_plane *plane,
* Src.x1 can be odd when do clip, but yuv plane start point
* need align with 2 pixel.
*/
- if (is_yuv_support(fb->format->format) && ((state->src.x1 >> 16) % 2))
+ if (is_support_fb(fb) && ((state->src.x1 >> 16) % 2))
return -EINVAL;
return 0;
@@ -723,6 +740,7 @@ static void vop_plane_atomic_update(struct drm_plane *plane,
dma_addr_t dma_addr;
uint32_t val;
bool rb_swap;
+ bool is_10_bits = false;
int format;
/*
@@ -739,6 +757,9 @@ static void vop_plane_atomic_update(struct drm_plane *plane,
return;
}
+ if (fb->modifier == DRM_FORMAT_MOD_ROCKCHIP_10BITS)
+ is_10_bits = true;
+
obj = rockchip_fb_get_gem_obj(fb, 0);
rk_obj = to_rockchip_obj(obj);
@@ -753,7 +774,10 @@ static void vop_plane_atomic_update(struct drm_plane
*plane,
dsp_sty = dest->y1 + crtc->mode.vtotal - crtc->mode.vsync_start;
dsp_st = dsp_sty << 16 | (dsp_stx & 0x);
- offset = (src->x1 >> 16) * fb->format->cpp[0];
+ if (is_10_bits)
+ offset = (src->x1 >> 16) * 10 / 8;
+ else
+ offset = (src->x1 >> 16) * fb->format->cpp[0];
offset += (src->y1 >> 16) * fb->pitches[0];
dma_addr = rk_obj->dma_addr + offset + fb->offsets[0];
@@ -764,6 +788,7 @@ static void vop_plane_atomic_update(struct drm_plane *plane,
VOP_WIN_SET(vop, win, format, format);
VOP_WIN_SET(vop, win, yrgb_vir, fb->pitches[0] >> 2);
VOP_WIN_SET(vop, win, yrgb_mst, dma_addr);
+ VOP_WIN_SET(vop, win, fmt_10, is_10_bits);
if (is_yuv_support(fb->format->format)) {
int hsub =
drm_format_horz_chroma_subsampling(fb->format->format);
int vsub =
drm_format_vert_chroma_subsampling(fb->format->format);
@@ -772,7 +797,10 @@ static void vop_plane_atomic_update(struct drm_plane
*plane,
uv_obj = rockchip_fb_get_gem_obj(fb, 1);
rk_uv_obj = to_rockchip_obj(uv_obj);
- offset = (src->x1 >> 16) * bpp / hsub;
+ if (is_10_bits)
+ offset = (src->x1 >> 16) * 10 / 8 / hsub;
+ else
+ offset = (src->x1 >> 16) * bpp / hsub;
[PATCH v6 1/3] drm_fourcc: Add new P010, P016 video format
P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
per channel video format.
P016 is a planar 4:2:0 YUV with interleaved UV plane, 16 bits
per channel video format.
V3: Added P012 and fixed cpp for P010
V4: format definition refined per review
V5: Format comment block for each new pixel format
V6: reversed Cb/Cr order in comments
v7: reversed Cb/Cr order in comments of header files, remove
the wrong part of commit message.
Cc: Daniel Stone <dan...@fooishbar.org>
Cc: Ville Syrjälä <ville.syrj...@linux.intel.com>
Signed-off-by: Randy Li <ay...@soulik.info>
Signed-off-by: Clint Taylor <clinton.a.tay...@intel.com>
---
drivers/gpu/drm/drm_fourcc.c | 3 +++
include/uapi/drm/drm_fourcc.h | 21 +
2 files changed, 24 insertions(+)
diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
index 90d2cc8..3e0fd58 100644
--- a/drivers/gpu/drm/drm_fourcc.c
+++ b/drivers/gpu/drm/drm_fourcc.c
@@ -165,6 +165,9 @@ const struct drm_format_info *__drm_format_info(u32 format)
{ .format = DRM_FORMAT_UYVY,.depth = 0,
.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
{ .format = DRM_FORMAT_VYUY,.depth = 0,
.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
{ .format = DRM_FORMAT_AYUV,.depth = 0,
.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+ { .format = DRM_FORMAT_P010,.depth = 0,
.num_planes = 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
+ { .format = DRM_FORMAT_P012,.depth = 0,
.num_planes = 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
+ { .format = DRM_FORMAT_P016,.depth = 0,
.num_planes = 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
};
unsigned int i;
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index ef20abb..306f979 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -128,6 +128,27 @@ extern "C" {
#define DRM_FORMAT_NV42fourcc_code('N', 'V', '4', '2') /*
non-subsampled Cb:Cr plane */
/*
+ * 2 plane YCbCr MSB aligned
+ * index 0 = Y plane, [15:0] Y:x [10:6] little endian
+ * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [10:6:10:6] little endian
+ */
+#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
subsampled Cb:Cr plane 10 bits per channel */
+
+/*
+ * 2 plane YCbCr MSB aligned
+ * index 0 = Y plane, [15:0] Y:x [12:4] little endian
+ * index 1 = Cb:Cr plane, [31:0] Cb:x:Cr:x [12:4:12:4] little endian
+ */
+#define DRM_FORMAT_P012fourcc_code('P', '0', '1', '2') /* 2x2
subsampled Cb:Cr plane 12 bits per channel */
+
+/*
+ * 2 plane YCbCr MSB aligned
+ * index 0 = Y plane, [15:0] Y little endian
+ * index 1 = Cb:Cr plane, [31:0] Cb:Cr [16:16] little endian
+ */
+#define DRM_FORMAT_P016fourcc_code('P', '0', '1', '6') /* 2x2
subsampled Cb:Cr plane 16 bits per channel */
+
+/*
* 3 plane YCbCr
* index 0: Y plane, [7:0] Y
* index 1: Cb plane, [7:0] Cb
--
2.7.4
[PATCH v6 0/3] Add pixel format for 10 bits YUV video
Thanks to Clint's work, this version just correct some wrong info in the comment. I also offer a driver sample here, but it have been verified with the 10 bits properly. I lacks of the userspace tool. And I am not sure whether it is a properly way to support the pixel format used in rockchip, although it is not common used pixel format, but it could save lots of memory, it may be welcome by the other vendor, also I think the 3GR serial from Intel would use the same pixel format as the video IP comes from rockchip. Randy Li (3): drm_fourcc: Add new P010, P016 video format v4l: Add 10/16-bits per channel YUV pixel formats drm/rockchip: Support 10 bits yuv format in vop Documentation/media/uapi/v4l/pixfmt-p010.rst | 126 Documentation/media/uapi/v4l/pixfmt-p010m.rst | 135 ++ Documentation/media/uapi/v4l/pixfmt-p016.rst | 125 Documentation/media/uapi/v4l/pixfmt-p016m.rst | 134 + Documentation/media/uapi/v4l/yuv-formats.rst | 4 + drivers/gpu/drm/drm_fourcc.c | 3 + drivers/gpu/drm/rockchip/rockchip_drm_fb.c| 1 + drivers/gpu/drm/rockchip/rockchip_drm_vop.c | 34 ++- drivers/gpu/drm/rockchip/rockchip_drm_vop.h | 1 + drivers/gpu/drm/rockchip/rockchip_vop_reg.c | 2 + include/uapi/drm/drm_fourcc.h | 32 ++ include/uapi/linux/videodev2.h| 5 + 12 files changed, 599 insertions(+), 3 deletions(-) create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010.rst create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010m.rst create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016.rst create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016m.rst -- 2.7.4
Request API: stateless VPU: the buffer mechanism and DPB management
Hello all: I have recently finish the learning of the H.264 codec and ready to write the driver. Although I have not get deep in syntax of H.264 but I think I just need to reuse and extended the VA-API H264 Parser from gstreamer. The whole plan in userspace is just injecting a parsing operation and set those v4l2 control in kernel before enqueue a buffer into OUTPUT, which would keep the most compatible with those stateful video IP(those with a firmware). But in order to do that, I can't avoid the management of DPB. I decided to moving the DPB management job from userspace in kernel. Also the video IP(On2 on rk3288 and the transition video IP on those future SoC than rk3288, rkv don't have this problem) would a special way to manage the DPB, which requests the same reference frame is storing in the same reference index in the runtime(actually it is its Motion Vector data appended in decoded YUV data would not be moved). I would suggest to keep those job in kernel, the userspace just to need update the list0 and list1 of DPB. DPB is self managed in kernel the userspace don't need to even dequeue the buffer from CAPTURE until the re-order is done. The kernel driver would also re-order the CAPTURE buffer into display order, and blocking the operation on CAPTURE until a buffer is ready to place in the very display order. If I don't do that, I have to get the buffer once it is decoded, and marking its result with the poc, I could only begin the processing of the next frame only after those thing are done. Which would effect the performance badly. That is what chromebook did(I hear that from the other staff, I didn't get invoke in chromium project yet). So I would suggest that doing the re-order job in kernel, and inform the the userspace the buffers are ready when the new I frame(key frame) is pushed into the video IP. Although moving those job into kernel would increase the loading, but I think it is worth to do that, but I don't know whether all those thought are correct and high performance(It is more important than API compatible especially for those 4K video). And I want to know more ideas about this topic. I would begin the writing the new driver after the coming culture new year vacation(I would go to the Europe), I wish we can decide the final mechanism before I begin this job. -- Randy Li The third produce department -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Request API: stateless VPU: the buffer mechanism and DPB management
Hello all: I have recently finish the learning of the H.264 codec and ready to write the driver. Although I have not get deep in syntax of H.264 but I think I just need to reuse and extended the VA-API H264 Parser from gstreamer. The whole plan in userspace is just injecting a parsing operation and set those v4l2 control in kernel before enqueue a buffer into OUTPUT, which would keep the most compatible with those stateful video IP(those with a firmware). But in order to do that, I can't avoid the management of DPB. I decided to moving the DPB management job from userspace in kernel. Also the video IP(On2 on rk3288 and the transition video IP on those future SoC than rk3288, rkv don't have this problem) would a special way to manage the DPB, which requests the same reference frame is storing in the same reference index in the runtime(actually it is its Motion Vector data appended in decoded YUV data would not be moved). I would suggest to keep those job in kernel, the userspace just to need update the list0 and list1 of DPB. DPB is self managed in kernel the userspace don't need to even dequeue the buffer from CAPTURE until the re-order is done. The kernel driver would also re-order the CAPTURE buffer into display order, and blocking the operation on CAPTURE until a buffer is ready to place in the very display order. If I don't do that, I have to get the buffer once it is decoded, and marking its result with the poc, I could only begin the processing of the next frame only after those thing are done. Which would effect the performance badly. That is what chromebook did(I hear that from the other staff, I didn't get invoke in chromium project yet). So I would suggest that doing the re-order job in kernel, and inform the the userspace the buffers are ready when the new I frame(key frame) is pushed into the video IP. Although moving those job into kernel would increase the loading, but I think it is worth to do that, but I don't know whether all those thought are correct and high performance(It is more important than API compatible especially for those 4K video). And I want to know more ideas about this topic. I would begin the writing the new driver after the coming culture new year vacation(I would go to the Europe), I wish we can decide the final mechanism before I begin this job. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 0/2] Add pixel formats for 10/16 bits YUV video
Those pixel formats mainly comes from Gstreamer and ffmpeg. Currently, the VOP(video output mixer) found on RK3288 and future support those 10 bits pixel formats are input. Also the decoder on RK3288 could use them as output. The fourcc is not enough to store the endian information for those pixel formats in v4l2, as it doesn't have a flag like drm does. I have not met the usage of those 16 bits per-channel format, it is a very large color range, even the DSLR only use 12 bits. Randy Li (2): drm_fourcc: Add new P010, P016 video format [media] v4l: Add 10/16-bits per channel YUV pixel formats Documentation/media/uapi/v4l/pixfmt-p010.rst | 86 Documentation/media/uapi/v4l/pixfmt-p010m.rst | 94 ++ Documentation/media/uapi/v4l/pixfmt-p016.rst | 126 Documentation/media/uapi/v4l/pixfmt-p016m.rst | 136 ++ drivers/gpu/drm/drm_fourcc.c | 3 + include/uapi/drm/drm_fourcc.h | 2 + include/uapi/linux/videodev2.h| 4 + 7 files changed, 451 insertions(+) create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010.rst create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010m.rst create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016.rst create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016m.rst -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 2/2] [media] v4l: Add 10/16-bits per channel YUV pixel formats
The formats added by this patch are:
V4L2_PIX_FMT_P010
V4L2_PIX_FMT_P010M
V4L2_PIX_FMT_P016
V4L2_PIX_FMT_P016M
Currently, none of driver uses those format, but some video device
has been confirmed with could as those format for video output.
The Rockchip's new decoder has supported those 10 bits format for
profile_10 HEVC/AVC video.
Signed-off-by: Randy Li <ay...@soulik.info>
v4l2
---
Documentation/media/uapi/v4l/pixfmt-p010.rst | 86
Documentation/media/uapi/v4l/pixfmt-p010m.rst | 94 ++
Documentation/media/uapi/v4l/pixfmt-p016.rst | 126
Documentation/media/uapi/v4l/pixfmt-p016m.rst | 136 ++
include/uapi/linux/videodev2.h| 4 +
5 files changed, 446 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p010m.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016.rst
create mode 100644 Documentation/media/uapi/v4l/pixfmt-p016m.rst
diff --git a/Documentation/media/uapi/v4l/pixfmt-p010.rst
b/Documentation/media/uapi/v4l/pixfmt-p010.rst
new file mode 100644
index 000..82b300c
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-p010.rst
@@ -0,0 +1,86 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-P010:
+
+**
+V4L2_PIX_FMT_P010 ('P010')
+**
+
+
+V4L2_PIX_FMT_P010
+Formats with ½ horizontal and vertical chroma resolution. One luminance and
+one chrominance plane with alternating
+chroma samples as simliar to ``V4L2_PIX_FMT_NV12``
+
+
+Description
+===
+
+It is a two-plane versions of the YUV 4:2:0 format. The three
+components are separated into two sub-images or planes. The Y plane is
+first. The Y plane has 10 bits per pixel. For ``V4L2_PIX_FMT_P010``, a
+combined CbCr plane immediately follows the Y plane in memory. The CbCr
+plane is the same width, in bytes, as the Y plane (and of the image),
+but is half as tall in pixels. Each CbCr pair belongs to four pixels.
+For example, Cb\ :sub:`0`/Cr\ :sub:`0` belongs to Y'\ :sub:`00`,
+Y'\ :sub:`01`, Y'\ :sub:`10`, Y'\ :sub:`11`.
+If the Y plane has pad bytes after each row, then the CbCr plane has as
+many pad bytes after its rows.
+
+
+**Color Sample Location..**
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+
+* -
+ - 0
+ -
+ - 1
+ - 2
+ -
+ - 3
+* - 0
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 1
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+* - 2
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 3
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
diff --git a/Documentation/media/uapi/v4l/pixfmt-p010m.rst
b/Documentation/media/uapi/v4l/pixfmt-p010m.rst
new file mode 100644
index 000..80194a1
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-p010m.rst
@@ -0,0 +1,94 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-P010M:
+
+***
+V4L2_PIX_FMT_P010 ('P010')
+***
+
+
+V4L2_PIX_FMT_P010M
+Variation of ``V4L2_PIX_FMT_P010`` with planes non contiguous in memory.
+
+
+Description
+===
+
+This is a multi-planar, two-plane version of the YUV 4:2:0 format. The
+three components are separated into two sub-images or planes.
+``V4L2_PIX_FMT_P010M`` differs from ``V4L2_PIX_FMT_P010`` in that the
+two planes are non-contiguous in memory, i.e. the chroma plane do not
+necessarily immediately follows the luma plane. The luminance data
+occupies the first plane. The Y plane has one byte per pixel. In the
+second plane there is a chrominance data with alternating chroma
+samples. The CbCr plane is the same width, in bytes, as the Y plane (and
+of the image), but is half as tall in pixels. Each CbCr pair belongs to
+four pixels. For example, Cb\ :sub:`0`/Cr\ :sub:`0` belongs to
+Y'\ :sub:`00`, Y'\ :sub:`01`, Y'\ :sub:`10`, Y'\ :sub:`11`.
+
+``V4L2_PIX_FMT_P010M`` is intended to be used only in drivers and
+applications that support the multi-planar API, described in
+:ref:`planar-apis`.
+
+If the Y plane has pad bytes after each row, then the CbCr plane has as
+many pad bytes after its rows.
+
+**Color Sample Location..**
+
+
+
+.. flat-table::
+:header-rows: 0
+:stub-columns: 0
+
+* -
+ - 0
+ -
+ - 1
+ - 2
+ -
+ - 3
+* - 0
+ - Y
+ -
+ - Y
+ - Y
+ -
+ - Y
+* -
+ -
+ - C
+ -
+ -
+ - C
+ -
+* - 1
+ - Y
+ -
+ - Y
+
[PATCH v2 1/2] drm_fourcc: Add new P010, P016 video format
P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
per channel video format. Rockchip's vop support this
video format(little endian only) as the input video format.
P016 is a planar 4:2:0 YUV with interleaved UV plane, 16 bits
per channel video format.
Signed-off-by: Randy Li <ay...@soulik.info>
drm
---
drivers/gpu/drm/drm_fourcc.c | 3 +++
include/uapi/drm/drm_fourcc.h | 2 ++
2 files changed, 5 insertions(+)
diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c
index 90d2cc8..23c8e99 100644
--- a/drivers/gpu/drm/drm_fourcc.c
+++ b/drivers/gpu/drm/drm_fourcc.c
@@ -165,6 +165,9 @@ const struct drm_format_info *__drm_format_info(u32 format)
{ .format = DRM_FORMAT_UYVY,.depth = 0,
.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
{ .format = DRM_FORMAT_VYUY,.depth = 0,
.num_planes = 1, .cpp = { 2, 0, 0 }, .hsub = 2, .vsub = 1 },
{ .format = DRM_FORMAT_AYUV,.depth = 0,
.num_planes = 1, .cpp = { 4, 0, 0 }, .hsub = 1, .vsub = 1 },
+ /* FIXME a pixel in Y for P010 is 10 bits */
+ { .format = DRM_FORMAT_P010,.depth = 0,
.num_planes = 2, .cpp = { 1, 2, 0 }, .hsub = 2, .vsub = 2 },
+ { .format = DRM_FORMAT_P016,.depth = 0,
.num_planes = 2, .cpp = { 2, 4, 0 }, .hsub = 2, .vsub = 2 },
};
unsigned int i;
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 9e1bb7f..ecc2e05e5 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -119,6 +119,8 @@ extern "C" {
#define DRM_FORMAT_NV61fourcc_code('N', 'V', '6', '1') /* 2x1
subsampled Cb:Cr plane */
#define DRM_FORMAT_NV24fourcc_code('N', 'V', '2', '4') /*
non-subsampled Cr:Cb plane */
#define DRM_FORMAT_NV42fourcc_code('N', 'V', '4', '2') /*
non-subsampled Cb:Cr plane */
+#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
subsampled Cr:Cb plane 10 bits per channel */
+#define DRM_FORMAT_P016fourcc_code('P', '0', '1', '6') /* 2x2
subsampled Cr:Cb plane 16 bits per channel */
/*
* 3 plane YCbCr
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] [media] v4l: Add 10-bits per channel YUV pixel formats
The formats added by this patch are:
V4L2_PIX_FMT_P010
V4L2_PIX_FMT_P010M
Currently, none of driver uses those format, but some video device
has been confirmed with could as those format for video output.
The Rockchip's new decoder has supported those format for profile_10
HEVC/AVC video.
Signed-off-by: Randy Li <ay...@soulik.info>
---
include/uapi/linux/videodev2.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 46e8a2e3..9e03f20 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -551,6 +551,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV61v4l2_fourcc('N', 'V', '6', '1') /* 16 Y/CrCb
4:2:2 */
#define V4L2_PIX_FMT_NV24v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr
4:4:4 */
#define V4L2_PIX_FMT_NV42v4l2_fourcc('N', 'V', '4', '2') /* 24 Y/CrCb
4:4:4 */
+#define V4L2_PIX_FMT_P010v4l2_fourcc('P', '0', '1', '0') /* 15 Y/CbCr
4:2:0, 10 bits per channel */
/* two non contiguous planes - one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr
4:2:0 */
@@ -559,6 +560,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV61M v4l2_fourcc('N', 'M', '6', '1') /* 16 Y/CrCb
4:2:2 */
#define V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr
4:2:0 64x32 macroblocks */
#define V4L2_PIX_FMT_NV12MT_16X16 v4l2_fourcc('V', 'M', '1', '2') /* 12
Y/CbCr 4:2:0 16x16 macroblocks */
+#define V4L2_PIX_FMT_P010M v4l2_fourcc('P', 'M', '1', '0') /* 15 Y/CbCr
4:2:0, 10 bits per channel */
/* three planes - Y Cb, Cr */
#define V4L2_PIX_FMT_YUV410 v4l2_fourcc('Y', 'U', 'V', '9') /* 9 YUV 4:1:0
*/
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] drm_fourcc: Add new P010 video format
P010 is a planar 4:2:0 YUV with interleaved UV plane, 10 bits
per channel video format. Rockchip's vop support this
video format(little endian only) as the input video format.
Signed-off-by: Randy Li <ay...@soulik.info>
---
include/uapi/drm/drm_fourcc.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index 9e1bb7f..d2721da 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -119,6 +119,7 @@ extern "C" {
#define DRM_FORMAT_NV61fourcc_code('N', 'V', '6', '1') /* 2x1
subsampled Cb:Cr plane */
#define DRM_FORMAT_NV24fourcc_code('N', 'V', '2', '4') /*
non-subsampled Cr:Cb plane */
#define DRM_FORMAT_NV42fourcc_code('N', 'V', '4', '2') /*
non-subsampled Cb:Cr plane */
+#define DRM_FORMAT_P010fourcc_code('P', '0', '1', '0') /* 2x2
subsampled Cr:Cb plane 10 bits per channel */
/*
* 3 plane YCbCr
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/2] Add pixel format for 10 bits YUV video
Those pixel formats comes from Gstreamer and ffmpeg. Currently, the VOP(video output mixer) found on RK3288 and future support those pixel formats are input. Also the decoder on RK3288 could use them as output. Randy Li (2): drm_fourcc: Add new P010 video format [media] v4l: Add 10-bits per channel YUV pixel formats include/uapi/drm/drm_fourcc.h | 1 + include/uapi/linux/videodev2.h | 2 ++ 2 files changed, 3 insertions(+) -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
The update of RockChip media framework plan
To those guy who cares Rockchip There is a update about Rockchip media framework, I decided not to continue of developing the VA-API driver for produces purpose(both official and private plan). As there is really to much problem with VA-API with V4L2. But I may develop the VA-API driver to know how to expend the parser in Gstreamer. The currently Gstreamer plugins using a rockchip video parser and HAL layer library could be located here https://github.com/rockchip-linux/gstreamer-rockchip For the future develop plan, I would try to move all the thing to Gstreamer V4L2 plugin, maybe I would create a new plugin. But it would be a long time to go. As there is no critical need of the Media framework for Rockchip in Linux, the official plan would be paused, if the kodi or LibreELEC is not interesting to my supervisor, this plan will become my private plan, but good news is that my plan about the ISP support would become official. Any the result would be there is a standard V4L2 API and a driver for those stateless video driver(Do remember we don't need firmware). And I would use the Gstreamer in userspace and welcome the other to offer the support in the other project. For the those project who have supported the ffmpeg, I would suggest you to add support for Gstreamer, the role of Gstreamer in a player is not different to the ffmpeg. The support of ffmpeg of Rockchip project would be dropped, as we need a power way to rendering decoding result for those 4K video. The Gstreamer is the only Media Framework who could offer a static interface and easy to extend. Even the Gstreamer, currently I found it could only cover the 80% solution of I would meet, but it is enough for Linux. On 12/23/2016 07:17 PM, Christian Hewitt wrote: Hello Randy, I would like LibreELEC to run great on a wide range of devices including rockchip so it is good to have direct contact - thanks Lionel. Our core focus is Kodi so I have cc'd Keith Herrington from the Kodi project board. Keith and senior developers on Kodi’s team can guide you on their technical direction and requirements for Linux. Once there is agreement on the right approach for your developers we can start looking at hardware support. Kind regards, Christian -- Christian Hewitt Project Lead chew...@libreelec.tv <mailto:chew...@libreelec.tv> +971 50 3570 499 On 22 Dec 2016, at 07:49 pm, LongChair . <longch...@hotmail.com <mailto:longch...@hotmail.com>> wrote: Hi Christian, I have been looking around recently to RockChip latest SoCs, which are RK3288 and RK3399. They seem great on paper, and I have some interest in those. What i know about those so far is that : - They Run Linux 4.4 - They Have some V4L2 compliance which was made by rockChip for Google/chromium - They also have a va-api wrapper (only supporting h264 so far, but intended to be extended in a near future to other codecs) - They have some muscles (A72 for RK3399) and Mali 8XX as GPU which makes them probably snappier than AML SoCs. I am considering investigating this as a potential platform for US, which means integrating it into LE. I am not sure if that would be a valid approach, or if that would meet any kodi eventual requirements as well. I'm available to discuss that with you anytime. I will cc also Ayaka to this email. He's a Rockchip employee in charge of Linux Support. Should you have deeper technical questions about this, he will probably be able to answer that :) Cheers, Lionel -- Randy Li The third produce department -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] V4L2 unified low-level decoder API
On 11/04/2016 09:55 PM, Hugues FRUCHET wrote:
Hi Randy,
thanks for reply, some comments below:
On 10/27/2016 03:08 AM, Randy Li wrote:
On 10/26/2016 11:09 PM, Hugues FRUCHET wrote:
Hi,
This RFC aims to start discussions in order to define the codec specific
controls structures to fulfill the low-level decoder API needed by non
“Stream API” based decoders (“stateless” or “Frame API” based decoders).
Several implementation exists now which runs on several SoC and various
software frameworks.
The idea is to find the communalities between all those implementations
and SoC to define a single unified interface in V4L2 includes.
Even if “Request API” is needed to pass those codec specific controls
from userspace down to kernel on a per-buffer basis, we can start
discussions and define the controls in parallel of its development.
Yes, I have sent a one for H.264 decoder and JPEG encoder.
We can even propose some implementations based on existing V4L2 control
framework (which doesn’t support “per-frame” basis) by ensuring
atomicity of sequence S_EXT_CTRL(header[i])/QBUF(stream[i]). Constraint
can then be relaxed when “Request API” is merged.
I would like to propose to work on a “per-codec” basis, having at least
2 different SoC and 2 different frameworks to test and validate controls.
To do so, I have tried to identify some people that have worked on this
subject and have proposed some implementations, feel free to correct me
and enhance the list if needed:
* MPEG2/MPEG4
- Florent Revest for Allwinner A13 CedarX support [1] tested with VLC
-> libVA + sunxi-cedrus-drv-video -> V4L2
- Myself for STMicroelectronics Delta support [2] tested with
GStreamer V4L2 -> libv4l2 + libv4l-delta plugin -> V4L2
* VP8
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [3] tested with
Chromium -> V4L2
- Jung Zhao for Rockchip RK3288 VPU support [4]
There is rockchip VDPAU driver supporting it, but it is .
Could you point out the code that is used ? Which application is used on
top of VDPAU ?
https://github.com/rockchip-linux/libvdpau-rockchip
* H264
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [5] tested with
Chromium -> V4L2
- Randy Li for Rockchip RK3288 VPU support [6] tested with VLC? ->
libVA + rockchip-va-driver -> V4L2
I only tested it with Gstreamer -> VA-API element -> Rockchip VA-API
driver -> V4L2
OK got it, thks !
VLC?
-> libVDPAU + rockchip-va-driver -> V4L2
I can work to define MPEG2/MPEG4 controls and propose functional
implementations for those codecs, and will be glad to co-work with you
Florent.
But it may not work with Rockchip's SoC, you may check the following branch
https://github.com/hizukiayaka/rockchip-video-driver/tree/rk_v4l2_mix
I have checked code and I have only found H264 support, do I miss
something ?
No, I have said above, only H264 decoder and JPEG encoder are supported
in currently Rockchip VA-API driver. And H264 decoder depends on a
Rockchip H264 parser. The rk_v4l2_mix just a branch make that clearly,
it could get what the VA-API doesn't offer from code.
I can help on H264 on a code review basis based on the functional H264
setup I have in-house and codec knowledge, but I cannot provide
implementation in a reasonable timeframe, same for VP8.
Apart of very details of each codec, we have also to state about generic
concerns such as:
- new pixel format introduction (VP8 => VP8F, H264 => S264,
MPG2 => MG2F, MPG4 => MG4F)
I don't think it is necessary.
But currently it is done that way in all patches proposals I have seen
so far, including rockchip:
rockchip_decoder_v4l2.c:{VAProfileH264Baseline,V4L2_PIX_FMT_H264_SLICE},
It is Google's idea, it would be removed with new version kernel driver
of mine. Also I don't like multiplanes image format from Google driver.
We have to state about it all together. Seems natural to me to do this
way instead of device caps.
Doing so user knows that the driver is based on "Frame API" -so
additional headers are required to decode input stream- and not
on "Stream API" -H264 stream can be decoded directly-.
> We should probably use something else then "STREAMING" in the
> capabilities instead of duplicating all the encoding formats (exception
> to H264 byte-stream and H264 AVC, that also applies to streaming
> drivers and there is not easy way to introduce stream-format in the API
> atm). Other then that, this solution works, so it could just be
> considered the right way, I just find it less elegant personally.
I agree with Nicolas.
Best regards,
Hugues.
[0] [ANN] Codec & Request API Brainstorm meeting Oct 10 & 11
https://www.spinics.net/lists/linux-media/msg106699.html
[1] MPEG2 A13 CedarX http://www.spinics.net/lists/l
Re: [RFC] V4L2 unified low-level decoder API
On 10/26/2016 11:09 PM, Hugues FRUCHET wrote: Hi, This RFC aims to start discussions in order to define the codec specific controls structures to fulfill the low-level decoder API needed by non “Stream API” based decoders (“stateless” or “Frame API” based decoders). Several implementation exists now which runs on several SoC and various software frameworks. The idea is to find the communalities between all those implementations and SoC to define a single unified interface in V4L2 includes. Even if “Request API” is needed to pass those codec specific controls from userspace down to kernel on a per-buffer basis, we can start discussions and define the controls in parallel of its development. Yes, I have sent a one for H.264 decoder and JPEG encoder. We can even propose some implementations based on existing V4L2 control framework (which doesn’t support “per-frame” basis) by ensuring atomicity of sequence S_EXT_CTRL(header[i])/QBUF(stream[i]). Constraint can then be relaxed when “Request API” is merged. I would like to propose to work on a “per-codec” basis, having at least 2 different SoC and 2 different frameworks to test and validate controls. To do so, I have tried to identify some people that have worked on this subject and have proposed some implementations, feel free to correct me and enhance the list if needed: * MPEG2/MPEG4 - Florent Revest for Allwinner A13 CedarX support [1] tested with VLC -> libVA + sunxi-cedrus-drv-video -> V4L2 - Myself for STMicroelectronics Delta support [2] tested with GStreamer V4L2 -> libv4l2 + libv4l-delta plugin -> V4L2 * VP8 - Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [3] tested with Chromium -> V4L2 - Jung Zhao for Rockchip RK3288 VPU support [4] There is rockchip VDPAU driver supporting it, but it is . * H264 - Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [5] tested with Chromium -> V4L2 - Randy Li for Rockchip RK3288 VPU support [6] tested with VLC? -> libVA + rockchip-va-driver -> V4L2 I only tested it with Gstreamer -> VA-API element -> Rockchip VA-API driver -> V4L2 VLC? -> libVDPAU + rockchip-va-driver -> V4L2 I can work to define MPEG2/MPEG4 controls and propose functional implementations for those codecs, and will be glad to co-work with you Florent. But it may not work with Rockchip's SoC, you may check the following branch https://github.com/hizukiayaka/rockchip-video-driver/tree/rk_v4l2_mix I can help on H264 on a code review basis based on the functional H264 setup I have in-house and codec knowledge, but I cannot provide implementation in a reasonable timeframe, same for VP8. Apart of very details of each codec, we have also to state about generic concerns such as: - new pixel format introduction (VP8 => VP8F, H264 => S264, MPG2 => MG2F, MPG4 => MG4F) I don't think it is necessary. - new device caps to indicate that driver requires extra headers ? maybe not needed because redundant with new pixel format I prefer this one. - continue to modify v4l2-controls.h ? or do we add some new specific header files (H264 is huge!) ? Not huge. You could check rockchip's kernel. - how to manage sequence header & picture header, optional/extended controls (MPEG2 sequence/picture extensions, H264 SEI, …). Personally I have added flags inside a single control structure, H264 is done in a different way using several controls (SPS/PPS/SLICE/DECODE/…) the last one is dpb, except the dpb, it would have the same numbers of controls to those structures defined in VA-API H264 decoder. Thanks you to all of you for your attention and feel free to react on this topic if you are interested to work on this subject. Currently, I have to pause the process of VA-API drive, and moving to the other idea I have said before, creating a new API in userspace(but won't archive the goal I set before in this step). There are some shortages in VA-API I have said in last email making the performance in 4K video and extending the Gstreamer VA-API is a little difficult job and need more time. And the development for the new VPU driver for rockchip would pause a while as well. It would not be a long time(a few weeks) and I am still available in my free time(at home). It is good to know the wheel begin to roll. And do feel free to assign job to me. Best regards, Hugues. [0] [ANN] Codec & Request API Brainstorm meeting Oct 10 & 11 https://www.spinics.net/lists/linux-media/msg106699.html [1] MPEG2 A13 CedarX http://www.spinics.net/lists/linux-media/msg104823.html [1] MPEG4 A13 CedarX http://www.spinics.net/lists/linux-media/msg104817.html [2] MPEG2 STi4xx Delta http://www.spinics.net/lists/linux-media/msg106240.html [2] MPEG4 STi4xx Delta is also supported
something different ideas from chromium's decoder settings API
Hello Hans and all media staff: Recently I have try to run the my VA-API driver in Rockchip RK3399, after ported the driver in chromium to request API, it works now. Thanks to the chromium project effort, both the decoder settings API and structures are the same between rk3288 and rk3399. However the those v4l2 decoder structures are too different between the VA-API, I know those VA-API structures would not enough for our situation. If we could expend the VA-API structures, it sounds more easy to start up a standard. Also creating a new v4l2 fourcc for each format is not convenience, also the data format may be a little different, but it is still a part of the original data right? The slice API and request API are still not clearly, I just put my ideas here and hoping more ideas coming. P.S Does somebody know where the chromium would switch to request API instead of the config store? -- Randy Li The third produce department === This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. [Fuzhou Rockchip Electronics, INC. China mainland] === -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: media: rockchip-vpu: I should place the huffman table at kernel or userspace?
On 09/27/2016 03:52 PM, Hans Verkuil wrote: On 27/09/16 05:43, Randy Li wrote: Hello: I have just done a JPEG HW encoder for the RK3288. I have been told that I can't use the standard way to generate huffman table, the VPU supports only 10 levels with a different huffman table. If I send the huffman table through the v4l2 extra control, the memory copy is requested, although the data is not very large(2 x 64 bytes) but still a overhead. The other way is to place them in the kernel driver, and just define the quality every time it encode a picture. But storing in kernel would make the driver a little bigger(2 x 11 x 64 bytes) and beyond the FIFO job. So where Should I place the huffman table? Put it in the driver. It's less than 1.5 kB, so really small. I see. I'm not sure what you mean with 'beyond the FIFO job' though. I always been taught that the kernel driver is just FIFO between userspace and hardware. Next time I would learn those small thing(even hard code) could be stored in kernel source code. My understanding is that there 10 quality levels, each with its own huffman table? Right. But I still asking what the relationship is to the standard quality in JPEG from 0 to 100 to the other staff. So the driver would implement the V4L2_CID_JPEG_COMPRESSION_QUALITY control and for each quality level it picks a table. Makes sense to me. Let looks I could omit a extra JPEG control then. Regards, Thank you. P.S the encoder for VP8 and H.264 is really very hard. I really need more time to do them. There is a new VA-API driver[1] which removed the most possible information from the third party library(but still need it) and using most decoder settings from the VA-API client. The new kernel driver is in schedule as well. [1] https://github.com/hizukiayaka/rockchip-video-driver/tree/rk_v4l2_mix Hans -- Randy Li The third produce department === This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. [Fuzhou Rockchip Electronics, INC. China mainland] === -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
media: rockchip-vpu: I should place the huffman table at kernel or userspace ?
Hello: I have just done a JPEG HW encoder for the RK3288. I have been told that I can't use the standard way to generate huffman table, the VPU supports only 10 levels with a different huffman table. If I send the huffman table through the v4l2 extra control, the memory copy is requested, although the data is not very large(2 x 64 bytes) but still a overhead. The other way is to place them in the kernel driver, and just define the quality every time it encode a picture. But storing in kernel would make the driver a little bigger(2 x 11 x 64 bytes) and beyond the FIFO job. So where Should I place the huffman table? -- Randy Li The third produce department === This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. [Fuzhou Rockchip Electronics, INC. China mainland] === -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Summary of the discussion about Rockchip VPU in Gstreamer
Hello all media staff Dear Mr.Verkuil Dear Mr.Osciak I talked with Nicolas and Mr.ceyusa in the yesterday and early morning of today. I think I have made them get the situation of state-less Video Acceleration Unit(VPU) and Rockchip for VA-API driver. We both agree that creating a new C API bindings to V4L2 is making wheel again. Mr.Ceyusa suggest that there could be a middle library to parse those codec settings to V4L2 extra controls array, and push back to Gstreamer, leaving the V4L2 related job to Gstreamer. I agree with him. I think the Gstreamer then could get rid of hardware detail, even not need to know internal data structure in kernel driver of codec parameters. Later, the ad-n770 joined us. He gave me some idea about the relationship with VA-API and DXVA2. I found we do need some extra data beyond those data used by VA-API to reconstruct a frame, it is a limitation in HW. We better regard this kind of HW to a Acceleration Unit rather than Full decoder/Encoder. Also ad-n770 pointer out that it seems that Rockchip HW could do the reodering job, which is not need actually as it is done by Gstreamer, but I am not sure whether the Hardware does and I could disable this logic. I am sorry I can't attend the conference in Berlin. But I hope we could keep in discussion in this topic, and offering more information to you before the meetings. Currently, I would still work on VA-API framework and I am learning something about codec through a book, I hope that it make me explaining the situation easily. -- Randy Li The third produce department === This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. [Fuzhou Rockchip Electronics, INC. China mainland] === -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] v4l2-ctrls: add generic H.264 decoder codec settings structure
The generic decoder settings for H.264. It is modified from
the VA-API. Adding the extra data required by the Rockchip.
Signed-off-by: Randy Li <ay...@soulik.info>
---
include/uapi/linux/videodev2.h | 103 +
1 file changed, 103 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 904c44c..3da 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2214,6 +2214,109 @@ struct v4l2_request_cmd {
} raw;
};
};
+
+/* H.264 Codec settings */
+struct v4l2_mpeg_video_h264_picture {
+ __u32 frame_idx;
+ __u32 flags;
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+};
+
+struct v4l2_mpeg_video_h264_picture_param {
+ struct v4l2_mpeg_video_h264_picture curr_pic;
+ struct v4l2_mpeg_video_h264_picture reference_frames[16];
+ __u16 picture_width_in_mbs_minus1;
+ __u16 picture_height_in_mbs_minus1;
+ __u8 bit_depth_luma_minus8;
+ __u8 bit_depth_chroma_minus8;
+ __u8 num_ref_frames;
+ union {
+ struct {
+ __u32 chroma_format_idc:2;
+ __u32 residual_colour_transform_flag:1;
+ __u32 gaps_in_frame_num_value_allowed_flag:1;
+ __u32 frame_mbs_only_flag:1;
+ __u32 mb_adaptive_frame_field_flag:1;
+ __u32 direct_8x8_inference_flag:1;
+ __u32 min_luma_bi_pred_size8x8:1;
+ __u32 log2_max_frame_num_minus4:4;
+ __u32 pic_order_cnt_type:2;
+ __u32 log2_max_pic_order_cnt_lsb_minus4:4;
+ __u32 delta_pic_order_always_zero_flag:1;
+ } bits;
+ __u32 value;
+ } seq_fields;
+ __u8 num_slice_groups_minus1;
+ __u8 slice_group_map_type;
+ __u16 slice_group_change_rate_minus1;
+ __s8 pic_init_qp_minus26;
+ __s8 pic_init_qs_minus26;
+ __s8 chroma_qp_index_offset;
+ __s8 second_chroma_qp_index_offset;
+ union {
+ struct {
+ __u32 entropy_coding_mode_flag:1;
+ __u32 weighted_pred_flag:1
+ __u32 weighted_bipred_idc:2;
+ __u32 transform_8x8_mode_flag:1;
+ __u32 field_pic_flag:1;
+ __u32 constrained_intra_pred_flag:1;
+ __u32 pic_order_present_flag:1;
+ __u32 deblocking_filter_control_present_flag:1;
+ __u32 redundant_pic_cnt_present_flag:1;
+ __u32 reference_pic_flag:1;
+ } bits;
+ __u32 value;
+ } pic_fields;
+ __u16 frame_num;
+ /*
+* Some extra data required by Rockchip, the decoder in RK serial
+* chip would omit some part of data
+*/
+ struct {
+ __u8 profile;
+ __u8 constraint_set_flags;
+ __u32 pic_order_cnt_bit_size;
+ __u32 dec_ref_pic_marking_bit_size;
+ __u16 idr_pic_id;
+ } extra;
+};
+
+struct v4l2_mpeg_video_h264_slice_param {
+{
+ __u32 slice_data_size;
+ __u32 slice_data_offset;
+ __u32 slice_data_flag;
+ __u16 slice_data_bit_offset;
+ __u16 first_mb_in_slice;
+ __u8 slice_type;
+ __u8 direct_spatial_mv_pred_flag;
+ __u8 num_ref_idx_l0_active_minus1;
+ __u8 num_ref_idx_l1_active_minus1;
+ __u8 cabac_init_idc;
+ __s8 slice_qp_delta;
+ __u8 disable_deblocking_filter_idc;
+ __s8 slice_alpha_c0_offset_div2;
+ __s8 slice_beta_offset_div2;
+ struct v4l2_mpeg_video_h264_picture ref_pic_list0[32];
+ struct v4l2_mpeg_video_h264_picture ref_pic_list1[32];
+ __u8 luma_log2_weight_denom;
+ __u8 chroma_log2_weight_denom;
+ __u8 luma_weight_l0_flag;
+ __s16 luma_weight_l0 [32];
+ __s16 luma_offset_l0 [32];
+ __u8 chroma_weight_l0_flag;
+ __s16 chroma_weight_l0 [32][2];
+ __s16 chroma_offset_l0 [32][2];
+ __u8 luma_weight_l1_flag;
+ __s16 luma_weight_l1 [32];
+ __s16 luma_offset_l1 [32];
+ __u8 chroma_weight_l1_flag;
+ __s16 chroma_weight_l1 [32][2];
+ __s16 chroma_offset_l1 [32][2];
+};
+
/*
* I O C T L C O D E S F O R V I D E O D E V I C E S
*
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH RFC 0/2] add the generic H.264 decoder settings controls
This is not done yet. The rockchip VA-API driver[1] still need a third part library to pre-parse the nalu data. Maybe after the third part library free version[2] had done, it would be clear that we else filed we may need. Those structures comes from VA-API SPCE. But still not enough to driver a stateless video processor. For the Rockchip VPU, it won't process some part of data, like pic_order_cnt. Then you could configure a length of that part in register in order to let the video processor skip it. You may look at extra fields in struct v4l2_mpeg_video_h264_picture_param. Also there is some problem with the mechiansm of VA-API. A VA-API Surface looks like a wrapper of the buffer in output side(CAPTURE in V4L2), but for the V4L2, which buffer is dequeue from the CAPTURE is unknown. So it is a little hard to link the surface to the vpu driver internal buffer. The CREATE_BUFS ioctl may solve the allocation problem but not the dequeue. But as most stateless video processor could only process one frame in one time, I don't know the reference frame is need to keep for future parse(not by video processor but by the upper layer software), allowing only a buffer may solve this problem Some patches also reqired to make the VA-API client(like Gstreamer) to support those extra data need by the video processor. It would be done in a short time. Need someone is very familiar with the codec standard. Currently, I am doing JPEG encoder for RK3288 now, I won't be avaiable in short time and I am lack of the knowledge of codec standard. But I am scheduling the plan of the new driver for both kernel[3] and VA-API, I could do it on my free time. [1] https://github.com/rockchip-linux/rockchip-va-driver v4l2-libvpu [2] https://github.com/rockchip-linux/rockchip-va-driver rk_v4l2 [3] https://github.com/hizukiayaka/linux-kernel rk3288-media Randy Li (2): [media] v4l2-ctrls: add H.264 decoder settings controls v4l2-ctrls: add generic H.264 decoder codec settings structure drivers/media/v4l2-core/v4l2-ctrls.c | 2 + include/uapi/linux/v4l2-controls.h | 2 + include/uapi/linux/videodev2.h | 103 +++ 3 files changed, 107 insertions(+) -- 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] [media] v4l2-ctrls: add H.264 decoder settings controls
These two controls would be used to set the H.264 codec settings
for decoder.
Signed-off-by: Randy Li <ay...@soulik.info>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++
include/uapi/linux/v4l2-controls.h | 2 ++
2 files changed, 4 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index 60056b0..789a5fc 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -740,6 +740,8 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264
Number of HC Layers";
case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
return "H264
Set QP Value for HC Layers";
+ case V4L2_CID_MPEG_VIDEO_H264_PICTURE_PARAM:return "H264
Picture Parameter";
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264
Slice Parameter";
case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4
I-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4
P-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4
B-Frame QP Value";
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index b6a357a..5b0bdc5 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -521,6 +521,8 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
};
#define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER
(V4L2_CID_MPEG_BASE+381)
#define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP
(V4L2_CID_MPEG_BASE+382)
+#define V4L2_CID_MPEG_VIDEO_H264_PICTURE_PARAM
(V4L2_CID_MPEG_BASE+383)
+#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM
(V4L2_CID_MPEG_BASE+384)
#define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
#define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
#define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
--
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: MFC: different h264 profile and level output the same size encoded result
On 08/29/2016 01:49 PM, Andrzej Hajda wrote: Hi, On 08/27/2016 11:55 AM, Randy Li wrote: Hi: I have been reported that the setting the profile, level and bitrate through the v4l2 extra controls would not make the encoded result different. I tried it recently, it is true. Although the h264 parser would tell me the result have been applied as different h264 profile and level, but size is the same. You may try this in Gstreamer. gst-launch-1.0 -v \ videotestsrc num-buffers=500 ! video/x-raw, width=1920,height=1080 ! \ videoconvert ! \ v4l2video4h264enc extra-controls="controls,h264_profile=1,video_bitrate=100;" ! \ h264parse ! matroskamux ! filesink location=/tmp/1.mkv Is there any way to reduce the size of MFC encoded data? There is control called rc_enable (rate control enable), it must be set to one if you want to control bitrate. This control confuses many users, I guess it cannot be removed as it is already part of UAPI, but enabling it internally by the driver if user sets bitrate, profille, etc, would make it more saner. I see, thank you so much. A guy told me that the "frame_level_rate_control_enable=1" in _ extra-controls="encode,h264_level=10,h264_profile=4,frame_level_rate_control_enable=1,video_bitrate=2097152" would also make it works. But I really know there is a switch need to turn on. Regards Andrzej -- Randy Li The third produce department -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
MFC: different h264 profile and level output the same size encoded result
Hi: I have been reported that the setting the profile, level and bitrate through the v4l2 extra controls would not make the encoded result different. I tried it recently, it is true. Although the h264 parser would tell me the result have been applied as different h264 profile and level, but size is the same. You may try this in Gstreamer. gst-launch-1.0 -v \ videotestsrc num-buffers=500 ! video/x-raw, width=1920,height=1080 ! \ videoconvert ! \ v4l2video4h264enc extra-controls="controls,h264_profile=1,video_bitrate=100;" ! \ h264parse ! matroskamux ! filesink location=/tmp/1.mkv Is there any way to reduce the size of MFC encoded data? -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Plan to support Rockchip VPU in DRM, is it a good idea
On 08/26/2016 06:56 PM, Hans Verkuil wrote: On 08/26/2016 12:05 PM, Randy Li wrote: On 08/26/2016 05:34 PM, Hans Verkuil wrote: Hi Randi, On 08/26/2016 04:13 AM, Randy Li wrote: Hello, We always use some kind of hack work to make our Video Process Unit(Multi-format Video Encoder/Decoder) work in kernel. From a customize driver(vpu service) to the customize V4L2 driver. The V4L2 subsystem is really not suitable for the stateless Video process or it could make driver too fat. After talking to some kindness Intel guys and moving our userspace library to VA-API driver, I find the DRM may the good choice for us. But I don't know whether it is welcome to to submit a video driver in DRM subsystem? Also our VPU(Video process unit) is not just like the Intel's, we don't have VCS, we based on registers to set the encoder/decoder. I think we may need a lots of IOCTL then. Also we do have a IOMMU in VPU but also not a isolated memory for VPU, I don't know I should use TT memory or GEM memory. I am actually not a member of the department in charge of VPU, and I am just beginning to learning DRM(thank the help from Intel again), I am not so good at memory part as well(I am more familiar with CMA not the IOMMU way), I may need know guide about the implementations when I am going to submit driver, I hope I could get help from someone. It makes no sense to do this in the DRM subsystem IMHO. There are already quite a few HW codecs implemented in the V4L2 subsystem and more are in the pipeline. Putting codec support in different subsystems will just make userspace software much harder to write. One of the codecs that was posted to linux-media was actually from Rockchip: https://lkml.org/lkml/2016/2/29/861 There is also a libVA driver (I think) that sits on top of it: https://github.com/rockchip-linux/rockchip-va-driver/tree/v4l2-libvpu It is old version, I am the author of this https://github.com/rockchip-linux/rockchip-va-driver For the Allwinner a patch series was posted yesterday: https://lkml.org/lkml/2016/8/25/246 They created a pretty generic libVA userspace that looks very promising at first glance. What these have in common is that they depend on the Request API and Frame API, neither of which has been merged. The problem is that the Request API requires more work since not only controls have to be part of a request, but also formats, selection rectangles, and even dynamic routing changes. While that is not relevant for codecs, it is relevant for Android CameraHAL in general and complex devices like Google's Project Ara. Actually just as the Intel did, our hardware decoder/encoder need full settings for them, most of them are relevant to the codec. You may notice that there is four extra control need to be set before. If the libvpu(a helper library we offered to parse each slice to generate decoder settings) is remove(in process now, only three decoder settings can't got from VA-API directly), it would be more clearly. We really a lots decoder settings information to make the decoder work. This is being worked on, but it is simply not yet ready. The core V4L2 developers involved in this plan to discuss this on the Monday before the ELCE in Berlin, to see if we can fast track this work somehow so this support can be merged. I am glad to hear that. I hope that I could have an opportunity to show our problems. If there are missing features in V4L2 (other that the two APIs discussed above) that prevent you from creating a good driver, then please discuss that with us. We are always open to suggestions and improvements and want to work with you on that. I have a few experience with the s5p-mfc, and I do wrote a V4L2 encoder plugin for Gstreamer. I don't think the V4L2 is good place for us stateless video processor, unless it would break the present implementation. The stateful and stateless are operated quite differently. The stateless must parse the header and set those settings for every frames. The request data may quite different from vendor to vendor, even chip to chip. It is impossible to make a common way to send those settings to driver.For the samsung MFC, you don't need to do any parse work at all. Anyway, I would like to follow what Intel does now, we are both stateless video processor. I don't see the problem. As I understand it what the hardware needs is the video data and settings (i.e. 'state'). It will process the video data (encode or decode) and return the result (probably with additional settings/state). V4L2 + Request API does exactly that. What does DRM offer you that makes life Actually I don't reject the new framework, I also have heard this new API before. But the last update of Request API is 2015 and still not been merged yet. The DRM looks more stable and less limit. I still doesn't know how to implement a new V4L2 Request API. I would like to last prototype of V4L2 Request API. easier for you compared to V4L2? I am not aware
Re: Plan to support Rockchip VPU in DRM, is it a good idea
On 08/26/2016 05:34 PM, Hans Verkuil wrote: Hi Randi, On 08/26/2016 04:13 AM, Randy Li wrote: Hello, We always use some kind of hack work to make our Video Process Unit(Multi-format Video Encoder/Decoder) work in kernel. From a customize driver(vpu service) to the customize V4L2 driver. The V4L2 subsystem is really not suitable for the stateless Video process or it could make driver too fat. After talking to some kindness Intel guys and moving our userspace library to VA-API driver, I find the DRM may the good choice for us. But I don't know whether it is welcome to to submit a video driver in DRM subsystem? Also our VPU(Video process unit) is not just like the Intel's, we don't have VCS, we based on registers to set the encoder/decoder. I think we may need a lots of IOCTL then. Also we do have a IOMMU in VPU but also not a isolated memory for VPU, I don't know I should use TT memory or GEM memory. I am actually not a member of the department in charge of VPU, and I am just beginning to learning DRM(thank the help from Intel again), I am not so good at memory part as well(I am more familiar with CMA not the IOMMU way), I may need know guide about the implementations when I am going to submit driver, I hope I could get help from someone. It makes no sense to do this in the DRM subsystem IMHO. There are already quite a few HW codecs implemented in the V4L2 subsystem and more are in the pipeline. Putting codec support in different subsystems will just make userspace software much harder to write. One of the codecs that was posted to linux-media was actually from Rockchip: https://lkml.org/lkml/2016/2/29/861 There is also a libVA driver (I think) that sits on top of it: https://github.com/rockchip-linux/rockchip-va-driver/tree/v4l2-libvpu It is old version, I am the author of this https://github.com/rockchip-linux/rockchip-va-driver For the Allwinner a patch series was posted yesterday: https://lkml.org/lkml/2016/8/25/246 They created a pretty generic libVA userspace that looks very promising at first glance. What these have in common is that they depend on the Request API and Frame API, neither of which has been merged. The problem is that the Request API requires more work since not only controls have to be part of a request, but also formats, selection rectangles, and even dynamic routing changes. While that is not relevant for codecs, it is relevant for Android CameraHAL in general and complex devices like Google's Project Ara. Actually just as the Intel did, our hardware decoder/encoder need full settings for them, most of them are relevant to the codec. You may notice that there is four extra control need to be set before. If the libvpu(a helper library we offered to parse each slice to generate decoder settings) is remove(in process now, only three decoder settings can't got from VA-API directly), it would be more clearly. We really a lots decoder settings information to make the decoder work. This is being worked on, but it is simply not yet ready. The core V4L2 developers involved in this plan to discuss this on the Monday before the ELCE in Berlin, to see if we can fast track this work somehow so this support can be merged. I am glad to hear that. I hope that I could have an opportunity to show our problems. If there are missing features in V4L2 (other that the two APIs discussed above) that prevent you from creating a good driver, then please discuss that with us. We are always open to suggestions and improvements and want to work with you on that. I have a few experience with the s5p-mfc, and I do wrote a V4L2 encoder plugin for Gstreamer. I don't think the V4L2 is good place for us stateless video processor, unless it would break the present implementation. The stateful and stateless are operated quite differently. The stateless must parse the header and set those settings for every frames. The request data may quite different from vendor to vendor, even chip to chip. It is impossible to make a common way to send those settings to driver.For the samsung MFC, you don't need to do any parse work at all. Anyway, I would like to follow what Intel does now, we are both stateless video processor. Regards, Hans ___ dri-devel mailing list dri-de...@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel -- Randy Li The third produce department -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Plan to support Rockchip VPU in DRM, is it a good idea
Hello, We always use some kind of hack work to make our Video Process Unit(Multi-format Video Encoder/Decoder) work in kernel. From a customize driver(vpu service) to the customize V4L2 driver. The V4L2 subsystem is really not suitable for the stateless Video process or it could make driver too fat. After talking to some kindness Intel guys and moving our userspace library to VA-API driver, I find the DRM may the good choice for us. But I don't know whether it is welcome to to submit a video driver in DRM subsystem? Also our VPU(Video process unit) is not just like the Intel's, we don't have VCS, we based on registers to set the encoder/decoder. I think we may need a lots of IOCTL then. Also we do have a IOMMU in VPU but also not a isolated memory for VPU, I don't know I should use TT memory or GEM memory. I am actually not a member of the department in charge of VPU, and I am just beginning to learning DRM(thank the help from Intel again), I am not so good at memory part as well(I am more familiar with CMA not the IOMMU way), I may need know guide about the implementations when I am going to submit driver, I hope I could get help from someone. -- Randy Li The third produce department -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
