Re: i.MX6 CSC and scaler
Le 2014-12-17 08:12, Jean-Michel Hautbois a écrit : [10005.819862] mx6-m2m ipum2m@ipu1: call capture S_FMT first to determine segmentation Please, find the part of the spec that would inforce that, if not inforced, do the spec work first. Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
2014-12-16 15:50 GMT+01:00 Nicolas Dufresne :
>
> Le 2014-12-16 09:27, Jean-Michel Hautbois a écrit :
>>
>> Have you already started something about that ?
>
> Yes, currently I do cropping right if there is no scaling. Doing this with
> S_CROP is really ackward, but very few m2m driver has been ported to
> G_SELECTION yet. The v4l2transform was tested to run with Exynos 4 FIMC
> driver. There was still issue with such driver doing middle rounding for
> alignment. This has been reported on this list. The proposed solution is to
> add flags to our internal alignment method, and update these driver to pick
> a direction.
Right now, I can get the driver probed (twice, as I have two IPUs) and
gstreamer creates two elements :
$> rm .cache/gstreamer-1.0/registry.arm.bin
$>
gst-inspect-1.0 |grep v4l2
video4linux2: v4l2video3videodec: V4L2 Video Decoder
video4linux2: v4l2video2h264enc: V4L2 H.264 Encoder
video4linux2: v4l2video1convert: V4L2 Video Converter
video4linux2: v4l2video0convert: V4L2 Video Converter
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2src: Video (video4linux2) Source
$> gst-inspect-1.0 v4l2video0convert
Factory Details:
Rank none (0)
Long-nameV4L2 Video Converter
KlassFilter/Converter/Video
Description Transform streams via V4L2 API
Author Nicolas Dufresne
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
Version 1.5.0.1
License LGPL
Source modulegst-plugins-good
Source release date 2014-12-15 11:41 (UTC)
Binary package GStreamer Good Plug-ins git
Origin URL Unknown package origin
GObject
+GInitiallyUnowned
+GstObject
+GstElement
+GstBaseTransform
+GstV4l2Transform
+v4l2video0convert
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { RGB16, RGB, BGR, xRGB, ARGB, BGRx, BGRA,
YUY2, UYVY, I420, YV12, NV12, NV21, Y42B }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 100/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { RGB16, RGB, BGR, xRGB, ARGB, BGRx, BGRA,
YUY2, UYVY, I420, YV12, NV12, NV21, Y42B }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 100/1 ]
Element Flags:
no flags set
Element Implementation:
Has change_state() function: gst_v4l2_transform_change_state
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name: The name of the object
flags: readable, writable
String. Default: "v4l2video0convert0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
qos : Handle Quality-of-Service events
flags: readable, writable
Boolean. Default: true
device : Device location
flags: readable
String. Default: "/dev/video0"
device-name : Name of the device
flags: readable
String. Default: "mx6-m2m"
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
output-io-mode : Output side I/O mode (matches sink pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import- GST_V4L2_IO_DMABUF_IMPORT
capture-io-mode : Capture I/O mode (matches src pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap
Re: i.MX6 CSC and scaler
2014-12-16 15:50 GMT+01:00 Nicolas Dufresne nicolas.dufre...@collabora.com:
Le 2014-12-16 09:27, Jean-Michel Hautbois a écrit :
Have you already started something about that ?
Yes, currently I do cropping right if there is no scaling. Doing this with
S_CROP is really ackward, but very few m2m driver has been ported to
G_SELECTION yet. The v4l2transform was tested to run with Exynos 4 FIMC
driver. There was still issue with such driver doing middle rounding for
alignment. This has been reported on this list. The proposed solution is to
add flags to our internal alignment method, and update these driver to pick
a direction.
Right now, I can get the driver probed (twice, as I have two IPUs) and
gstreamer creates two elements :
$ rm .cache/gstreamer-1.0/registry.arm.bin
$
gst-inspect-1.0 |grep v4l2
video4linux2: v4l2video3videodec: V4L2 Video Decoder
video4linux2: v4l2video2h264enc: V4L2 H.264 Encoder
video4linux2: v4l2video1convert: V4L2 Video Converter
video4linux2: v4l2video0convert: V4L2 Video Converter
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2src: Video (video4linux2) Source
$ gst-inspect-1.0 v4l2video0convert
Factory Details:
Rank none (0)
Long-nameV4L2 Video Converter
KlassFilter/Converter/Video
Description Transform streams via V4L2 API
Author Nicolas Dufresne nicolas.dufre...@collabora.com
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
Version 1.5.0.1
License LGPL
Source modulegst-plugins-good
Source release date 2014-12-15 11:41 (UTC)
Binary package GStreamer Good Plug-ins git
Origin URL Unknown package origin
GObject
+GInitiallyUnowned
+GstObject
+GstElement
+GstBaseTransform
+GstV4l2Transform
+v4l2video0convert
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { RGB16, RGB, BGR, xRGB, ARGB, BGRx, BGRA,
YUY2, UYVY, I420, YV12, NV12, NV21, Y42B }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 100/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { RGB16, RGB, BGR, xRGB, ARGB, BGRx, BGRA,
YUY2, UYVY, I420, YV12, NV12, NV21, Y42B }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 100/1 ]
Element Flags:
no flags set
Element Implementation:
Has change_state() function: gst_v4l2_transform_change_state
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name: The name of the object
flags: readable, writable
String. Default: v4l2video0convert0
parent : The parent of the object
flags: readable, writable
Object of type GstObject
qos : Handle Quality-of-Service events
flags: readable, writable
Boolean. Default: true
device : Device location
flags: readable
String. Default: /dev/video0
device-name : Name of the device
flags: readable
String. Default: mx6-m2m
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
output-io-mode : Output side I/O mode (matches sink pad)
flags: readable, writable
Enum GstV4l2IOMode Default: 0, auto
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import- GST_V4L2_IO_DMABUF_IMPORT
capture-io-mode : Capture I/O mode (matches src pad)
flags: readable, writable
Enum GstV4l2IOMode Default: 0, auto
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
Re: i.MX6 CSC and scaler
Le 2014-12-17 08:12, Jean-Michel Hautbois a écrit : [10005.819862] mx6-m2m ipum2m@ipu1: call capture S_FMT first to determine segmentation Please, find the part of the spec that would inforce that, if not inforced, do the spec work first. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Le 2014-12-16 09:27, Jean-Michel Hautbois a écrit : Have you already started something about that ? Yes, currently I do cropping right if there is no scaling. Doing this with S_CROP is really ackward, but very few m2m driver has been ported to G_SELECTION yet. The v4l2transform was tested to run with Exynos 4 FIMC driver. There was still issue with such driver doing middle rounding for alignment. This has been reported on this list. The proposed solution is to add flags to our internal alignment method, and update these driver to pick a direction. Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Hi, 2014-12-16 15:15 GMT+01:00 Nicolas Dufresne : > > Le 2014-12-16 08:30, Philipp Zabel a écrit : >> >> For the IPU IC mem2mem scaler/CSC, v4l2transform with scaling capability >> would be the perfect fit > > Thanks you for clarifying. Jean-Michel, are you going to work on this ? If > so, feel free to contact me directly for pointers. You'll find me as stormer > on IRC freenode on both #v4l2 and #gstreamer. Well, yes, we are going to work on it, but first we need to have Steve's module probed. It seems to be using DT, but figuring out how to use it properly. > What would need to be done: > a) Import scaler negotiation code from videoscale element, and merge it into > v4l2transform negotiation. It is likely that wtay (Wim Tayman) have some > code already for you, has on the SW side he is currently working on merging > videoscale and videoconvert together as it's faster this way. OK, so better to go on #gstreamer and talk together with Wim of this. > b) Complete the handling of G_CROP (if required add G_SELECTION). Most HW > scaler will allocate aligned buffers, but can reach a 1 or 2 pixels > precision through CROP or SELECTION. Have you already started something about that ? JM -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Le 2014-12-16 08:30, Philipp Zabel a écrit : For the IPU IC mem2mem scaler/CSC, v4l2transform with scaling capability would be the perfect fit Thanks you for clarifying. Jean-Michel, are you going to work on this ? If so, feel free to contact me directly for pointers. You'll find me as stormer on IRC freenode on both #v4l2 and #gstreamer. What would need to be done: a) Import scaler negotiation code from videoscale element, and merge it into v4l2transform negotiation. It is likely that wtay (Wim Tayman) have some code already for you, has on the SW side he is currently working on merging videoscale and videoconvert together as it's faster this way. b) Complete the handling of G_CROP (if required add G_SELECTION). Most HW scaler will allocate aligned buffers, but can reach a 1 or 2 pixels precision through CROP or SELECTION. cheers, Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Am Montag, den 15.12.2014, 22:27 +0100 schrieb Jean-Michel Hautbois: > 2014-12-15 19:52 GMT+01:00 Nicolas Dufresne : > > Le 12/15/2014 01:23 PM, Steve Longerbeam a écrit : > >> You would need to write a gstreamer plugin to make use of the > >> mem2mem device for h/w CSC and scaling. > > Thank you Steve, this is of a great help ! > > > There is an element already (v4l2transform), though it currently only > > handle color conversion. I'd like to see scaling in that too instead of > > having a separate element that only do scaling like: > > > > https://bugzilla.gnome.org/show_bug.cgi?id=726104 > > Nice idea. You are probably right Nicolas, it would make sense to be > able to scale and convert color in the same element. > > > I think Michael and Philipp idea was that scaling could be done directly > > out of the decoder, something supported by CODA driver. There is some > > work require to get that in GStreamer, see: > > > > https://bugzilla.gnome.org/show_bug.cgi?id=733827 (pixel format nego) > > https://bugzilla.gnome.org/show_bug.cgi?id=733828 (display size nego) > > There is a need for a separate element too. So it could be handled in > front of the encoder if needed, on decoder output if needed, but > should be a single element too, instanciable multiple times if > multiple inputs/outputs are needed. Scaling in the decoder element is desirable, but unrelated. I wanted that for the CODA JPEG decoder, which can produce decimated output right during the decoding process. For the IPU IC mem2mem scaler/CSC, v4l2transform with scaling capability would be the perfect fit. regards Philipp -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Am Montag, den 15.12.2014, 22:27 +0100 schrieb Jean-Michel Hautbois: 2014-12-15 19:52 GMT+01:00 Nicolas Dufresne nicolas.dufre...@collabora.com: Le 12/15/2014 01:23 PM, Steve Longerbeam a écrit : You would need to write a gstreamer plugin to make use of the mem2mem device for h/w CSC and scaling. Thank you Steve, this is of a great help ! There is an element already (v4l2transform), though it currently only handle color conversion. I'd like to see scaling in that too instead of having a separate element that only do scaling like: https://bugzilla.gnome.org/show_bug.cgi?id=726104 Nice idea. You are probably right Nicolas, it would make sense to be able to scale and convert color in the same element. I think Michael and Philipp idea was that scaling could be done directly out of the decoder, something supported by CODA driver. There is some work require to get that in GStreamer, see: https://bugzilla.gnome.org/show_bug.cgi?id=733827 (pixel format nego) https://bugzilla.gnome.org/show_bug.cgi?id=733828 (display size nego) There is a need for a separate element too. So it could be handled in front of the encoder if needed, on decoder output if needed, but should be a single element too, instanciable multiple times if multiple inputs/outputs are needed. Scaling in the decoder element is desirable, but unrelated. I wanted that for the CODA JPEG decoder, which can produce decimated output right during the decoding process. For the IPU IC mem2mem scaler/CSC, v4l2transform with scaling capability would be the perfect fit. regards Philipp -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Le 2014-12-16 08:30, Philipp Zabel a écrit : For the IPU IC mem2mem scaler/CSC, v4l2transform with scaling capability would be the perfect fit Thanks you for clarifying. Jean-Michel, are you going to work on this ? If so, feel free to contact me directly for pointers. You'll find me as stormer on IRC freenode on both #v4l2 and #gstreamer. What would need to be done: a) Import scaler negotiation code from videoscale element, and merge it into v4l2transform negotiation. It is likely that wtay (Wim Tayman) have some code already for you, has on the SW side he is currently working on merging videoscale and videoconvert together as it's faster this way. b) Complete the handling of G_CROP (if required add G_SELECTION). Most HW scaler will allocate aligned buffers, but can reach a 1 or 2 pixels precision through CROP or SELECTION. cheers, Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Hi, 2014-12-16 15:15 GMT+01:00 Nicolas Dufresne nicolas.dufre...@collabora.com: Le 2014-12-16 08:30, Philipp Zabel a écrit : For the IPU IC mem2mem scaler/CSC, v4l2transform with scaling capability would be the perfect fit Thanks you for clarifying. Jean-Michel, are you going to work on this ? If so, feel free to contact me directly for pointers. You'll find me as stormer on IRC freenode on both #v4l2 and #gstreamer. Well, yes, we are going to work on it, but first we need to have Steve's module probed. It seems to be using DT, but figuring out how to use it properly. What would need to be done: a) Import scaler negotiation code from videoscale element, and merge it into v4l2transform negotiation. It is likely that wtay (Wim Tayman) have some code already for you, has on the SW side he is currently working on merging videoscale and videoconvert together as it's faster this way. OK, so better to go on #gstreamer and talk together with Wim of this. b) Complete the handling of G_CROP (if required add G_SELECTION). Most HW scaler will allocate aligned buffers, but can reach a 1 or 2 pixels precision through CROP or SELECTION. Have you already started something about that ? JM -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Le 2014-12-16 09:27, Jean-Michel Hautbois a écrit : Have you already started something about that ? Yes, currently I do cropping right if there is no scaling. Doing this with S_CROP is really ackward, but very few m2m driver has been ported to G_SELECTION yet. The v4l2transform was tested to run with Exynos 4 FIMC driver. There was still issue with such driver doing middle rounding for alignment. This has been reported on this list. The proposed solution is to add flags to our internal alignment method, and update these driver to pick a direction. Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
2014-12-15 19:52 GMT+01:00 Nicolas Dufresne : > > > Le 12/15/2014 01:23 PM, Steve Longerbeam a écrit : >> You would need to write a gstreamer plugin to make use of the >> mem2mem device for h/w CSC and scaling. Thank you Steve, this is of a great help ! > There is an element already (v4l2transform), though it currently only > handle color conversion. I'd like to see scaling in that too instead of > having a separate element that only do scaling like: > > https://bugzilla.gnome.org/show_bug.cgi?id=726104 Nice idea. You are probably right Nicolas, it would make sense to be able to scale and convert color in the same element. > I think Michael and Philipp idea was that scaling could be done directly > out of the decoder, something supported by CODA driver. There is some > work require to get that in GStreamer, see: > > https://bugzilla.gnome.org/show_bug.cgi?id=733827 (pixel format nego) > https://bugzilla.gnome.org/show_bug.cgi?id=733828 (display size nego) There is a need for a separate element too. So it could be handled in front of the encoder if needed, on decoder output if needed, but should be a single element too, instanciable multiple times if multiple inputs/outputs are needed. JM -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
Le 12/15/2014 01:23 PM, Steve Longerbeam a écrit : > You would need to write a gstreamer plugin to make use of the > mem2mem device for h/w CSC and scaling. There is an element already (v4l2transform), though it currently only handle color conversion. I'd like to see scaling in that too instead of having a separate element that only do scaling like: https://bugzilla.gnome.org/show_bug.cgi?id=726104 I think Michael and Philipp idea was that scaling could be done directly out of the decoder, something supported by CODA driver. There is some work require to get that in GStreamer, see: https://bugzilla.gnome.org/show_bug.cgi?id=733827 (pixel format nego) https://bugzilla.gnome.org/show_bug.cgi?id=733828 (display size nego) cheers, Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
On 12/15/2014 09:03 AM, Jean-Michel Hautbois wrote:
> Hi Steve, Philipp,
>
> I see in the kernel sources you created a ipu-ic.c file which helps
> working with the Color Space Converter on i.MX6.
> We would like to use it on our board with GStreamer, as the conversion
> done by the CPU is not very efficient :).
> What is the easiest/best approach to this ?
> Should we create a video device using a new driver, which would be
> instanciated as a /dev/videoX and see as a transform element in
> GStreamer ?
> The idea is to have a separate element, ideally without memory copy or
> anything like that.
Hi JM,
I've written a mem2mem driver for this. It does tiling to support
> 1024x1024 scaled output frames. I've attached it, feel free to use
it, it was pulled from a 3.14 kernel. I haven't submitted this driver to
community yet, mostly because I want to improve it, most importantly
move the tiling support into ipu-ic, so that tiling could eventually be used by
any media-device enabled pipeline elements (instead of only mem2mem).
Also Philipp probably also has a mem2mem device as well.
You would need to write a gstreamer plugin to make use of the
mem2mem device for h/w CSC and scaling.
Steve
/*
* This is a mem2mem driver for the Freescale i.MX6 SOC. It carries out
* color-space conversion, downsizing, resizing, and rotation transformations
* on input buffers using the IPU Image Converter's Post-Processing task.
*
* Based on mem2mem_testdev.c by Pawel Osciak.
*
* Copyright (c) 2012-2013 Mentor Graphics Inc.
* Steve Longerbeam
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the
* License, or (at your option) any later version
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
MODULE_DESCRIPTION("i.MX6 Post-Processing mem2mem device");
MODULE_AUTHOR("Steve Longerbeam ");
MODULE_LICENSE("GPL");
MODULE_VERSION("0.1");
static int instrument;
module_param(instrument, int, 0);
MODULE_PARM_DESC(instrument, "1 = enable conversion time measurement");
#define MIN_W 128
#define MIN_H 128
#define MAX_W 4096
#define MAX_H 4096
#define S_ALIGN1 /* multiple of 2 pixels */
/*
* The IC Resizer has a restriction that the output frame from the
* resizer must be 1024 or less in both width (pixels) and height
* (lines).
*
* This driver attempts to split up a conversion when the desired
* capture (output) frame resolution exceeds the IC resizer limit
* of 1024 in either dimension.
*
* If either dimension of the output frame exceeds the limit, the
* dimension is split into 1, 2, or 4 equal stripes, for a maximum
* of 4*4 or 16 segments. A conversion is then carried out for each
* segment (but taking care to pass the full frame stride length to
* the DMA channel's parameter memory!). IDMA double-buffering is used
* to convert each segment back-to-back when possible (see note below
* when double_buffering boolean is set).
*
* Note that the input frame must be split up into the same number
* of segments as the output frame.
*/
#define MAX_SEG_W4
#define MAX_SEG_H4
#define MAX_SEGMENTS (MAX_SEG_W * MAX_SEG_H)
/* Flags that indicate a format can be used for capture/output */
#define MEM2MEM_CAPTURE (1 << 0)
#define MEM2MEM_OUTPUT (1 << 1)
#define MEM2MEM_NAME "mx6-m2m"
/* Per queue */
#define MEM2MEM_DEF_NUM_BUFS VIDEO_MAX_FRAME
/* In bytes, per queue */
#define MEM2MEM_VID_MEM_LIMIT SZ_256M
#define dprintk(dev, fmt, arg...) \
v4l2_dbg(1, 1, >v4l2_dev, "%s: " fmt, __func__, ## arg)
struct m2mx6_pixfmt {
char *name;
u32 fourcc;
int depth; /* total bpp */
int y_depth; /* depth of Y plane for planar formats */
int uv_width_dec; /* decimation in width for U/V planes */
int uv_height_dec; /* decimation in height for U/V planes */
booluv_packed; /* partial planar (U and V in same plane) */
u32 types; /* Types the format can be used for */
};
struct m2mx6_seg_off {
/* start Y or packed offset of this segment */
u32 offset;
/* offset from start to segment in U plane, for planar formats */
u32 u_off;
/* offset from start to segment in V plane, for planar formats */
u32 v_off;
};
/* Per-queue, driver-specific private data */
struct m2mx6_q_data {
unsigned int width;
unsigned int height;
unsigned int bytesperline;
unsigned int stride;
unsigned int rot_stride;
unsigned int sizeimage;
struct m2mx6_pixfmt *fmt;
dma_addr_tphys_start;
struct m2mx6_seg_off seg_off[MAX_SEGMENTS];
/* width of each segment */
unsigned int seg_width;
/* height of each segment */
unsigned int seg_height;
};
struct m2mx6_dev {
struct v4l2_device v4l2_dev;
struct video_device *vfd;
struct mutex dev_mutex;
spinlock_t
i.MX6 CSC and scaler
Hi Steve, Philipp, I see in the kernel sources you created a ipu-ic.c file which helps working with the Color Space Converter on i.MX6. We would like to use it on our board with GStreamer, as the conversion done by the CPU is not very efficient :). What is the easiest/best approach to this ? Should we create a video device using a new driver, which would be instanciated as a /dev/videoX and see as a transform element in GStreamer ? The idea is to have a separate element, ideally without memory copy or anything like that. I have a similar question, regarding the scaler part. It seems to be limited by nature to 1024x1024 frames. In FSL BSP there is a split to have greater frames. Is there something doable right now with the kernel, or should we write something else ? Thanks ! JM -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
i.MX6 CSC and scaler
Hi Steve, Philipp, I see in the kernel sources you created a ipu-ic.c file which helps working with the Color Space Converter on i.MX6. We would like to use it on our board with GStreamer, as the conversion done by the CPU is not very efficient :). What is the easiest/best approach to this ? Should we create a video device using a new driver, which would be instanciated as a /dev/videoX and see as a transform element in GStreamer ? The idea is to have a separate element, ideally without memory copy or anything like that. I have a similar question, regarding the scaler part. It seems to be limited by nature to 1024x1024 frames. In FSL BSP there is a split to have greater frames. Is there something doable right now with the kernel, or should we write something else ? Thanks ! JM -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
On 12/15/2014 09:03 AM, Jean-Michel Hautbois wrote:
Hi Steve, Philipp,
I see in the kernel sources you created a ipu-ic.c file which helps
working with the Color Space Converter on i.MX6.
We would like to use it on our board with GStreamer, as the conversion
done by the CPU is not very efficient :).
What is the easiest/best approach to this ?
Should we create a video device using a new driver, which would be
instanciated as a /dev/videoX and see as a transform element in
GStreamer ?
The idea is to have a separate element, ideally without memory copy or
anything like that.
Hi JM,
I've written a mem2mem driver for this. It does tiling to support
1024x1024 scaled output frames. I've attached it, feel free to use
it, it was pulled from a 3.14 kernel. I haven't submitted this driver to
community yet, mostly because I want to improve it, most importantly
move the tiling support into ipu-ic, so that tiling could eventually be used by
any media-device enabled pipeline elements (instead of only mem2mem).
Also Philipp probably also has a mem2mem device as well.
You would need to write a gstreamer plugin to make use of the
mem2mem device for h/w CSC and scaling.
Steve
/*
* This is a mem2mem driver for the Freescale i.MX6 SOC. It carries out
* color-space conversion, downsizing, resizing, and rotation transformations
* on input buffers using the IPU Image Converter's Post-Processing task.
*
* Based on mem2mem_testdev.c by Pawel Osciak.
*
* Copyright (c) 2012-2013 Mentor Graphics Inc.
* Steve Longerbeam steve_longerb...@mentor.com
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the
* License, or (at your option) any later version
*/
#include linux/module.h
#include linux/delay.h
#include linux/fs.h
#include linux/timer.h
#include linux/sched.h
#include linux/slab.h
#include linux/log2.h
#include linux/platform_device.h
#include linux/interrupt.h
#include media/v4l2-mem2mem.h
#include media/v4l2-device.h
#include media/v4l2-ioctl.h
#include media/videobuf2-dma-contig.h
#include video/imx-ipu-v3.h
#include media/imx6.h
MODULE_DESCRIPTION(i.MX6 Post-Processing mem2mem device);
MODULE_AUTHOR(Steve Longerbeam steve_longerb...@mentor.com);
MODULE_LICENSE(GPL);
MODULE_VERSION(0.1);
static int instrument;
module_param(instrument, int, 0);
MODULE_PARM_DESC(instrument, 1 = enable conversion time measurement);
#define MIN_W 128
#define MIN_H 128
#define MAX_W 4096
#define MAX_H 4096
#define S_ALIGN1 /* multiple of 2 pixels */
/*
* The IC Resizer has a restriction that the output frame from the
* resizer must be 1024 or less in both width (pixels) and height
* (lines).
*
* This driver attempts to split up a conversion when the desired
* capture (output) frame resolution exceeds the IC resizer limit
* of 1024 in either dimension.
*
* If either dimension of the output frame exceeds the limit, the
* dimension is split into 1, 2, or 4 equal stripes, for a maximum
* of 4*4 or 16 segments. A conversion is then carried out for each
* segment (but taking care to pass the full frame stride length to
* the DMA channel's parameter memory!). IDMA double-buffering is used
* to convert each segment back-to-back when possible (see note below
* when double_buffering boolean is set).
*
* Note that the input frame must be split up into the same number
* of segments as the output frame.
*/
#define MAX_SEG_W4
#define MAX_SEG_H4
#define MAX_SEGMENTS (MAX_SEG_W * MAX_SEG_H)
/* Flags that indicate a format can be used for capture/output */
#define MEM2MEM_CAPTURE (1 0)
#define MEM2MEM_OUTPUT (1 1)
#define MEM2MEM_NAME mx6-m2m
/* Per queue */
#define MEM2MEM_DEF_NUM_BUFS VIDEO_MAX_FRAME
/* In bytes, per queue */
#define MEM2MEM_VID_MEM_LIMIT SZ_256M
#define dprintk(dev, fmt, arg...) \
v4l2_dbg(1, 1, dev-v4l2_dev, %s: fmt, __func__, ## arg)
struct m2mx6_pixfmt {
char *name;
u32 fourcc;
int depth; /* total bpp */
int y_depth; /* depth of Y plane for planar formats */
int uv_width_dec; /* decimation in width for U/V planes */
int uv_height_dec; /* decimation in height for U/V planes */
booluv_packed; /* partial planar (U and V in same plane) */
u32 types; /* Types the format can be used for */
};
struct m2mx6_seg_off {
/* start Y or packed offset of this segment */
u32 offset;
/* offset from start to segment in U plane, for planar formats */
u32 u_off;
/* offset from start to segment in V plane, for planar formats */
u32 v_off;
};
/* Per-queue, driver-specific private data */
struct m2mx6_q_data {
unsigned int width;
unsigned int height;
unsigned int bytesperline;
unsigned int stride;
unsigned int rot_stride;
unsigned int sizeimage;
struct m2mx6_pixfmt *fmt;
dma_addr_tphys_start;
struct
Re: i.MX6 CSC and scaler
Le 12/15/2014 01:23 PM, Steve Longerbeam a écrit : You would need to write a gstreamer plugin to make use of the mem2mem device for h/w CSC and scaling. There is an element already (v4l2transform), though it currently only handle color conversion. I'd like to see scaling in that too instead of having a separate element that only do scaling like: https://bugzilla.gnome.org/show_bug.cgi?id=726104 I think Michael and Philipp idea was that scaling could be done directly out of the decoder, something supported by CODA driver. There is some work require to get that in GStreamer, see: https://bugzilla.gnome.org/show_bug.cgi?id=733827 (pixel format nego) https://bugzilla.gnome.org/show_bug.cgi?id=733828 (display size nego) cheers, Nicolas -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: i.MX6 CSC and scaler
2014-12-15 19:52 GMT+01:00 Nicolas Dufresne nicolas.dufre...@collabora.com: Le 12/15/2014 01:23 PM, Steve Longerbeam a écrit : You would need to write a gstreamer plugin to make use of the mem2mem device for h/w CSC and scaling. Thank you Steve, this is of a great help ! There is an element already (v4l2transform), though it currently only handle color conversion. I'd like to see scaling in that too instead of having a separate element that only do scaling like: https://bugzilla.gnome.org/show_bug.cgi?id=726104 Nice idea. You are probably right Nicolas, it would make sense to be able to scale and convert color in the same element. I think Michael and Philipp idea was that scaling could be done directly out of the decoder, something supported by CODA driver. There is some work require to get that in GStreamer, see: https://bugzilla.gnome.org/show_bug.cgi?id=733827 (pixel format nego) https://bugzilla.gnome.org/show_bug.cgi?id=733828 (display size nego) There is a need for a separate element too. So it could be handled in front of the encoder if needed, on decoder output if needed, but should be a single element too, instanciable multiple times if multiple inputs/outputs are needed. JM -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
