Re: [PATCH v5 00/12] imx-media: Fixes for interlaced capture
Hi Hans, On 12/7/18 5:35 AM, Hans Verkuil wrote: Hi Steve, How to proceed with this w.r.t. the two gpu ipu patches? Are those going in first through the gpu tree? Or do they have to go in through our tree? There is only one remaining gpu ipu patch in this series that is awaiting an ack from Philipp: "gpu: ipu-csi: Swap fields according to input/output field types" I pinged him again. Philipp agreed to allow the two ipu patches in the series to be merged to media tree. Steve In that case I need Acks from whoever maintains that code. Regards, Hans On 10/17/2018 02:00 AM, Steve Longerbeam wrote: A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. History: v5: - Added a regression fix to allow empty endpoints to CSI (fix for imx6q SabreAuto). - Cleaned up some convoluted code in ipu_csi_init_interface(), suggested by Philipp Zabel. - Fixed a regression in csi_setup(), caught by Philipp. - Removed interweave_offset and replace with boolean interweave_swap, suggested by Philipp. - Make clear that it is IDMAC channel that does pixel reordering and interweave, not the CSI, in the imx.rst doc, caught by Philipp. v4: - rebased to latest media-tree master branch. - Make patch author and SoB email addresses the same. v3: - add support for/fix interweaved scan with YUV planar output. - fix bug in 4:2:0 U/V offset macros. - add patch that generalizes behavior of field swap in ipu_csi_init_interface(). - add support for interweaved scan with field order swap. Suggested by Philipp Zabel. - in v2, inteweave scan was determined using field types of CSI (and PRPENCVF) at the sink and source pads. In v3, this has been moved one hop downstream: interweave is now determined using field type at source pad, and field type selected at capture interface. Suggested by Philipp. - make sure to double CSI crop target height when input field type in alternate. - more updates to media driver doc to reflect above. v2: - update media driver doc. - enable idmac interweave only if input field is sequential/alternate, and output field is 'interlaced*'. - move field try logic out of *try_fmt and into separate function. - fix bug with resetting crop/compose rectangles. - add a patch that fixes a field order bug in VDIC indirect mode. - remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro Suggested-by: Nicolas Dufresne . - add macro V4L2_FIELD_IS_INTERLACED(). Steve Longerbeam (12): media: videodev2.h: Add more field helper macros gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Add planar support to interlaced scan media: imx: Fix field negotiation media: imx-csi: Input connections to CSI should be optional media: imx-csi: Double crop height for alternate fields at sink media: imx: interweave and odd-chroma-row skip are incompatible media: imx-csi: Allow skipping odd chroma rows for YVU420 media: imx: vdic: rely on VDIC for correct field order media: imx-csi: Move crop/compose reset after filling default mbus fields media: imx: Allow interweave with top/bottom lines swapped media: imx.rst: Update doc to reflect fixes to interlaced capture Documentation/media/v4l-drivers/imx.rst | 103 +++ drivers/gpu/ipu-v3/ipu-cpmem.c| 26 ++- drivers/gpu/ipu-v3/ipu-csi.c | 119 + drivers/staging/media/imx/imx-ic-prpencvf.c | 46 +++-- drivers/staging/media/imx/imx-media-capture.c | 14 ++ drivers/staging/media/imx/imx-media-csi.c | 168 +- drivers/staging/media/imx/imx-media-vdic.c| 12 +- include/uapi/linux/videodev2.h| 7 + include/video/imx-ipu-v3.h| 6 +- 9 files changed, 354 insertions(+), 147 deletions(-)
Re: [PATCH v5] media: imx: add mem2mem device
Hi Hans,
On 12/6/18 4:32 AM, Hans Verkuil wrote:
On 12/06/18 00:13, Steve Longerbeam wrote:
On 12/5/18 10:50 AM, Hans Verkuil wrote:
On 12/05/2018 02:20 AM, Steve Longerbeam wrote:
Hi Hans, Philipp,
One comment on my side...
On 12/3/18 7:21 AM, Hans Verkuil wrote:
+void imx_media_mem2mem_device_unregister(struct imx_media_video_dev *vdev)
+{
+ struct mem2mem_priv *priv = to_mem2mem_priv(vdev);
+ struct video_device *vfd = priv->vdev.vfd;
+
+ mutex_lock(>mutex);
+
+ if (video_is_registered(vfd)) {
+ video_unregister_device(vfd);
+ media_entity_cleanup(>entity);
Is this needed?
If this is to be part of the media controller, then I expect to see a call
to v4l2_m2m_register_media_controller() somewhere.
Yes, I agree there should be a call to
v4l2_m2m_register_media_controller(). This driver does not connect with
any of the imx-media entities, but calling it will at least make the
mem2mem output/capture device entities (and processing entity) visible
in the media graph.
Philipp, can you pick/squash the following from my media-tree github fork?
6fa05f5170 ("media: imx: mem2mem: Add missing media-device header")
d355bf8b15 ("media: imx: Add missing unregister and remove of mem2mem
device")
6787a50cdc ("media: imx: mem2mem: Register with media control")
Steve
Why is this driver part of the imx driver? Since it doesn't connect with
any of the imx-media entities, doesn't that mean that this is really a
stand-alone driver?
It is basically a stand-alone m2m driver, but it makes use of some
imx-media utility functions like imx_media_enum_format(). Also making it
a true stand-alone driver would require creating a second /dev/mediaN
device.
If it is standalone, is it reused in newer iMX versions? (7 or 8)
No, this driver makes use of the Image Converter in IPUv3, so it will
only run on iMX 5/6. The IPU has been dropped in iMX 7 and 8.
And if it is just a regular m2m device, then it doesn't need to create a
media device either (doesn't hurt, but it is not required).
Ok, I'll leave that up to Philipp. I don't mind either way whether it is
folded into imx-media device, or whether it is made stand-alone with or
without a new media device.
Steve
Re: [PATCH v5] media: imx: add mem2mem device
On 12/5/18 10:50 AM, Hans Verkuil wrote:
On 12/05/2018 02:20 AM, Steve Longerbeam wrote:
Hi Hans, Philipp,
One comment on my side...
On 12/3/18 7:21 AM, Hans Verkuil wrote:
+void imx_media_mem2mem_device_unregister(struct imx_media_video_dev *vdev)
+{
+ struct mem2mem_priv *priv = to_mem2mem_priv(vdev);
+ struct video_device *vfd = priv->vdev.vfd;
+
+ mutex_lock(>mutex);
+
+ if (video_is_registered(vfd)) {
+ video_unregister_device(vfd);
+ media_entity_cleanup(>entity);
Is this needed?
If this is to be part of the media controller, then I expect to see a call
to v4l2_m2m_register_media_controller() somewhere.
Yes, I agree there should be a call to
v4l2_m2m_register_media_controller(). This driver does not connect with
any of the imx-media entities, but calling it will at least make the
mem2mem output/capture device entities (and processing entity) visible
in the media graph.
Philipp, can you pick/squash the following from my media-tree github fork?
6fa05f5170 ("media: imx: mem2mem: Add missing media-device header")
d355bf8b15 ("media: imx: Add missing unregister and remove of mem2mem
device")
6787a50cdc ("media: imx: mem2mem: Register with media control")
Steve
Why is this driver part of the imx driver? Since it doesn't connect with
any of the imx-media entities, doesn't that mean that this is really a
stand-alone driver?
It is basically a stand-alone m2m driver, but it makes use of some
imx-media utility functions like imx_media_enum_format(). Also making it
a true stand-alone driver would require creating a second /dev/mediaN
device.
Steve
Re: [PATCH v5] media: imx: add mem2mem device
Hi Hans, Philipp,
One comment on my side...
On 12/3/18 7:21 AM, Hans Verkuil wrote:
+void imx_media_mem2mem_device_unregister(struct imx_media_video_dev *vdev)
+{
+ struct mem2mem_priv *priv = to_mem2mem_priv(vdev);
+ struct video_device *vfd = priv->vdev.vfd;
+
+ mutex_lock(>mutex);
+
+ if (video_is_registered(vfd)) {
+ video_unregister_device(vfd);
+ media_entity_cleanup(>entity);
Is this needed?
If this is to be part of the media controller, then I expect to see a call
to v4l2_m2m_register_media_controller() somewhere.
Yes, I agree there should be a call to
v4l2_m2m_register_media_controller(). This driver does not connect with
any of the imx-media entities, but calling it will at least make the
mem2mem output/capture device entities (and processing entity) visible
in the media graph.
Philipp, can you pick/squash the following from my media-tree github fork?
6fa05f5170 ("media: imx: mem2mem: Add missing media-device header")
d355bf8b15 ("media: imx: Add missing unregister and remove of mem2mem
device")
6787a50cdc ("media: imx: mem2mem: Register with media control")
Steve
Re: [PATCH] media: v4l2-fwnode: Demote warning to debug level
Reviewed-by: Steve Longerbeam
On 11/23/18 4:50 AM, Fabio Estevam wrote:
On a imx6q-wandboard the following warnings are observed:
[4.327794] video-mux 20e.iomuxc-gpr:ipu1_csi0_mux: bad remote port
parent
[4.336118] video-mux 20e.iomuxc-gpr:ipu2_csi1_mux: bad remote port
parent
As explained by Philipp Zabel:
"There are empty endpoint nodes (without remote-endpoint property)
labeled ipu1_csi[01]_mux_from_parallel_sensor in the i.MX6 device trees
for board DT implementers' convenience. See commit 2539f517acbdc ("ARM:
dts: imx6qdl: Add video multiplexers, mipi_csi, and their connections")."
So demote the warning to debug level and make the wording a bit
less misleading.
Suggested-by: Philipp Zabel
Signed-off-by: Fabio Estevam
---
drivers/media/v4l2-core/v4l2-fwnode.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-fwnode.c
b/drivers/media/v4l2-core/v4l2-fwnode.c
index 218f0da..7a3cc10 100644
--- a/drivers/media/v4l2-core/v4l2-fwnode.c
+++ b/drivers/media/v4l2-core/v4l2-fwnode.c
@@ -613,7 +613,7 @@ v4l2_async_notifier_fwnode_parse_endpoint(struct device
*dev,
asd->match.fwnode =
fwnode_graph_get_remote_port_parent(endpoint);
if (!asd->match.fwnode) {
- dev_warn(dev, "bad remote port parent\n");
+ dev_dbg(dev, "no remote endpoint found\n");
ret = -ENOTCONN;
goto out_err;
}
Re: 'bad remote port parent' warnings
Hi Fabio,
On 11/22/18 11:17 AM, Fabio Estevam wrote:
Hi Philipp,
On Thu, Nov 22, 2018 at 2:27 PM Philipp Zabel wrote:
There are empty endpoint nodes (without remote-endpoint property)
labeled ipu1_csi[01]_mux_from_parallel_sensor in the i.MX6 device trees
for board DT implementers' convenience. See commit 2539f517acbdc ("ARM:
dts: imx6qdl: Add video multiplexers, mipi_csi, and their connections").
We had a discussion about this issue in February when this caused a
probing error: https://patchwork.kernel.org/patch/10234469/
Thanks for the clarification.
We could demote the warning to a debug message, make the wording a bit
less misleading (there is no bad remote port parent, there is just no
remote endpoint at all), or we could just accept the error message for
Something like this?
--- a/drivers/media/v4l2-core/v4l2-fwnode.c
+++ b/drivers/media/v4l2-core/v4l2-fwnode.c
@@ -613,7 +613,7 @@ v4l2_async_notifier_fwnode_parse_endpoint(struct
device *dev,
asd->match.fwnode =
fwnode_graph_get_remote_port_parent(endpoint);
if (!asd->match.fwnode) {
- dev_warn(dev, "bad remote port parent\n");
+ dev_dbg(dev, "no remote endpoint found\n");
ret = -ENOTCONN;
goto out_err;
}
And how should we treat these error probes?
[3.449564] imx-ipuv3 240.ipu: driver could not parse
port@1/endpoint@0 (-22)
[3.457342] imx-ipuv3-csi: probe of imx-ipuv3-csi.1 failed with error -22
[3.464498] imx-ipuv3 280.ipu: driver could not parse
port@0/endpoint@0 (-22)
[3.472120] imx-ipuv3-csi: probe of imx-ipuv3-csi.4 failed with error -22
Yes, this is a regression caused by the imx subdev notifier patches.
I've already sent a patch to the list for this, see
https://www.spinics.net/lists/linux-media/msg141809.html
Steve
Re: [PATCH 3/3] media: imx: lift CSI width alignment restriction
On 11/9/18 6:50 AM, Philipp Zabel wrote:
On Thu, 2018-11-08 at 21:46 -0800, Steve Longerbeam wrote:
On 11/5/18 7:20 AM, Philipp Zabel wrote:
The CSI subdevice shouldn't have to care about IDMAC line start
address alignment. With compose rectangle support in the capture
driver, it doesn't have to anymore.
Signed-off-by: Philipp Zabel
---
drivers/staging/media/imx/imx-media-capture.c | 9 -
drivers/staging/media/imx/imx-media-csi.c | 2 +-
drivers/staging/media/imx/imx-media-utils.c | 15 ---
3 files changed, 17 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 2d49d9573056..f87d6e8019e5 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -204,10 +204,9 @@ static int capture_g_fmt_vid_cap(struct file *file, void
*fh,
}
static int __capture_try_fmt_vid_cap(struct capture_priv *priv,
-struct v4l2_subev_format *fmt_src,
+struct v4l2_subdev_format *fmt_src,
struct v4l2_format *f)
{
- struct capture_priv *priv = video_drvdata(file);
const struct imx_media_pixfmt *cc, *cc_src;
cc_src = imx_media_find_ipu_format(fmt_src->format.code, CS_SEL_ANY);
@@ -250,7 +249,7 @@ static int capture_try_fmt_vid_cap(struct file *file, void
*fh,
if (ret)
return ret;
- return __capture_try_fmt(priv, _src, f);
+ return __capture_try_fmt_vid_cap(priv, _src, f);
}
static int capture_s_fmt_vid_cap(struct file *file, void *fh,
@@ -280,8 +279,8 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
CS_SEL_ANY, true);
priv->vdev.compose.left = 0;
priv->vdev.compose.top = 0;
- priv->vdev.compose.width = fmt_src.width;
- priv->vdev.compose.height = fmt_src.height;
+ priv->vdev.compose.width = fmt_src.format.width;
+ priv->vdev.compose.height = fmt_src.format.height;
return 0;
}
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index c4523afe7b48..d39682192a67 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -41,7 +41,7 @@
#define MIN_H 144
#define MAX_W 4096
#define MAX_H 4096
-#define W_ALIGN4 /* multiple of 16 pixels */
+#define W_ALIGN1 /* multiple of 2 pixels */
This works for the IDMAC output pad because the channel's cpmem width
and stride can be rounded up, but width align at the CSI sink still
needs to be 8 pixels when directed to the IC via the CSI_SRC_PAD_DIRECT
pad, in order to support the 8x8 block rotator in the IC PRP, and
there's no way AFAIK to do the same trick of rounding up width and
stride for non-IDMAC direct paths through the IPU.
Actually, this is not necessary at all. csi_try_crop takes care of this
by setting:
crop->width &= ~0x7;
Which is then used to set compose rectangle and source pad formats.
Ah you are right, I had forgotten about that line.
So this should be relaxed as well, if the SRC_DIRECT pad is not enabled.
Agreed, crop->width align can be relaxed but only if SRC_DIRECT pad
not enabled.
And further, I think there is no reason to align crop->left to multiples
of 4 pixels?
Honestly, I don't know the reason for that either. Along with crop->top
this defines the CSI active image frame position (HSC/VSC fields in
IPU_CSI_OUT_FRM_CTRL register), and I don't see any h/w restrictions on
HSC in the ref manual. I do remember that this alignment exists in
FSL/NXP BSP's though, so maybe it is an undocumented restriction.
Steve
Re: [PATCH 3/3] media: imx: lift CSI width alignment restriction
On 11/5/18 7:20 AM, Philipp Zabel wrote:
The CSI subdevice shouldn't have to care about IDMAC line start
address alignment. With compose rectangle support in the capture
driver, it doesn't have to anymore.
Signed-off-by: Philipp Zabel
---
drivers/staging/media/imx/imx-media-capture.c | 9 -
drivers/staging/media/imx/imx-media-csi.c | 2 +-
drivers/staging/media/imx/imx-media-utils.c | 15 ---
3 files changed, 17 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 2d49d9573056..f87d6e8019e5 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -204,10 +204,9 @@ static int capture_g_fmt_vid_cap(struct file *file, void
*fh,
}
static int __capture_try_fmt_vid_cap(struct capture_priv *priv,
-struct v4l2_subev_format *fmt_src,
+struct v4l2_subdev_format *fmt_src,
struct v4l2_format *f)
{
- struct capture_priv *priv = video_drvdata(file);
const struct imx_media_pixfmt *cc, *cc_src;
cc_src = imx_media_find_ipu_format(fmt_src->format.code, CS_SEL_ANY);
@@ -250,7 +249,7 @@ static int capture_try_fmt_vid_cap(struct file *file, void
*fh,
if (ret)
return ret;
- return __capture_try_fmt(priv, _src, f);
+ return __capture_try_fmt_vid_cap(priv, _src, f);
}
static int capture_s_fmt_vid_cap(struct file *file, void *fh,
@@ -280,8 +279,8 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
CS_SEL_ANY, true);
priv->vdev.compose.left = 0;
priv->vdev.compose.top = 0;
- priv->vdev.compose.width = fmt_src.width;
- priv->vdev.compose.height = fmt_src.height;
+ priv->vdev.compose.width = fmt_src.format.width;
+ priv->vdev.compose.height = fmt_src.format.height;
return 0;
}
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index c4523afe7b48..d39682192a67 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -41,7 +41,7 @@
#define MIN_H 144
#define MAX_W 4096
#define MAX_H 4096
-#define W_ALIGN4 /* multiple of 16 pixels */
+#define W_ALIGN1 /* multiple of 2 pixels */
This works for the IDMAC output pad because the channel's cpmem width
and stride can be rounded up, but width align at the CSI sink still
needs to be 8 pixels when directed to the IC via the CSI_SRC_PAD_DIRECT
pad, in order to support the 8x8 block rotator in the IC PRP, and
there's no way AFAIK to do the same trick of rounding up width and
stride for non-IDMAC direct paths through the IPU.
Also, the imx-ic-prpencvf.c W_ALIGN_SRC can be relaxed to 2 pixels as well.
Steve
#define H_ALIGN1 /* multiple of 2 lines */
#define S_ALIGN1 /* multiple of 2 */
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 0eaa353d5cb3..5f110d90a4ef 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -580,6 +580,7 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format
*pix,
struct v4l2_mbus_framefmt *mbus,
const struct imx_media_pixfmt *cc)
{
+ u32 width;
u32 stride;
if (!cc) {
@@ -602,9 +603,16 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format
*pix,
cc = imx_media_find_mbus_format(code, CS_SEL_YUV, false);
}
- stride = cc->planar ? mbus->width : (mbus->width * cc->bpp) >> 3;
+ /* Round up width for minimum burst size */
+ width = round_up(mbus->width, 8);
- pix->width = mbus->width;
+ /* Round up stride for IDMAC line start address alignment */
+ if (cc->planar)
+ stride = round_up(width, 16);
+ else
+ stride = round_up((width * cc->bpp) >> 3, 8);
+
+ pix->width = width;
pix->height = mbus->height;
pix->pixelformat = cc->fourcc;
pix->colorspace = mbus->colorspace;
@@ -613,7 +621,8 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format
*pix,
pix->quantization = mbus->quantization;
pix->field = mbus->field;
pix->bytesperline = stride;
- pix->sizeimage = (pix->width * pix->height * cc->bpp) >> 3;
+ pix->sizeimage = cc->planar ? ((stride * pix->height * cc->bpp) >> 3) :
+stride * pix->height;
return 0;
}
Re: [PATCH 2/3] media: imx: set compose rectangle to mbus format
Hi Philipp,
On 11/5/18 7:20 AM, Philipp Zabel wrote:
Prepare for mbus format being smaller than the written rectangle
due to burst size.
Signed-off-by: Philipp Zabel
---
drivers/staging/media/imx/imx-media-capture.c | 55 +--
1 file changed, 38 insertions(+), 17 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index cace8a51aca8..2d49d9573056 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -203,21 +203,14 @@ static int capture_g_fmt_vid_cap(struct file *file, void
*fh,
return 0;
}
-static int capture_try_fmt_vid_cap(struct file *file, void *fh,
- struct v4l2_format *f)
+static int __capture_try_fmt_vid_cap(struct capture_priv *priv,
+struct v4l2_subev_format *fmt_src,
typo: struct v4l2_subdev_format *fmt_src,
+struct v4l2_format *f)
{
struct capture_priv *priv = video_drvdata(file);
- struct v4l2_subdev_format fmt_src;
const struct imx_media_pixfmt *cc, *cc_src;
- int ret;
-
- fmt_src.pad = priv->src_sd_pad;
- fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
- ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, _src);
- if (ret)
- return ret;
- cc_src = imx_media_find_ipu_format(fmt_src.format.code, CS_SEL_ANY);
+ cc_src = imx_media_find_ipu_format(fmt_src->format.code, CS_SEL_ANY);
if (cc_src) {
u32 fourcc, cs_sel;
@@ -231,7 +224,7 @@ static int capture_try_fmt_vid_cap(struct file *file, void *fh,
cc = imx_media_find_format(fourcc, cs_sel, false);
}
} else {
- cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+ cc_src = imx_media_find_mbus_format(fmt_src->format.code,
CS_SEL_ANY, true);
if (WARN_ON(!cc_src))
return -EINVAL;
@@ -239,15 +232,32 @@ static int capture_try_fmt_vid_cap(struct file *file,
void *fh,
cc = cc_src;
}
- imx_media_mbus_fmt_to_pix_fmt(>fmt.pix, _src.format, cc);
+ imx_media_mbus_fmt_to_pix_fmt(>fmt.pix, _src->format, cc);
return 0;
}
+static int capture_try_fmt_vid_cap(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_format fmt_src;
+ int ret;
+
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, _src);
+ if (ret)
+ return ret;
+
+ return __capture_try_fmt(priv, _src, f);
typo: return __capture_try_fmt_vid_cap(priv, _src, f);
+}
+
static int capture_s_fmt_vid_cap(struct file *file, void *fh,
struct v4l2_format *f)
{
struct capture_priv *priv = video_drvdata(file);
+ struct v4l2_subdev_format fmt_src;
int ret;
if (vb2_is_busy(>q)) {
@@ -255,7 +265,13 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
return -EBUSY;
}
- ret = capture_try_fmt_vid_cap(file, priv, f);
+ fmt_src.pad = priv->src_sd_pad;
+ fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, _src);
+ if (ret)
+ return ret;
+
+ ret = __capture_try_fmt_vid_cap(priv, _src, f);
if (ret)
return ret;
@@ -264,8 +280,8 @@ static int capture_s_fmt_vid_cap(struct file *file, void *fh,
CS_SEL_ANY, true);
priv->vdev.compose.left = 0;
priv->vdev.compose.top = 0;
- priv->vdev.compose.width = f->fmt.fmt.pix.width;
- priv->vdev.compose.height = f->fmt.fmt.pix.height;
+ priv->vdev.compose.width = fmt_src.width;
+ priv->vdev.compose.height = fmt_src.height;
return 0;
}
@@ -307,9 +323,14 @@ static int capture_g_selection(struct file *file, void *fh,
case V4L2_SEL_TGT_COMPOSE:
case V4L2_SEL_TGT_COMPOSE_DEFAULT:
case V4L2_SEL_TGT_COMPOSE_BOUNDS:
- case V4L2_SEL_TGT_COMPOSE_PADDED:
s->r = priv->vdev.compose;
break;
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = priv->vdev.fmt.fmt.pix.width;
+ s->r.height = priv->vdev.fmt.fmt.pix.height;
+ break;
default:
return -EINVAL;
}
Re: [PATCH 1/3] media: imx: add capture compose rectangle
Hi Philipp,
On 11/5/18 7:20 AM, Philipp Zabel wrote:
Allowing to compose captured images into larger memory buffers
will let us lift alignment restrictions on CSI crop width.
Signed-off-by: Philipp Zabel
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 3 +-
drivers/staging/media/imx/imx-media-capture.c | 38 +++
drivers/staging/media/imx/imx-media-csi.c | 3 +-
drivers/staging/media/imx/imx-media-vdic.c| 4 +-
drivers/staging/media/imx/imx-media.h | 2 +
5 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 28f41caba05d..fe5a77baa592 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -366,8 +366,7 @@ static int prp_setup_channel(struct prp_priv *priv,
memset(, 0, sizeof(image));
image.pix = vdev->fmt.fmt.pix;
- image.rect.width = image.pix.width;
- image.rect.height = image.pix.height;
+ image.rect = vdev->compose;
if (rot_swap_width_height) {
swap(image.pix.width, image.pix.height);
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index b37e1186eb2f..cace8a51aca8 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -262,6 +262,10 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
priv->vdev.fmt.fmt.pix = f->fmt.pix;
priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat,
CS_SEL_ANY, true);
+ priv->vdev.compose.left = 0;
+ priv->vdev.compose.top = 0;
+ priv->vdev.compose.width = f->fmt.fmt.pix.width;
+ priv->vdev.compose.height = f->fmt.fmt.pix.height;
this should be:
priv->vdev.compose.width = fmt_src.format.width;
priv->vdev.compose.height = fmt_src.format.height;
(corrected in the next patches but needs to be corrected here).
return 0;
}
@@ -290,6 +294,35 @@ static int capture_s_std(struct file *file, void *fh,
v4l2_std_id std)
return v4l2_subdev_call(priv->src_sd, video, s_std, std);
}
+static int capture_g_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ struct capture_priv *priv = video_drvdata(file);
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ case V4L2_SEL_TGT_NATIVE_SIZE:
+ case V4L2_SEL_TGT_COMPOSE:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ s->r = priv->vdev.compose;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int capture_s_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ return capture_g_selection(file, fh, s);
+}
+
static int capture_g_parm(struct file *file, void *fh,
struct v4l2_streamparm *a)
{
@@ -350,6 +383,9 @@ static const struct v4l2_ioctl_ops capture_ioctl_ops = {
.vidioc_g_std = capture_g_std,
.vidioc_s_std = capture_s_std,
+ .vidioc_g_selection = capture_g_selection,
+ .vidioc_s_selection = capture_s_selection,
+
.vidioc_g_parm = capture_g_parm,
.vidioc_s_parm = capture_s_parm,
@@ -687,6 +723,8 @@ int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
vdev->fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
imx_media_mbus_fmt_to_pix_fmt(>fmt.fmt.pix,
_src.format, NULL);
+ vdev->compose.width = fmt_src.format.width;
+ vdev->compose.height = fmt_src.format.height;
vdev->cc = imx_media_find_format(vdev->fmt.fmt.pix.pixelformat,
CS_SEL_ANY, false);
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index 4223f8d418ae..c4523afe7b48 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -413,8 +413,7 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
memset(, 0, sizeof(image));
image.pix = vdev->fmt.fmt.pix;
- image.rect.width = image.pix.width;
- image.rect.height = image.pix.height;
+ image.rect = vdev->compose;
csi_idmac_setup_vb2_buf(priv, phys);
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 482250d47e7c..e08d296cf4eb 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -263,10 +263,10 @@ static int setup_vdi_channel(struct vdic_priv *priv,
memset(, 0,
Re: [RFC] media: imx: queue subdevice events on the video device in the same pipeline
Hi Philipp,
Thanks, I've been meaning this too. Comments below.
On 11/5/18 7:03 AM, Philipp Zabel wrote:
While subdevice and video device are in the same pipeline, pass
subdevice events on to userspace via the video device node.
Signed-off-by: Philipp Zabel
---
This would allow to see source change events from the source subdevice
on the video device node, for example.
---
drivers/staging/media/imx/imx-media-dev.c | 18 ++
1 file changed, 18 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media-dev.c
b/drivers/staging/media/imx/imx-media-dev.c
index 4b344a4a3706..2fe6fdf2faf1 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -442,6 +442,23 @@ static const struct media_device_ops imx_media_md_ops = {
.link_notify = imx_media_link_notify,
};
+static void imx_media_notify(struct v4l2_subdev *sd, unsigned int notification,
+void *arg)
+{
+ struct imx_media_dev *imxmd;
+ struct imx_media_video_dev *vdev;
+
+ imxmd = container_of(sd->v4l2_dev, struct imx_media_dev, v4l2_dev);
+ list_for_each_entry(vdev, >vdev_list, list) {
+ if (sd->entity.pipe &&
+ sd->entity.pipe == vdev->vfd->entity.pipe &&
The problem with this check is that a sensor can be streaming to
multiple video capture devices simultaneously (to prpenc and prpvf). The
media framework doesn't support an entity being a member of multiple
pipelines (there's only one pipe object in 'struct media_entity') so the
pipe object ends up just pointing to whichever stream was started last
that included that entity. The result being the event will only get
queued to whichever video device who's stream was enabled last.
So I think there should be no if statement in the loop. Also it's better
to loop through the sub-devices vdev_list, because that list contains
only the video devices reachable from that sub-device. So the function
should read:
static void imx_media_notify(struct v4l2_subdev *sd,
unsigned int notification,
void *arg)
{
struct media_entity *entity = >entity;
int i;
if (notification != V4L2_DEVICE_NOTIFY_EVENT)
return;
for (i = 0; i < entity->num_pads; i++) {
struct media_pad *pad = >pads[i];
struct imx_media_pad_vdev *pad_vdev;
struct list_head *pad_vdev_list;
pad_vdev_list = to_pad_vdev_list(sd, pad->index);
if (!pad_vdev_list)
continue;
list_for_each_entry(pad_vdev, pad_vdev_list, list)
v4l2_event_queue(pad_vdev->vdev->vfd, arg);
}
}
I posted this to my media-tree fork, see
da05ccab97 ("media: imx: queue subdevice events to the reachable video devices")
and this Note in the commit header:
Note this will queue the event to a video device even if there is
no actual _enabled_ path from the sub-device to the video device.
So a future fix is to skip the video device if there is no enabled
path to it from the sub-device. The entity->pipe pointer can't be
used for this check because in imx-media a sub-device can be a
member to more than one streaming pipeline at a time.
Steve
+ notification == V4L2_DEVICE_NOTIFY_EVENT) {
+ v4l2_event_queue(vdev->vfd, arg);
+ break;
+ }
+ }
+}
+
static int imx_media_probe(struct platform_device *pdev)
{
struct device *dev = >dev;
@@ -464,6 +481,7 @@ static int imx_media_probe(struct platform_device *pdev)
imxmd->v4l2_dev.mdev = >md;
strscpy(imxmd->v4l2_dev.name, "imx-media",
sizeof(imxmd->v4l2_dev.name));
+ imxmd->v4l2_dev.notify = imx_media_notify;
media_device_init(>md);
Re: i.MX6: can't capture on MIPI-CSI2 with DS90UB954
Hi Jean-Michel,
We've done some work with another FPD-Link de-serializer (ds90ux940) and
IIRC we had some trouble figuring out how to coax the lanes into LP-11
state. But on the ds90ux940 it can be done by setting bit 7 in the CSI
Enable Port registers (offsets 0x13 and 0x14). But the "imx6-mipi-csi2:
clock lane timeout" message is something else and indicates the
de-serializer is not activating the clock lane during its s_stream(ON)
subdev call.
Hope that helps,
Steve
On 10/30/18 9:41 AM, Jean-Michel Hautbois wrote:
Hi there,
I am using the i.MX6D from Digi (connect core 6 sbc) with a mailine
kernel (well, 4.14 right now) and have an issue with mipi-csi2
capture.
First I will give brief explanation of my setup, and then I will
detail the issue.
I have a camera sensor (OV2732, but could be any other sensor)
connected on a DS90UB953 FPD-Link III serializer.
Then a coax cable propagates the signal to a DS90UB954 FPD-Link III
deserializer.
The DS90UB954 has the ability to work in a pattern generation mode,
and I will use it for the rest of the discussion.
It is an I²C device, and I have written a basic driver (for the moment
;)) in order to make it visible on the imx6-mipi-csi2 bus as a camera
sensor.
I can give an access to the driver if necessary.
I then program the MC pipeline :
media-ctl -l "'ds90ub954 2-0034':0 -> 'imx6-mipi-csi2':0[1]" -v
media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]" -v
media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" -v
media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
media-ctl -V "'ds90ub954 2-0034':0 [fmt:RGB888_1X24/1280x720 field:none]"
media-ctl -V "'imx6-mipi-csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"
media-ctl -V "'ipu1_csi0_mux':2 [fmt:RGB888_1X24/1280x720 field:none]"
media-ctl -V "'ipu1_csi0':2 [fmt:RGB888_1X24/1280x720 field:none]"
Everything works fine, all nodes are correctly configured, and the
DS90UB954 is normaly sending data on 2 lanes, with VC-ID=0.
The pattern is 1280x720@30 RGB888.
Then, I start a Gstreamer pipeline (I tried with v4l2-ctl and have the
same issue though) :
GST_DEBUG="v4l2:5" gst-launch-1.0 v4l2src device=/dev/video4 !
video/x-raw, width=1280, height=720, format=RGB ! fakesink
And... well, I had to use this patch
https://lkml.org/lkml/2017/3/11/270 in order to go further, but I am
finishing into :
[ 164.077302] imx-ipuv3-csi imx-ipuv3-csi.0: stream ON
[ 164.097955] imx-ipuv3-csi imx-ipuv3-csi.0: FI=3 usec
[ 165.129424] ipu1_csi0: EOF timeout
[ 165.142395] imx-ipuv3-csi imx-ipuv3-csi.0: stream OFF
[ 166.169299] ipu1_csi0: wait last EOF timeout
Sounds like a recurrent issue on this ML :).
I can think of several things which could explain this, but I tried a
lot and am a bit stuck.
The clock is set to 800MHz on DS90UB954 side.
=> Should CSI2_PHY_TST_CTRL1 be 0x32 ? 0x12 ? or 0x4a (ie 400MHz) ?
I think I have tried all but still the same issue.
Maybe this could be a hint, when booting, the first stream-on leads to:
imx6-mipi-csi2: LP-11 timeout, phy_state = 0x0200 => just a warning now
imx6-mipi-csi2: clock lane timeout, phy_state = 0x0230
The next one leads to the EOF timeout.
Here is the dts part BTW :
{
status = "okay";
ds90ub954: camera@34 {
compatible = "ti,ds90ub954";
status = "okay";
reg = <0x34>;
clocks = < IMX6QDL_CLK_CKO>;
clock-names = "xclk";
port {
#address-cells = <1>;
#size-cells = <0>;
ds90ub954_out0: endpoint {
remote-endpoint = <_csi2_in>;
clock-lanes = <0>;
data-lanes = <1 2>;
};
};
};
};
_csi {
status = "okay";
port@0 {
reg = <0>;
mipi_csi2_in: endpoint {
remote-endpoint = <_out0>;
clock-lanes = <0>;
data-lanes = <1 2>;
};
};
};
If ayone has a suggestion...
Thanks a lot !
Regards,
JM
Re: [PATCH v3 01/16] media: imx: add mem2mem device
(resending as plain text) On 10/21/18 10:43 AM, Philipp Zabel wrote: On Fri, Oct 19, 2018 at 01:19:10PM -0700, Steve Longerbeam wrote: On 10/19/18 2:53 AM, Philipp Zabel wrote: Hi Tim, On Thu, 2018-10-18 at 15:53 -0700, Tim Harvey wrote: [...] Philipp, Thanks for submitting this! I'm hoping this lets us use non-IMX capture devices along with the IMX media controller entities to so we can use hardware CSC,scaling,pixel-format-conversions and ultimately coda based encode. I've built this on top of linux-media and see that it registers as /dev/video8 but I'm not clear how to use it? I don't see it within the media controller graph. It's a V4L2 mem2mem device that can be handled by the GstV4l2Transform element, for example. GStreamer should create a v4l2video8convert element of that type. The mem2mem device is not part of the media controller graph on purpose. There is no interaction with any of the entities in the media controller graph apart from the fact that the IC PP task we are using for mem2mem scaling is sharing hardware resources with the IC PRP tasks used for the media controller scaler entitites. It would be nice in the future to link mem2mem output-side to the ipu_vdic:1 pad, to make use of h/w VDIC de-interlace as part of mem2mem operations. The progressive output from a new ipu_vdic:3 pad can then be sent to the image_convert APIs by the mem2mem driver for further tiled scaling, CSC, and rotation by the IC PP task. The ipu_vdic:1 pad makes use of pure DMA-based de-interlace, that is, all input frames (N-1, N, N+1) to the VDIC are sent from DMA buffers, and this VDIC mode of operation is well understood and produces clean de-interlace output. The risk is that this would require iDMAC channel 5 for ipu_vdic:3, which IFAIK is not verified to work yet. Tiled mem2mem deinterlacing support would be nice, I'm not sure yet how though. I'd limit media controller links to marking VDIC as unavailable for the capture pipeline. The V4L2 subdev API is too lowlevel for tiling mem2mem purposes, as we'd need to change the subdev format multiple times per frame. I wasn't considering tiled deinterlacing, only deinterlacing at the hardware limited input frame size to the VDIC. Also I'd like to keep the option of scheduling tile jobs to both IPUs on i.MX6Q, which will become difficult to describe via MC, as both IPUs' ipu_vdics would have to be involved. Agreed, it would be good to add to the mem2mem driver the ability to schedule jobs to whichever IPU is least busy. The other problem with that currently is that mem2mem would have to be split into separate device nodes: a /dev/videoN for output-side (linked to ipu_vdic:1), and a /dev/videoM for capture-side (linked from ipu_vdic:3). And then it no longer presents to userspace as a mem2mem device with a single device node for both output and capture sides. I don't understand why we'd need separate video devices for output and capture, deinterlacing is still single input single (double rate) output. As soon as we begin tiling, we are one layer of abstraction away from the hardware pads anyway. Now if we want to support combining on the other hand... Again I wasn't thinking of doing tiled deinterlace. Or is there another way? I recall work to integrate mem2mem with media control. There is v4l2_m2m_register_media_controller(), but that create three entities: source, processing, and sink. The VDIC entity would be part of mem2mem processing but this entity already exists for the current graph. This function could however be used as a guide to incorporate the VDIC entity into m2m device. I'm not sure if this is the right abstraction. Without tiling or multi-IPU scheduling, sure. But the mem2mem driver does not directly describe hardware operation anyway. What I'm thinking is separate mem2mem devices from this proposed tiled-scaling post-processing mem2mem, that solely do motion compensated deinterlace using the VDIC as the processing entity. [1] is the dot graph that demonstrates the idea, on imx6q SabreSD. Two new mem2mem devices are attached to VDIC sink and source IDMAC pads. Also the PP tiled scaling mem2mem is shown in the graph. As of now, the VDIC is only available for video capture pipelines, so the advantage would be to allow the use of hardware deinterlace gstreamer pipelines from other types of interlaced sources like file or network streams. So something like the following example which would do hardware deinterlace, followed by tiled scaling/CSC/rotation, then h.264 encode, the VDIC mem2mem device is at /dev/video0, and the PP mem2mem device is at /dev/video12 in this example: gst-launch-1.0 \ v4l2src ! \ v4l2video0convert output-io-mode=dmabuf-import ! \ v4l2video12convert output-io-mode=dmabuf-import ! \ v4l2h264enc output-io-mode=dmabuf-import ! \ h264parse ! \ matroskamux ! \ filesink I'm probably missing some stream properties in that example, but you get the idea. Steve [1] digraph board
Re: [PATCH v3 01/16] media: imx: add mem2mem device
On 10/19/18 2:53 AM, Philipp Zabel wrote: Hi Tim, On Thu, 2018-10-18 at 15:53 -0700, Tim Harvey wrote: [...] Philipp, Thanks for submitting this! I'm hoping this lets us use non-IMX capture devices along with the IMX media controller entities to so we can use hardware CSC,scaling,pixel-format-conversions and ultimately coda based encode. I've built this on top of linux-media and see that it registers as /dev/video8 but I'm not clear how to use it? I don't see it within the media controller graph. It's a V4L2 mem2mem device that can be handled by the GstV4l2Transform element, for example. GStreamer should create a v4l2video8convert element of that type. The mem2mem device is not part of the media controller graph on purpose. There is no interaction with any of the entities in the media controller graph apart from the fact that the IC PP task we are using for mem2mem scaling is sharing hardware resources with the IC PRP tasks used for the media controller scaler entitites. It would be nice in the future to link mem2mem output-side to the ipu_vdic:1 pad, to make use of h/w VDIC de-interlace as part of mem2mem operations. The progressive output from a new ipu_vdic:3 pad can then be sent to the image_convert APIs by the mem2mem driver for further tiled scaling, CSC, and rotation by the IC PP task. The ipu_vdic:1 pad makes use of pure DMA-based de-interlace, that is, all input frames (N-1, N, N+1) to the VDIC are sent from DMA buffers, and this VDIC mode of operation is well understood and produces clean de-interlace output. The risk is that this would require iDMAC channel 5 for ipu_vdic:3, which IFAIK is not verified to work yet. The other problem with that currently is that mem2mem would have to be split into separate device nodes: a /dev/videoN for output-side (linked to ipu_vdic:1), and a /dev/videoM for capture-side (linked from ipu_vdic:3). And then it no longer presents to userspace as a mem2mem device with a single device node for both output and capture sides. Or is there another way? I recall work to integrate mem2mem with media control. There is v4l2_m2m_register_media_controller(), but that create three entities: source, processing, and sink. The VDIC entity would be part of mem2mem processing but this entity already exists for the current graph. This function could however be used as a guide to incorporate the VDIC entity into m2m device. Steve
Re: i.MX6 IPU CSI analog video input on Ventana
On 10/18/18 10:56 AM, Tim Harvey wrote: On Wed, Oct 17, 2018 at 4:37 PM Steve Longerbeam wrote: On 10/17/18 4:05 PM, Tim Harvey wrote: On Wed, Oct 17, 2018 at 2:33 PM Steve Longerbeam wrote: Hi Tim, On 10/17/18 1:38 PM, Tim Harvey wrote: On Mon, Jun 4, 2018 at 1:58 AM Krzysztof Hałasa wrote: I've just tested the PAL setup: in currect situation (v4.17 + Steve's fix-csi-interlaced.2 + "media: adv7180: fix field type" + a small cheap PAL camera) the following produces bottom-first interlaced frames: media-ctl -r -l '"adv7180 2-0020":0->"ipu2_csi1_mux":1[1], "ipu2_csi1_mux":2->"ipu2_csi1":0[1], "ipu2_csi1":2->"ipu2_csi1 capture":0[1]' media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x576 field:alternate]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x576]" media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x576 field:interlaced]" "adv7180 2-0020":0 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1_mux":1 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1_mux":2 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1":0 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1":2 [fmt:AYUV32/720x576 field:interlaced] I think it would be great if these changes make their way upstream. The details could be refined then. Krzysztof / Steve / Philipp, I jumped back onto IMX6 video capture from the adv7180 the other day trying to help out a customer that's using mainline and found things are still not working right. Where is all of this at these days? If I use v4.19 with Steves 'imx-media: Fixes for interlaced capture' v3 series (https://patchwork.kernel.org/cover/10626499/) I rolling/split (un-synchronized) video using: # Setup links media-ctl -r media-ctl -l '"adv7180 2-0020":0 -> "ipu2_csi1_mux":1[1]' media-ctl -l '"ipu2_csi1_mux":2 -> "ipu2_csi1":0[1]' media-ctl -l '"ipu2_csi1":1 -> "ipu2_ic_prp":0[1]' media-ctl -l '"ipu2_ic_prp":2 -> "ipu2_ic_prpvf":0[1]' media-ctl -l '"ipu2_ic_prpvf":1 -> "ipu2_ic_prpvf capture":0[1]' # Configure pads media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" media-ctl -V "'ipu2_csi1':1 [fmt:UYVY2X8/720x480 field:interlaced]" media-ctl -V "'ipu2_ic_prp':2 [fmt:UYVY2X8/720x480 field:interlaced]" media-ctl -V "'ipu2_ic_prpvf':1 [fmt:UYVY2X8/720x480 field:none]" # stream JPEG/RTP/UDP gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw,format=UYVY ! jpegenc ! rtpjpegpay ! udpsink host=$SERVER port=$PORT ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video3' does not support progressive interlacing I'm doing the above on a Gateworks GW5404 IMXQ which has a tda1997x HDMI receiver sensor and an adv7180 Analog CVBS sensor - media graph is here: http://dev.gateworks.com/docs/linux/media/imx6q-gw54xx-media.png Are there other patches I need or different field formats above with 4.19? Do any of the other kernels work without patchsets that you know of between 4.16 and 4.19? First, the v3 series is out of date. Please apply the latest v5 posting of that series. See the imx.rst doc regarding field type negotiation, all pads starting at ipu2_csi1:1 should be 'seq-bt' or 'seq-tb' until the capture device, which should be set to 'interlaced' to enable IDMAC interweave. The ADV7180 now correctly sets its field type to alternate, which imx-media-csi.c translates to seq-tb or seq-bt at its output pad. See the SabreAuto examples in the doc. For the rolling/split image problem, try the attached somewhat hackish patch. There used to be code in imx-media-csi.c that searched for the backend sensor and queries via .g_skip_frames whether the sensor produces bad frames at first stream-on. But there was push-back on that, so the attached is another approach that doesn't require searching for a backend sensor. Steve, Thanks - I hadn't noticed the updated series. I've built it on top of linux-media/master and tested with: - Testing linux-media/master + your v5 now: # Use simple interweaving media-ctl -r # Setup links media-ctl -l '"adv7180 2-0020":0 -> "ipu2_csi1_mux":1[1]' media-ctl -l '"ipu2_csi1_mux":2 -> "ipu2_csi1":0[1]' media-ctl -l '"ipu2_csi1":2 -> "ipu2_csi1 capture":0[1]' # Configure pads media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480 field:seq-bt]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480]" media-ctl -V "'ipu2_csi1':1 [fmt:AYUV32/720x480]" # Configure ipu_csi capture interface (/dev/video7) v4l2-ctl -d7 --set-fmt-video=field=interlaced_bt # Stream JPEG/RTP/UDP gst-launch-1.0 v4l2src device=/dev/video7
Re: [PATCH v4 00/22] i.MX media mem2mem scaler
Awesome, thanks Philipp. For the whole series: Acked-by: Steve Longerbeam Tested-by: Steve Longerbeam on i.MX6q SabreSD. On 10/19/18 5:15 AM, Philipp Zabel wrote: Hi, this is the fourth version of the i.MX mem2mem scaler series. An alignment issue with 24-bit RGB formats has been corrected in the seam position selection patch and a few new fixes by Steve have been added. If there are no more issues, I'll pick up the ipu-v3 patches via imx-drm/next. The first patch could be merged via the media tree independently. Changes since v3: - Fix tile_left_align for 24-bit RGB formats and reduce alignment restrictions for U/V packed planar YUV formats - Catch unaligned tile offsets in image-convert - Add chroma plane offset overrides to ipu_cpmem_set_image() to prevent a false positive warning in some cases - Fix a race between run and unprepare and make abort reentrant. Changes since v2: - Rely on ipu_image_convert_adjust() in mem2mem_try_fmt() for format adjustments. This makes the mem2mem driver mostly a V4L2 mem2mem API wrapper around the IPU image converter, and independent of the internal image converter implementation. - Remove the source and destination buffers on error in device_run(). Otherwise the conversion is re-attempted apparently over and over again (with WARN() backtraces). - Allow subscribing to control changes. - Fix seam position selection for more corner cases: - Switch width/height properly and align tile top left positions to 8x8 IRT block size when rotating. - Align input width to input burst length in case the scaling step flips horizontally. - Fix bottom edge calculation. Changes since v1: - Fix inverted allow_overshoot logic - Correctly switch horizontal / vertical tile alignment when determining seam positions with the 90° rotator active. - Fix SPDX-License-Identifier and remove superfluous license text. - Fix uninitialized walign in try_fmt Previous cover letter: we have image conversion code for scaling and colorspace conversion in the IPUv3 base driver for a while. Since the IC hardware can only write up to 1024x1024 pixel buffers, it scales to larger output buffers by splitting the input and output frame into similarly sized tiles. This causes the issue that the bilinear interpolation resets at the tile boundary: instead of smoothly interpolating across the seam, there is a jump in the input sample position that is very apparent for high upscaling factors. This can be avoided by slightly changing the scaling coefficients to let the left/top tiles overshoot their input sampling into the first pixel / line of their right / bottom neighbors. The error can be further reduced by letting tiles be differently sized and by selecting seam positions that minimize the input sampling position error at tile boundaries. This is complicated by different DMA start address, burst size, and rotator block size alignment requirements, depending on the input and output pixel formats, and the fact that flipping happens in different places depending on the rotation. This series implements optimal seam position selection and seam hiding with per-tile resizing coefficients and adds a scaling mem2mem device to the imx-media driver. regards Philipp Philipp Zabel (15): media: imx: add mem2mem device gpu: ipu-v3: ipu-ic: allow to manually set resize coefficients gpu: ipu-v3: image-convert: prepare for per-tile configuration gpu: ipu-v3: image-convert: calculate per-tile resize coefficients gpu: ipu-v3: image-convert: reconfigure IC per tile gpu: ipu-v3: image-convert: store tile top/left position gpu: ipu-v3: image-convert: calculate tile dimensions and offsets outside fill_image gpu: ipu-v3: image-convert: move tile alignment helpers gpu: ipu-v3: image-convert: select optimal seam positions gpu: ipu-v3: image-convert: fix debug output for varying tile sizes gpu: ipu-v3: image-convert: relax alignment restrictions gpu: ipu-v3: image-convert: fix bytesperline adjustment gpu: ipu-v3: image-convert: add some ASCII art to the exposition gpu: ipu-v3: image-convert: disable double buffering if necessary gpu: ipu-v3: image-convert: allow three rows or columns Steve Longerbeam (7): gpu: ipu-cpmem: add WARN_ON_ONCE() for unaligned dma buffers gpu: ipu-v3: Add chroma plane offset overrides to ipu_cpmem_set_image() gpu: ipu-v3: image-convert: Prevent race between run and unprepare gpu: ipu-v3: image-convert: Only wait for abort completion if active run gpu: ipu-v3: image-convert: Allow reentrancy into abort gpu: ipu-v3: image-convert: Remove need_abort flag gpu: ipu-v3: image-convert: Catch unaligned tile offsets drivers/gpu/ipu-v3/ipu-cpmem.c| 52 +- drivers/gpu/ipu-v3/ipu-ic.c | 52 +- drivers/gpu/ipu-v3/ipu-image-convert.c| 1019 ++--- drivers/staging/media/imx/Kconfig
Re: [PATCH v3 00/16] i.MX media mem2mem scaler
Hi Philipp, On 10/12/18 5:29 PM, Steve Longerbeam wrote: But one last thing. Conversions to and from YV12 are producing images with wrong colors, it looks like the .uv_swapped boolean needs to be checked additionally somewhere. Any ideas? Sorry, this was my fault. I fixed this in "gpu: ipu-v3: Add chroma plane offset overrides to ipu_cpmem_set_image()" in my fork g...@github.com:slongerbeam/mediatree.git, branch imx-mem2mem.3. Steve
Re: [PATCH v3 10/16] gpu: ipu-v3: image-convert: select optimal seam positions
On 10/17/18 4:10 AM, Philipp Zabel wrote:
On Fri, 2018-10-12 at 17:33 -0700, Steve Longerbeam wrote:
On 09/18/2018 02:34 AM, Philipp Zabel wrote:
+/*
+ * Tile left edges are required to be aligned to multiples of 8 bytes
+ * by the IDMAC.
+ */
+static inline u32 tile_left_align(const struct ipu_image_pixfmt *fmt)
+{
+ return fmt->planar ? 8 * fmt->uv_width_dec : 64 / fmt->bpp;
+}
As I indicated, shouldn't this be
return fmt->planar ? 8 * fmt->uv_width_dec : 8;
?
Just from a unit analysis perspective, "64 / fmt->bp" has
units of pixels / 8-bytes, it should have units of bytes.
The tile alignment is in pixels, not in bytes.
Ah, yes of course you are right, I used to know this :) I am
loosing track of this code.
For 16-bit and 32-bit
packed formats, we only need to align to 4 or 2 pixels, respectively,
as the LCM of 8-byte alignment and 2-byte or 4-byte pixel size is
always 8 bytes.
Yes I agree, the LCM of 8-byte alignment and bytes-per-pixel should
be the tile left edge alignment in pixels.
But now that you pointed it out, it is quite obvious that this can't
work for 24-bit packed formats. Here the LCM of 8-byte alignment and 3-
byte pixels is 24 bytes, or 8 pixels.
How about:
if (fmt->planar)
return fmt->uv_packed ? 8 : 8 * fmt->uv_width_dec;
else
return fmt->bpp == 32 ? 2 : fmt->bpp == 16 ? 4 : 8;
Yep, that looks better. I tested this and it works fine.
Steve
Re: i.MX6 IPU CSI analog video input on Ventana
On 10/17/18 4:05 PM, Tim Harvey wrote: On Wed, Oct 17, 2018 at 2:33 PM Steve Longerbeam wrote: Hi Tim, On 10/17/18 1:38 PM, Tim Harvey wrote: On Mon, Jun 4, 2018 at 1:58 AM Krzysztof Hałasa wrote: I've just tested the PAL setup: in currect situation (v4.17 + Steve's fix-csi-interlaced.2 + "media: adv7180: fix field type" + a small cheap PAL camera) the following produces bottom-first interlaced frames: media-ctl -r -l '"adv7180 2-0020":0->"ipu2_csi1_mux":1[1], "ipu2_csi1_mux":2->"ipu2_csi1":0[1], "ipu2_csi1":2->"ipu2_csi1 capture":0[1]' media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x576 field:alternate]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x576]" media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x576 field:interlaced]" "adv7180 2-0020":0 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1_mux":1 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1_mux":2 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1":0 [fmt:UYVY2X8/720x576 field:alternate] "ipu2_csi1":2 [fmt:AYUV32/720x576 field:interlaced] I think it would be great if these changes make their way upstream. The details could be refined then. Krzysztof / Steve / Philipp, I jumped back onto IMX6 video capture from the adv7180 the other day trying to help out a customer that's using mainline and found things are still not working right. Where is all of this at these days? If I use v4.19 with Steves 'imx-media: Fixes for interlaced capture' v3 series (https://patchwork.kernel.org/cover/10626499/) I rolling/split (un-synchronized) video using: # Setup links media-ctl -r media-ctl -l '"adv7180 2-0020":0 -> "ipu2_csi1_mux":1[1]' media-ctl -l '"ipu2_csi1_mux":2 -> "ipu2_csi1":0[1]' media-ctl -l '"ipu2_csi1":1 -> "ipu2_ic_prp":0[1]' media-ctl -l '"ipu2_ic_prp":2 -> "ipu2_ic_prpvf":0[1]' media-ctl -l '"ipu2_ic_prpvf":1 -> "ipu2_ic_prpvf capture":0[1]' # Configure pads media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" media-ctl -V "'ipu2_csi1':1 [fmt:UYVY2X8/720x480 field:interlaced]" media-ctl -V "'ipu2_ic_prp':2 [fmt:UYVY2X8/720x480 field:interlaced]" media-ctl -V "'ipu2_ic_prpvf':1 [fmt:UYVY2X8/720x480 field:none]" # stream JPEG/RTP/UDP gst-launch-1.0 v4l2src device=/dev/video3 ! video/x-raw,format=UYVY ! jpegenc ! rtpjpegpay ! udpsink host=$SERVER port=$PORT ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video3' does not support progressive interlacing I'm doing the above on a Gateworks GW5404 IMXQ which has a tda1997x HDMI receiver sensor and an adv7180 Analog CVBS sensor - media graph is here: http://dev.gateworks.com/docs/linux/media/imx6q-gw54xx-media.png Are there other patches I need or different field formats above with 4.19? Do any of the other kernels work without patchsets that you know of between 4.16 and 4.19? First, the v3 series is out of date. Please apply the latest v5 posting of that series. See the imx.rst doc regarding field type negotiation, all pads starting at ipu2_csi1:1 should be 'seq-bt' or 'seq-tb' until the capture device, which should be set to 'interlaced' to enable IDMAC interweave. The ADV7180 now correctly sets its field type to alternate, which imx-media-csi.c translates to seq-tb or seq-bt at its output pad. See the SabreAuto examples in the doc. For the rolling/split image problem, try the attached somewhat hackish patch. There used to be code in imx-media-csi.c that searched for the backend sensor and queries via .g_skip_frames whether the sensor produces bad frames at first stream-on. But there was push-back on that, so the attached is another approach that doesn't require searching for a backend sensor. Steve, Thanks - I hadn't noticed the updated series. I've built it on top of linux-media/master and tested with: - Testing linux-media/master + your v5 now: # Use simple interweaving media-ctl -r # Setup links media-ctl -l '"adv7180 2-0020":0 -> "ipu2_csi1_mux":1[1]' media-ctl -l '"ipu2_csi1_mux":2 -> "ipu2_csi1":0[1]' media-ctl -l '"ipu2_csi1":2 -> "ipu2_csi1 capture":0[1]' # Configure pads media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480 field:seq-bt]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480]" media-ctl -V "'ipu2_csi1':1 [fmt:AYUV32/720x480]" # Configure ipu_csi capture interface (/dev/video7) v4l2-ctl -d7 --set-fmt-video=field=interlaced_bt # Stream JPEG/RTP/UDP gst-launch-1.0 v4l2src device=/dev/video7 ! video/x-raw,format=UYVY ! jpegenc ! rtpjpegpay ! udpsink host=$SERVER port=5000 ^^ gives me ER
[PATCH v5 00/12] imx-media: Fixes for interlaced capture
A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. History: v5: - Added a regression fix to allow empty endpoints to CSI (fix for imx6q SabreAuto). - Cleaned up some convoluted code in ipu_csi_init_interface(), suggested by Philipp Zabel. - Fixed a regression in csi_setup(), caught by Philipp. - Removed interweave_offset and replace with boolean interweave_swap, suggested by Philipp. - Make clear that it is IDMAC channel that does pixel reordering and interweave, not the CSI, in the imx.rst doc, caught by Philipp. v4: - rebased to latest media-tree master branch. - Make patch author and SoB email addresses the same. v3: - add support for/fix interweaved scan with YUV planar output. - fix bug in 4:2:0 U/V offset macros. - add patch that generalizes behavior of field swap in ipu_csi_init_interface(). - add support for interweaved scan with field order swap. Suggested by Philipp Zabel. - in v2, inteweave scan was determined using field types of CSI (and PRPENCVF) at the sink and source pads. In v3, this has been moved one hop downstream: interweave is now determined using field type at source pad, and field type selected at capture interface. Suggested by Philipp. - make sure to double CSI crop target height when input field type in alternate. - more updates to media driver doc to reflect above. v2: - update media driver doc. - enable idmac interweave only if input field is sequential/alternate, and output field is 'interlaced*'. - move field try logic out of *try_fmt and into separate function. - fix bug with resetting crop/compose rectangles. - add a patch that fixes a field order bug in VDIC indirect mode. - remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro Suggested-by: Nicolas Dufresne . - add macro V4L2_FIELD_IS_INTERLACED(). Steve Longerbeam (12): media: videodev2.h: Add more field helper macros gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Add planar support to interlaced scan media: imx: Fix field negotiation media: imx-csi: Input connections to CSI should be optional media: imx-csi: Double crop height for alternate fields at sink media: imx: interweave and odd-chroma-row skip are incompatible media: imx-csi: Allow skipping odd chroma rows for YVU420 media: imx: vdic: rely on VDIC for correct field order media: imx-csi: Move crop/compose reset after filling default mbus fields media: imx: Allow interweave with top/bottom lines swapped media: imx.rst: Update doc to reflect fixes to interlaced capture Documentation/media/v4l-drivers/imx.rst | 103 +++ drivers/gpu/ipu-v3/ipu-cpmem.c| 26 ++- drivers/gpu/ipu-v3/ipu-csi.c | 119 + drivers/staging/media/imx/imx-ic-prpencvf.c | 46 +++-- drivers/staging/media/imx/imx-media-capture.c | 14 ++ drivers/staging/media/imx/imx-media-csi.c | 168 +- drivers/staging/media/imx/imx-media-vdic.c| 12 +- include/uapi/linux/videodev2.h| 7 + include/video/imx-ipu-v3.h| 6 +- 9 files changed, 354 insertions(+), 147 deletions(-) -- 2.17.1
[PATCH v5 01/12] media: videodev2.h: Add more field helper macros
Adds two helper macros:
V4L2_FIELD_IS_SEQUENTIAL: returns true if the given field type is
'sequential', that is a full frame is transmitted, or exists in
memory, as all top field lines followed by all bottom field lines,
or vice-versa.
V4L2_FIELD_IS_INTERLACED: returns true if the given field type is
'interlaced', that is a full frame is transmitted, or exists in
memory, as top field lines interlaced with bottom field lines.
Signed-off-by: Steve Longerbeam
---
Changes since v3:
- none
Changes since v2:
- none
Changes since v1:
- add the complement macro V4L2_FIELD_IS_INTERLACED
- remove V4L2_FIELD_ALTERNATE from V4L2_FIELD_IS_SEQUENTIAL macro.
- moved new macros past end of existing V4L2_FIELD_HAS_* macros.
---
include/uapi/linux/videodev2.h | 7 +++
1 file changed, 7 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 29729d580452..f7f031736d91 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -130,6 +130,13 @@ enum v4l2_field {
((field) == V4L2_FIELD_BOTTOM ||\
(field) == V4L2_FIELD_TOP ||\
(field) == V4L2_FIELD_ALTERNATE)
+#define V4L2_FIELD_IS_INTERLACED(field) \
+ ((field) == V4L2_FIELD_INTERLACED ||\
+(field) == V4L2_FIELD_INTERLACED_TB ||\
+(field) == V4L2_FIELD_INTERLACED_BT)
+#define V4L2_FIELD_IS_SEQUENTIAL(field) \
+ ((field) == V4L2_FIELD_SEQ_TB ||\
+(field) == V4L2_FIELD_SEQ_BT)
enum v4l2_buf_type {
V4L2_BUF_TYPE_VIDEO_CAPTURE= 1,
--
2.17.1
Re: [PATCH v3 10/16] gpu: ipu-v3: image-convert: select optimal seam positions
On 09/18/2018 02:34 AM, Philipp Zabel wrote:
+/*
+ * Tile left edges are required to be aligned to multiples of 8 bytes
+ * by the IDMAC.
+ */
+static inline u32 tile_left_align(const struct ipu_image_pixfmt *fmt)
+{
+ return fmt->planar ? 8 * fmt->uv_width_dec : 64 / fmt->bpp;
+}
As I indicated, shouldn't this be
return fmt->planar ? 8 * fmt->uv_width_dec : 8;
?
Just from a unit analysis perspective, "64 / fmt->bp" has
units of pixels / 8-bytes, it should have units of bytes.
Steve
Re: [PATCH v3 00/16] i.MX media mem2mem scaler
v3: image-convert: prepare for per-tile configuration gpu: ipu-v3: image-convert: calculate per-tile resize coefficients gpu: ipu-v3: image-convert: reconfigure IC per tile gpu: ipu-v3: image-convert: store tile top/left position gpu: ipu-v3: image-convert: calculate tile dimensions and offsets outside fill_image gpu: ipu-v3: image-convert: move tile alignment helpers gpu: ipu-v3: image-convert: select optimal seam positions gpu: ipu-v3: image-convert: fix debug output for varying tile sizes gpu: ipu-v3: image-convert: relax alignment restrictions gpu: ipu-v3: image-convert: fix bytesperline adjustment gpu: ipu-v3: image-convert: add some ASCII art to the exposition gpu: ipu-v3: image-convert: disable double buffering if necessary gpu: ipu-v3: image-convert: allow three rows or columns Steve Longerbeam (1): gpu: ipu-cpmem: add WARN_ON_ONCE() for unaligned dma buffers drivers/gpu/ipu-v3/ipu-cpmem.c| 6 + drivers/gpu/ipu-v3/ipu-ic.c | 52 +- drivers/gpu/ipu-v3/ipu-image-convert.c| 919 +++--- drivers/staging/media/imx/Kconfig | 1 + drivers/staging/media/imx/Makefile| 1 + drivers/staging/media/imx/imx-media-dev.c | 11 + drivers/staging/media/imx/imx-media-mem2mem.c | 873 + drivers/staging/media/imx/imx-media.h | 10 + include/video/imx-ipu-v3.h| 6 + 9 files changed, 1727 insertions(+), 152 deletions(-) create mode 100644 drivers/staging/media/imx/imx-media-mem2mem.c
Re: [PATCH v4 00/11] imx-media: Fixes for interlaced capture
On 10/04/2018 12:34 PM, Hans Verkuil wrote: On 10/04/2018 08:53 PM, Steve Longerbeam wrote: A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. History: v4: - rebased to latest media-tree master branch. - Make patch author and SoB email addresses the same. v3: - add support for/fix interweaved scan with YUV planar output. - fix bug in 4:2:0 U/V offset macros. - add patch that generalizes behavior of field swap in ipu_csi_init_interface(). - add support for interweaved scan with field order swap. Suggested by Philipp Zabel. - in v2, inteweave scan was determined using field types of CSI (and PRPENCVF) at the sink and source pads. In v3, this has been moved one hop downstream: interweave is now determined using field type at source pad, and field type selected at capture interface. Suggested by Philipp. - make sure to double CSI crop target height when input field type in alternate. - more updates to media driver doc to reflect above. v2: - update media driver doc. - enable idmac interweave only if input field is sequential/alternate, and output field is 'interlaced*'. - move field try logic out of *try_fmt and into separate function. - fix bug with resetting crop/compose rectangles. - add a patch that fixes a field order bug in VDIC indirect mode. - remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro Suggested-by: Nicolas Dufresne . - add macro V4L2_FIELD_IS_INTERLACED(). Steve Longerbeam (11): media: videodev2.h: Add more field helper macros gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Add planar support to interlaced scan What should I do with these patches? Do they go through us? Or the drm subsystem (or whoever handles this)? If it goes through another subsystem, then I can Ack them. Hi Hans, sorry you are right. Philipp Zabel needs to merge these to his imx-drm/fixes tree. Then we need to wait for them to filter over to media-tree. Same old slow process, I wish this were faster, but that is the drawback of changes that span subsystems. I will submit the above patches to dri-devel ML. And resubmit this series once they hit media-tree. Steve media: imx: Fix field negotiation media: imx-csi: Double crop height for alternate fields at sink media: imx: interweave and odd-chroma-row skip are incompatible media: imx-csi: Allow skipping odd chroma rows for YVU420 media: imx: vdic: rely on VDIC for correct field order media: imx-csi: Move crop/compose reset after filling default mbus fields media: imx: Allow interweave with top/bottom lines swapped media: imx.rst: Update doc to reflect fixes to interlaced capture Documentation/media/v4l-drivers/imx.rst | 93 ++ drivers/gpu/ipu-v3/ipu-cpmem.c| 26 ++- drivers/gpu/ipu-v3/ipu-csi.c | 132 ++ drivers/staging/media/imx/imx-ic-prpencvf.c | 48 +++-- drivers/staging/media/imx/imx-media-capture.c | 14 ++ drivers/staging/media/imx/imx-media-csi.c | 166 -- drivers/staging/media/imx/imx-media-vdic.c| 12 +- include/uapi/linux/videodev2.h| 7 + include/video/imx-ipu-v3.h| 6 +- 9 files changed, 359 insertions(+), 145 deletions(-)
[PATCH v4 00/11] imx-media: Fixes for interlaced capture
A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. History: v4: - rebased to latest media-tree master branch. - Make patch author and SoB email addresses the same. v3: - add support for/fix interweaved scan with YUV planar output. - fix bug in 4:2:0 U/V offset macros. - add patch that generalizes behavior of field swap in ipu_csi_init_interface(). - add support for interweaved scan with field order swap. Suggested by Philipp Zabel. - in v2, inteweave scan was determined using field types of CSI (and PRPENCVF) at the sink and source pads. In v3, this has been moved one hop downstream: interweave is now determined using field type at source pad, and field type selected at capture interface. Suggested by Philipp. - make sure to double CSI crop target height when input field type in alternate. - more updates to media driver doc to reflect above. v2: - update media driver doc. - enable idmac interweave only if input field is sequential/alternate, and output field is 'interlaced*'. - move field try logic out of *try_fmt and into separate function. - fix bug with resetting crop/compose rectangles. - add a patch that fixes a field order bug in VDIC indirect mode. - remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro Suggested-by: Nicolas Dufresne . - add macro V4L2_FIELD_IS_INTERLACED(). Steve Longerbeam (11): media: videodev2.h: Add more field helper macros gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Add planar support to interlaced scan media: imx: Fix field negotiation media: imx-csi: Double crop height for alternate fields at sink media: imx: interweave and odd-chroma-row skip are incompatible media: imx-csi: Allow skipping odd chroma rows for YVU420 media: imx: vdic: rely on VDIC for correct field order media: imx-csi: Move crop/compose reset after filling default mbus fields media: imx: Allow interweave with top/bottom lines swapped media: imx.rst: Update doc to reflect fixes to interlaced capture Documentation/media/v4l-drivers/imx.rst | 93 ++ drivers/gpu/ipu-v3/ipu-cpmem.c| 26 ++- drivers/gpu/ipu-v3/ipu-csi.c | 132 ++ drivers/staging/media/imx/imx-ic-prpencvf.c | 48 +++-- drivers/staging/media/imx/imx-media-capture.c | 14 ++ drivers/staging/media/imx/imx-media-csi.c | 166 -- drivers/staging/media/imx/imx-media-vdic.c| 12 +- include/uapi/linux/videodev2.h| 7 + include/video/imx-ipu-v3.h| 6 +- 9 files changed, 359 insertions(+), 145 deletions(-) -- 2.17.1
Re: [PATCH v3 00/14] imx-media: Fixes for interlaced capture
On 10/04/2018 06:41 AM, Hans Verkuil wrote: On 10/04/18 01:21, Steve Longerbeam wrote: Hi Hans, On 10/01/2018 03:07 AM, Hans Verkuil wrote: Hi Steve, On 08/01/2018 09:12 PM, Steve Longerbeam wrote: A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. I reviewed this series and it looks fine to me. Cool. It appears that the ipu* patches are already merged, so can you rebase and repost? Done. There are still two ipu* patches that still need a merge: gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Add planar support to interlaced scan so those will still be included in the v4 submission. I would also like to see the 'v4l2-compliance -f' for an interlaced source, if at all possible. Sure, I've run 'v4l2-compliance -f' on two configured pipelines: unprocessed capture (no scaling, CSC, rotation using ipu), and a VDIC de-interlace pipeline. I have the text output, the output is huge but here is the abbreviated results: Unprocessed pipeline: root@mx6q:/home/fu# v4l2-compliance -d4 -f v4l2-compliance SHA : 2d35de61ac90b030fe15439809b807014e9751fe test VIDIOC_G/S/ENUMINPUT: FAIL test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL This looks like something that should work. Not relevant for this patch series, but something you should look into. Yes, I've been meaning to implement (UN)SUBSCRIBE_EVENT/DQEVENT at the capture interface. I'll send a patch soon. Total: 715, Succeeded: 713, Failed: 2, Warnings: 0 VDIC de-interlace pipeline: root@mx6q:/home/fu# v4l2-compliance -d1 -f v4l2-compliance SHA : 2d35de61ac90b030fe15439809b807014e9751fe test VIDIOC_G/S/ENUMINPUT: FAIL test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL test VIDIOC_G/S_PARM: FAIL Same here: this appears to be an actual bug. But also not related to this patch series. It's because the capture interface passes vidioc_[gs]_parm down to its connected source subdevice as [gs]_frame_interval, which in this case is PRPVF, which just accepts whatever frame interval is requested. Not sure why v4l2-compliance reports an error, but perhaps [gs]_frame_interval should be chained, until it reaches a subdevice that actually cares about frame intervals (in this case CSI and VDIC), similar to how [gs]_stream is chained. Anyway something else to look at later. Steve
Re: [PATCH v3 00/14] imx-media: Fixes for interlaced capture
Hi Hans, On 10/01/2018 03:07 AM, Hans Verkuil wrote: Hi Steve, On 08/01/2018 09:12 PM, Steve Longerbeam wrote: A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. I reviewed this series and it looks fine to me. Cool. It appears that the ipu* patches are already merged, so can you rebase and repost? Done. There are still two ipu* patches that still need a merge: gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Add planar support to interlaced scan so those will still be included in the v4 submission. I would also like to see the 'v4l2-compliance -f' for an interlaced source, if at all possible. Sure, I've run 'v4l2-compliance -f' on two configured pipelines: unprocessed capture (no scaling, CSC, rotation using ipu), and a VDIC de-interlace pipeline. I have the text output, the output is huge but here is the abbreviated results: Unprocessed pipeline: root@mx6q:/home/fu# v4l2-compliance -d4 -f v4l2-compliance SHA : 2d35de61ac90b030fe15439809b807014e9751fe test VIDIOC_G/S/ENUMINPUT: FAIL test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL Total: 715, Succeeded: 713, Failed: 2, Warnings: 0 VDIC de-interlace pipeline: root@mx6q:/home/fu# v4l2-compliance -d1 -f v4l2-compliance SHA : 2d35de61ac90b030fe15439809b807014e9751fe test VIDIOC_G/S/ENUMINPUT: FAIL test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL test VIDIOC_G/S_PARM: FAIL Total: 50, Succeeded: 47, Failed: 3, Warnings: 1 I will send you the full output privately. For that matter, were you able to test all the field formats? Yes. I've tested on imx6q SabreAuto with the ADV7180 alternate source, all of the following are tested and produce good video: ntsc alternate -> interlaced-tb ntsc alternate -> interlaced-bt ntsc alternate -> none (VDIC pipeline) ntsc alternate -> none (VDIC pipeline) pal alternate -> interlaced-tb pal alternate -> interlaced-bt pal alternate -> none (VDIC pipeline) pal alternate -> none (VDIC pipeline) Steve History: v3: - add support for/fix interweaved scan with YUV planar output. - fix bug in 4:2:0 U/V offset macros. - add patch that generalizes behavior of field swap in ipu_csi_init_interface(). - add support for interweaved scan with field order swap. Suggested by Philipp Zabel. - in v2, inteweave scan was determined using field types of CSI (and PRPENCVF) at the sink and source pads. In v3, this has been moved one hop downstream: interweave is now determined using field type at source pad, and field type selected at capture interface. Suggested by Philipp. - make sure to double CSI crop target height when input field type in alternate. - more updates to media driver doc to reflect above. v2: - update media driver doc. - enable idmac interweave only if input field is sequential/alternate, and output field is 'interlaced*'. - move field try logic out of *try_fmt and into separate function. - fix bug with resetting crop/compose rectangles. - add a patch that fixes a field order bug in VDIC indirect mode. - remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro Suggested-by: Nicolas Dufresne . - add macro V4L2_FIELD_IS_INTERLACED(). Philipp Zabel (1): gpu: ipu-v3: Allow negative offsets for interlaced scanning Steve Longerbeam (13): media: videodev2.h: Add more field helper macros gpu: ipu-csi: Check for field type alternate gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Fix U/V offset macros for planar 4:2:0 gpu: ipu-v3: Add planar support to interlaced scan media: imx: Fix field negotiation media: imx-csi: Double crop height for alternate fields at sink media: imx: interweave and odd-chroma-row skip are incompatible media: imx-csi: Allow skipping odd chroma rows for YVU420 media: imx: vdic: rely on VDIC for correct field order media: imx-csi: Move crop/compose reset after filling default mbus fields media: imx: Allow interweave with top/bottom lines swapped media: imx.rst: Update doc to reflect fixes to interlaced capture Documentation/media/v4l-drivers/imx.rst | 93 ++- drivers/gpu/ipu-v3/ipu-cpmem.c| 45 ++- drivers/gpu/ipu-v3/ipu-csi.c | 136 ++--- drivers/staging/media/imx/imx-ic-prpencvf.c | 48 ++-- drivers/staging/media/imx/imx-media-capture.c | 14 +++ drivers/staging/media/imx/imx-media-csi.c | 166 ++ drivers/staging/media/imx/imx-media-vdic.c| 12 +- include/uapi/linux/videodev2.h| 7 ++ include/video/imx-ipu-v3.h| 6 +- 9 files changed, 377 insertions(+), 150 deletions(-)
[RESEND PATCH v7 13/17] media: staging/imx: Switch to v4l2_async_notifier_add_*_subdev
Switch to v4l2_async_notifier_add_*_subdev() when adding async subdevs
to the imx-media root notifier. This removes the need to check for
an already added asd, since v4l2_async_notifier_add_*_subdev() does this
check. Also no need to allocate a subdevs array when registering the
root notifier, or keeping an internal master asd_list, since this is
moved to the notifier's asd_list.
Signed-off-by: Steve Longerbeam
---
Changes since v6:
- none
Changes since v5:
- remove reference to notifier.num_subdevs and call
v4l2_async_notifier_init(). Suggested by Sakari Ailus.
---
drivers/staging/media/imx/imx-media-dev.c | 121 +-
.../staging/media/imx/imx-media-internal-sd.c | 5 +-
drivers/staging/media/imx/imx-media.h | 4 +-
3 files changed, 36 insertions(+), 94 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-dev.c
b/drivers/staging/media/imx/imx-media-dev.c
index a9faee4b2495..481840195071 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -33,43 +33,10 @@ static inline struct imx_media_dev *notifier2dev(struct
v4l2_async_notifier *n)
}
/*
- * Find an asd by fwnode or device name. This is called during
- * driver load to form the async subdev list and bind them.
- */
-static struct v4l2_async_subdev *
-find_async_subdev(struct imx_media_dev *imxmd,
- struct fwnode_handle *fwnode,
- const char *devname)
-{
- struct imx_media_async_subdev *imxasd;
- struct v4l2_async_subdev *asd;
-
- list_for_each_entry(imxasd, >asd_list, list) {
- asd = >asd;
- switch (asd->match_type) {
- case V4L2_ASYNC_MATCH_FWNODE:
- if (fwnode && asd->match.fwnode == fwnode)
- return asd;
- break;
- case V4L2_ASYNC_MATCH_DEVNAME:
- if (devname && !strcmp(asd->match.device_name,
- devname))
- return asd;
- break;
- default:
- break;
- }
- }
-
- return NULL;
-}
-
-
-/*
- * Adds a subdev to the async subdev list. If fwnode is non-NULL, adds
- * the async as a V4L2_ASYNC_MATCH_FWNODE match type, otherwise as
- * a V4L2_ASYNC_MATCH_DEVNAME match type using the dev_name of the
- * given platform_device. This is called during driver load when
+ * Adds a subdev to the root notifier's async subdev list. If fwnode is
+ * non-NULL, adds the async as a V4L2_ASYNC_MATCH_FWNODE match type,
+ * otherwise as a V4L2_ASYNC_MATCH_DEVNAME match type using the dev_name
+ * of the given platform_device. This is called during driver load when
* forming the async subdev list.
*/
int imx_media_add_async_subdev(struct imx_media_dev *imxmd,
@@ -80,47 +47,34 @@ int imx_media_add_async_subdev(struct imx_media_dev *imxmd,
struct imx_media_async_subdev *imxasd;
struct v4l2_async_subdev *asd;
const char *devname = NULL;
- int ret = 0;
-
- mutex_lock(>mutex);
+ int ret;
- if (pdev)
+ if (fwnode) {
+ asd = v4l2_async_notifier_add_fwnode_subdev(
+ >notifier, fwnode, sizeof(*imxasd));
+ } else {
devname = dev_name(>dev);
-
- /* return -EEXIST if this asd already added */
- if (find_async_subdev(imxmd, fwnode, devname)) {
- dev_dbg(imxmd->md.dev, "%s: already added %s\n",
- __func__, np ? np->name : devname);
- ret = -EEXIST;
- goto out;
+ asd = v4l2_async_notifier_add_devname_subdev(
+ >notifier, devname, sizeof(*imxasd));
}
- imxasd = devm_kzalloc(imxmd->md.dev, sizeof(*imxasd), GFP_KERNEL);
- if (!imxasd) {
- ret = -ENOMEM;
- goto out;
- }
- asd = >asd;
-
- if (fwnode) {
- asd->match_type = V4L2_ASYNC_MATCH_FWNODE;
- asd->match.fwnode = fwnode;
- } else {
- asd->match_type = V4L2_ASYNC_MATCH_DEVNAME;
- asd->match.device_name = devname;
- imxasd->pdev = pdev;
+ if (IS_ERR(asd)) {
+ ret = PTR_ERR(asd);
+ if (ret == -EEXIST)
+ dev_dbg(imxmd->md.dev, "%s: already added %s\n",
+ __func__, np ? np->name : devname);
+ return ret;
}
- list_add_tail(>list, >asd_list);
+ imxasd = to_imx_media_asd(asd);
- imxmd->notifier.num_subdevs++;
+ if (devname)
+ imxasd->pdev = pdev;
dev_dbg(imxmd->md.dev, "%s: added %s, match type %s\n",
__func__, np ? n
[RESEND PATCH v7 00/17] media: imx: Switch to subdev notifiers
This patchset converts the imx-media driver and its dependent subdevs to use subdev notifiers. There are a couple shortcomings in v4l2-core that prevented subdev notifiers from working correctly in imx-media: 1. v4l2_async_notifier_fwnode_parse_endpoint() treats a fwnode endpoint that is not connected to a remote device as an error. But in the case of the video-mux subdev, this is not an error, it is OK if some of the mux inputs have no connection. Also, Documentation/devicetree/bindings/media/video-interfaces.txt explicitly states that the 'remote-endpoint' property is optional. So the first patch is a small modification to ignore empty endpoints in v4l2_async_notifier_fwnode_parse_endpoint() and allow __v4l2_async_notifier_parse_fwnode_endpoints() to continue to parse the remaining port endpoints of the device. 2. In the imx-media graph, multiple subdevs will encounter the same upstream subdev (such as the imx6-mipi-csi2 receiver), and so v4l2_async_notifier_parse_fwnode_endpoints() will add imx6-mipi-csi2 multiple times. This is treated as an error by v4l2_async_notifier_register() later. To get around this problem, add an v4l2_async_notifier_add_subdev() which first verifies the provided asd does not already exist in the given notifier asd list or in other registered notifiers. If the asd exists, the function returns -EEXIST and it's up to the caller to decide if that is an error (in imx-media case it is never an error). Patches 2-5 deal with adding that support. 3. Patch 6 adds v4l2_async_register_fwnode_subdev(), which is a convenience function for parsing a subdev's fwnode port endpoints for connected remote subdevs, registering a subdev notifier, and then registering the sub-device itself. 4. Patches 7-14 update the subdev drivers to register a subdev notifier with endpoint parsing, and the changes to imx-media to support that. 5. Finally, the last 3 patches endeavor to completely remove support for the notifier->subdevs[] array in platform drivers and v4l2 core. All platform drivers are modified to make use of v4l2_async_notifier_add_subdev() and its related convenience functions to add asd's to the notifier @asd_list, and any allocation or reference to the notifier->subdevs[] array removed. After that large patch, notifier->subdevs[] array is stripped from v4l2-async and v4l2-subdev docs are updated to reflect the new method of adding asd's to notifiers. Signed-off-by: Steve Longerbeam Patches 07-14 (video-mux and the imx patches) are Reviewed-by: Philipp Zabel Patches 01-14 are Tested-by: Philipp Zabel on i.MX6 with Toshiba TC358743 connected via MIPI CSI-2. History: v7: - Comment, error message, and whitespace fixes in patch 02/17. Suggested by Mauro Chehab. - Fixed merge conflict in drivers/media/platform/ti-vpe/cal.c. - Make patch author and SoB email addresses the same. v6: - Export v4l2_async_notifier_init(), which must be called by all drivers before the first call to v4l2_async_notifier_add_subdev(). Suggested by Sakari Ailus. - Removed @num_subdevs from struct v4l2_async_notifier, and the macro V4L2_MAX_SUBDEVS. Sugested by Sakari. - Fixed a double device node put in vpif_capture.c. Reported by Sakari. - Fixed wrong printk format qualifiers in xilinx-vipp.c. Reported by Dan Carpenter. v5: - see point 5 above. v4: - small non-functional code cleanup in video-mux.c. - strip TODO for comparing custom asd's for equivalence. - add three more convenience functions: v4l2_async_notifier_add_fwnode_subdev, v4l2_async_notifier_add_i2c_subdev, v4l2_async_notifier_add_devname_subdev. - strip support in v4l2_async_register_fwnode_subdev for sub-devices that register from port nodes. v3: - code optimization in asd_equal(), and remove unneeded braces, suggested by Sakari Ailus. - add a NULL asd pointer check to v4l2_async_notifier_asd_valid(). - fix an error-out path in v4l2_async_register_fwnode_subdev() that forgot to put device. v2: - don't pass an empty endpoint to the parse_endpoint callback, v4l2_async_notifier_fwnode_parse_endpoint() now just ignores them and returns success. - Fix a couple compile warnings and errors seen in i386 and sh archs. Steve Longerbeam (17): media: v4l2-fwnode: ignore endpoints that have no remote port parent media: v4l2: async: Allow searching for asd of any type media: v4l2: async: Add v4l2_async_notifier_add_subdev media: v4l2: async: Add convenience functions to allocate and add asd's media: v4l2-fwnode: Switch to v4l2_async_notifier_add_subdev media: v4l2-fwnode: Add a convenience function for registering subdevs with notifiers media: platform: video-mux: Register a subdev notifier media: imx: csi: Register a subdev notifier media: imx: mipi csi-2: Register a subdev notifier media: staging/imx: of: Remove recursive graph walk media: staging/imx: Loop through all registered subdevs for media
[RESEND PATCH v7 01/17] media: v4l2-fwnode: ignore endpoints that have no remote port parent
Documentation/devicetree/bindings/media/video-interfaces.txt states that
the 'remote-endpoint' property is optional.
So v4l2_async_notifier_fwnode_parse_endpoint() should not return error
if the endpoint has no remote port parent. Just ignore the endpoint,
skip adding an asd to the notifier and return 0.
__v4l2_async_notifier_parse_fwnode_endpoints() will then continue
parsing the remaining port endpoints of the device.
Signed-off-by: Steve Longerbeam
Acked-by: Hans Verkuil
---
Changes since v6:
- none
Changes since v5:
- none
Changes since v4:
- none
Changes since v3:
- none
Changes since v2:
- none
Changes since v1:
- don't pass an empty endpoint to the parse_endpoint callback,
v4l2_async_notifier_fwnode_parse_endpoint() now just ignores them
and returns success. The current users of
v4l2_async_notifier_parse_fwnode_endpoints() (omap3isp, rcar-vin,
intel-ipu3) no longer need modification.
---
drivers/media/v4l2-core/v4l2-fwnode.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-fwnode.c
b/drivers/media/v4l2-core/v4l2-fwnode.c
index 169bdbb1f61a..0b8c736b1606 100644
--- a/drivers/media/v4l2-core/v4l2-fwnode.c
+++ b/drivers/media/v4l2-core/v4l2-fwnode.c
@@ -367,7 +367,7 @@ static int v4l2_async_notifier_fwnode_parse_endpoint(
fwnode_graph_get_remote_port_parent(endpoint);
if (!asd->match.fwnode) {
dev_warn(dev, "bad remote port parent\n");
- ret = -EINVAL;
+ ret = -ENOTCONN;
goto out_err;
}
--
2.17.1
[PATCH v7 00/17] media: imx: Switch to subdev notifiers
This patchset converts the imx-media driver and its dependent subdevs to use subdev notifiers. There are a couple shortcomings in v4l2-core that prevented subdev notifiers from working correctly in imx-media: 1. v4l2_async_notifier_fwnode_parse_endpoint() treats a fwnode endpoint that is not connected to a remote device as an error. But in the case of the video-mux subdev, this is not an error, it is OK if some of the mux inputs have no connection. Also, Documentation/devicetree/bindings/media/video-interfaces.txt explicitly states that the 'remote-endpoint' property is optional. So the first patch is a small modification to ignore empty endpoints in v4l2_async_notifier_fwnode_parse_endpoint() and allow __v4l2_async_notifier_parse_fwnode_endpoints() to continue to parse the remaining port endpoints of the device. 2. In the imx-media graph, multiple subdevs will encounter the same upstream subdev (such as the imx6-mipi-csi2 receiver), and so v4l2_async_notifier_parse_fwnode_endpoints() will add imx6-mipi-csi2 multiple times. This is treated as an error by v4l2_async_notifier_register() later. To get around this problem, add an v4l2_async_notifier_add_subdev() which first verifies the provided asd does not already exist in the given notifier asd list or in other registered notifiers. If the asd exists, the function returns -EEXIST and it's up to the caller to decide if that is an error (in imx-media case it is never an error). Patches 2-5 deal with adding that support. 3. Patch 6 adds v4l2_async_register_fwnode_subdev(), which is a convenience function for parsing a subdev's fwnode port endpoints for connected remote subdevs, registering a subdev notifier, and then registering the sub-device itself. 4. Patches 7-14 update the subdev drivers to register a subdev notifier with endpoint parsing, and the changes to imx-media to support that. 5. Finally, the last 3 patches endeavor to completely remove support for the notifier->subdevs[] array in platform drivers and v4l2 core. All platform drivers are modified to make use of v4l2_async_notifier_add_subdev() and its related convenience functions to add asd's to the notifier @asd_list, and any allocation or reference to the notifier->subdevs[] array removed. After that large patch, notifier->subdevs[] array is stripped from v4l2-async and v4l2-subdev docs are updated to reflect the new method of adding asd's to notifiers. Signed-off-by: Steve Longerbeam Patches 07-14 (video-mux and the imx patches) are Reviewed-by: Philipp Zabel Patches 01-14 are Tested-by: Philipp Zabel on i.MX6 with Toshiba TC358743 connected via MIPI CSI-2. History: v7: - Comment, error message, and whitespace fixes in patch 02/17. Suggested by Mauro Chehab. - Fixed merge conflict in drivers/media/platform/ti-vpe/cal.c. - Make patch author and SoB email addresses the same. v6: - Export v4l2_async_notifier_init(), which must be called by all drivers before the first call to v4l2_async_notifier_add_subdev(). Suggested by Sakari Ailus. - Removed @num_subdevs from struct v4l2_async_notifier, and the macro V4L2_MAX_SUBDEVS. Sugested by Sakari. - Fixed a double device node put in vpif_capture.c. Reported by Sakari. - Fixed wrong printk format qualifiers in xilinx-vipp.c. Reported by Dan Carpenter. v5: - see point 5 above. v4: - small non-functional code cleanup in video-mux.c. - strip TODO for comparing custom asd's for equivalence. - add three more convenience functions: v4l2_async_notifier_add_fwnode_subdev, v4l2_async_notifier_add_i2c_subdev, v4l2_async_notifier_add_devname_subdev. - strip support in v4l2_async_register_fwnode_subdev for sub-devices that register from port nodes. v3: - code optimization in asd_equal(), and remove unneeded braces, suggested by Sakari Ailus. - add a NULL asd pointer check to v4l2_async_notifier_asd_valid(). - fix an error-out path in v4l2_async_register_fwnode_subdev() that forgot to put device. v2: - don't pass an empty endpoint to the parse_endpoint callback, v4l2_async_notifier_fwnode_parse_endpoint() now just ignores them and returns success. - Fix a couple compile warnings and errors seen in i386 and sh archs. Steve Longerbeam (17): media: v4l2-fwnode: ignore endpoints that have no remote port parent media: v4l2: async: Allow searching for asd of any type media: v4l2: async: Add v4l2_async_notifier_add_subdev media: v4l2: async: Add convenience functions to allocate and add asd's media: v4l2-fwnode: Switch to v4l2_async_notifier_add_subdev media: v4l2-fwnode: Add a convenience function for registering subdevs with notifiers media: platform: video-mux: Register a subdev notifier media: imx: csi: Register a subdev notifier media: imx: mipi csi-2: Register a subdev notifier media: staging/imx: of: Remove recursive graph walk media: staging/imx: Loop through all registered subdevs for media
Re: [PATCH] media: imx: Skip every second frame in VDIC DIRECT mode
On 09/17/2018 03:27 AM, Hans Verkuil wrote: On 05/07/2018 11:54 AM, Hans Verkuil wrote: On 07/04/18 15:04, Marek Vasut wrote: In VDIC direct mode, the VDIC applies combing filter during and doubles the framerate, that is, after the first two half-frames are received and the first frame is emitted by the VDIC, every subsequent half-frame is patched into the result and a full frame is produced. The half-frame order in the full frames is as follows 12 32 34 54 etc. Drop every second frame to trim the framerate back to the original one of the signal and skip the odd patched frames. Signed-off-by: Marek Vasut Cc: Steve Longerbeam Cc: Philipp Zabel Steve, Philipp, I saw there was a discussion about this patch, but no clear answer whether or not this patch is OK. If it is, then please Ack this patch. Marking this patch as Obsoleted since I have no seen any activity for a long time. Hi Hans, yes that's fine. This needs to be re-worked to allow configuration of input/output frame-rates from the VDIC via [gs]_frame_interval. Steve --- drivers/staging/media/imx/imx-media-vdic.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c index 482250d47e7c..b538bbebedc5 100644 --- a/drivers/staging/media/imx/imx-media-vdic.c +++ b/drivers/staging/media/imx/imx-media-vdic.c @@ -289,6 +289,7 @@ static int vdic_setup_direct(struct vdic_priv *priv) /* set VDIC to receive from CSI for direct path */ ipu_fsu_link(priv->ipu, IPUV3_CHANNEL_CSI_DIRECT, IPUV3_CHANNEL_CSI_VDI_PREV); + ipu_set_vdi_skip(priv->ipu, 0x2); return 0; } @@ -313,6 +314,8 @@ static int vdic_setup_indirect(struct vdic_priv *priv) const struct imx_media_pixfmt *incc; int in_size, ret; + ipu_set_vdi_skip(priv->ipu, 0x0); + infmt = >format_mbus[VDIC_SINK_PAD_IDMAC]; incc = priv->cc[VDIC_SINK_PAD_IDMAC];
Re: [PATCH] media: staging/imx: Handle CSI->VDIC->PRPVF pipeline
On 09/17/2018 03:25 AM, Hans Verkuil wrote:
On 05/07/2018 11:55 AM, Hans Verkuil wrote:
On 07/04/18 15:05, Marek Vasut wrote:
In case the PRPVF is not connected directly to CSI, the PRPVF subdev
driver won't find the CSI subdev and will not configure the CSI input
mux. This is not noticable on the IPU1-CSI0 interface with parallel
camera, since the mux is set "correctly" by default and the parallel
camera will work just fine. This is however noticable on IPU2-CSI1,
where the mux is not set to the correct position by default and the
pipeline will fail.
Add similar code to what is in PRPVF to VDIC driver, so that the VDIC
can locate the CSI subdev and configure the mux correctly if the CSI
is connected to the VDIC. Make the PRPVF driver configure the CSI mux
only in case it's connected directly to CSI and not in case it is
connected to VDIC.
Signed-off-by: Marek Vasut
Cc: Philipp Zabel
Cc: Steve Longerbeam
Same here, I cannot merge with out Acks since I don't know the details
of the imx hardware.
I'm marking this patch as Obsoleted since there has been no activity for a long
time.
Hi Hans, yes that's fine. IIRC this issue has been fixed a different way.
Steve
---
drivers/staging/media/imx/imx-ic-prp.c | 6 ++
drivers/staging/media/imx/imx-media-vdic.c | 24
2 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 98923fc844ce..84fa66dae21a 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -72,14 +72,12 @@ static inline struct prp_priv *sd_to_priv(struct
v4l2_subdev *sd)
static int prp_start(struct prp_priv *priv)
{
struct imx_ic_priv *ic_priv = priv->ic_priv;
- bool src_is_vdic;
priv->ipu = priv->md->ipu[ic_priv->ipu_id];
/* set IC to receive from CSI or VDI depending on source */
- src_is_vdic = !!(priv->src_sd->grp_id & IMX_MEDIA_GRP_ID_VDIC);
-
- ipu_set_ic_src_mux(priv->ipu, priv->csi_id, src_is_vdic);
+ if (!(priv->src_sd->grp_id & IMX_MEDIA_GRP_ID_VDIC))
+ ipu_set_ic_src_mux(priv->ipu, priv->csi_id, false);
return 0;
}
diff --git a/drivers/staging/media/imx/imx-media-vdic.c
b/drivers/staging/media/imx/imx-media-vdic.c
index b538bbebedc5..e660911e7024 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -117,6 +117,9 @@ struct vdic_priv {
bool csi_direct; /* using direct CSI->VDIC->IC pipeline */
+ /* the CSI id at link validate */
+ int csi_id;
+
/* motion select control */
struct v4l2_ctrl_handler ctrl_hdlr;
enum ipu_motion_sel motion;
@@ -388,6 +391,9 @@ static int vdic_start(struct vdic_priv *priv)
if (ret)
return ret;
+ /* set IC to receive from CSI or VDI depending on source */
+ ipu_set_ic_src_mux(priv->ipu, priv->csi_id, true);
+
/*
* init the VDIC.
*
@@ -778,6 +784,7 @@ static int vdic_link_validate(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sink_fmt)
{
struct vdic_priv *priv = v4l2_get_subdevdata(sd);
+ struct imx_media_subdev *csi;
int ret;
ret = v4l2_subdev_link_validate_default(sd, link,
@@ -785,6 +792,23 @@ static int vdic_link_validate(struct v4l2_subdev *sd,
if (ret)
return ret;
+ csi = imx_media_find_upstream_subdev(priv->md, priv->src,
+IMX_MEDIA_GRP_ID_CSI);
+ if (!IS_ERR(csi)) {
+ switch (csi->sd->grp_id) {
+ case IMX_MEDIA_GRP_ID_CSI0:
+ priv->csi_id = 0;
+ break;
+ case IMX_MEDIA_GRP_ID_CSI1:
+ priv->csi_id = 1;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+ } else {
+ priv->csi_id = 0;
+ }
+
mutex_lock(>lock);
if (priv->csi_direct && priv->motion != HIGH_MOTION) {
Re: [PATCH] media: imx: use well defined 32-bit RGB pixel format
On 09/18/2018 02:42 AM, Philipp Zabel wrote:
The documentation in Documentation/media/uapi/v4l/pixfmt-packed-rgb.rst
tells us that the V4L2_PIX_FMT_RGB32 format is deprecated and must not
be used by new drivers. Replace it with V4L2_PIX_FMT_XRGB32.
Signed-off-by: Philipp Zabel
Acked-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-utils.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index 8aa13403b09d..0eaa353d5cb3 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -88,7 +88,7 @@ static const struct imx_media_pixfmt rgb_formats[] = {
.cs = IPUV3_COLORSPACE_RGB,
.bpp= 24,
}, {
- .fourcc = V4L2_PIX_FMT_RGB32,
+ .fourcc = V4L2_PIX_FMT_XRGB32,
.codes = {MEDIA_BUS_FMT_ARGB_1X32},
.cs = IPUV3_COLORSPACE_RGB,
.bpp= 32,
@@ -212,7 +212,7 @@ static const struct imx_media_pixfmt ipu_yuv_formats[] = {
static const struct imx_media_pixfmt ipu_rgb_formats[] = {
{
- .fourcc = V4L2_PIX_FMT_RGB32,
+ .fourcc = V4L2_PIX_FMT_XRGB32,
.codes = {MEDIA_BUS_FMT_ARGB_1X32},
.cs = IPUV3_COLORSPACE_RGB,
.bpp= 32,
Re: [PATCH v3 0/5] Fix OV5640 exposure & gain
Hi Hughes, The whole series, Acked-by: Steve Longerbeam and Tested-by: Steve Longerbeam on i.MX6q SabreSD with MIPI CSI-2 OV5640 module On 09/11/2018 06:48 AM, Hugues Fruchet wrote: This patch serie fixes some problems around exposure & gain in OV5640 driver. The 4th patch about autocontrols requires also a fix in v4l2-ctrls.c: https://www.mail-archive.com/linux-media@vger.kernel.org/msg133164.html Here is the test procedure used for exposure & gain controls check: 1) Preview in background $> gst-launch-1.0 v4l2src ! "video/x-raw, width=640, Height=480" ! queue ! waylandsink -e & 2) Check gain & exposure values $> v4l2-ctl --all | grep -e exposure -e gain | grep "(int)" exposure (int): min=0 max=65535 step=1 default=0 value=330 flags=inactive, volatile gain (int): min=0 max=1023 step=1 default=0 value=19 flags=inactive, volatile 3) Put finger in front of camera and check that gain/exposure values are changing: $> v4l2-ctl --all | grep -e exposure -e gain | grep "(int)" exposure (int): min=0 max=65535 step=1 default=0 value=660 flags=inactive, volatile gain (int): min=0 max=1023 step=1 default=0 value=37 flags=inactive, volatile 4) switch to manual mode, image exposition must not change $> v4l2-ctl --set-ctrl=gain_automatic=0 $> v4l2-ctl --set-ctrl=auto_exposure=1 Note the "1" for manual exposure. 5) Check current gain/exposure values: $> v4l2-ctl --all | grep -e exposure -e gain | grep "(int)" exposure (int): min=0 max=65535 step=1 default=0 value=330 gain (int): min=0 max=1023 step=1 default=0 value=20 6) Put finger behind camera and check that gain/exposure values are NOT changing: $> v4l2-ctl --all | grep -e exposure -e gain | grep "(int)" exposure (int): min=0 max=65535 step=1 default=0 value=330 gain (int): min=0 max=1023 step=1 default=0 value=20 7) Update exposure, check that it is well changed on display and that same value is returned: $> v4l2-ctl --set-ctrl=exposure=100 $> v4l2-ctl --get-ctrl=exposure exposure: 100 9) Update gain, check that it is well changed on display and that same value is returned: $> v4l2-ctl --set-ctrl=gain=10 $> v4l2-ctl --get-ctrl=gain gain: 10 10) Switch back to auto gain/exposure, verify that image is correct and values returned are correct: $> v4l2-ctl --set-ctrl=gain_automatic=1 $> v4l2-ctl --set-ctrl=auto_exposure=0 $> v4l2-ctl --all | grep -e exposure -e gain | grep "(int)" exposure (int): min=0 max=65535 step=1 default=0 value=330 flags=inactive, volatile gain (int): min=0 max=1023 step=1 default=0 value=22 flags=inactive, volatile Note the "0" for auto exposure. === = history = === version 3: - Change patch 5/5 by removing set_mode() orig_mode parameter as per jacopo' suggestion: https://www.spinics.net/lists/linux-media/msg139457.html version 2: - Fix patch 3/5 commit comment and rename binning function as per jacopo' suggestion: https://www.mail-archive.com/linux-media@vger.kernel.org/msg133272.html Hugues Fruchet (5): media: ov5640: fix exposure regression media: ov5640: fix auto gain & exposure when changing mode media: ov5640: fix wrong binning value in exposure calculation media: ov5640: fix auto controls values when switching to manual mode media: ov5640: fix restore of last mode set drivers/media/i2c/ov5640.c | 128 ++--- 1 file changed, 73 insertions(+), 55 deletions(-)
Re: [PATCH v4 2/2] media: ov5640: Fix timings setup code
On 09/14/2018 08:58 AM, Jacopo Mondi wrote:
From: Jacopo Mondi
As of:
commit 476dec012f4c ("media: ov5640: Add horizontal and vertical totals")
the timings parameters gets programmed separately from the static register
values array.
When changing capture mode, the vertical and horizontal totals gets inspected
by the set_mode_exposure_calc() functions, and only later programmed with the
new values. This means exposure, light banding filter and shutter gain are
calculated using the previous timings, and are thus not correct.
Fix this by programming timings right after the static register value table
has been sent to the sensor in the ov5640_load_regs() function.
Fixes: 476dec012f4c ("media: ov5640: Add horizontal and vertical totals")
Tested-by: Steve Longerbeam
on i.MX6q SabreSD with MIPI CSI-2 OV5640 module
Acked-by: Steve Longerbeam
Tested-by: Loic Poulain
on Dragonboard-410c with MIPI CSI-2 OV5640 module
Signed-off-by: Samuel Bobrowicz
Signed-off-by: Maxime Ripard
Signed-off-by: Jacopo Mondi
---
drivers/media/i2c/ov5640.c | 50 +++---
1 file changed, 21 insertions(+), 29 deletions(-)
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 7ade416..2b9e84f 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -908,6 +908,26 @@ static int ov5640_mod_reg(struct ov5640_dev *sensor, u16
reg,
}
/* download ov5640 settings to sensor through i2c */
+static int ov5640_set_timings(struct ov5640_dev *sensor,
+ const struct ov5640_mode_info *mode)
+{
+ int ret;
+
+ ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
+ if (ret < 0)
+ return ret;
+
+ ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, mode->vact);
+ if (ret < 0)
+ return ret;
+
+ ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->htot);
+ if (ret < 0)
+ return ret;
+
+ return ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, mode->vtot);
+}
+
static int ov5640_load_regs(struct ov5640_dev *sensor,
const struct ov5640_mode_info *mode)
{
@@ -935,7 +955,7 @@ static int ov5640_load_regs(struct ov5640_dev *sensor,
usleep_range(1000 * delay_ms, 1000 * delay_ms + 100);
}
- return ret;
+ return ov5640_set_timings(sensor, mode);
}
/* read exposure, in number of line periods */
@@ -1398,30 +1418,6 @@ static int ov5640_set_virtual_channel(struct ov5640_dev
*sensor)
return ov5640_write_reg(sensor, OV5640_REG_DEBUG_MODE, temp);
}
-static int ov5640_set_timings(struct ov5640_dev *sensor,
- const struct ov5640_mode_info *mode)
-{
- int ret;
-
- ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPHO, mode->hact);
- if (ret < 0)
- return ret;
-
- ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_DVPVO, mode->vact);
- if (ret < 0)
- return ret;
-
- ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_HTS, mode->htot);
- if (ret < 0)
- return ret;
-
- ret = ov5640_write_reg16(sensor, OV5640_REG_TIMING_VTS, mode->vtot);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
static const struct ov5640_mode_info *
ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
int width, int height, bool nearest)
@@ -1665,10 +1661,6 @@ static int ov5640_set_mode(struct ov5640_dev *sensor,
if (ret < 0)
return ret;
- ret = ov5640_set_timings(sensor, mode);
- if (ret < 0)
- return ret;
-
ret = ov5640_set_binning(sensor, dn_mode != SCALING);
if (ret < 0)
return ret;
--
2.7.4
Re: [PATCH v4 1/2] media: ov5640: Re-work MIPI startup sequence
On 09/14/2018 08:58 AM, Jacopo Mondi wrote:
From: Jacopo Mondi
Rework the MIPI interface startup sequence with the following changes:
- Remove MIPI bus initialization from the initial settings blob
- At set_power(1) time power up MIPI Tx/Rx and set data and clock lanes in
LP11 during 'sleep' and 'idle' with MIPI clock in non-continuous mode.
- At s_stream time enable/disable the MIPI interface output.
- Restore default settings at set_power(0) time.
Before this commit the sensor MIPI interface was initialized with settings
that require a start/stop sequence at power-up time in order to force lanes
into LP11 state, as they were initialized in LP00 when in 'sleep mode',
which is assumed to be the sensor manual definition for the D-PHY defined
stop mode.
The stream start/stop was performed by enabling disabling clock gating,
and had the side effect to change the lanes sleep mode configuration when
stream was stopped.
Clock gating/ungating:
- ret = ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
-on ? 0 : BIT(5));
- if (ret)
Set lanes in LP11 when in 'sleep mode':
- ret = ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
- on ? 0x00 : 0x70);
This commit fixes an issue reported by Jagan Teki on i.MX6 platforms that
prevents the host interface from powering up correctly:
https://lkml.org/lkml/2018/6/1/38
It also improves MIPI capture operations stability on my testing platform
where MIPI capture often failed and returned all-purple frames.
fixes: f22996db44e2 ("media: ov5640: add support of DVP parallel interface")
Tested-by: Steve Longerbeam
on i.MX6q SabreSD with MIPI CSI-2 OV5640 module
Acked-by: Steve Longerbeam
Tested-by: Loic Poulain
on Dragonboard-410c with MIPI CSI-2 OV5640 module
Reported-by: Jagan Teki
Signed-off-by: Jacopo Mondi
---
drivers/media/i2c/ov5640.c | 99 --
1 file changed, 79 insertions(+), 20 deletions(-)
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 071f4bc..7ade416 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -286,10 +286,10 @@ static const struct reg_value
ov5640_init_setting_30fps_VGA[] = {
{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x3000, 0x00, 0, 0},
{0x3002, 0x1c, 0, 0}, {0x3004, 0xff, 0, 0}, {0x3006, 0xc3, 0, 0},
- {0x300e, 0x45, 0, 0}, {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
+ {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
{0x501f, 0x00, 0, 0}, {0x4713, 0x03, 0, 0}, {0x4407, 0x04, 0, 0},
{0x440e, 0x00, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
- {0x4837, 0x0a, 0, 0}, {0x4800, 0x04, 0, 0}, {0x3824, 0x02, 0, 0},
+ {0x4837, 0x0a, 0, 0}, {0x3824, 0x02, 0, 0},
{0x5000, 0xa7, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x5180, 0xff, 0, 0},
{0x5181, 0xf2, 0, 0}, {0x5182, 0x00, 0, 0}, {0x5183, 0x14, 0, 0},
{0x5184, 0x25, 0, 0}, {0x5185, 0x24, 0, 0}, {0x5186, 0x09, 0, 0},
@@ -1102,12 +1102,25 @@ static int ov5640_set_stream_mipi(struct ov5640_dev
*sensor, bool on)
{
int ret;
- ret = ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
-on ? 0 : BIT(5));
- if (ret)
- return ret;
- ret = ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
- on ? 0x00 : 0x70);
+ /*
+* Enable/disable the MIPI interface
+*
+* 0x300e = on ? 0x45 : 0x40
+*
+* FIXME: the sensor manual (version 2.03) reports
+* [7:5] = 000 : 1 data lane mode
+* [7:5] = 001 : 2 data lanes mode
+* But this settings do not work, while the following ones
+* have been validated for 2 data lanes mode.
+*
+* [7:5] = 010 : 2 data lanes mode
+* [4] = 0 : Power up MIPI HS Tx
+* [3] = 0 : Power up MIPI LS Rx
+* [2] = 1/0: MIPI interface enable/disable
+* [1:0] = 01/00: FIXME: 'debug'
+*/
+ ret = ov5640_write_reg(sensor, OV5640_REG_IO_MIPI_CTRL00,
+ on ? 0x45 : 0x40);
if (ret)
return ret;
@@ -1786,23 +1799,69 @@ static int ov5640_set_power(struct ov5640_dev *sensor,
bool on)
if (ret)
goto power_off;
+ /* We're done here for DVP bus, while CSI-2 needs setup. */
+ if (sensor->ep.bus_type != V4L2_MBUS_CSI2)
+ return 0;
+
+ /*
+* Power up MIPI HS Tx and LS Rx; 2 data lanes mode
+*
+* 0x300e = 0x40
+* [7:5] = 010 : 2 data lanes mode (see FIXME note in
+*"ov5640_set_stream_mipi()")
+* [4] = 0 : Power up MIPI HS Tx
+*
Re: [PATCH v2 00/23] V4L2 fwnode rework; support for default configuration
Hi Sakari,
On 08/27/2018 02:29 AM, Sakari Ailus wrote:
Hello everyone,
I've long thought the V4L2 fwnode framework requires some work (it's buggy
and it does not adequately serve common needs). This set should address in
particular these matters:
- Most devices support a particular media bus type but the V4L2 fwnode
framework was not able to use such information, but instead tried to
guess the bus type with varying levels of success while drivers
generally ignored the results. This patchset makes that possible ---
setting a bus type enables parsing configuration for only that bus.
Failing that check results in returning -ENXIO to be returned.
- Support specifying default configuration. If the endpoint has no
configuration, the defaults set by the driver (as documented in DT
bindings) will prevail. Any available configuration will still be read
from the endpoint as one could expect. A common use case for this is
e.g. the number of CSI-2 lanes. Few devices support lane mapping, and
default 1:1 mapping is provided in absence of a valid default or
configuration read OF.
- Debugging information is greatly improved.
- Recognition of the differences between CSI-2 D-PHY and C-PHY. All
currently supported hardware (or at least drivers) is D-PHY only, so
this change is still easy.
The smiapp driver is converted to use the new functionality. This patchset
does not address remaining issues such as supporting setting defaults for
e.g. bridge drivers with multiple ports, but with Steve Longerbeam's
patchset we're much closer to that goal. I've rebased this set on top of
Steve's. Albeit the two deal with the same files, there were only a few
trivial conflicts.
Note that I've only tested parsing endpoints for the CSI-2 bus (no
parallel IF hardware). Jacopo has tested an earlier version of the set
with a few changes to the parallel bus handling compared to this one.
Comments are welcome.
I got around to testing this. The following diff needs to be added
to initialize bus_type before calling v4l2_fwnode_endpoint_parse()
in imx-media driver, this should probably be squashed with
"v4l: fwnode: Initialise the V4L2 fwnode endpoints to zero":
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 539159d..ac9d718 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1050,7 +1050,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sink_fmt)
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
- struct v4l2_fwnode_endpoint upstream_ep = {};
+ struct v4l2_fwnode_endpoint upstream_ep = { .bus_type = 0 };
bool is_csi2;
int ret;
@@ -1164,7 +1164,7 @@ static int csi_enum_mbus_code(struct v4l2_subdev *sd,
struct v4l2_subdev_mbus_code_enum *code)
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
- struct v4l2_fwnode_endpoint upstream_ep;
+ struct v4l2_fwnode_endpoint upstream_ep = { .bus_type = 0 };
const struct imx_media_pixfmt *incc;
struct v4l2_mbus_framefmt *infmt;
int ret = 0;
@@ -1403,7 +1403,7 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
struct imx_media_video_dev *vdev = priv->vdev;
- struct v4l2_fwnode_endpoint upstream_ep;
+ struct v4l2_fwnode_endpoint upstream_ep = { .bus_type = 0 };
const struct imx_media_pixfmt *cc;
struct v4l2_pix_format vdev_fmt;
struct v4l2_mbus_framefmt *fmt;
@@ -1542,7 +1542,7 @@ static int csi_set_selection(struct v4l2_subdev *sd,
struct v4l2_subdev_selection *sel)
{
struct csi_priv *priv = v4l2_get_subdevdata(sd);
- struct v4l2_fwnode_endpoint upstream_ep;
+ struct v4l2_fwnode_endpoint upstream_ep = { .bus_type = 0 };
struct v4l2_mbus_framefmt *infmt;
struct v4l2_rect *crop, *compose;
int pad, ret;
After making that change, capture from CSI-2 OV5640 and parallel
OV5642 on the imx6q Sabrelite is working fine. Feel free to add my
Tested-by on that platform.
I've pushed the patches (including Steve's) here:
https://git.linuxtv.org/sailus/media_tree.git/log/?h=v4l2-fwnode-next>
since v1:
- Rebase it all on current media tree master --- there was a conflict in
drivers/media/platform/qcom/camss/camss.c in Steve's patch "media:
platform: Switch to v4l2_async_notifier_add_subdev"; I hope the
resolution was fine.
I checked your resolution to camss.c and it was the same resolution I
made as well.
Thanks,
Steve
- Default to Bt.656 bus in guessing the bus type if no properties
suggesting otherwise are set. In v1 and error was returned, which would
have been troublesome for the existing drivers.
- Set the bus_type field to zero (i.e. guess) for existing callers of
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
On 08/14/2018 10:38 AM, jacopo mondi wrote: Hi Steve, On Tue, Aug 14, 2018 at 09:51:04AM -0700, Steve Longerbeam wrote: Hi Jacopo, On 08/14/2018 08:35 AM, jacopo mondi wrote: Hi Steve, sorry for resurecting this. I'm sorry I'm not sur I'm following. Does this mean that with that bug you are referring to up here fixed by my last patch you have capture working? No, capture still not working for me on SabreSD, even after fixing the bug in 476dec0 "media: ov5640: Add horizontal and vertical totals", by either using your patchset, or by running version 476dec0 of ov5640.c with the call to ov5640_set_timings() moved to the correct places as described below. I've been reported a bug on exposure handling that makes the first captured frames all black. Both me and Hugues have tried to fix the issue (him with a more complete series, but that's another topic). See [1] and [2] It might be possible that you're getting blank frames with this series applied? I never seen them as I'm skipping the first frames when capturing, but I've now tested and without the exposure fixes (either [1] or [2]) I actually have blank frames. If that's the case for you too (which I hope so much) would you be available to test again this series with exposure fixes on top? On my platform that actually makes all frames correct. Thanks j [1] [PATCH 0/2] media: ov5640: Fix set_timings and auto-exposure [2] [PATCH v2 0/5] Fix OV5640 exposure & gain It's not clear to me which patch sets you would like me to test. Just [1] and [2], or [1], [2], and "media: i2c: ov5640: Re-work MIPI startup sequence"? I have tested on my board the following: v4.18-rc2 + MIPI Fix + Timings + Hugues' exposure fix Without Hugues' patches I get blank frames (the first ones at least) Without MIPI startup reowkr and timings I get the LP-11 error on the CSI-2 bus. As Hugues' series has to be rebased on mine, I have prepared a branch here for you if you feel like testing it: git://jmondi.org/linux ov5640/timings_exposure Hi Jacopo, that branch works on SabreSD! Feel free to add Tested-by: Steve Longerbeam on i.MX6q SabreSD with MIPI CSI-2 OV5640 module to whichever ov5640 patches are appropriate. Steve
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
Hi Jacopo, On 08/14/2018 08:35 AM, jacopo mondi wrote: Hi Steve, sorry for resurecting this. I'm sorry I'm not sur I'm following. Does this mean that with that bug you are referring to up here fixed by my last patch you have capture working? No, capture still not working for me on SabreSD, even after fixing the bug in 476dec0 "media: ov5640: Add horizontal and vertical totals", by either using your patchset, or by running version 476dec0 of ov5640.c with the call to ov5640_set_timings() moved to the correct places as described below. I've been reported a bug on exposure handling that makes the first captured frames all black. Both me and Hugues have tried to fix the issue (him with a more complete series, but that's another topic). See [1] and [2] It might be possible that you're getting blank frames with this series applied? I never seen them as I'm skipping the first frames when capturing, but I've now tested and without the exposure fixes (either [1] or [2]) I actually have blank frames. If that's the case for you too (which I hope so much) would you be available to test again this series with exposure fixes on top? On my platform that actually makes all frames correct. Thanks j [1] [PATCH 0/2] media: ov5640: Fix set_timings and auto-exposure [2] [PATCH v2 0/5] Fix OV5640 exposure & gain It's not clear to me which patch sets you would like me to test. Just [1] and [2], or [1], [2], and "media: i2c: ov5640: Re-work MIPI startup sequence"? Steve
[PATCH v3 00/14] imx-media: Fixes for interlaced capture
A set of patches that fixes some bugs with capturing from an interlaced source, and incompatibilites between IDMAC interlace interweaving and 4:2:0 data write reduction. History: v3: - add support for/fix interweaved scan with YUV planar output. - fix bug in 4:2:0 U/V offset macros. - add patch that generalizes behavior of field swap in ipu_csi_init_interface(). - add support for interweaved scan with field order swap. Suggested by Philipp Zabel. - in v2, inteweave scan was determined using field types of CSI (and PRPENCVF) at the sink and source pads. In v3, this has been moved one hop downstream: interweave is now determined using field type at source pad, and field type selected at capture interface. Suggested by Philipp. - make sure to double CSI crop target height when input field type in alternate. - more updates to media driver doc to reflect above. v2: - update media driver doc. - enable idmac interweave only if input field is sequential/alternate, and output field is 'interlaced*'. - move field try logic out of *try_fmt and into separate function. - fix bug with resetting crop/compose rectangles. - add a patch that fixes a field order bug in VDIC indirect mode. - remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro Suggested-by: Nicolas Dufresne . - add macro V4L2_FIELD_IS_INTERLACED(). Philipp Zabel (1): gpu: ipu-v3: Allow negative offsets for interlaced scanning Steve Longerbeam (13): media: videodev2.h: Add more field helper macros gpu: ipu-csi: Check for field type alternate gpu: ipu-csi: Swap fields according to input/output field types gpu: ipu-v3: Fix U/V offset macros for planar 4:2:0 gpu: ipu-v3: Add planar support to interlaced scan media: imx: Fix field negotiation media: imx-csi: Double crop height for alternate fields at sink media: imx: interweave and odd-chroma-row skip are incompatible media: imx-csi: Allow skipping odd chroma rows for YVU420 media: imx: vdic: rely on VDIC for correct field order media: imx-csi: Move crop/compose reset after filling default mbus fields media: imx: Allow interweave with top/bottom lines swapped media: imx.rst: Update doc to reflect fixes to interlaced capture Documentation/media/v4l-drivers/imx.rst | 93 ++- drivers/gpu/ipu-v3/ipu-cpmem.c| 45 ++- drivers/gpu/ipu-v3/ipu-csi.c | 136 ++--- drivers/staging/media/imx/imx-ic-prpencvf.c | 48 ++-- drivers/staging/media/imx/imx-media-capture.c | 14 +++ drivers/staging/media/imx/imx-media-csi.c | 166 ++ drivers/staging/media/imx/imx-media-vdic.c| 12 +- include/uapi/linux/videodev2.h| 7 ++ include/video/imx-ipu-v3.h| 6 +- 9 files changed, 377 insertions(+), 150 deletions(-) -- 2.7.4
[PATCH RESEND v6 15/17] media: platform: Switch to v4l2_async_notifier_add_subdev
Switch all media platform drivers to call v4l2_async_notifier_add_subdev()
to add asd's to a notifier, in place of referencing the notifier->subdevs[]
array. These drivers also must now call v4l2_async_notifier_init() before
adding asd's to their notifiers.
There may still be cases where a platform driver maintains a list of
asd's that is a duplicate of the notifier asd_list, in which case its
possible the platform driver list can be removed, and can reference the
notifier asd_list instead. One example of where a duplicate list has
been removed in this patch is xilinx-vipp.c. If there are such cases
remaining, those drivers should be optimized to remove the duplicate
platform driver asd lists.
None of the changes to the platform drivers in this patch have been
tested. Verify that the async subdevices needed by the platform are
bound at load time, and that the driver unloads and reloads correctly
with no memory leaking of asd objects.
Suggested-by: Sakari Ailus
Signed-off-by: Steve Longerbeam
---
Changes since v5:
- remove reference to notifier.num_subdevs, and call
v4l2_async_notifier_init(). Suggested by Sakari Ailus.
- removed "OF" qualifier when passing fwnode pointers to printk in
xilinx-vipp.c. Reported by Dan Carpenter.
- fixed double node put in vpif_capture.c. Reported by Sakari.
---
drivers/media/pci/intel/ipu3/ipu3-cio2.c | 2 +-
drivers/media/platform/am437x/am437x-vpfe.c| 82 ++--
drivers/media/platform/atmel/atmel-isc.c | 15 ++-
drivers/media/platform/atmel/atmel-isi.c | 17 +--
drivers/media/platform/cadence/cdns-csi2rx.c | 28 ++--
drivers/media/platform/davinci/vpif_capture.c | 71 +-
drivers/media/platform/davinci/vpif_display.c | 25 ++--
drivers/media/platform/exynos4-is/media-dev.c | 32 +++--
drivers/media/platform/exynos4-is/media-dev.h | 1 -
drivers/media/platform/pxa_camera.c| 25 ++--
drivers/media/platform/qcom/camss-8x16/camss.c | 86 ++--
drivers/media/platform/qcom/camss-8x16/camss.h | 2 +-
drivers/media/platform/rcar-vin/rcar-core.c| 2 +-
drivers/media/platform/rcar-vin/rcar-csi2.c| 22 ++--
drivers/media/platform/rcar_drif.c | 18 ++-
drivers/media/platform/renesas-ceu.c | 53
drivers/media/platform/soc_camera/soc_camera.c | 35 +++--
drivers/media/platform/stm32/stm32-dcmi.c | 24 ++--
drivers/media/platform/ti-vpe/cal.c| 48 +--
drivers/media/platform/xilinx/xilinx-vipp.c| 173 -
drivers/media/platform/xilinx/xilinx-vipp.h| 4 -
21 files changed, 423 insertions(+), 342 deletions(-)
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
index 4a5f7c3..ca3ad11 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
@@ -1508,7 +1508,7 @@ static int cio2_notifier_init(struct cio2_device *cio2)
if (ret < 0)
return ret;
- if (!cio2->notifier.num_subdevs)
+ if (list_empty(>notifier.asd_list))
return -ENODEV; /* no endpoint */
cio2->notifier.ops = _async_ops;
diff --git a/drivers/media/platform/am437x/am437x-vpfe.c
b/drivers/media/platform/am437x/am437x-vpfe.c
index b05738a..b19e0ab 100644
--- a/drivers/media/platform/am437x/am437x-vpfe.c
+++ b/drivers/media/platform/am437x/am437x-vpfe.c
@@ -2423,30 +2423,32 @@ static const struct v4l2_async_notifier_operations
vpfe_async_ops = {
};
static struct vpfe_config *
-vpfe_get_pdata(struct platform_device *pdev)
+vpfe_get_pdata(struct vpfe_device *vpfe)
{
struct device_node *endpoint = NULL;
struct v4l2_fwnode_endpoint bus_cfg;
+ struct device *dev = vpfe->pdev;
struct vpfe_subdev_info *sdinfo;
struct vpfe_config *pdata;
unsigned int flags;
unsigned int i;
int err;
- dev_dbg(>dev, "vpfe_get_pdata\n");
+ dev_dbg(dev, "vpfe_get_pdata\n");
- if (!IS_ENABLED(CONFIG_OF) || !pdev->dev.of_node)
- return pdev->dev.platform_data;
+ v4l2_async_notifier_init(>notifier);
- pdata = devm_kzalloc(>dev, sizeof(*pdata), GFP_KERNEL);
+ if (!IS_ENABLED(CONFIG_OF) || !dev->of_node)
+ return dev->platform_data;
+
+ pdata = devm_kzalloc(dev, sizeof(*pdata), GFP_KERNEL);
if (!pdata)
return NULL;
for (i = 0; ; i++) {
struct device_node *rem;
- endpoint = of_graph_get_next_endpoint(pdev->dev.of_node,
- endpoint);
+ endpoint = of_graph_get_next_endpoint(dev->of_node, endpoint);
if (!endpoint)
break;
@@ -2473,16 +2475,16 @@ vpfe_get_pdata(struct platform_device *pdev)
err = v4l2_fwnode_endpoint_parse(of_fwn
Re: [PATCH 16/16] media: imx: add mem2mem device
On 07/23/2018 12:31 AM, Philipp Zabel wrote:
ipu_image_convert_adjust tries to adjust both input and output image at
the same time, here we just have the format of either input or output
image. Do you suggest to split this function into an input and an output
version?
See b4362162c0 ("media: imx: mem2mem: Use ipu_image_convert_adjust
in try format")
Alright, this looks fine to me. I was worried about inter-format
limitations, but the only one seems to be the output size lower bound to
1/4 of the input size. Should S_FMT(OUT) also update the capture format
if adjustments were made to keep a consistent state?
That's a good question, I don't know if the mem2mem API allows for
that, but if it does we should do that for consistent state as you said.
In b4362162c0, the current capture format is used to adjust output
format during S_FMT(OUT) but any capture format changes are
dropped, and vice-versa.
Steve
Re: [PATCH v2 00/16] i.MX media mem2mem scaler
On 07/22/2018 11:30 AM, Steve Longerbeam wrote:
Hi Philipp,
On 07/19/2018 08:30 AM, Philipp Zabel wrote:
Hi,
this is the second version of the i.MX mem2mem scaler series.
Patches 8 and 16 have been modified.
Changes since v1:
- Fix inverted allow_overshoot logic
- Correctly switch horizontal / vertical tile alignment when
determining seam positions with the 90° rotator active.
Yes, this fixes the specific rotation test that was broken
(720x480, UYVY --> 1280x768, UYVY, rotate 90).
But running more tests on this v2 reveals more issues. I chose a
somewhat random upscaling-only example as a first try:
640x480, YV12 --> full HD 2560x1600, YV12 (no rotation or flip).
This produces division by zero backtraces and the conversion hangs:
The hang is apparently because the conversion is re-attempted over and
over again, with an endless WARN() from
drivers/media/common/videobuf2/videobuf2-core.c:900.
I fixed the hang with an additional patch:
50026cbe08 ("media: imx: mem2mem: Remove buffers on device_run failures")
With this the conversion completes, but the below div-by-zero errors
persist, and the resultant image is blank.
Steve
[ 131.079978] Division by zero in kernel.
[ 131.083853] CPU: 0 PID: 683 Comm: mx6-m2m Tainted: G W
4.18.0-rc2-13448-g678218d #7
[ 131.092830] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 131.099372] Backtrace:
[ 131.101858] [] (dump_backtrace) from []
(show_stack+0x18/0x1c)
[ 131.109450] r7: r6:600f0013 r5: r4:c107db3c
[ 131.115135] [] (show_stack) from []
(dump_stack+0xb4/0xe8)
[ 131.122380] [] (dump_stack) from []
(__div0+0x18/0x20)
[ 131.129274] r9:ec37d800 r8:0003 r7: r6:
r5: r4:ec37dae8
[ 131.137036] [] (__div0) from [] (Ldiv0+0x8/0x10)
[ 131.143425] [] (ipu_image_convert_prepare) from
[] (mem2mem_start_streaming+0xe0/0x1c0)
[ 131.153186] r10:c0b9f640 r9:c071958c r8:0280 r7:01e0
r6:32315659 r5:c1008908
[ 131.161030] r4:ecf9c800
[ 131.163588] [] (mem2mem_start_streaming) from
[] (vb2_start_streaming+0x64/0x160)
[ 131.172826] r8:c1008908 r7:0001 r6:ed01b808 r5:ed01b934
r4:ed01b810
[ 131.179547] [] (vb2_start_streaming) from []
(vb2_core_streamon+0x10c/0x164)
[ 131.188351] r9:c071958c r8:c1008908 r7:0001 r6:ec1038f8
r5: r4:ed01b808
[ 131.196114] [] (vb2_core_streamon) from []
(vb2_streamon+0x34/0x58)
[ 131.204133] r5:40045612 r4:ed01b800
[ 131.207733] [] (vb2_streamon) from []
(v4l2_m2m_streamon+0x24/0x3c)
[ 131.215758] [] (v4l2_m2m_streamon) from []
(v4l2_m2m_ioctl_streamon+0x18/0x1c)
[ 131.224732] r5:40045612 r4:c072f354
[ 131.228330] [] (v4l2_m2m_ioctl_streamon) from
[] (v4l_streamon+0x24/0x28)
[ 131.236878] [] (v4l_streamon) from []
(__video_do_ioctl+0x284/0x4f8)
[ 131.244984] r5:40045612 r4:ecc50800
[ 131.248583] [] (__video_do_ioctl) from []
(video_usercopy+0x260/0x55c)
[ 131.256866] r10:0004 r9: r8:c1008908 r7:ed747dfc
r6: r5:0004
[ 131.264709] r4:40045612
[ 131.267265] [] (video_usercopy) from []
(video_ioctl2+0x14/0x1c)
[ 131.275026] r10:0036 r9:0003 r8:ed480068 r7:c0254840
r6:ecc76000 r5:bea6ab38
[ 131.282869] r4:c071fcd8
[ 131.285424] [] (video_ioctl2) from []
(v4l2_ioctl+0x44/0x5c)
[ 131.292845] [] (v4l2_ioctl) from []
(do_vfs_ioctl+0xa8/0xa4c)
[ 131.300343] r5:bea6ab38 r4:c1008908
[ 131.303940] [] (do_vfs_ioctl) from []
(ksys_ioctl+0x3c/0x60)
[ 131.311355] r10:0036 r9:ed746000 r8:bea6ab38 r7:40045612
r6:0003 r5:ecc76000
[ 131.319198] r4:ecc76000
[ 131.321752] [] (ksys_ioctl) from []
(sys_ioctl+0x10/0x14)
[ 131.328907] r9:ed746000 r8:c01011e4 r7:0036 r6:00010960
r5: r4:00012620
[ 131.336672] [] (sys_ioctl) from []
(ret_fast_syscall+0x0/0x28)
[ 131.344256] Exception stack(0xed747fa8 to 0xed747ff0)
[ 131.349327] 7fa0: 00012620 0003
40045612 bea6ab38 0003
[ 131.357524] 7fc0: 00012620 00010960 0036
45d8 bea6abac
[ 131.365717] 7fe0: 0002312c bea6aaa4 00012308 45e58d5c
To aid in debugging this I created branch 'imx-mem2mem.stevel' in my
mediatree fork on github. I moved the mem2mem driver to the beginning
and added a few patches:
d317a7771c ("gpu: ipu-cpmem: add WARN_ON_ONCE() for unaligned dma
buffers")
b4362162c0 ("media: imx: mem2mem: Use ipu_image_convert_adjust in try
format")
4758be0cf8 ("gpu: ipu-v3: image-convert: Fix width/height alignment")
d069163c7f ("gpu: ipu-v3: image-convert: Fix input bytesperline clamp
in adjust")
(feel free to squash some of those if you agree with them for v3).
By moving the mem2mem driver before the seam avoidance patches, and
making
it independent of the image converter implementation, the driver can
be tested with
and without the seam avoidance changes.
If you run a git rebase and build/run the kernel when stopped at
b4362162c0
Re: [PATCH v2 00/16] i.MX media mem2mem scaler
Hi Philipp,
On 07/19/2018 08:30 AM, Philipp Zabel wrote:
Hi,
this is the second version of the i.MX mem2mem scaler series.
Patches 8 and 16 have been modified.
Changes since v1:
- Fix inverted allow_overshoot logic
- Correctly switch horizontal / vertical tile alignment when
determining seam positions with the 90° rotator active.
Yes, this fixes the specific rotation test that was broken
(720x480, UYVY --> 1280x768, UYVY, rotate 90).
But running more tests on this v2 reveals more issues. I chose a
somewhat random upscaling-only example as a first try:
640x480, YV12 --> full HD 2560x1600, YV12 (no rotation or flip).
This produces division by zero backtraces and the conversion hangs:
[ 131.079978] Division by zero in kernel.
[ 131.083853] CPU: 0 PID: 683 Comm: mx6-m2m Tainted: G W
4.18.0-rc2-13448-g678218d #7
[ 131.092830] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 131.099372] Backtrace:
[ 131.101858] [] (dump_backtrace) from []
(show_stack+0x18/0x1c)
[ 131.109450] r7: r6:600f0013 r5: r4:c107db3c
[ 131.115135] [] (show_stack) from []
(dump_stack+0xb4/0xe8)
[ 131.122380] [] (dump_stack) from []
(__div0+0x18/0x20)
[ 131.129274] r9:ec37d800 r8:0003 r7: r6:
r5: r4:ec37dae8
[ 131.137036] [] (__div0) from [] (Ldiv0+0x8/0x10)
[ 131.143425] [] (ipu_image_convert_prepare) from
[] (mem2mem_start_streaming+0xe0/0x1c0)
[ 131.153186] r10:c0b9f640 r9:c071958c r8:0280 r7:01e0
r6:32315659 r5:c1008908
[ 131.161030] r4:ecf9c800
[ 131.163588] [] (mem2mem_start_streaming) from []
(vb2_start_streaming+0x64/0x160)
[ 131.172826] r8:c1008908 r7:0001 r6:ed01b808 r5:ed01b934 r4:ed01b810
[ 131.179547] [] (vb2_start_streaming) from []
(vb2_core_streamon+0x10c/0x164)
[ 131.188351] r9:c071958c r8:c1008908 r7:0001 r6:ec1038f8
r5: r4:ed01b808
[ 131.196114] [] (vb2_core_streamon) from []
(vb2_streamon+0x34/0x58)
[ 131.204133] r5:40045612 r4:ed01b800
[ 131.207733] [] (vb2_streamon) from []
(v4l2_m2m_streamon+0x24/0x3c)
[ 131.215758] [] (v4l2_m2m_streamon) from []
(v4l2_m2m_ioctl_streamon+0x18/0x1c)
[ 131.224732] r5:40045612 r4:c072f354
[ 131.228330] [] (v4l2_m2m_ioctl_streamon) from []
(v4l_streamon+0x24/0x28)
[ 131.236878] [] (v4l_streamon) from []
(__video_do_ioctl+0x284/0x4f8)
[ 131.244984] r5:40045612 r4:ecc50800
[ 131.248583] [] (__video_do_ioctl) from []
(video_usercopy+0x260/0x55c)
[ 131.256866] r10:0004 r9: r8:c1008908 r7:ed747dfc
r6: r5:0004
[ 131.264709] r4:40045612
[ 131.267265] [] (video_usercopy) from []
(video_ioctl2+0x14/0x1c)
[ 131.275026] r10:0036 r9:0003 r8:ed480068 r7:c0254840
r6:ecc76000 r5:bea6ab38
[ 131.282869] r4:c071fcd8
[ 131.285424] [] (video_ioctl2) from []
(v4l2_ioctl+0x44/0x5c)
[ 131.292845] [] (v4l2_ioctl) from []
(do_vfs_ioctl+0xa8/0xa4c)
[ 131.300343] r5:bea6ab38 r4:c1008908
[ 131.303940] [] (do_vfs_ioctl) from []
(ksys_ioctl+0x3c/0x60)
[ 131.311355] r10:0036 r9:ed746000 r8:bea6ab38 r7:40045612
r6:0003 r5:ecc76000
[ 131.319198] r4:ecc76000
[ 131.321752] [] (ksys_ioctl) from []
(sys_ioctl+0x10/0x14)
[ 131.328907] r9:ed746000 r8:c01011e4 r7:0036 r6:00010960
r5: r4:00012620
[ 131.336672] [] (sys_ioctl) from []
(ret_fast_syscall+0x0/0x28)
[ 131.344256] Exception stack(0xed747fa8 to 0xed747ff0)
[ 131.349327] 7fa0: 00012620 0003
40045612 bea6ab38 0003
[ 131.357524] 7fc0: 00012620 00010960 0036
45d8 bea6abac
[ 131.365717] 7fe0: 0002312c bea6aaa4 00012308 45e58d5c
To aid in debugging this I created branch 'imx-mem2mem.stevel' in my
mediatree fork on github. I moved the mem2mem driver to the beginning
and added a few patches:
d317a7771c ("gpu: ipu-cpmem: add WARN_ON_ONCE() for unaligned dma buffers")
b4362162c0 ("media: imx: mem2mem: Use ipu_image_convert_adjust in try
format")
4758be0cf8 ("gpu: ipu-v3: image-convert: Fix width/height alignment")
d069163c7f ("gpu: ipu-v3: image-convert: Fix input bytesperline clamp in
adjust")
(feel free to squash some of those if you agree with them for v3).
By moving the mem2mem driver before the seam avoidance patches, and making
it independent of the image converter implementation, the driver can be
tested with
and without the seam avoidance changes.
If you run a git rebase and build/run the kernel when stopped at
b4362162c0 (e.g.
without the seam avoidance patches), you will find that the above
640x480 -->
2560x1600 conversion succeeds, albeit with the expected visible seams at the
tile boundaries.
Also, I'm trying to parse the functions find_best_seam() and
find_seams(). Can
you provide some more background on the behavior of those functions?
Steve
- Fix SPDX-License-Identifier and remove superfluous license
text.
- Fix uninitialized walign in try_fmt
Previous cover letter:
we have image conversion code for scaling
Re: [PATCH 16/16] media: imx: add mem2mem device
On 07/16/2018 07:12 AM, Philipp Zabel wrote:
Hi Steve,
On Thu, 2018-07-05 at 15:09 -0700, Steve Longerbeam wrote:
[...]
[...]
+ halign = 0;
+ break;
+ }
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) {
+ /*
+* The IC burst reads 8 pixels at a time. Reading beyond the
+* end of the line is usually acceptable. Those pixels are
+* ignored, unless the IC has to write the scaled line in
+* reverse.
+*/
+ if (!ipu_rot_mode_is_irt(ctx->rot_mode) &&
+ ctx->rot_mode && IPU_ROT_BIT_HFLIP)
+ walign = 3;
This looks wrong. Do you mean:
if (ipu_rot_mode_is_irt(ctx->rot_mode) || (ctx->rot_mode & IPU_ROT_BIT_HFLIP))
walign = 3;
else
walign = 1;
The input DMA burst width alignment is only necessary if the lines are
scanned from right to left (that is, if HF is enabled) in the scaling
step.
Ok, thanks for the explanation, that makes sense.
If the rotator is used, the flipping is done in the rotation step
instead,
Ah, I missed or forgot about that detail in the ref manual,
I reviewed it again and you are right...
so the alignment restriction would be on the width of the
intermediate tile (and thus on the output height). This is already
covered by the rotator 8x8 pixel block alignment.
so this makes sense too.
That is, require 8 byte width alignment for IRT or if HFLIP is enabled.
No, I specifically meant (!IRT && HFLIP).
Right, but there is still a typo:
if (!ipu_rot_mode_is_irt(ctx->rot_mode) && ctx->rot_mode && IPU_ROT_BIT_HFLIP)
should be:
if (!ipu_rot_mode_is_irt(ctx->rot_mode) && (ctx->rot_mode & IPU_ROT_BIT_HFLIP))
The rotator itself doesn't cause any input alignment restrictions, we
just have to make sure that the intermediate tiles after scaling are 8x8
aligned.
Also, why not simply call ipu_image_convert_adjust() in
mem2mem_try_fmt()? If there is something missing in the former
function, then it should be added there, instead of adding the
missing checks in mem2mem_try_fmt().
ipu_image_convert_adjust tries to adjust both input and output image at
the same time, here we just have the format of either input or output
image. Do you suggest to split this function into an input and an output
version?
See b4362162c0 ("media: imx: mem2mem: Use ipu_image_convert_adjust
in try format")
in my mediatree fork at g...@github.com:slongerbeam/mediatree.git.
Let's discuss this further in the v2 patches.
Steve
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
On 07/16/2018 01:29 AM, jacopo mondi wrote: Hi Steve, thanks for keep testing it On Sat, Jul 14, 2018 at 01:02:32PM -0700, Steve Longerbeam wrote: On 07/14/2018 12:41 PM, Steve Longerbeam wrote: Hi Jacopo, On 07/14/2018 11:57 AM, Steve Longerbeam wrote: Hi Jacopo, Pardon the late reply, see below. On 07/11/2018 12:21 AM, jacopo mondi wrote: Hi Steve, On Tue, Jul 10, 2018 at 02:10:54PM -0700, Steve Longerbeam wrote: Hi Jacopo, Sorry to report my testing on SabreSD has same result as last time. This series fixes the LP-11 timeout at stream on but captured images are still blank. I tried the 640x480 mode with UYVY2X8. Here is the pad config: This saddens me :( I'm capturing with the same format and sizes... this shouldn't be the issue Could you confirm this matches what you have in your tree? 5dc2c80 media: ov5640: Fix timings setup code b35e757 media: i2c: ov5640: Re-work MIPI startup sequence 3c4a737 media: ov5640: fix frame interval enumeration 41cb1c7 media: ov5640: adjust xclk_max c3f3ba3 media: ov5640: add support of module orientation ce85705 media: ov5640: add HFLIP/VFLIP controls support 8663341 media: ov5640: Program the visible resolution 476dec0 media: ov5640: Add horizontal and vertical totals dba13a0 media: ov5640: Change horizontal and vertical resolutions name 8f57c2f media: ov5640: Init properly the SCLK dividers Yes, I have that commit sequence. FWIW, I can verify what Jagan Teki reported earlier, that the driver still works on the SabreSD platform at: dba13a0 media: ov5640: Change horizontal and vertical resolutions name and is broken at: 476dec0 media: ov5640: Add horizontal and vertical totals with LP-11 timeout at the mipi csi-2 receiver: [ 80.763189] imx6-mipi-csi2: LP-11 timeout, phy_state = 0x0230 [ 80.769599] ipu1_csi1: pipeline start failed with -110 And I discovered the bug in 476dec0 "media: ov5640: Add horizontal and vertical totals". The call to ov5640_set_timings() needs to be moved before the calls to ov5640_get_vts() and ov5640_get_hts(). But I see you have discovered that as well, and fixed in the second patch in your series. I'm sorry I'm not sur I'm following. Does this mean that with that bug you are referring to up here fixed by my last patch you have capture working? No, capture still not working for me on SabreSD, even after fixing the bug in 476dec0 "media: ov5640: Add horizontal and vertical totals", by either using your patchset, or by running version 476dec0 of ov5640.c with the call to ov5640_set_timings() moved to the correct places as described below. Steve But strangely, if I revert to 476dec0, and then move the call to ov5640_set_timings() to just after ov5640_load_regs() in ov5640_set_mode_exposure_calc() and ov5640_set_mode_direct(), the LP-11 timeouts are still present. So I can confirm this strangeness which you already pointed out below [1]. The version I'm sending here re-introduces some of the timings parameters in the initial configuration blob (not in the single mode ones), which apparently has to be at least initially programmed to allow the driver to later program them singularly in the 'set_timings()' function. Unfortunately I do not have a real rationale behind this which explains why it has to be done this way :( [1] here :) Steve
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
On 07/14/2018 12:41 PM, Steve Longerbeam wrote: Hi Jacopo, On 07/14/2018 11:57 AM, Steve Longerbeam wrote: Hi Jacopo, Pardon the late reply, see below. On 07/11/2018 12:21 AM, jacopo mondi wrote: Hi Steve, On Tue, Jul 10, 2018 at 02:10:54PM -0700, Steve Longerbeam wrote: Hi Jacopo, Sorry to report my testing on SabreSD has same result as last time. This series fixes the LP-11 timeout at stream on but captured images are still blank. I tried the 640x480 mode with UYVY2X8. Here is the pad config: This saddens me :( I'm capturing with the same format and sizes... this shouldn't be the issue Could you confirm this matches what you have in your tree? 5dc2c80 media: ov5640: Fix timings setup code b35e757 media: i2c: ov5640: Re-work MIPI startup sequence 3c4a737 media: ov5640: fix frame interval enumeration 41cb1c7 media: ov5640: adjust xclk_max c3f3ba3 media: ov5640: add support of module orientation ce85705 media: ov5640: add HFLIP/VFLIP controls support 8663341 media: ov5640: Program the visible resolution 476dec0 media: ov5640: Add horizontal and vertical totals dba13a0 media: ov5640: Change horizontal and vertical resolutions name 8f57c2f media: ov5640: Init properly the SCLK dividers Yes, I have that commit sequence. FWIW, I can verify what Jagan Teki reported earlier, that the driver still works on the SabreSD platform at: dba13a0 media: ov5640: Change horizontal and vertical resolutions name and is broken at: 476dec0 media: ov5640: Add horizontal and vertical totals with LP-11 timeout at the mipi csi-2 receiver: [ 80.763189] imx6-mipi-csi2: LP-11 timeout, phy_state = 0x0230 [ 80.769599] ipu1_csi1: pipeline start failed with -110 And I discovered the bug in 476dec0 "media: ov5640: Add horizontal and vertical totals". The call to ov5640_set_timings() needs to be moved before the calls to ov5640_get_vts() and ov5640_get_hts(). But I see you have discovered that as well, and fixed in the second patch in your series. But strangely, if I revert to 476dec0, and then move the call to ov5640_set_timings() to just after ov5640_load_regs() in ov5640_set_mode_exposure_calc() and ov5640_set_mode_direct(), the LP-11 timeouts are still present. So I can confirm this strangeness which you already pointed out below [1]. The version I'm sending here re-introduces some of the timings parameters in the initial configuration blob (not in the single mode ones), which apparently has to be at least initially programmed to allow the driver to later program them singularly in the 'set_timings()' function. Unfortunately I do not have a real rationale behind this which explains why it has to be done this way :( [1] here :) Steve
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
Hi Jacopo, On 07/14/2018 11:57 AM, Steve Longerbeam wrote: Hi Jacopo, Pardon the late reply, see below. On 07/11/2018 12:21 AM, jacopo mondi wrote: Hi Steve, On Tue, Jul 10, 2018 at 02:10:54PM -0700, Steve Longerbeam wrote: Hi Jacopo, Sorry to report my testing on SabreSD has same result as last time. This series fixes the LP-11 timeout at stream on but captured images are still blank. I tried the 640x480 mode with UYVY2X8. Here is the pad config: This saddens me :( I'm capturing with the same format and sizes... this shouldn't be the issue Could you confirm this matches what you have in your tree? 5dc2c80 media: ov5640: Fix timings setup code b35e757 media: i2c: ov5640: Re-work MIPI startup sequence 3c4a737 media: ov5640: fix frame interval enumeration 41cb1c7 media: ov5640: adjust xclk_max c3f3ba3 media: ov5640: add support of module orientation ce85705 media: ov5640: add HFLIP/VFLIP controls support 8663341 media: ov5640: Program the visible resolution 476dec0 media: ov5640: Add horizontal and vertical totals dba13a0 media: ov5640: Change horizontal and vertical resolutions name 8f57c2f media: ov5640: Init properly the SCLK dividers Yes, I have that commit sequence. FWIW, I can verify what Jagan Teki reported earlier, that the driver still works on the SabreSD platform at: dba13a0 media: ov5640: Change horizontal and vertical resolutions name and is broken at: 476dec0 media: ov5640: Add horizontal and vertical totals with LP-11 timeout at the mipi csi-2 receiver: [ 80.763189] imx6-mipi-csi2: LP-11 timeout, phy_state = 0x0230 [ 80.769599] ipu1_csi1: pipeline start failed with -110 And I discovered the bug in 476dec0 "media: ov5640: Add horizontal and vertical totals". The call to ov5640_set_timings() needs to be moved before the calls to ov5640_get_vts() and ov5640_get_hts(). But I see you have discovered that as well, and fixed in the second patch in your series. Steve # media-ctl --get-v4l2 "'ov5640 1-003c':0" [fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] Steve On 07/10/2018 11:36 AM, Jacopo Mondi wrote: Hello, this series fixes capture operations on i.MX6Q platforms (and possible other platforms reported not working) using MIPI CSI-2 interface. This iteration expands the v1 version with an additional fix, initially submitted by Maxime in his series: [PATCH v3 00/12] media: ov5640: Misc cleanup and improvements https://www.spinics.net/lists/linux-media/msg134436.html The original patch has been reported not fully fixing the issues by Daniel Mack in his comment here below (on a Qualcomm platform if I'm not wrong): https://www.spinics.net/lists/linux-media/msg134524.html On my i.MX6Q testing platform that patch alone does not fix MIPI capture neither. The version I'm sending here re-introduces some of the timings parameters in the initial configuration blob (not in the single mode ones), which apparently has to be at least initially programmed to allow the driver to later program them singularly in the 'set_timings()' function. Unfortunately I do not have a real rationale behind this which explains why it has to be done this way :( For the MIPI startup sequence re-work patch, no changes compared to v1. Steve reported he has verified the LP-11 timout issue is solved on his testing platform too. For more details, please refer to the v1 cover letter: https://www.mail-archive.com/linux-media@vger.kernel.org/msg133352.html Thanks j Jacopo Mondi (1): media: i2c: ov5640: Re-work MIPI startup sequence Samuel Bobrowicz (1): media: ov5640: Fix timings setup code drivers/media/i2c/ov5640.c | 107 ++--- 1 file changed, 82 insertions(+), 25 deletions(-) -- 2.7.4
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
Hi Jacopo, Pardon the late reply, see below. On 07/11/2018 12:21 AM, jacopo mondi wrote: Hi Steve, On Tue, Jul 10, 2018 at 02:10:54PM -0700, Steve Longerbeam wrote: Hi Jacopo, Sorry to report my testing on SabreSD has same result as last time. This series fixes the LP-11 timeout at stream on but captured images are still blank. I tried the 640x480 mode with UYVY2X8. Here is the pad config: This saddens me :( I'm capturing with the same format and sizes... this shouldn't be the issue Could you confirm this matches what you have in your tree? 5dc2c80 media: ov5640: Fix timings setup code b35e757 media: i2c: ov5640: Re-work MIPI startup sequence 3c4a737 media: ov5640: fix frame interval enumeration 41cb1c7 media: ov5640: adjust xclk_max c3f3ba3 media: ov5640: add support of module orientation ce85705 media: ov5640: add HFLIP/VFLIP controls support 8663341 media: ov5640: Program the visible resolution 476dec0 media: ov5640: Add horizontal and vertical totals dba13a0 media: ov5640: Change horizontal and vertical resolutions name 8f57c2f media: ov5640: Init properly the SCLK dividers Yes, I have that commit sequence. FWIW, I can verify what Jagan Teki reported earlier, that the driver still works on the SabreSD platform at: dba13a0 media: ov5640: Change horizontal and vertical resolutions name and is broken at: 476dec0 media: ov5640: Add horizontal and vertical totals with LP-11 timeout at the mipi csi-2 receiver: [ 80.763189] imx6-mipi-csi2: LP-11 timeout, phy_state = 0x0230 [ 80.769599] ipu1_csi1: pipeline start failed with -110 Steve # media-ctl --get-v4l2 "'ov5640 1-003c':0" [fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] Steve On 07/10/2018 11:36 AM, Jacopo Mondi wrote: Hello, this series fixes capture operations on i.MX6Q platforms (and possible other platforms reported not working) using MIPI CSI-2 interface. This iteration expands the v1 version with an additional fix, initially submitted by Maxime in his series: [PATCH v3 00/12] media: ov5640: Misc cleanup and improvements https://www.spinics.net/lists/linux-media/msg134436.html The original patch has been reported not fully fixing the issues by Daniel Mack in his comment here below (on a Qualcomm platform if I'm not wrong): https://www.spinics.net/lists/linux-media/msg134524.html On my i.MX6Q testing platform that patch alone does not fix MIPI capture neither. The version I'm sending here re-introduces some of the timings parameters in the initial configuration blob (not in the single mode ones), which apparently has to be at least initially programmed to allow the driver to later program them singularly in the 'set_timings()' function. Unfortunately I do not have a real rationale behind this which explains why it has to be done this way :( For the MIPI startup sequence re-work patch, no changes compared to v1. Steve reported he has verified the LP-11 timout issue is solved on his testing platform too. For more details, please refer to the v1 cover letter: https://www.mail-archive.com/linux-media@vger.kernel.org/msg133352.html Thanks j Jacopo Mondi (1): media: i2c: ov5640: Re-work MIPI startup sequence Samuel Bobrowicz (1): media: ov5640: Fix timings setup code drivers/media/i2c/ov5640.c | 107 ++--- 1 file changed, 82 insertions(+), 25 deletions(-) -- 2.7.4
Re: [PATCH v2 0/2] media: i2c: ov5640: Re-work MIPI startup sequence
Hi Jacopo, Sorry to report my testing on SabreSD has same result as last time. This series fixes the LP-11 timeout at stream on but captured images are still blank. I tried the 640x480 mode with UYVY2X8. Here is the pad config: # media-ctl --get-v4l2 "'ov5640 1-003c':0" [fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] Steve On 07/10/2018 11:36 AM, Jacopo Mondi wrote: Hello, this series fixes capture operations on i.MX6Q platforms (and possible other platforms reported not working) using MIPI CSI-2 interface. This iteration expands the v1 version with an additional fix, initially submitted by Maxime in his series: [PATCH v3 00/12] media: ov5640: Misc cleanup and improvements https://www.spinics.net/lists/linux-media/msg134436.html The original patch has been reported not fully fixing the issues by Daniel Mack in his comment here below (on a Qualcomm platform if I'm not wrong): https://www.spinics.net/lists/linux-media/msg134524.html On my i.MX6Q testing platform that patch alone does not fix MIPI capture neither. The version I'm sending here re-introduces some of the timings parameters in the initial configuration blob (not in the single mode ones), which apparently has to be at least initially programmed to allow the driver to later program them singularly in the 'set_timings()' function. Unfortunately I do not have a real rationale behind this which explains why it has to be done this way :( For the MIPI startup sequence re-work patch, no changes compared to v1. Steve reported he has verified the LP-11 timout issue is solved on his testing platform too. For more details, please refer to the v1 cover letter: https://www.mail-archive.com/linux-media@vger.kernel.org/msg133352.html Thanks j Jacopo Mondi (1): media: i2c: ov5640: Re-work MIPI startup sequence Samuel Bobrowicz (1): media: ov5640: Fix timings setup code drivers/media/i2c/ov5640.c | 107 ++--- 1 file changed, 82 insertions(+), 25 deletions(-) -- 2.7.4
Re: [PATCH] media: i2c: ov5640: Re-work MIPI startup sequence
On 07/10/2018 10:52 AM, jacopo mondi wrote:
Hi Steve,
I've done some more testing, and finally got MIPI capture work
properly on the i.MX6Q platform I'm testing on.
Apparently commit:
476dec012f4c6545b0b7599cd9adba2ed819ad3b
("media: ov5640: Add horizontal and vertical totals")
which several people, Jagan included, has reported introducing
regressions on MIPI actually breaks image capture.
I have a fix based on Maxime's and Sam's
[PATCH v3 01/12] media: ov5640: Fix timings setup code
I will send the two patches out soon.
Thanks J!
Steve
On Tue, Jul 10, 2018 at 09:33:31AM +0200, jacopo mondi wrote:
Hi Steve,
thanks for testing!
On Mon, Jul 09, 2018 at 02:52:09PM -0700, Steve Longerbeam wrote:
Hi Jacopo,
I tested this patch on the i.MX6Q SabreSD with the OV5640 module.
It fixes the LP-11 timeout at stream on, but the captured images
are completely blank/black.
Intersting that the module was not starting up properly on other
(all?) i.MX6Q platforms, not only on Engicam's one. I didn't get this
initially.
Ok, so that's a step forward, but not enough probably. I'll keep
looking into this and get back!
Thanks
j
Steve
On 07/06/2018 04:00 AM, Jacopo Mondi wrote:
From: Jacopo Mondi
Rework the MIPI interface startup sequence with the following changes:
- Remove MIPI bus initialization from the initial settings blob
- At set_power(1) time power up MIPI Tx/Rx and set data and clock lanes in
LP11 during 'sleep' and 'idle' with MIPI clock in non-continuous mode.
- At s_stream time enable/disable the MIPI interface output.
- Restore default settings at set_power(0) time.
Before this commit the sensor MIPI interface was initialized with settings
that require a start/stop sequence at power-up time in order to force lanes
into LP11 state, as they were initialized in LP00 when in 'sleep mode',
which is assumed to be the sensor manual definition for the D-PHY defined
stop mode.
The stream start/stop was performed by enabling disabling clock gating,
and had the side effect to change the lanes sleep mode configuration when
stream was stopped.
Clock gating/ungating:
- ret = ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
-on ? 0 : BIT(5));
- if (ret)
Set lanes in LP11 when in 'sleep mode':
- ret = ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
- on ? 0x00 : 0x70);
This commit fixes an issue reported by Jagan Teki on i.MX6 platforms that
prevents the host interface from powering up correctly:
https://lkml.org/lkml/2018/6/1/38
It also improves MIPI capture operations stability on my testing platform
where MIPI capture often (silently) failed and returned all-purple frames.
fixes: f22996db44e2 ("media: ov5640: add support of DVP parallel interface")
Reported-by: Jagan Teki
Signed-off-by: Jacopo Mondi
---
Hello,
I'm sending this one as new patch instead of a v2 of the previously sent
series "media: i2c: ov5640: Re-work MIPI interface configuration" as the
previous one was not working on the Engicam i.Mx6 platform where Jagan
initially reported issues on.
I've been able to test that capture now starts on said platform, but I've not
been able to visually verify any of the image content as I have no way yet to
transfer the raw images to my development host and verify their content (network
still not working for me on that platform :/ )
On my other testing platform images are correct, but they already were with the
previous version of this patches too, so I assume the CSI-2 receiver is far more
tolerant there.
Jagan, is there any way you could verify images? I would appreciate your
Tested-by tag in case they're correct.
Also, as there seems to be a lot of people interested in ov5640 these days, I
have expanded the receivers list. Anyone that could give these patches a spin?
(ie. Sam reported issues too with the current MIPI startup sequence in a patch
series he shared on dropbox iirc...)
Thanks
j
---
drivers/media/i2c/ov5640.c | 91 --
1 file changed, 71 insertions(+), 20 deletions(-)
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 1ecbb7a..7bbd1d7 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -286,10 +286,10 @@ static const struct reg_value
ov5640_init_setting_30fps_VGA[] = {
{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x3000, 0x00, 0, 0},
{0x3002, 0x1c, 0, 0}, {0x3004, 0xff, 0, 0}, {0x3006, 0xc3, 0, 0},
- {0x300e, 0x45, 0, 0}, {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
+ {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
{0x501f, 0x00, 0, 0}, {0x4713, 0x03, 0, 0}, {0x4407, 0x04, 0, 0},
{0x440e, 0x00, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
- {0x4837, 0x0a, 0, 0}, {0x4800, 0x04, 0, 0}, {0x3824, 0x02, 0, 0},
+ {0x4837,
[PATCH v6 00/17] media: imx: Switch to subdev notifiers
This patchset converts the imx-media driver and its dependent subdevs to use subdev notifiers. There are a couple shortcomings in v4l2-core that prevented subdev notifiers from working correctly in imx-media: 1. v4l2_async_notifier_fwnode_parse_endpoint() treats a fwnode endpoint that is not connected to a remote device as an error. But in the case of the video-mux subdev, this is not an error, it is OK if some of the mux inputs have no connection. Also, Documentation/devicetree/bindings/media/video-interfaces.txt explicitly states that the 'remote-endpoint' property is optional. So the first patch is a small modification to ignore empty endpoints in v4l2_async_notifier_fwnode_parse_endpoint() and allow __v4l2_async_notifier_parse_fwnode_endpoints() to continue to parse the remaining port endpoints of the device. 2. In the imx-media graph, multiple subdevs will encounter the same upstream subdev (such as the imx6-mipi-csi2 receiver), and so v4l2_async_notifier_parse_fwnode_endpoints() will add imx6-mipi-csi2 multiple times. This is treated as an error by v4l2_async_notifier_register() later. To get around this problem, add an v4l2_async_notifier_add_subdev() which first verifies the provided asd does not already exist in the given notifier asd list or in other registered notifiers. If the asd exists, the function returns -EEXIST and it's up to the caller to decide if that is an error (in imx-media case it is never an error). Patches 2-5 deal with adding that support. 3. Patch 6 adds v4l2_async_register_fwnode_subdev(), which is a convenience function for parsing a subdev's fwnode port endpoints for connected remote subdevs, registering a subdev notifier, and then registering the sub-device itself. 4. Patches 7-14 update the subdev drivers to register a subdev notifier with endpoint parsing, and the changes to imx-media to support that. 5. Finally, the last 3 patches endeavor to completely remove support for the notifier->subdevs[] array in platform drivers and v4l2 core. All platform drivers are modified to make use of v4l2_async_notifier_add_subdev() and its related convenience functions to add asd's to the notifier @asd_list, and any allocation or reference to the notifier->subdevs[] array removed. After that large patch, notifier->subdevs[] array is stripped from v4l2-async and v4l2-subdev docs are updated to reflect the new method of adding asd's to notifiers. Signed-off-by: Steve Longerbeam Patches 07-14 (video-mux and the imx patches) are Reviewed-by: Philipp Zabel Patches 01-14 are Tested-by: Philipp Zabel on i.MX6 with Toshiba TC358743 connected via MIPI CSI-2. History: v6: - Export v4l2_async_notifier_init(), which must be called by all drivers before the first call to v4l2_async_notifier_add_subdev(). Suggested by Sakari Ailus. - Removed @num_subdevs from struct v4l2_async_notifier, and the macro V4L2_MAX_SUBDEVS. Sugested by Sakari. - Fixed a double device node put in vpif_capture.c. Reported by Sakari. - Fixed wrong printk format qualifiers in xilinx-vipp.c. Reported by Dan Carpenter. v5: - see point 5 above. v4: - small non-functional code cleanup in video-mux.c. - strip TODO for comparing custom asd's for equivalence. - add three more convenience functions: v4l2_async_notifier_add_fwnode_subdev, v4l2_async_notifier_add_i2c_subdev, v4l2_async_notifier_add_devname_subdev. - strip support in v4l2_async_register_fwnode_subdev for sub-devices that register from port nodes. v3: - code optimization in asd_equal(), and remove unneeded braces, suggested by Sakari Ailus. - add a NULL asd pointer check to v4l2_async_notifier_asd_valid(). - fix an error-out path in v4l2_async_register_fwnode_subdev() that forgot to put device. v2: - don't pass an empty endpoint to the parse_endpoint callback, v4l2_async_notifier_fwnode_parse_endpoint() now just ignores them and returns success. - Fix a couple compile warnings and errors seen in i386 and sh archs. Steve Longerbeam (17): media: v4l2-fwnode: ignore endpoints that have no remote port parent media: v4l2: async: Allow searching for asd of any type media: v4l2: async: Add v4l2_async_notifier_add_subdev media: v4l2: async: Add convenience functions to allocate and add asd's media: v4l2-fwnode: Switch to v4l2_async_notifier_add_subdev media: v4l2-fwnode: Add a convenience function for registering subdevs with notifiers media: platform: video-mux: Register a subdev notifier media: imx: csi: Register a subdev notifier media: imx: mipi csi-2: Register a subdev notifier media: staging/imx: of: Remove recursive graph walk media: staging/imx: Loop through all registered subdevs for media links media: staging/imx: Rename root notifier media: staging/imx: Switch to v4l2_async_notifier_add_*_subdev media: staging/imx: TODO: Remove one assumption about OF graph parsing media: platfor
Re: [PATCH] media: i2c: ov5640: Re-work MIPI startup sequence
Hi Jacopo,
I tested this patch on the i.MX6Q SabreSD with the OV5640 module.
It fixes the LP-11 timeout at stream on, but the captured images
are completely blank/black.
Steve
On 07/06/2018 04:00 AM, Jacopo Mondi wrote:
From: Jacopo Mondi
Rework the MIPI interface startup sequence with the following changes:
- Remove MIPI bus initialization from the initial settings blob
- At set_power(1) time power up MIPI Tx/Rx and set data and clock lanes in
LP11 during 'sleep' and 'idle' with MIPI clock in non-continuous mode.
- At s_stream time enable/disable the MIPI interface output.
- Restore default settings at set_power(0) time.
Before this commit the sensor MIPI interface was initialized with settings
that require a start/stop sequence at power-up time in order to force lanes
into LP11 state, as they were initialized in LP00 when in 'sleep mode',
which is assumed to be the sensor manual definition for the D-PHY defined
stop mode.
The stream start/stop was performed by enabling disabling clock gating,
and had the side effect to change the lanes sleep mode configuration when
stream was stopped.
Clock gating/ungating:
- ret = ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
-on ? 0 : BIT(5));
- if (ret)
Set lanes in LP11 when in 'sleep mode':
- ret = ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
- on ? 0x00 : 0x70);
This commit fixes an issue reported by Jagan Teki on i.MX6 platforms that
prevents the host interface from powering up correctly:
https://lkml.org/lkml/2018/6/1/38
It also improves MIPI capture operations stability on my testing platform
where MIPI capture often (silently) failed and returned all-purple frames.
fixes: f22996db44e2 ("media: ov5640: add support of DVP parallel interface")
Reported-by: Jagan Teki
Signed-off-by: Jacopo Mondi
---
Hello,
I'm sending this one as new patch instead of a v2 of the previously sent
series "media: i2c: ov5640: Re-work MIPI interface configuration" as the
previous one was not working on the Engicam i.Mx6 platform where Jagan
initially reported issues on.
I've been able to test that capture now starts on said platform, but I've not
been able to visually verify any of the image content as I have no way yet to
transfer the raw images to my development host and verify their content (network
still not working for me on that platform :/ )
On my other testing platform images are correct, but they already were with the
previous version of this patches too, so I assume the CSI-2 receiver is far more
tolerant there.
Jagan, is there any way you could verify images? I would appreciate your
Tested-by tag in case they're correct.
Also, as there seems to be a lot of people interested in ov5640 these days, I
have expanded the receivers list. Anyone that could give these patches a spin?
(ie. Sam reported issues too with the current MIPI startup sequence in a patch
series he shared on dropbox iirc...)
Thanks
j
---
drivers/media/i2c/ov5640.c | 91 --
1 file changed, 71 insertions(+), 20 deletions(-)
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 1ecbb7a..7bbd1d7 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -286,10 +286,10 @@ static const struct reg_value
ov5640_init_setting_30fps_VGA[] = {
{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
{0x4001, 0x02, 0, 0}, {0x4004, 0x02, 0, 0}, {0x3000, 0x00, 0, 0},
{0x3002, 0x1c, 0, 0}, {0x3004, 0xff, 0, 0}, {0x3006, 0xc3, 0, 0},
- {0x300e, 0x45, 0, 0}, {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
+ {0x302e, 0x08, 0, 0}, {0x4300, 0x3f, 0, 0},
{0x501f, 0x00, 0, 0}, {0x4713, 0x03, 0, 0}, {0x4407, 0x04, 0, 0},
{0x440e, 0x00, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
- {0x4837, 0x0a, 0, 0}, {0x4800, 0x04, 0, 0}, {0x3824, 0x02, 0, 0},
+ {0x4837, 0x0a, 0, 0}, {0x3824, 0x02, 0, 0},
{0x5000, 0xa7, 0, 0}, {0x5001, 0xa3, 0, 0}, {0x5180, 0xff, 0, 0},
{0x5181, 0xf2, 0, 0}, {0x5182, 0x00, 0, 0}, {0x5183, 0x14, 0, 0},
{0x5184, 0x25, 0, 0}, {0x5185, 0x24, 0, 0}, {0x5186, 0x09, 0, 0},
@@ -1102,12 +1102,18 @@ static int ov5640_set_stream_mipi(struct ov5640_dev
*sensor, bool on)
{
int ret;
- ret = ov5640_mod_reg(sensor, OV5640_REG_MIPI_CTRL00, BIT(5),
-on ? 0 : BIT(5));
- if (ret)
- return ret;
- ret = ov5640_write_reg(sensor, OV5640_REG_PAD_OUTPUT00,
- on ? 0x00 : 0x70);
+ /*
+* Enable/disable the MIPI interface
+*
+* 0x300e = on ? 0x45 : 0x40
+* [7:5] = 001 : 2 data lanes mode
+* [4] = 0 : Power up MIPI HS Tx
+* [3] = 0 : Power up MIPI LS Rx
+* [2] = 1/0: MIPI interface enable/disable
+* [1:0] = 01/00: FIXME: 'debug'
+*/
+ ret =
Re: [PATCH 16/16] media: imx: add mem2mem device
Hi Philipp,
On 06/22/2018 08:52 AM, Philipp Zabel wrote:
Add a single imx-media mem2mem video device that uses the IPU IC PP
(image converter post processing) task for scaling and colorspace
conversion.
On i.MX6Q/DL SoCs with two IPUs currently only the first IPU is used.
The hardware only supports writing to destination buffers up to
1024x1024 pixels in a single pass, so the mem2mem video device is
limited to this resolution. After fixing the tiling code it should
be possible to extend this to arbitrary sizes by rendering multiple
tiles per frame.
Signed-off-by: Philipp Zabel
---
drivers/staging/media/imx/Kconfig | 1 +
drivers/staging/media/imx/Makefile| 1 +
drivers/staging/media/imx/imx-media-dev.c | 11 +
drivers/staging/media/imx/imx-media-mem2mem.c | 953 ++
drivers/staging/media/imx/imx-media.h | 10 +
5 files changed, 976 insertions(+)
create mode 100644 drivers/staging/media/imx/imx-media-mem2mem.c
diff --git a/drivers/staging/media/imx/Kconfig
b/drivers/staging/media/imx/Kconfig
index bfc17de56b17..07013cb3cb66 100644
--- a/drivers/staging/media/imx/Kconfig
+++ b/drivers/staging/media/imx/Kconfig
@@ -6,6 +6,7 @@ config VIDEO_IMX_MEDIA
depends on HAS_DMA
select VIDEOBUF2_DMA_CONTIG
select V4L2_FWNODE
+ select V4L2_MEM2MEM_DEV
---help---
Say yes here to enable support for video4linux media controller
driver for the i.MX5/6 SOC.
diff --git a/drivers/staging/media/imx/Makefile
b/drivers/staging/media/imx/Makefile
index 698a4210316e..f2e722d0fa19 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -6,6 +6,7 @@ imx-media-ic-objs := imx-ic-common.o imx-ic-prp.o
imx-ic-prpencvf.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-mem2mem.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
diff --git a/drivers/staging/media/imx/imx-media-dev.c b/drivers/staging/media/imx/imx-media-dev.c
index 289d775c4820..7a9aabcae3ee 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -359,6 +359,17 @@ static int imx_media_probe_complete(struct
v4l2_async_notifier *notifier)
goto unlock;
ret = v4l2_device_register_subdev_nodes(>v4l2_dev);
+ if (ret)
+ goto unlock;
+
+ /* TODO: check whether we have IC subdevices first */
+ imxmd->m2m_vdev = imx_media_mem2mem_device_init(imxmd);
+ if (IS_ERR(imxmd->m2m_vdev)) {
+ ret = PTR_ERR(imxmd->m2m_vdev);
+ goto unlock;
+ }
+
+ ret = imx_media_mem2mem_device_register(imxmd->m2m_vdev);
unlock:
mutex_unlock(>mutex);
if (ret)
diff --git a/drivers/staging/media/imx/imx-media-mem2mem.c
b/drivers/staging/media/imx/imx-media-mem2mem.c
new file mode 100644
index ..8830f77f0407
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-mem2mem.c
@@ -0,0 +1,953 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * i.MX IPUv3 mem2mem Scaler/CSC driver
+ *
+ * Copyright (C) 2011 Pengutronix, Sascha Hauer
+ * Copyright (C) 2018 Pengutronix, Philipp Zabel
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+
+#include "imx-media.h"
+
+#define MIN_W 16
+#define MIN_H 16
+#define MAX_W 4096
+#define MAX_H 4096
+
+#define fh_to_ctx(__fh)container_of(__fh, struct mem2mem_ctx, fh)
+
+enum {
+ V4L2_M2M_SRC = 0,
+ V4L2_M2M_DST = 1,
+};
+
+struct mem2mem_priv {
+ struct imx_media_video_dev vdev;
+
+ struct v4l2_m2m_dev *m2m_dev;
+ struct device *dev;
+
+ struct imx_media_dev *md;
+
+ struct mutex mutex; /* mem2mem device mutex */
+
+ atomic_t num_inst;
+};
+
+#define to_mem2mem_priv(v) container_of(v, struct mem2mem_priv, vdev)
+
+/* Per-queue, driver-specific private data */
+struct mem2mem_q_data {
+ struct v4l2_pix_format cur_fmt;
+ struct v4l2_rectrect;
+};
+
+struct mem2mem_ctx {
+ struct mem2mem_priv *priv;
+
+ struct v4l2_fh fh;
+ struct mem2mem_q_data q_data[2];
+ int error;
+ struct ipu_image_convert_ctx *icc;
+
+ struct v4l2_ctrl_handler ctrl_hdlr;
+ int rotate;
+ bool hflip;
+ bool vflip;
+ enum ipu_rotate_moderot_mode;
+};
+
+static struct mem2mem_q_data *get_q_data(struct
Re: [PATCH 00/16] i.MX media mem2mem scaler
Hi Philipp, Thanks for this great patchset! Finally we have improved seams with tiled conversions, and relaxed width alignment requirements. Unfortunately this patchset isn't working correctly yet. It breaks tiled conversions with rotation. Trying the following conversion: input: 720x480, UYVY output: 1280x768, UYVY, rotation=90 degrees causes non-8-byte aligned tile buffers at the output: [ 129.578210] imx-ipuv3 240.ipu: task 2: ctx 8955dec9: Input@[0,0]: phys [ 129.585980] imx-ipuv3 240.ipu: task 2: ctx 8955dec9: Input@[1,0]: phys 00051360 [ 129.593736] imx-ipuv3 240.ipu: task 2: ctx 8955dec9: Output@[0,0]: phys [ 129.601556] imx-ipuv3 240.ipu: task 2: ctx 8955dec9: Output@[0,1]: phys 052e resulting in hung conversion and abort timeout: [ 147.689220] imx-ipuv3 240.ipu: ipu_image_convert_abort: timeout Note that when converting to a planar format, the final (rotated) chroma tile buffers are also mis-aligned, in addition to the Y buffers. I have some additional comments to follow in the patches. Steve On 06/22/2018 08:52 AM, Philipp Zabel wrote: Hi, we have image conversion code for scaling and colorspace conversion in the IPUv3 base driver for a while. Since the IC hardware can only write up to 1024x1024 pixel buffers, it scales to larger output buffers by splitting the input and output frame into similarly sized tiles. This causes the issue that the bilinear interpolation resets at the tile boundary: instead of smoothly interpolating across the seam, there is a jump in the input sample position that is very apparent for high upscaling factors. This can be avoided by slightly changing the scaling coefficients to let the left/top tiles overshoot their input sampling into the first pixel / line of their right / bottom neighbors. The error can be further reduced by letting tiles be differently sized and by selecting seam positions that minimize the input sampling position error at tile boundaries. This is complicated by different DMA start address, burst size, and rotator block size alignment requirements, depending on the input and output pixel formats, and the fact that flipping happens in different places depending on the rotation. This series implements optimal seam position selection and seam hiding with per-tile resizing coefficients and adds a scaling mem2mem device to the imx-media driver. regards Philipp Philipp Zabel (16): gpu: ipu-v3: ipu-ic: allow to manually set resize coefficients gpu: ipu-v3: image-convert: prepare for per-tile configuration gpu: ipu-v3: image-convert: calculate per-tile resize coefficients gpu: ipu-v3: image-convert: reconfigure IC per tile gpu: ipu-v3: image-convert: store tile top/left position gpu: ipu-v3: image-convert: calculate tile dimensions and offsets outside fill_image gpu: ipu-v3: image-convert: move tile alignment helpers gpu: ipu-v3: image-convert: select optimal seam positions gpu: ipu-v3: image-convert: fix debug output for varying tile sizes gpu: ipu-v3: image-convert: relax tile width alignment for NV12 and NV16 gpu: ipu-v3: image-convert: relax input alignment restrictions gpu: ipu-v3: image-convert: relax output alignment restrictions gpu: ipu-v3: image-convert: fix bytesperline adjustment gpu: ipu-v3: image-convert: add some ASCII art to the exposition gpu: ipu-v3: image-convert: disable double buffering if necessary media: imx: add mem2mem device drivers/gpu/ipu-v3/ipu-ic.c | 52 +- drivers/gpu/ipu-v3/ipu-image-convert.c| 865 +--- drivers/staging/media/imx/Kconfig | 1 + drivers/staging/media/imx/Makefile| 1 + drivers/staging/media/imx/imx-media-dev.c | 11 + drivers/staging/media/imx/imx-media-mem2mem.c | 953 ++ drivers/staging/media/imx/imx-media.h | 10 + include/video/imx-ipu-v3.h| 6 + 8 files changed, 1760 insertions(+), 139 deletions(-) create mode 100644 drivers/staging/media/imx/imx-media-mem2mem.c
[PATCH] media: v4l2-ctrls: Fix CID base conflict between MAX217X and IMX
When the imx-media driver was initially merged, there was a conflict
with 8d67ae25 ("media: v4l2-ctrls: Reserve controls for MAX217X") which
was not fixed up correctly, resulting in V4L2_CID_USER_MAX217X_BASE and
V4L2_CID_USER_IMX_BASE taking on the same value. Fix by assigning imx
CID base the next available range at 0x10b0.
Signed-off-by: Steve Longerbeam
---
include/uapi/linux/v4l2-controls.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 8d473c9..8a75ad7 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -188,7 +188,7 @@ enum v4l2_colorfx {
/* The base for the imx driver controls.
* We reserve 16 controls for this driver. */
-#define V4L2_CID_USER_IMX_BASE (V4L2_CID_USER_BASE + 0x1090)
+#define V4L2_CID_USER_IMX_BASE (V4L2_CID_USER_BASE + 0x10b0)
/* MPEG-class control IDs */
/* The MPEG controls are applicable to all codec controls
--
2.7.4
Re: V4L2_CID_USER_MAX217X_BASE == V4L2_CID_USER_IMX_BASE
Hello Helmut,
On 06/22/2018 12:51 AM, Helmut Grohne wrote:
Hi,
I found it strange that the macros V4L2_CID_USER_MAX217X_BASE and
V4L2_CID_USER_IMX_BASE have equal value even though each of them state
that they reserved a range. Those reservations look conflicting to me.
Yes, they conflict.
The macro V4L2_CID_USER_MAX217X_BASE came first,
No, imx came first. e1302912 ("media: Add i.MX media core driver")
is dated June 10, 2017. 8d67ae25 ("media: v4l2-ctrls: Reserve controls for
MAX217X") is dated two days later.
and
V4L2_CID_USER_IMX_BASE was introduced in e130291212df ("media: Add i.MX
media core driver") with the conflicting assignment (not a merge error).
The authors of that patch mostly make up the recipient list.
There were 8 revisions of the imx-media driver posted. In all of
those postings, V4L2_CID_USER_MAX217X_BASE did not exist yet.
So it looks like 8d67ae25 was merged at the same time as e1302912
but the conflict went unnoticed.
Steve
Re: [PATCH v3 05/13] media: v4l2-fwnode: Add a convenience function for registering subdevs with notifiers
On 06/26/2018 12:45 AM, Sakari Ailus wrote:
On Tue, May 08, 2018 at 08:55:04PM -0700, Steve Longerbeam wrote:
On 05/08/2018 03:28 AM, Sakari Ailus wrote:
Hi Steve,
Again, sorry about the delay. This thread got buried in my inbox. :-(
Please see my reply below.
On Mon, Apr 23, 2018 at 11:00:22AM -0700, Steve Longerbeam wrote:
On 04/23/2018 12:14 AM, Sakari Ailus wrote:
Hi Steve,
On Tue, Mar 20, 2018 at 05:37:21PM -0700, Steve Longerbeam wrote:
Adds v4l2_async_register_fwnode_subdev(), which is a convenience function
for parsing a sub-device's fwnode port endpoints for connected remote
sub-devices, registering a sub-device notifier, and then registering
the sub-device itself.
Signed-off-by: Steve Longerbeam
---
Changes since v2:
- fix error-out path in v4l2_async_register_fwnode_subdev() that forgot
to put device.
Changes since v1:
- add #include to v4l2-fwnode.h for
'struct v4l2_subdev' declaration.
---
drivers/media/v4l2-core/v4l2-fwnode.c | 101
++
include/media/v4l2-fwnode.h | 43 +++
2 files changed, 144 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-fwnode.c
b/drivers/media/v4l2-core/v4l2-fwnode.c
index 99198b9..d42024d 100644
--- a/drivers/media/v4l2-core/v4l2-fwnode.c
+++ b/drivers/media/v4l2-core/v4l2-fwnode.c
@@ -880,6 +880,107 @@ int v4l2_async_register_subdev_sensor_common(struct
v4l2_subdev *sd)
}
EXPORT_SYMBOL_GPL(v4l2_async_register_subdev_sensor_common);
+int v4l2_async_register_fwnode_subdev(
+ struct v4l2_subdev *sd, size_t asd_struct_size,
+ unsigned int *ports, unsigned int num_ports,
+ int (*parse_endpoint)(struct device *dev,
+ struct v4l2_fwnode_endpoint *vep,
+ struct v4l2_async_subdev *asd))
+{
+ struct v4l2_async_notifier *notifier;
+ struct device *dev = sd->dev;
+ struct fwnode_handle *fwnode;
+ unsigned int subdev_port;
+ bool is_port;
+ int ret;
+
+ if (WARN_ON(!dev))
+ return -ENODEV;
+
+ fwnode = dev_fwnode(dev);
+ if (!fwnode_device_is_available(fwnode))
+ return -ENODEV;
+
+ is_port = (is_of_node(fwnode) &&
+ of_node_cmp(to_of_node(fwnode)->name, "port") == 0);
What's the intent of this and the code below? You may not parse the graph
data structure here, it should be done in the actual firmware
implementation instead.
The i.MX6 CSI sub-device registers itself from a port fwnode, so
the intent of the is_port code below is to support the i.MX6 CSI.
I can remove the is_port checks, but it means
v4l2_async_register_fwnode_subdev() won't be usable by the CSI
sub-device.
This won't scale.
The vast majority of sub-devices register themselves as
port parent nodes. So for now at least, I think
v4l2_async_register_fwnode_subdev() could be useful to many
platforms.
Instead, I think we'd need to separate registering
sub-devices (through async sub-devices) and binding them with the driver
that registered the notifier. Or at least change how that process works: a
single sub-device can well be bound to multiple notifiers,
Ok, that is certainly not the case now, a sub-device can only
be bound to a single notifier.
or multiple
times to the same notifier while it may be registered only once.
Anyway, this is a future generalization if I understand you
correctly. Not something to deal with here.
Also, sub-devices generally do not match ports.
Yes that's generally true, sub-devices generally match to port parent
nodes. But I do know of one other sub-device that buck that trend.
The ADV748x CSI-2 output sub-devices match against endpoint nodes.
Endpoints, yes, but not ports.
Well, the imx CSI registers from a port node.
I looked at the imx driver and it appears to be parsing DT without much
help from the frameworks; graph or V4L2 fwnode. Is there a reason to do so,
other than it's a staging driver? :-)
That's the whole point of this patchset. It gets rid of the code in imx
that discovers subdevices by recursively walking the graph, replacing
with the subdev notifier framework.
Do you happen to have any DT source (or bindings) for this?
Documentation/devicetree/bindings/media/imx.txt.
For example DT source, it's all in the imx6 reference boards,
see imx6qdl-sabresd.dtsi for example.
It'd help to understand what is the aim here.
The aim of what? Of this specific commit? I'll reprint it here:
Adds v4l2_async_register_fwnode_subdev(), which is a convenience function
for parsing a sub-device's fwnode port endpoints for connected remote
sub-devices, registering a sub-device notifier, and then registering
the sub-device itself.
Steve
Re: [PATCH v3 03/13] media: v4l2: async: Add v4l2_async_notifier_add_subdev
On 06/26/2018 12:12 AM, Sakari Ailus wrote:
On Wed, May 09, 2018 at 04:06:32PM -0700, Steve Longerbeam wrote:
On 05/08/2018 03:12 AM, Sakari Ailus wrote:
On Fri, Apr 20, 2018 at 10:12:33AM -0700, Steve Longerbeam wrote:
Hi Sakari,
On 04/20/2018 05:24 AM, Sakari Ailus wrote:
Hi Steve,
Thanks for the patchset.
On Tue, Mar 20, 2018 at 05:37:19PM -0700, Steve Longerbeam wrote:
v4l2_async_notifier_add_subdev() adds an asd to the notifier. It checks
that the asd's match_type is valid and that no other equivalent asd's
have already been added to this notifier's asd list, or to other
registered notifier's waiting or done lists, and increments num_subdevs.
v4l2_async_notifier_add_subdev() does not make use of the notifier subdevs
array, otherwise it would have to re-allocate the array every time the
function was called. In place of the subdevs array, the function adds
the asd to a new master asd_list. The function will return error with a
WARN() if it is ever called with the subdevs array allocated.
In v4l2_async_notifier_has_async_subdev(), __v4l2_async_notifier_register(),
and v4l2_async_notifier_cleanup(), alternatively operate on the subdevs
array or a non-empty notifier->asd_list.
I do agree with the approach, but this patch leaves the remaining users of
the subdevs array in the notifier intact. Could you rework them to use the
v4l2_async_notifier_add_subdev() as well?
There seem to be just a few of them --- exynos4-is and rcar_drif.
I count more than a few :)
% cd drivers/media && grep -l -r --include "*.[ch]"
'notifier[\.\-]>*subdevs[ ]*='
v4l2-core/v4l2-async.c
platform/pxa_camera.c
platform/ti-vpe/cal.c
platform/exynos4-is/media-dev.c
platform/qcom/camss-8x16/camss.c
platform/soc_camera/soc_camera.c
platform/atmel/atmel-isi.c
platform/atmel/atmel-isc.c
platform/stm32/stm32-dcmi.c
platform/davinci/vpif_capture.c
platform/davinci/vpif_display.c
platform/renesas-ceu.c
platform/am437x/am437x-vpfe.c
platform/xilinx/xilinx-vipp.c
platform/rcar_drif.c
So not including v4l2-core, the list is:
pxa_camera
ti-vpe
exynos4-is
qcom
soc_camera
atmel
stm32
davinci
renesas-ceu
am437x
xilinx
rcar_drif
Given such a large list of the users of the notifier->subdevs array,
I think this should be done is two steps: submit this patchset first,
that keeps the backward compatibility, and then a subsequent patchset
that converts all drivers to use v4l2_async_notifier_add_subdev(), at
which point the subdevs array can be removed from struct
v4l2_async_notifier.
Or, do you still think this should be done all at once? I could add a
large patch to this patchset that does the conversion and removes
the subdevs array.
Sorry for the delay. I grepped for this, too, but I guess I ended up using
an expression that missed most of the users. :-(
I think it'd be very good to perform that conversion --- the code in the
framework would be quite a bit cleaner and easier to maintain whereas the
per-driver conversions seem pretty simple, such as this on in
drivers/media/platform/atmel/atmel-isi.c :
/* Register the subdevices notifier. */
subdevs = devm_kzalloc(isi->dev, sizeof(*subdevs), GFP_KERNEL);
if (!subdevs) {
of_node_put(isi->entity.node);
return -ENOMEM;
}
subdevs[0] = >entity.asd;
isi->notifier.subdevs = subdevs;
isi->notifier.num_subdevs = 1;
isi->notifier.ops = _graph_notify_ops;
Yes, the conversions are fairly straightforward. I've completed that work,
but it was a very manual conversion, every platform is different and needed
careful study.
Although I was careful about getting the error-out paths correct, there
could
be mistakes there, which would result in memory leaks. And obviously I can't
re-test all these platforms. So this patch is very high-risk. More eyes are
needed on it, ideally the maintainer(s) of each affected platform.
I just submitted v4 of this series, but did not include this large un-tested
patch in v4 for those reasons.
Instead, this patch, and follow-up patches that strips support for subdevs
array altogether from v4l2-async.c, and updates rst docs, are available at
my
media-tree mirror on github:
g...@github.com:slongerbeam/mediatree.git
in the branch 'remove-subdevs-array'. The branch is based off this series
(branch 'imx-subdev-notifiers.6').
Would you be able to post these to the list? I'd really like this being
done as part of the related patchset, rather than leaving the mess in the
framework.
Backward compatibility can look messy.
I can include the patch that converts all platforms at once. But as I
said it is completely untested.
So I'm curious, what is the policy in V4L2 community regarding
merging untested patches? Do we go ahead and merge and then
fixup errors as they are discovered, or should the patch get basic
validation by everyone who has access to the affected hardware
first?
Steve
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
On 06/20/2018 01:54 AM, Philipp Zabel wrote: Hi Steve, On Tue, 2018-06-19 at 18:30 -0700, Steve Longerbeam wrote: I've found some time to diagnose the behavior of interweave with B/T line swapping (to support interlaced-bt) with planar formats. There are a couple problems (one known and one unknown): 1. This requires 32 pixel alignment to meet the IDMAC 8-byte alignment of the planar U/V buffer offsets, and 32 pixel alignment precludes capturing raw NTSC/PAL at 720 pixel line stride. What needs to be aligned to multiples of 32 pixels? I thought 8 pixel width alignment should be good enough for NV12/NV16, for which luma and chroma strides are equal to the width in pixels, and 16 pixel alignment for YUV420/YVU420/YUV422P, where chroma stride is half the width in pixels. I see where the problem is now. I was basing my mistaken 32 pixel alignment from a read of the U_OFFSET/V_OFFSET macros in ipu-cpmem.c: u_offset = pix->width * pix->height + pix->width * y / 4 + x / 2 But you can probably see the bug now. This does not produce a correct offset for odd values of y. It should read: u_offset = pix->width * pix->height + pix->width * (y / 2) / 2 + x / 2 With that fix, interweave line swap with planar 4:2:0 is working now. That includes YUV420, YVU420, and NV12. NV16 is also working after programming SLUV with double the chroma line stride. 2. Even with 32 pixel aligned frames, for example by using the prpenc scaler to generate 704 pixel strides from 720, the colors are still wrong when capturing interlaced-bt. As a side note, we can't properly scale seq-tb/bt direct input from the CSI vertically with the IC, as the bottom line of the first field will be blended with the top line of the second field... I thought for sure this must be because we also need to double the SLUV line strides in addition to doubling SLY line stride. But I tried this and the results are that it works only for YUV 4:2:2. For 4:2:0 it causes system hard lockups. (Aside note: interweave without line swap apparently has never worked for 4:2:2, even when doubling SLUV, so it's quite bizarre to me why 4:2:2 interweave _with_ line swap _does_ work after doubling SLUV). When you say 4:2:2 you only mean YUV422P, not NV16 or YUYV/UYVY ? Correct, I meant planar YUV422P. For these reasons I think we should disallow interlaced-bt with planar formats. Does that include NV12/NV16? Capturing to NV12 could be desirable if at all possible, because the result can be encoded by the CODA. The other formats are bandwidth inefficient anyway. Never mind, I found the bug described above in the U_OFFSET/V_OFFSET macros. In summary, at this point all planar formats are working with interlaced bt and tb, except for YUV422P. Steve
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
Hi Philipp, Krzysztof, On 06/15/2018 01:33 AM, Krzysztof Hałasa wrote: Steve Longerbeam writes: Right, the selection of interweave is moved to the capture devices, so the following will enable interweave: v4l2-ctl -dN --set-fmt-video=field=interlaced_tb and So the patch to adv7180 needs to be modified to report # field lines. Try the following: --- a/drivers/media/i2c/adv7180.c +++ b/drivers/media/i2c/adv7180.c With this patch, fix-csi-interlaced.3 seems to work for me. "ipu2_csi1":2 reports [fmt:AYUV32/720x576 field:seq-tb], but the /dev/videoX shows (when requested) 720 x 576 NV12 interlaced, top field first, and I'm getting valid output. Thanks for your work. I've found some time to diagnose the behavior of interweave with B/T line swapping (to support interlaced-bt) with planar formats. There are a couple problems (one known and one unknown): 1. This requires 32 pixel alignment to meet the IDMAC 8-byte alignment of the planar U/V buffer offsets, and 32 pixel alignment precludes capturing raw NTSC/PAL at 720 pixel line stride. 2. Even with 32 pixel aligned frames, for example by using the prpenc scaler to generate 704 pixel strides from 720, the colors are still wrong when capturing interlaced-bt. I thought for sure this must be because we also need to double the SLUV line strides in addition to doubling SLY line stride. But I tried this and the results are that it works only for YUV 4:2:2. For 4:2:0 it causes system hard lockups. (Aside note: interweave without line swap apparently has never worked for 4:2:2, even when doubling SLUV, so it's quite bizarre to me why 4:2:2 interweave _with_ line swap _does_ work after doubling SLUV). For these reasons I think we should disallow interlaced-bt with planar formats. If the user needs NTSC interlaced capture with planar, the fields can be swapped at the CSI, by selecting seq-tb at the CSI source pad, which allows for interlaced-tb at the capture interface, which doesn't require interweave line swapping. Steve
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
Hi Krzysztof,
On 06/14/2018 02:39 AM, Krzysztof Hałasa wrote:
Reporting from the field :-)
The fix-csi-interlaced.3 branch is still a bit off the track I guess:
Yes, it's still a WIP. A couple things are remaining:
- fix interweave with negative offsets for planar pixel formats.
- update the doc again.
media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/576 field:seq-tb]"
media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x576]"
media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x576 field:interlaced-tb]"
does:
"adv7180 2-0020":0 [fmt:UYVY2X8/720x576 field:alternate]
"ipu2_csi1_mux":1 [fmt:UYVY2X8/720x576 field:alternate]
"ipu2_csi1_mux":2 [fmt:UYVY2X8/720x576 field:alternate]
"ipu2_csi1":0 [fmt:UYVY2X8/720x576 field:alternate
crop.bounds:(0,0)/720x1152
crop:(0,0)/720x1152
compose.bounds:(0,0)/720x1152
compose:(0,0)/720x1152]
"ipu2_csi1":2 [fmt:AYUV32/720x1152 field:seq-tb]
... and not interlaced[-*], as with fix-csi-interlaced.2.
Right, the selection of interweave is moved to the capture devices,
so the following will enable interweave:
v4l2-ctl -dN --set-fmt-video=field=interlaced_tb
The double heights are funny, too - probably an ADV7180 issue.
That's because it's been confirmed that for sources that
report ALTERNATE, mbus format height must be the number
of lines per field, not the total frame lines.
See
0018147c964e ("media: v4l: doc: Clarify v4l2_mbus_fmt height definition")
So the patch to adv7180 needs to be modified to report # field lines.
Try the following:
--- a/drivers/media/i2c/adv7180.c
+++ b/drivers/media/i2c/adv7180.c
@@ -503,6 +503,9 @@ static int adv7180_set_power(struct adv7180_state
*state, bool on)
}
}
+ if (on)
+ msleep(500);
+
return 0;
}
@@ -643,6 +646,8 @@ static int adv7180_mbus_fmt(struct v4l2_subdev *sd,
fmt->colorspace = V4L2_COLORSPACE_SMPTE170M;
fmt->width = 720;
fmt->height = state->curr_norm & V4L2_STD_525_60 ? 480 : 576;
+ if (fmt->field == V4L2_FIELD_ALTERNATE)
+ fmt->height /= 2;
return 0;
}
@@ -694,8 +699,8 @@ static int adv7180_get_pad_format(struct v4l2_subdev
*sd,
if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
format->format = *v4l2_subdev_get_try_format(sd, cfg, 0);
} else {
- adv7180_mbus_fmt(sd, >format);
format->format.field = state->field;
+ adv7180_mbus_fmt(sd, >format);
}
return 0;
@@ -712,10 +717,10 @@ static int adv7180_set_pad_format(struct
v4l2_subdev *sd,
switch (format->format.field) {
case V4L2_FIELD_NONE:
if (!(state->chip_info->flags & ADV7180_FLAG_I2P))
- format->format.field = V4L2_FIELD_INTERLACED;
+ format->format.field = V4L2_FIELD_ALTERNATE;
break;
default:
- format->format.field = V4L2_FIELD_INTERLACED;
+ format->format.field = V4L2_FIELD_ALTERNATE;
break;
}
@@ -1291,7 +1296,7 @@ static int adv7180_probe(struct i2c_client *client,
return -ENOMEM;
state->client = client;
- state->field = V4L2_FIELD_INTERLACED;
+ state->field = V4L2_FIELD_ALTERNATE;
state->chip_info = (struct adv7180_chip_info *)id->driver_data;
state->pwdn_gpio = devm_gpiod_get_optional(>dev, "powerdown",
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
Hi Javier, thanks for the confirmations. I'm working on a fix for imx-media. Steve On 06/12/2018 10:27 AM, Javier Martinez Canillas wrote: Hi Steve, On 06/11/2018 11:06 PM, Steve Longerbeam wrote: [snip] I've been made aware [1] that recently V4L2_FIELD_ALTERNATE has been clarified [2] to specify that v4l2_mbus_fmt.height should contain the number of lines per field, not per frame: Yep! That was nagging at me as well. I noticed at least one other platform (omap3isp) that doubles the source pad frame height Coincidentally I noticed this problem when testing on a board with an omap3isp. This is the pipeline setup I've been using for a long time: $ media-ctl -l '"tvp5150 1-005c":1->"OMAP3 ISP CCDC":0[1]' $ media-ctl -l '"OMAP3 ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]' $ media-ctl -V '"OMAP3 ISP CCDC":0 [UYVY2X8 720x240 field:alternate]' $ media-ctl -V '"OMAP3 ISP CCDC":1 [UYVY 720x480 field:interlaced-tb]' when the sensor reports ALTERNATE field mode, to capture a whole frame. Makes sense. I think the crop height will need to As you said, the ISP doubles the source pad height, and so the sink pad is meant to have half of the frame height and this should match the camera sensor height. But since the tvp5150 had the full frame height (720x480) in its source pad, this didn't match the CCDC sink pads height which lead to .link_validate callback to return -EPIPE: ioctl(3, VIDIOC_STREAMON, 0xbeabea18) = -1 EPIPE (Broken pipe) After the revert, link validation / STREAMON works correctly and the following is what the relevant media entities look like in the graph: - entity 15: OMAP3 ISP CCDC (3 pads, 9 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2 pad0: Sink [fmt:UYVY2X8/720x240 field:alternate] <- "OMAP3 ISP CCP2":1 [] <- "OMAP3 ISP CSI2a":1 [] <- "tvp5150 1-005c":1 [ENABLED] pad1: Source [fmt:UYVY/720x480 field:interlaced-tb crop.bounds:(0,0)/720x240 crop:(0,0)/720x240] -> "OMAP3 ISP CCDC output":0 [ENABLED] -> "OMAP3 ISP resizer":0 [] - entity 81: tvp5150 1-005c (4 pads, 1 link) type V4L2 subdev subtype Decoder flags 0 device node name /dev/v4l-subdev8 pad0: Sink pad1: Source [fmt:UYVY2X8/720x240 field:alternate crop.bounds:(0,0)/720x480 crop:(0,0)/720x480] -> "OMAP3 ISP CCDC":0 [ENABLED] Best regards,
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
On 06/11/2018 02:19 AM, Philipp Zabel wrote:
Hi Steve,
On Sun, 2018-06-10 at 17:08 -0700, Steve Longerbeam wrote:
Hi Philipp,
On 06/01/2018 06:13 AM, Philipp Zabel wrote:
The IPU also supports interlaced buffers that start with the bottom field.
To achieve this, the the base address EBA has to be increased by a stride
length and the interlace offset ILO has to be set to the negative stride.
Signed-off-by: Philipp Zabel
---
drivers/gpu/ipu-v3/ipu-cpmem.c | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/ipu-v3/ipu-cpmem.c b/drivers/gpu/ipu-v3/ipu-cpmem.c
index 9f2d9ec42add..c1028f38c553 100644
--- a/drivers/gpu/ipu-v3/ipu-cpmem.c
+++ b/drivers/gpu/ipu-v3/ipu-cpmem.c
@@ -269,8 +269,14 @@ EXPORT_SYMBOL_GPL(ipu_cpmem_set_uv_offset);
void ipu_cpmem_interlaced_scan(struct ipuv3_channel *ch, int stride)
{
+ u32 ilo;
+
ipu_ch_param_write_field(ch, IPU_FIELD_SO, 1);
- ipu_ch_param_write_field(ch, IPU_FIELD_ILO, stride / 8);
+ if (stride >= 0)
+ ilo = stride / 8;
+ else
+ ilo = 0x10 - (-stride / 8);
+ ipu_ch_param_write_field(ch, IPU_FIELD_ILO, ilo);
ipu_ch_param_write_field(ch, IPU_FIELD_SLY, (stride * 2) - 1);
This should be "(-stride * 2) - 1" for SLY when stride is negative.
After fixing that, interweaving seq-bt -> interlaced-bt works fine for
packed pixel formats,
Ouch, thanks.
but there is still some problem for planar.
I haven't tracked down the issue with planar yet,
Just with the negative stride line?
Correct, planar is broken (bad colors in captured frames) when
negative stride is enabled for interweave. Planar works fine otherwise.
Only for YUV420 or also for NV12?
I tested YV12 (three planes YUV420). I can't remember if I tested
NV12 (two planes). I'm currently not able to test as I'm away from
the hardware but I will try on Wednesday.
The problem could be that we also have to change UBO/VBO for planar
formats with a chroma stride (SLUV) that is half the luma stride (SLY)
when we move both Y and U/V frame start by a line length.
Right, and I think I am doing that, by setting image.rect.top = 1
before calling ipu_cpmem_set_image(). And when dequeuing a
new v4l2_buffer, I am adding one luma stride to the buffer address
when calling ipu_cpmem_set_buffer() (grep for interweave_offset).
but the corresponding
changes to imx-media that allow interweaving with line swapping are at
e9dd81da20 ("media: imx: Allow interweave with top/bottom lines swapped")
in branch fix-csi-interlaced.3 in my media-tree fork on github. Please
have a
look and let me know if you see anything obvious.
In commit a19126a80d13 ("gpu: ipu-csi: Swap fields according to
input/output field types"), swap_fields = true is only set for
seq-bt -> seq-tb and seq-tb -> seq-bt. I think it should also be
enabled for alternate -> seq-bt (PAL) and alternate -> seq-tb (NTSC).
It is, see ipu_csi_translate_field() -- it will translate alternate
to seq-bt or seq-tb before determining swap_fields.
I've been made aware [1] that recently V4L2_FIELD_ALTERNATE has been
clarified [2] to specify that v4l2_mbus_fmt.height should contain the
number of lines per field, not per frame:
Yep! That was nagging at me as well. I noticed at least one other
platform (omap3isp) that doubles the source pad frame height
when the sensor reports ALTERNATE field mode, to capture a
whole frame. Makes sense. I think the crop height will need to
be doubled from the sink height in imx-media-csi.c to support
ALTERNATE. That also means sink height can't be used to
guess at input video standard (480 becomes 240 for NTSC).
Steve
[1] https://patchwork.linuxtv.org/patch/50166/
[2] 0018147c964e ("media: v4l: doc: Clarify v4l2_mbus_fmt height
definition")
regards
Philipp
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
Hi Philipp,
On 06/01/2018 06:13 AM, Philipp Zabel wrote:
The IPU also supports interlaced buffers that start with the bottom field.
To achieve this, the the base address EBA has to be increased by a stride
length and the interlace offset ILO has to be set to the negative stride.
Signed-off-by: Philipp Zabel
---
drivers/gpu/ipu-v3/ipu-cpmem.c | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/ipu-v3/ipu-cpmem.c b/drivers/gpu/ipu-v3/ipu-cpmem.c
index 9f2d9ec42add..c1028f38c553 100644
--- a/drivers/gpu/ipu-v3/ipu-cpmem.c
+++ b/drivers/gpu/ipu-v3/ipu-cpmem.c
@@ -269,8 +269,14 @@ EXPORT_SYMBOL_GPL(ipu_cpmem_set_uv_offset);
void ipu_cpmem_interlaced_scan(struct ipuv3_channel *ch, int stride)
{
+ u32 ilo;
+
ipu_ch_param_write_field(ch, IPU_FIELD_SO, 1);
- ipu_ch_param_write_field(ch, IPU_FIELD_ILO, stride / 8);
+ if (stride >= 0)
+ ilo = stride / 8;
+ else
+ ilo = 0x10 - (-stride / 8);
+ ipu_ch_param_write_field(ch, IPU_FIELD_ILO, ilo);
ipu_ch_param_write_field(ch, IPU_FIELD_SLY, (stride * 2) - 1);
This should be "(-stride * 2) - 1" for SLY when stride is negative.
After fixing that, interweaving seq-bt -> interlaced-bt works fine for
packed pixel formats, but there is still some problem for planar.
I haven't tracked down the issue with planar yet, but the corresponding
changes to imx-media that allow interweaving with line swapping are at
e9dd81da20 ("media: imx: Allow interweave with top/bottom lines swapped")
in branch fix-csi-interlaced.3 in my media-tree fork on github. Please
have a
look and let me know if you see anything obvious.
Steve
[PATCH v2] media: i2c: adv748x: csi2: set entity function to video interface bridge
The ADV748x CSI-2 subdevices are HDMI/AFE to MIPI CSI-2 bridges. Signed-off-by: Steve Longerbeam Acked-by: Kieran Bingham --- Changes since v1: - fix typo in commit message. --- drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c index 820b44e..469be87 100644 --- a/drivers/media/i2c/adv748x/adv748x-csi2.c +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx) adv748x_csi2_set_virtual_channel(tx, 0); adv748x_subdev_init(>sd, state, _csi2_ops, - MEDIA_ENT_F_UNKNOWN, + MEDIA_ENT_F_VID_IF_BRIDGE, is_txa(tx) ? "txa" : "txb"); /* Ensure that matching is based upon the endpoint fwnodes */ -- 2.7.4
Re: [PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
On 06/08/2018 02:39 PM, Kieran Bingham wrote: Hi Steve, On 08/06/18 18:43, Steve Longerbeam wrote: The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. Just spotted this :D s/HMDI/HDMI/ Sigh, sending v2. Steve Signed-off-by: Steve Longerbeam --- drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c index 820b44e..469be87 100644 --- a/drivers/media/i2c/adv748x/adv748x-csi2.c +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx) adv748x_csi2_set_virtual_channel(tx, 0); adv748x_subdev_init(>sd, state, _csi2_ops, - MEDIA_ENT_F_UNKNOWN, + MEDIA_ENT_F_VID_IF_BRIDGE, is_txa(tx) ? "txa" : "txb"); /* Ensure that matching is based upon the endpoint fwnodes */
Re: [PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
On 06/08/2018 02:37 PM, Kieran Bingham wrote: Hi Steve, On 08/06/18 22:34, Steve Longerbeam wrote: Hi Kieran, On 06/08/2018 02:29 PM, Kieran Bingham wrote: Hi Steve, Thankyou for the patch. On 08/06/18 18:43, Steve Longerbeam wrote: The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. Reading the documentation for MEDIA_ENT_F_VID_IF_BRIDGE, this seems reasonable. Out of interest, have you stumbled across this as part of your other work on CSI2 drivers - or have you been looking to test the ADV748x with your CSI2 receiver? I'd love to know if the driver works with other (non-renesas) platforms! This isn't really related to my other work on the i.MX CSI2 receiver driver in imx-media. I've only tested this on Renesas (Salvator-X). No problem. I was just curious :D And this will get rid of that annoying warning message that I've been ignoring! Yeah, darn those pesky warnings :) Steve Signed-off-by: Steve Longerbeam Acked-by: Kieran Bingham --- drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c index 820b44e..469be87 100644 --- a/drivers/media/i2c/adv748x/adv748x-csi2.c +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx) adv748x_csi2_set_virtual_channel(tx, 0); adv748x_subdev_init(>sd, state, _csi2_ops, - MEDIA_ENT_F_UNKNOWN, + MEDIA_ENT_F_VID_IF_BRIDGE, is_txa(tx) ? "txa" : "txb"); /* Ensure that matching is based upon the endpoint fwnodes */
Re: [PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
Hi Kieran, On 06/08/2018 02:29 PM, Kieran Bingham wrote: Hi Steve, Thankyou for the patch. On 08/06/18 18:43, Steve Longerbeam wrote: The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. Reading the documentation for MEDIA_ENT_F_VID_IF_BRIDGE, this seems reasonable. Out of interest, have you stumbled across this as part of your other work on CSI2 drivers - or have you been looking to test the ADV748x with your CSI2 receiver? I'd love to know if the driver works with other (non-renesas) platforms! This isn't really related to my other work on the i.MX CSI2 receiver driver in imx-media. I've only tested this on Renesas (Salvator-X). Steve Signed-off-by: Steve Longerbeam Acked-by: Kieran Bingham --- drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c index 820b44e..469be87 100644 --- a/drivers/media/i2c/adv748x/adv748x-csi2.c +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx) adv748x_csi2_set_virtual_channel(tx, 0); adv748x_subdev_init(>sd, state, _csi2_ops, - MEDIA_ENT_F_UNKNOWN, + MEDIA_ENT_F_VID_IF_BRIDGE, is_txa(tx) ? "txa" : "txb"); /* Ensure that matching is based upon the endpoint fwnodes */
[PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. Signed-off-by: Steve Longerbeam --- drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c index 820b44e..469be87 100644 --- a/drivers/media/i2c/adv748x/adv748x-csi2.c +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx) adv748x_csi2_set_virtual_channel(tx, 0); adv748x_subdev_init(>sd, state, _csi2_ops, - MEDIA_ENT_F_UNKNOWN, + MEDIA_ENT_F_VID_IF_BRIDGE, is_txa(tx) ? "txa" : "txb"); /* Ensure that matching is based upon the endpoint fwnodes */ -- 2.7.4
Re: [PATCH v2 04/10] media: imx: interweave only for sequential input/interlaced output fields
Hi Philipp, I think now I understand your interpretation of the v4l2_field enums: SEQ_BT/TB is specifying both field order and field dominance. But the TB/BT in INTERLACED_TB/BT has a different interpretation, in this case it is specifying _only_ field dominance, and top field is first in memory for both cases. I wasn't interpreting it this way. My interpretation was that BT/TB for all field types referred only to field order and does not say anything about field dominance. So in summary: SEQ_BT = bottom field first in memory AND bottom field is dominant (B lines are older lines in time, e.g. recorded first, then T lines). SEQ_TB = top field first in memory AND top field is dominant. INTERLACED_BT = top field first in memory and bottom field is dominant. INTERLACED_TB = top field first in memory and top field is dominant. With that, your explanations below make sense... On 06/06/2018 02:05 AM, Philipp Zabel wrote: Hi Steve, On Tue, 2018-06-05 at 12:00 -0700, Steve Longerbeam wrote: I'm probably misunderstanding you, so at the risk of overexplaining: There is no way we can ever produce a correct interlaced-tb frame in memory from a seq-bt frame output by the CSI, as the interweaving step only has access to a single frame. I don't follow you, yes the interweaving step only has access to a single frame, but why would interweave need access to another frame to carry out seq-bt -> interlaced-tb ? See below... A seq-bt frame has a bottom field (first in memory) with an older timestamp than the top field (second in memory). Without access to a second input frame we can only ever produce an interlaced frame where the bottom lines are older than the top lines, which is interlaced-bt. interlaced-tb requires the top lines to have the older timestamp, and the bottom lines to be newer. Right, in seq-bt the CSI can skip bottom field and capture top field, then bottom field of next frame, which results in flipping field dominance. So seq-bt becomes seq-tb and indeed the top field in seq-tb is now the older (dominant) field, and the bottom field is now newer. Which means we need to span two frames to accomplish seq-bt -> seq-tb in order to flip field dominance. A seq-tb PAL frame has the older top field in lines 0-287 and the newer bottom field in lines 288-576. From that interlaced-tb can be written via 0-287 -> 0,2,4,...,286 and 288-575 -> 1,3,5,...,287 [1]. This is what interweaving does if the interlace offset is set to positive line stride. Right, that was my understanding as well. And how interweave actually works in the IDMAC to achieve the above is : By turning on SO bit in cpmem, the IDMAC will write the first one-half lines of the frame received by the IDMAC channel to memory, starting at the EBA address, with a line stride equal to cpmem SLY. When it completes writing out the first half lines of the frame, the IDMAC begins to write the lines from the second half of frame to memory, but starts again at EBA address, and with an offset equal to cpmem ILO. So by setting SO=1, SLY=2*linestride, and ILO=linestride, we achieve interweave where: lines from first half of frame are written to lines 0,2,4,...,HEIGHT-2 in memory, and lines from the second half of the frame are written to lines 1,3,...,HEIGHT-1 in memory. So this setting achieves seq-bt -> interlaced-bt This is incorrect. Since the bottom field comes first in memory, with this setting the bottom lines are written to where the top lines should be and the top lines end up where the bottom lines should be. Since odd and even lines are switched, this will not produce a correct frame. Yes, if 'interlaced-bt' is interpreted as: bottom field is dominant and top field must be first in memory. or seq-tb -> interlaced-tb, This is correct. Yes, a simple interweave accomplishes this, and since odd/even lines are not switched, interlaced-tb would look correct if displayed. And top is still dominant. e.g. interweave *does not change field order*, bottom lines 1,3,,, are written to 0,2,4,,, in memory if IDMAC receives seq-bt, or top lines 0,2,4,,, are written to 0,2,4,,, in memory if IDMAC receives seq-tb. Of course we can't change field order, we are arguing the same point here. I think our misunderstanding comes from the definition of interlaced-bt [1]: "Images contain both fields, interleaved line by line, top field first. The bottom field is transmitted first." [1] https://linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/field-order.html?highlight=field#enum-v4l2-field The BT or TB part of the v4l2_field enums refer to the order in time, not to the order in memory. Yes BT/TB refers to field dominance for both 'seq-*' and 'interlaced-*'. But TB/BT only refers to field order for the 'seq-*' definitions, in 'interlaced-*' top field is fixed as first field in memory. But I'm still not so sure because of the sentence "The bottom/top field is transmitted first" in interl
Re: [PATCH v2 04/10] media: imx: interweave only for sequential input/interlaced output fields
Hi Philipp, On 06/05/2018 01:07 AM, Philipp Zabel wrote: Hi Steve, On Mon, 2018-06-04 at 17:56 -0700, Steve Longerbeam wrote: On 06/04/2018 01:27 AM, Philipp Zabel wrote: On Mon, 2018-06-04 at 07:35 +0200, Krzysztof Hałasa wrote: Philipp Zabel writes: This is ok in this patch, but we can't use this check in the following TRY_FMT patch as there is no way to interweave SEQ_TB -> INTERLACED_BT (because in SEQ_TB the B field is newer than T, but in INTERLACED_BT it has to be older) or SEQ_BT -> INTERLACED_TB (the other way around). Actually we can do SEQ_TB -> INTERLACED_BT and SEQ_BT -> INTERLACED_TB rather easily. We only need to skip a single field at start :-) That's what CCIR_CODE_* registers do. To be honest, SEQ_TB and SEQ_BT are precisely the same thing (i.e., SEQUENTIAL). It's up to the user to say which field is the first. There is the progressive sensor exception, though, and the TB/BT could be a hint for downstream elements (i.e., setting the default field order). But I think we should be able to request INTERLACED_TB or INTERLACED_BT (with any analog signal on input) and the CCIR_CODE registers should be set accordingly. This should all magically work fine. The CSI subdevice itself can't interweave at all, this is done in the IDMAC. In my opinion the CSI subdev should allow the following src -> sink field transformations for BT.656: none -> none seq-tb -> seq-tb seq-tb -> seq-bt seq-bt -> seq-bt seq-bt -> seq-tb alternate -> seq-tb alternate -> seq-bt interlaced -> interlaced interlaced-tb -> interlaced-tb interlaced-bt -> interlaced-bt The capture video device should then additionally allow selecting the field order that can be produced by IDMAC interweaving: INTERLACED_TB if the pad is seq-tb and INTERLACED_BT if the pad is seq- bt, as that is what the IDMAC can convert. Good idea. This is also in-line with how planar YUV is selected at the capture interface instead of at the CSI/PRPENCVF source pad. Philipp, Krzysztof, please see branch fix-csi-interlaced.3 in my github mediatree fork. I've implemented the above and it works great for both NTSC and PAL sources to the ADV7180. Thanks! I'll have a look. seq-tb -> seq-tb and seq-bt -> seq-bt should always capture field 0 first, as we currently do for PAL. seq->tb -> seq-bt and seq-bt -> seq-tb should always capture field 1 first, as we currently do for NTSC. alternate -> seq-tb and alternate -> seq-bt should match seq-tb -> * for PAL and seq-bt -> * for NTSC. Yes, I had already implemented this idea yesterday, I've added it to branch fix-csi-interlaced.3. The CSI will swap field capture (field 1 first, then field 2, by inverting F bit in CCIR registers) if the field order input to the CSI is different from the requested output field order. Philipp, a word about the idea of using negative ILO line stride and an extra line added to EBA start address, for interweaving. I believe the result of this is to also invert field order when interweaving 'seq-bt/tb', which would produce 'interlaced-tb/bt' in memory. I'm probably misunderstanding you, so at the risk of overexplaining: There is no way we can ever produce a correct interlaced-tb frame in memory from a seq-bt frame output by the CSI, as the interweaving step only has access to a single frame. I don't follow you, yes the interweaving step only has access to a single frame, but why would interweave need access to another frame to carry out seq-bt -> interlaced-tb ? See below... A seq-tb PAL frame has the older top field in lines 0-287 and the newer bottom field in lines 288-576. From that interlaced-tb can be written via 0-287 -> 0,2,4,...,286 and 288-575 -> 1,3,5,...,287 [1]. This is what interweaving does if the interlace offset is set to positive line stride. Right, that was my understanding as well. And how interweave actually works in the IDMAC to achieve the above is : By turning on SO bit in cpmem, the IDMAC will write the first one-half lines of the frame received by the IDMAC channel to memory, starting at the EBA address, with a line stride equal to cpmem SLY. When it completes writing out the first half lines of the frame, the IDMAC begins to write the lines from the second half of frame to memory, but starts again at EBA address, and with an offset equal to cpmem ILO. So by setting SO=1, SLY=2*linestride, and ILO=linestride, we achieve interweave where: lines from first half of frame are written to lines 0,2,4,...,HEIGHT-2 in memory, and lines from the second half of the frame are written to lines 1,3,...,HEIGHT-1 in memory. So this setting achieves seq-bt -> interlaced-bt or seq-tb -> interlaced-tb, e.g. interweave *does not change field order*, bottom lines 1,3,,, are written to 0,2,4,,, in memory if IDMAC receives seq-bt, or top lines 0,2,4,,, are written to 0,2,4,,, in memory if IDMAC receives seq-tb. And by setting SO=1, SLY=2*linestride, ILO=-linest
Re: [PATCH v2 04/10] media: imx: interweave only for sequential input/interlaced output fields
On 06/04/2018 01:27 AM, Philipp Zabel wrote: On Mon, 2018-06-04 at 07:35 +0200, Krzysztof Hałasa wrote: Philipp Zabel writes: This is ok in this patch, but we can't use this check in the following TRY_FMT patch as there is no way to interweave SEQ_TB -> INTERLACED_BT (because in SEQ_TB the B field is newer than T, but in INTERLACED_BT it has to be older) or SEQ_BT -> INTERLACED_TB (the other way around). Actually we can do SEQ_TB -> INTERLACED_BT and SEQ_BT -> INTERLACED_TB rather easily. We only need to skip a single field at start :-) That's what CCIR_CODE_* registers do. To be honest, SEQ_TB and SEQ_BT are precisely the same thing (i.e., SEQUENTIAL). It's up to the user to say which field is the first. There is the progressive sensor exception, though, and the TB/BT could be a hint for downstream elements (i.e., setting the default field order). But I think we should be able to request INTERLACED_TB or INTERLACED_BT (with any analog signal on input) and the CCIR_CODE registers should be set accordingly. This should all magically work fine. The CSI subdevice itself can't interweave at all, this is done in the IDMAC. In my opinion the CSI subdev should allow the following src -> sink field transformations for BT.656: none -> none seq-tb -> seq-tb seq-tb -> seq-bt seq-bt -> seq-bt seq-bt -> seq-tb alternate -> seq-tb alternate -> seq-bt interlaced -> interlaced interlaced-tb -> interlaced-tb interlaced-bt -> interlaced-bt The capture video device should then additionally allow selecting the field order that can be produced by IDMAC interweaving: INTERLACED_TB if the pad is seq-tb and INTERLACED_BT if the pad is seq- bt, as that is what the IDMAC can convert. Good idea. This is also in-line with how planar YUV is selected at the capture interface instead of at the CSI/PRPENCVF source pad. Philipp, Krzysztof, please see branch fix-csi-interlaced.3 in my github mediatree fork. I've implemented the above and it works great for both NTSC and PAL sources to the ADV7180. seq-tb -> seq-tb and seq-bt -> seq-bt should always capture field 0 first, as we currently do for PAL. seq->tb -> seq-bt and seq-bt -> seq-tb should always capture field 1 first, as we currently do for NTSC. alternate -> seq-tb and alternate -> seq-bt should match seq-tb -> * for PAL and seq-bt -> * for NTSC. Yes, I had already implemented this idea yesterday, I've added it to branch fix-csi-interlaced.3. The CSI will swap field capture (field 1 first, then field 2, by inverting F bit in CCIR registers) if the field order input to the CSI is different from the requested output field order. Philipp, a word about the idea of using negative ILO line stride and an extra line added to EBA start address, for interweaving. I believe the result of this is to also invert field order when interweaving 'seq-bt/tb', which would produce 'interlaced-tb/bt' in memory. I don't think this is necessary now, because field order swapping can already be done earlier at the CSI sink->src using the CCIR registers. For example here is a pipeline for an NTSC adv7180 source that swapped NTSC 'seq-bt' (well assumed NTSC 'seq-bt' since adv7180 is 'alternate') to 'seq-tb' at the CSI source pad: 'adv7180 3-0021':0 [fmt:UYVY8_2X8/720x480 field:alternate colorspace:smpte170m] 'ipu1_csi0_mux':1 [fmt:UYVY8_2X8/720x480 field:alternate colorspace:smpte170m] 'ipu1_csi0_mux':2 [fmt:UYVY8_2X8/720x480 field:alternate colorspace:smpte170m] 'ipu1_csi0':0 [fmt:UYVY8_2X8/720x480@1/30 field:alternate ...] crop.bounds:(0,0)/720x480 crop:(0,2)/720x480 compose.bounds:(0,0)/720x480 compose:(0,0)/720x480] 'ipu1_csi0':2 [fmt:AYUV8_1X32/720x480@1/30 field:seq-tb ...] And at the capture interface: # v4l2-ctl -d4 -V Format Video Capture: Width/Height : 720/480 Pixel Format : 'YV12' Field : Interlaced Top-Bottom Bytes per Line : 1440 Size Image : 691200 Colorspace : SMPTE 170M Transfer Function : Rec. 709 YCbCr/HSV Encoding: ITU-R 601 Quantization : Limited Range Flags : So we've accomplished 'seq-bt' -> 'interlaced-tb' without needing to swap field order using the modified interweave idea. I've run tests for both PAL and NTSC inputs to the adv7180 on SabreAuto, and the results are consistent: NTSC seq-bt -> interlaced-tb produces good interweave images as expected NTSC seq-bt -> interlaced-bt produces interweave images with a "mauve" artifact as expected PAL seq-tb -> interlaced-tb produces good interweave images as expected PAL seq-tb -> interlaced-bt produces interweave images with a "mauve" artifact as expected Steve
Re: [PATCH v2 01/10] media: imx-csi: Pass sink pad field to ipu_csi_init_interface
On 06/03/2018 10:25 PM, Krzysztof Hałasa wrote: Steve Longerbeam writes: I think we should return to enforcing field order to userspace that matches field order from the source, which is what I had implemented previously. I agree with you that we should put off allowing inverting field order. There is no any particular field order at the source, most of the time. The odd field is followed by the even field, and so on, sure. But there is no "first" and "second" field, any field can be the "first". I think you misunderstood me. Of course there is a first and second field. By first I am referring to the first field transmitted, which could be top or bottom. The exception to this is a camera with a progressive sensor - both "fields" are taken at the same time and transmitted one after the other, so in this case the order is defined (by the camera, e.g. B-T on DV even with PAL version). But this isn't exactly PAL/NTSC. Progressive sensors have no fields, the entire image is captured at once as you said. Steve
Re: [PATCH v2 10/10] media: imx.rst: Update doc to reflect fixes to interlaced capture
On 06/01/2018 06:44 AM, Philipp Zabel wrote: On Thu, 2018-05-31 at 17:30 -0700, Steve Longerbeam wrote: + +.. code-block:: none + + # Setup links + media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" + # Configure pads + media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]" + media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480 field:interlaced]" Could the example suggest using interlaced-bt to be explicit here? Actually, I don't think we should allow interlaced on the CSI src pads at all in this case. Technically it always writes either seq-tb or seq- bt into the smfc, never interlaced (unless the input is already interlaced). Hmm, if the sink is 'alternate', and the requested source is 'interlaced*', perhaps we should allow the source to be 'interlaced*' and not override it. For example, if requested is 'interlaced-tb', let it be that. IOW assume user knows something we don't about the original field order, or is experimenting with finding the correct field order. Steve
Re: [PATCH v2 10/10] media: imx.rst: Update doc to reflect fixes to interlaced capture
On 06/01/2018 06:44 AM, Philipp Zabel wrote: On Thu, 2018-05-31 at 17:30 -0700, Steve Longerbeam wrote: Also add an example pipeline for unconverted capture with interweave on SabreAuto. Signed-off-by: Steve Longerbeam --- Documentation/media/v4l-drivers/imx.rst | 51 - 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst index 65d3d15..4149b76 100644 --- a/Documentation/media/v4l-drivers/imx.rst +++ b/Documentation/media/v4l-drivers/imx.rst @@ -179,9 +179,10 @@ sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8. If the sink pad is receiving YUV, the output at the capture device can also be converted to a planar YUV format such as YUV420. -It will also perform simple de-interlace without motion compensation, -which is activated if the sink pad's field type is an interlaced type, -and the IDMAC source pad field type is set to none. +It will also perform simple interweave without motion compensation, +which is activated if the sink pad's field type is sequential top-bottom +or bottom-top or alternate, and the IDMAC source pad field type is +interlaced (t-b, b-t, or unqualified interlaced). I think sink pad alternate behaviour should be equal to sink pad top- bottom for PAL and sink pad bottom-top for NTSC. If we agree on this, we should mention that here. Agreed. This subdev can generate the following event when enabling the second IDMAC source pad: @@ -383,13 +384,13 @@ and CSC operations and flip/rotation controls. It will receive and process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is receiving from ipuX_vdic. -Like the ipuX_csiY IDMAC source, it can perform simple de-interlace +Like the ipuX_csiY IDMAC source, it can perform simple interweaving without motion compensation. However, note that if the ipuX_vdic is included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic), -it's not possible to use simple de-interlace in ipuX_ic_prpvf, since -the ipuX_vdic has already carried out de-interlacing (with motion -compensation) and therefore the field type output from ipuX_ic_prp can -only be none. +it's not possible to use interweave in ipuX_ic_prpvf, since the +ipuX_vdic has already carried out de-interlacing (with motion +compensation) and therefore the field type output from ipuX_vdic +can only be none (progressive). Capture Pipelines - @@ -514,10 +515,32 @@ On the SabreAuto, an on-board ADV7180 SD decoder is connected to the parallel bus input on the internal video mux to IPU1 CSI0. The following example configures a pipeline to capture from the ADV7180 +video decoder, assuming NTSC 720x480 input signals, using simple +interweave (unconverted and without motion compensation). The adv7180 +must output sequential or alternating fields (field type 'seq-tb', +'seq-bt', or 'alternate'): + +.. code-block:: none + + # Setup links + media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" + # Configure pads + media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]" + media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480 field:interlaced]" Could the example suggest using interlaced-bt to be explicit here? Actually, I don't think we should allow interlaced on the CSI src pads at all in this case. Technically it always writes either seq-tb or seq- bt into the smfc, never interlaced (unless the input is already interlaced). Agreed, I'll make that change in v3 and update the doc. Steve +Streaming can then begin on the capture device node at +"ipu1_csi0 capture". The v4l2-ctl tool can be used to select any +supported YUV pixelformat on the capture device node. + +This example configures a pipeline to capture from the ADV7180 video decoder, assuming NTSC 720x480 input signals, with Motion -Compensated de-interlacing. Pad field types assume the adv7180 outputs -"interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf -entity at its output pad: +Compensated de-interlacing. The adv7180 must output sequential or +alternating fields (field type 'seq-tb', 'seq-bt', or 'alternate'). +$outputfmt can be any format supported by the ipu1_ic_prpvf entity +at its output pad: .. code-block:: none @@ -529,9 +552,9 @@ entity at its output pad: media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" # Configure pads - media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]" - media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" - me
Re: i.MX6 IPU CSI analog video input on Ventana
On 06/01/2018 06:13 AM, Philipp Zabel wrote:
Hi Krzysztof,
On Fri, 2018-06-01 at 12:02 +0200, Krzysztof Hałasa wrote:
Steve Longerbeam writes:
I tend to agree, I've found conflicting info out there regarding
PAL vs. NTSC field order. And I've never liked having to guess
at input analog standard based on input # lines. I will go ahead
and remove the field order override code.
I've merged your current fix-csi-interlaced.2 branch (2018-06-01
00:06:45 UTC 22ad9f30454b6e46979edf6f8122243591910a3e) along with
"media: adv7180: fix field type" commit and NTSC camera:
media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480 field:alternate]"
media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480]"
media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x480 field:interlaced/-bt/-tb]"
correctly sets:
"adv7180 2-0020":0 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1_mux":1 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1_mux":2 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1":0 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1":2 [fmt:AYUV32/720x480 field:interlaced/-bt/-tb]
but all 3 cases seem to produce top-first interlaced frames.
The CCIR_CODE_* register dump shows no differences:
2a38014: 010D07DF 00040596 00FF
...it's because the code in drivers/gpu/ipu-v3/ipu-csi.c still sets the
registers depending on the height of the image.
Exactly.
Hovewer, I'm using 480
lines here, so it should be B-T instead.
My understanding is that the CCIR codes for height == 480 (NTSC)
currently capture the second field (top) first, assuming that for NTSC
the EAV/SAV codes are bottom-field-first.
So the CSI captures SEQ_TB for both PAL and NTSC: The three-bit values
in CCIR_CODE_2/3 are in H,V,F order, and the NTSC case has F=1 for the
field that is captured first, where F=1 is the field that is marked as
second field on the wire, so top. Which means that the captured frame
has two fields captured across frame boundaries, which might be
problematic if the source data was originally progressive.
I agree, for NTSC the CSI will drop the first B field and start capturing
at the T field, and then capture fields across frame boundaries. At
least, that is if we understand how these CCIR registers work: the
CSI will look for H-S-V codes for the start and end of active and blanking
lines, that match the codes written to CCIR_CODE_1/2 for fields 0/1.
I think this must be legacy code from a Freescale BSP requirement
that the CSI must always capture in T-B order. We should remove this
code, so that the CSI always captures field 0 followed by field 1,
irrespective
of field affinity, as in:
diff --git a/drivers/gpu/ipu-v3/ipu-csi.c b/drivers/gpu/ipu-v3/ipu-csi.c
index 5450a2d..b8b9b6d 100644
--- a/drivers/gpu/ipu-v3/ipu-csi.c
+++ b/drivers/gpu/ipu-v3/ipu-csi.c
@@ -398,41 +398,20 @@ int ipu_csi_init_interface(struct ipu_csi *csi,
break;
case IPU_CSI_CLK_MODE_CCIR656_INTERLACED:
if (mbus_fmt->width == 720 && mbus_fmt->height == 576) {
- /*
- * PAL case
- *
- * Field0BlankEnd = 0x6, Field0BlankStart = 0x2,
- * Field0ActiveEnd = 0x4, Field0ActiveStart = 0
- * Field1BlankEnd = 0x7, Field1BlankStart = 0x3,
- * Field1ActiveEnd = 0x5, Field1ActiveStart = 0x1
- */
height = 625; /* framelines for PAL */
-
- ipu_csi_write(csi, 0x40596 | CSI_CCIR_ERR_DET_EN,
- CSI_CCIR_CODE_1);
- ipu_csi_write(csi, 0xD07DF, CSI_CCIR_CODE_2);
- ipu_csi_write(csi, 0xFF, CSI_CCIR_CODE_3);
} else if (mbus_fmt->width == 720 && mbus_fmt->height
== 480) {
- /*
- * NTSC case
- *
- * Field0BlankEnd = 0x7, Field0BlankStart = 0x3,
- * Field0ActiveEnd = 0x5, Field0ActiveStart = 0x1
- * Field1BlankEnd = 0x6, Field1BlankStart = 0x2,
- * Field1ActiveEnd = 0x4, Field1ActiveStart = 0
- */
height = 525; /* framelines for NTSC */
-
- ipu_csi_write(csi, 0xD07DF | CSI_CCIR_ERR_DET_EN,
- CSI_CCIR_CODE_1);
- ipu_csi_write(csi, 0x40596, CSI_CCIR_CODE_2);
- ipu_csi_write(csi, 0xFF, CSI_CCIR_CODE_3);
} else {
dev_err(csi->ipu->dev,
"Unsupported CCIR656 interlaced video
mode\n");
spin_unlock_irqrestore(>lock, flags);
return -EI
Re: i.MX6 IPU CSI analog video input on Ventana
On 05/30/2018 11:29 PM, Philipp Zabel wrote: On Wed, May 30, 2018 at 01:56:34PM -0700, Steve Longerbeam wrote: On 05/30/2018 11:46 AM, Krzysztof Hałasa wrote: Steve Longerbeam writes: but it should be possible for the user to explicitly request the field order on CSI output (I can make a patch I guess). If you think that is the correct behavior, I will remove the override code. I suppose it makes sense to allow user to select field order even if that order does not make sense given the input standard. I'm fine either way, Philipp what is your opinion? I'll go with the popular vote :) I think it should be up to the user. Actually, PAL and NTSC aren't valid names in the digital world. Their meaning ends in the ADV7180 (or equivalent). I don't know if PAL and/or NTSC specify the field order in the analog frame (meaningful when someone hooks a camera with progressive sensor and analog, interlaced output), but the digital YUV422 from ADV to CSI isn't NTSC/PAL anymore. It's just WxH @ framerate + possible interlacing, sequential fields, top-bottom or otherwise, etc. The analog standard names could be used here but just as defaults. If we were strict (and we don't want to force it), then we should set NTSC/PAL thing on ADV7180 input, 720x480@29.97i (or 720x576@50i, or 704x... etc) on the input parts of the CSI/IPU (where there are no video frames yet), and 720x480@29.97i B-T or T-B (or default, or separate fields - whatever suits the user) on the output from CSI. I remember the reversed field order was sometimes needed - for example, PAL DV (the casette camcorder thing) produced B-T frames (same as NTSC), and to avoid (slight) additional quality loss one had to process it (up to e.g. .MP4, DVD, and then to HDMI, SCART etc) as B-T. It wasn't a problem in otherwise-PAL-centric environment. I tend to agree, I've found conflicting info out there regarding PAL vs. NTSC field order. And I've never liked having to guess at input analog standard based on input # lines. I will go ahead and remove the field order override code. Note that the code in ipu_csi_init_interface currently hard-codes field order depending on frame size. It could be that selecting opposite field order is as easy as switching the relevant parts of writes to registers CCIR_CODE_2 and _3, but we'd have to pass the desired output field order to this function. I'd welcome if somebody would verify that this works. As I said in the other thread, I think we should put this off to some other time, and remove the code in ipu_csi_init_interface() that inverts field order according to frame size. This way, CSI will not be lying to userspace when we tell it the order is BT but the CSI has actually inverted that to TB. Also I have concerns about the CSI capturing field 1 _before_ field 0 for NTSC. Doesn't that mean the CSI will drop the B-field in the first captured frame on stream on, and thereafter mix fields from different adjacent frames? Steve
Re: [PATCH v2 06/10] media: imx: Fix field setting logic in try_fmt
On 06/01/2018 06:34 AM, Philipp Zabel wrote:
On Thu, 2018-05-31 at 17:30 -0700, Steve Longerbeam wrote:
The logic for setting field type in try_fmt at CSI and PRPENCVF
entities wasn't quite right. The behavior should be:
- No restrictions on field type at sink pads (except ANY, which is filled
with current sink pad field by imx_media_fill_default_mbus_fields()).
- At IDMAC output pads, if the caller asks for an interlaced output, and
the input is sequential fields, the IDMAC output channel can accommodate
by interweaving. The CSI can also interweave if input is alternate
fields.
- If final source pad field type is alternate, translate to seq_bt or
seq_tb. But the field order translation was backwards, SD NTSC is BT
order, SD PAL is TB.
Move this logic to new functions csi_try_field() and prp_try_field().
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 +++--
drivers/staging/media/imx/imx-media-csi.c | 50 +
2 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 7e1e0c3..1002eb1 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -833,6 +833,21 @@ static int prp_get_fmt(struct v4l2_subdev *sd,
return ret;
}
+static void prp_try_field(struct prp_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct v4l2_mbus_framefmt *infmt =
+ __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD, sdformat->which);
+
+ /* no restrictions on sink pad field type */
+ if (sdformat->pad == PRPENCVF_SINK_PAD)
+ return;
+
+ if (!idmac_interweave(sdformat->format.field, infmt->field))
+ sdformat->format.field = infmt->field;
This is not strict enough. As I wrote in reply to patch 4, we can only
do SEQ_TB -> INTERLACED_TB and SEQ_BT -> INTERLACED_BT interweaving.
Agreed.
Steve
Re: [PATCH v2 04/10] media: imx: interweave only for sequential input/interlaced output fields
On 06/01/2018 06:33 AM, Philipp Zabel wrote:
Hi Steve,
On Thu, 2018-05-31 at 17:30 -0700, Steve Longerbeam wrote:
IDMAC interlaced scan, a.k.a. interweave, should be enabled at the
IDMAC output pads only if the input field type is 'seq-bt' or 'seq-tb',
and the IDMAC output pad field type is 'interlaced*'. Move this
determination to a new macro idmac_interweave().
V4L2_FIELD_HAS_BOTH() macro should not be used on the input to determine
enabling interlaced/interweave scan. That macro includes the 'interlaced'
field types, and in those cases the data is already interweaved with
top/bottom field lines.
The CSI will capture whole frames when the source specifies alternate
field mode. So the CSI also enables interweave at the IDMAC output pad
for alternate input field type.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 ++
drivers/staging/media/imx/imx-media-csi.c | 22 ++
2 files changed, 36 insertions(+), 8 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index ae453fd..894db21 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -132,6 +132,18 @@ static inline struct prp_priv *sd_to_priv(struct
v4l2_subdev *sd)
return ic_priv->task_priv;
}
+/*
+ * If the field type at IDMAC output pad is interlaced, and
+ * the input is sequential fields, the IDMAC output channel
+ * can accommodate by interweaving.
+ */
+static inline bool idmac_interweave(enum v4l2_field outfield,
+ enum v4l2_field infield)
+{
+ return V4L2_FIELD_IS_INTERLACED(outfield) &&
+ V4L2_FIELD_IS_SEQUENTIAL(infield);
+}
This is ok in this patch, but we can't use this check in the following
TRY_FMT patch as there is no way to interweave
SEQ_TB -> INTERLACED_BT (because in SEQ_TB the B field is newer than T,
but in INTERLACED_BT it has to be older) or SEQ_BT -> INTERLACED_TB (the
other way around).
Agreed, let's enforce userspace field order to same order from
the originating source.
Steve
+
static void prp_put_ipu_resources(struct prp_priv *priv)
{
if (priv->ic)
@@ -353,6 +365,7 @@ static int prp_setup_channel(struct prp_priv *priv,
struct v4l2_mbus_framefmt *infmt;
unsigned int burst_size;
struct ipu_image image;
+ bool interweave;
int ret;
infmt = >format_mbus[PRPENCVF_SINK_PAD];
@@ -365,6 +378,9 @@ static int prp_setup_channel(struct prp_priv *priv,
image.rect.width = image.pix.width;
image.rect.height = image.pix.height;
+ interweave = (idmac_interweave(image.pix.field, infmt->field) &&
+ channel == priv->out_ch);
+
if (rot_swap_width_height) {
swap(image.pix.width, image.pix.height);
swap(image.rect.width, image.rect.height);
@@ -405,9 +421,7 @@ static int prp_setup_channel(struct prp_priv *priv,
if (rot_mode)
ipu_cpmem_set_rotation(channel, rot_mode);
- if (image.pix.field == V4L2_FIELD_NONE &&
- V4L2_FIELD_HAS_BOTH(infmt->field) &&
- channel == priv->out_ch)
+ if (interweave)
ipu_cpmem_interlaced_scan(channel, image.pix.bytesperline);
This only works for SEQ_TB -> INTERLACED_TB interweave.
For SEQ_BT -> INTERLACED_BT we have to start at +1 line offset and set
ipu_cpmem_interlaced_scan to -image.pix.bytesperline.
regards
Philipp
Re: [PATCH v2 01/10] media: imx-csi: Pass sink pad field to ipu_csi_init_interface
On 06/01/2018 06:22 AM, Philipp Zabel wrote:
On Thu, 2018-05-31 at 17:30 -0700, Steve Longerbeam wrote:
The output pad's field type was being passed to ipu_csi_init_interface(),
in order to deal with field type 'alternate' at the sink pad, which
is not understood by ipu_csi_init_interface().
Remove that code and pass the sink pad field to ipu_csi_init_interface().
The latter function will have to explicity deal with field type 'alternate'
when setting up the CSI interface for BT.656 busses.
I fear this won't be enough. If we want to capture
sink:ALTERNATE/SEQ_TB/SEQ_BT -> src:SEQ_TB we have to configure the CSI
differently than if we want to capture
ALTERNATE/SEQ_TB/SEQ_BT -> src:SEQ_BT. (And differently for NTSC and
PAL). For NTSC sink:ALTERNATE should behave like sink:SEQ_BT, and for
PAL sink:ALTERNATE should behave like sink:SEQ_TB.
I think we should return to enforcing field order to userspace that
matches field order from the source, which is what I had implemented
previously. I agree with you that we should put off allowing inverting
field order.
Interweaving SEQ_TB to INTERLACED_TB should work right now, but to
interweave SEQ_BT to INTERLACED_BT, we need to add one line offset to
the frame start and use a negative interlaced scanline offset.
Is that because ipu_csi_init_interface() is inverting the F-bit for
NTSC? I think we should remove that code, I will comment on
that in another thread.
Steve
Reported-by: Krzysztof Hałasa
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 13 ++---
1 file changed, 2 insertions(+), 11 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 95d7805..9bc555c 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -629,12 +629,10 @@ static void csi_idmac_stop(struct csi_priv *priv)
/* Update the CSI whole sensor and active windows */
static int csi_setup(struct csi_priv *priv)
{
- struct v4l2_mbus_framefmt *infmt, *outfmt;
+ struct v4l2_mbus_framefmt *infmt;
struct v4l2_mbus_config mbus_cfg;
- struct v4l2_mbus_framefmt if_fmt;
infmt = >format_mbus[CSI_SINK_PAD];
- outfmt = >format_mbus[priv->active_output_pad];
/* compose mbus_config from the upstream endpoint */
mbus_cfg.type = priv->upstream_ep.bus_type;
@@ -642,20 +640,13 @@ static int csi_setup(struct csi_priv *priv)
priv->upstream_ep.bus.mipi_csi2.flags :
priv->upstream_ep.bus.parallel.flags;
- /*
-* we need to pass input frame to CSI interface, but
-* with translated field type from output format
-*/
- if_fmt = *infmt;
- if_fmt.field = outfmt->field;
-
ipu_csi_set_window(priv->csi, >crop);
ipu_csi_set_downsize(priv->csi,
priv->crop.width == 2 * priv->compose.width,
priv->crop.height == 2 * priv->compose.height);
- ipu_csi_init_interface(priv->csi, _cfg, _fmt);
+ ipu_csi_init_interface(priv->csi, _cfg, infmt);
ipu_csi_set_dest(priv->csi, priv->dest);
[PATCH v2 02/10] gpu: ipu-csi: Check for field type alternate
When the CSI is receiving from a bt.656 bus, include a check for field type 'alternate' when determining whether to set CSI clock mode to CCIR656_INTERLACED or CCIR656_PROGRESSIVE. Signed-off-by: Steve Longerbeam --- drivers/gpu/ipu-v3/ipu-csi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/ipu-v3/ipu-csi.c b/drivers/gpu/ipu-v3/ipu-csi.c index caa05b0..5450a2d 100644 --- a/drivers/gpu/ipu-v3/ipu-csi.c +++ b/drivers/gpu/ipu-v3/ipu-csi.c @@ -339,7 +339,8 @@ static void fill_csi_bus_cfg(struct ipu_csi_bus_config *csicfg, break; case V4L2_MBUS_BT656: csicfg->ext_vsync = 0; - if (V4L2_FIELD_HAS_BOTH(mbus_fmt->field)) + if (V4L2_FIELD_HAS_BOTH(mbus_fmt->field) || + mbus_fmt->field == V4L2_FIELD_ALTERNATE) csicfg->clk_mode = IPU_CSI_CLK_MODE_CCIR656_INTERLACED; else csicfg->clk_mode = IPU_CSI_CLK_MODE_CCIR656_PROGRESSIVE; -- 2.7.4
[PATCH v2 06/10] media: imx: Fix field setting logic in try_fmt
The logic for setting field type in try_fmt at CSI and PRPENCVF
entities wasn't quite right. The behavior should be:
- No restrictions on field type at sink pads (except ANY, which is filled
with current sink pad field by imx_media_fill_default_mbus_fields()).
- At IDMAC output pads, if the caller asks for an interlaced output, and
the input is sequential fields, the IDMAC output channel can accommodate
by interweaving. The CSI can also interweave if input is alternate
fields.
- If final source pad field type is alternate, translate to seq_bt or
seq_tb. But the field order translation was backwards, SD NTSC is BT
order, SD PAL is TB.
Move this logic to new functions csi_try_field() and prp_try_field().
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 +++--
drivers/staging/media/imx/imx-media-csi.c | 50 +
2 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 7e1e0c3..1002eb1 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -833,6 +833,21 @@ static int prp_get_fmt(struct v4l2_subdev *sd,
return ret;
}
+static void prp_try_field(struct prp_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct v4l2_mbus_framefmt *infmt =
+ __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD, sdformat->which);
+
+ /* no restrictions on sink pad field type */
+ if (sdformat->pad == PRPENCVF_SINK_PAD)
+ return;
+
+ if (!idmac_interweave(sdformat->format.field, infmt->field))
+ sdformat->format.field = infmt->field;
+}
+
static void prp_try_fmt(struct prp_priv *priv,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat,
@@ -852,12 +867,11 @@ static void prp_try_fmt(struct prp_priv *priv,
infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD, sdformat->which);
if (sdformat->pad == PRPENCVF_SRC_PAD) {
- if (!V4L2_FIELD_IS_INTERLACED(sdformat->format.field))
- sdformat->format.field = infmt->field;
-
prp_bound_align_output(>format, infmt,
priv->rot_mode);
+ prp_try_field(priv, cfg, sdformat);
+
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
@@ -870,6 +884,8 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
+ prp_try_field(priv, cfg, sdformat);
+
imx_media_fill_default_mbus_fields(>format, infmt,
true);
}
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 6a2a47c..6101e2ed 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1272,6 +1272,38 @@ static int csi_get_fmt(struct v4l2_subdev *sd,
return ret;
}
+static void csi_try_field(struct csi_priv *priv,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sdformat)
+{
+ struct v4l2_mbus_framefmt *infmt =
+ __csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
+
+ switch (sdformat->pad) {
+ case CSI_SRC_PAD_DIRECT:
+ sdformat->format.field = infmt->field;
+ break;
+ case CSI_SRC_PAD_IDMAC:
+ if (!idmac_interweave(sdformat->format.field, infmt->field))
+ sdformat->format.field = infmt->field;
+ break;
+ case CSI_SINK_PAD:
+ /* no restrictions on sink pad field type */
+ return;
+ }
+
+ /*
+* This driver does not support alternate field mode, and the
+* CSI captures a whole frame, so translate ALTERNATE at either
+* source pad to SEQ_TB or SEQ_BT depending on input height
+* (assume NTSC bt order if 480 lines, otherwise PAL tb order).
+*/
+ if (sdformat->format.field == V4L2_FIELD_ALTERNATE) {
+ sdformat->format.field = (infmt->height == 480) ?
+ V4L2_FIELD_SEQ_BT : V4L2_FIELD_SEQ_TB;
+ }
+}
+
static void csi_try_fmt(struct csi_priv *priv,
struct v4l2_fwnode_endpoint *upstream_ep,
struct v4l2_subdev_pad_config *cfg,
@@ -1311,25 +1343,14 @@ static void csi_try_fmt(struct csi_priv
[PATCH v2 05/10] media: imx: interweave and odd-chroma-row skip are incompatible
If IDMAC interweaving is enabled in a write channel, the channel must
write the odd chroma rows for 4:2:0 formats. Skipping writing the odd
chroma rows produces corrupted captured 4:2:0 images when interweave
is enabled.
Reported-by: Krzysztof Hałasa
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 9 +++--
drivers/staging/media/imx/imx-media-csi.c | 8 ++--
2 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 894db21..7e1e0c3 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -393,12 +393,17 @@ static int prp_setup_channel(struct prp_priv *priv,
image.phys0 = addr0;
image.phys1 = addr1;
- if (channel == priv->out_ch || channel == priv->rot_out_ch) {
+ /*
+* Skip writing U and V components to odd rows in the output
+* channels for planar 4:2:0 (but not when enabling IDMAC
+* interweaving, they are incompatible).
+*/
+ if (!interweave && (channel == priv->out_ch ||
+ channel == priv->rot_out_ch)) {
switch (image.pix.pixelformat) {
case V4L2_PIX_FMT_YUV420:
case V4L2_PIX_FMT_YVU420:
case V4L2_PIX_FMT_NV12:
- /* Skip writing U and V components to odd rows */
ipu_cpmem_skip_odd_chroma_rows(channel);
break;
}
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 2c77ef9..6a2a47c 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -436,8 +436,12 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
((image.pix.width & 0xf) ? 8 : 16) : 32) : 64;
passthrough = is_parallel_16bit_bus(>upstream_ep);
passthrough_bits = 16;
- /* Skip writing U and V components to odd rows */
- ipu_cpmem_skip_odd_chroma_rows(priv->idmac_ch);
+ /*
+* Skip writing U and V components to odd rows (but not
+* when enabling IDMAC interweaving, they are incompatible).
+*/
+ if (!interweave)
+ ipu_cpmem_skip_odd_chroma_rows(priv->idmac_ch);
break;
case V4L2_PIX_FMT_YUYV:
case V4L2_PIX_FMT_UYVY:
--
2.7.4
[PATCH v2 07/10] media: imx-csi: Allow skipping odd chroma rows for YVU420
Skip writing U/V components to odd rows for YVU420 in addition to YUV420 and NV12. Signed-off-by: Steve Longerbeam --- drivers/staging/media/imx/imx-media-csi.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c index 6101e2ed..c878a00 100644 --- a/drivers/staging/media/imx/imx-media-csi.c +++ b/drivers/staging/media/imx/imx-media-csi.c @@ -430,6 +430,7 @@ static int csi_idmac_setup_channel(struct csi_priv *priv) passthrough_bits = 16; break; case V4L2_PIX_FMT_YUV420: + case V4L2_PIX_FMT_YVU420: case V4L2_PIX_FMT_NV12: burst_size = (image.pix.width & 0x3f) ? ((image.pix.width & 0x1f) ? -- 2.7.4
[PATCH v2 08/10] media: imx: vdic: rely on VDIC for correct field order
prepare_vdi_in_buffers() was setting up the dma pointers as if the
VDIC is always programmed to receive the fields in bottom-top order,
i.e. as if ipu_vdi_set_field_order() only programs BT order in the VDIC.
But that's not true, ipu_vdi_set_field_order() is working correctly.
So fix prepare_vdi_in_buffers() to give the VDIC the fields in whatever
order they were received by the video source, and rely on the VDIC to
sort out which is top and which is bottom.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-vdic.c | 12 ++--
1 file changed, 2 insertions(+), 10 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-vdic.c
b/drivers/staging/media/imx/imx-media-vdic.c
index 482250d..4a89071 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -219,26 +219,18 @@ static void __maybe_unused prepare_vdi_in_buffers(struct
vdic_priv *priv,
switch (priv->fieldtype) {
case V4L2_FIELD_SEQ_TB:
- prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0);
- curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + fs;
- next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
- break;
case V4L2_FIELD_SEQ_BT:
prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0) + fs;
curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + fs;
break;
+ case V4L2_FIELD_INTERLACED_TB:
case V4L2_FIELD_INTERLACED_BT:
+ case V4L2_FIELD_INTERLACED:
prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0) + is;
curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
break;
- default:
- /* assume V4L2_FIELD_INTERLACED_TB */
- prev_phys = vb2_dma_contig_plane_dma_addr(prev_vb, 0);
- curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
- next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
- break;
}
ipu_cpmem_set_buffer(priv->vdi_in_ch_p, 0, prev_phys);
--
2.7.4
[PATCH v2 10/10] media: imx.rst: Update doc to reflect fixes to interlaced capture
Also add an example pipeline for unconverted capture with interweave on SabreAuto. Signed-off-by: Steve Longerbeam --- Documentation/media/v4l-drivers/imx.rst | 51 - 1 file changed, 37 insertions(+), 14 deletions(-) diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst index 65d3d15..4149b76 100644 --- a/Documentation/media/v4l-drivers/imx.rst +++ b/Documentation/media/v4l-drivers/imx.rst @@ -179,9 +179,10 @@ sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8. If the sink pad is receiving YUV, the output at the capture device can also be converted to a planar YUV format such as YUV420. -It will also perform simple de-interlace without motion compensation, -which is activated if the sink pad's field type is an interlaced type, -and the IDMAC source pad field type is set to none. +It will also perform simple interweave without motion compensation, +which is activated if the sink pad's field type is sequential top-bottom +or bottom-top or alternate, and the IDMAC source pad field type is +interlaced (t-b, b-t, or unqualified interlaced). This subdev can generate the following event when enabling the second IDMAC source pad: @@ -383,13 +384,13 @@ and CSC operations and flip/rotation controls. It will receive and process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is receiving from ipuX_vdic. -Like the ipuX_csiY IDMAC source, it can perform simple de-interlace +Like the ipuX_csiY IDMAC source, it can perform simple interweaving without motion compensation. However, note that if the ipuX_vdic is included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic), -it's not possible to use simple de-interlace in ipuX_ic_prpvf, since -the ipuX_vdic has already carried out de-interlacing (with motion -compensation) and therefore the field type output from ipuX_ic_prp can -only be none. +it's not possible to use interweave in ipuX_ic_prpvf, since the +ipuX_vdic has already carried out de-interlacing (with motion +compensation) and therefore the field type output from ipuX_vdic +can only be none (progressive). Capture Pipelines - @@ -514,10 +515,32 @@ On the SabreAuto, an on-board ADV7180 SD decoder is connected to the parallel bus input on the internal video mux to IPU1 CSI0. The following example configures a pipeline to capture from the ADV7180 +video decoder, assuming NTSC 720x480 input signals, using simple +interweave (unconverted and without motion compensation). The adv7180 +must output sequential or alternating fields (field type 'seq-tb', +'seq-bt', or 'alternate'): + +.. code-block:: none + + # Setup links + media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" + media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" + media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" + # Configure pads + media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]" + media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480 field:interlaced]" + +Streaming can then begin on the capture device node at +"ipu1_csi0 capture". The v4l2-ctl tool can be used to select any +supported YUV pixelformat on the capture device node. + +This example configures a pipeline to capture from the ADV7180 video decoder, assuming NTSC 720x480 input signals, with Motion -Compensated de-interlacing. Pad field types assume the adv7180 outputs -"interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf -entity at its output pad: +Compensated de-interlacing. The adv7180 must output sequential or +alternating fields (field type 'seq-tb', 'seq-bt', or 'alternate'). +$outputfmt can be any format supported by the ipu1_ic_prpvf entity +at its output pad: .. code-block:: none @@ -529,9 +552,9 @@ entity at its output pad: media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" # Configure pads - media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]" - media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" - media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]" + media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" + media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480]" + media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]" -- 2.7.4
[PATCH v2 09/10] media: imx-csi: Move crop/compose reset after filling default mbus fields
If caller passes un-initialized field type V4L2_FIELD_ANY to CSI
sink pad, the reset CSI crop window would not be correct, because
the crop window depends on the current input field type. To fix move
the reset of crop and compose windows to after the call to
imx_media_fill_default_mbus_fields().
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 22 +++---
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index c878a00..471130a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1358,17 +1358,6 @@ static void csi_try_fmt(struct csi_priv *priv,
W_ALIGN, >format.height,
MIN_H, MAX_H, H_ALIGN, S_ALIGN);
- /* Reset crop and compose rectangles */
- crop->left = 0;
- crop->top = 0;
- crop->width = sdformat->format.width;
- crop->height = sdformat->format.height;
- csi_try_crop(priv, crop, cfg, >format, upstream_ep);
- compose->left = 0;
- compose->top = 0;
- compose->width = crop->width;
- compose->height = crop->height;
-
*cc = imx_media_find_mbus_format(sdformat->format.code,
CS_SEL_ANY, true);
if (!*cc) {
@@ -1385,6 +1374,17 @@ static void csi_try_fmt(struct csi_priv *priv,
>format, infmt,
priv->active_output_pad == CSI_SRC_PAD_DIRECT);
+ /* Reset crop and compose rectangles */
+ crop->left = 0;
+ crop->top = 0;
+ crop->width = sdformat->format.width;
+ crop->height = sdformat->format.height;
+ csi_try_crop(priv, crop, cfg, >format, upstream_ep);
+ compose->left = 0;
+ compose->top = 0;
+ compose->width = crop->width;
+ compose->height = crop->height;
+
break;
}
}
--
2.7.4
[PATCH v2 04/10] media: imx: interweave only for sequential input/interlaced output fields
IDMAC interlaced scan, a.k.a. interweave, should be enabled at the
IDMAC output pads only if the input field type is 'seq-bt' or 'seq-tb',
and the IDMAC output pad field type is 'interlaced*'. Move this
determination to a new macro idmac_interweave().
V4L2_FIELD_HAS_BOTH() macro should not be used on the input to determine
enabling interlaced/interweave scan. That macro includes the 'interlaced'
field types, and in those cases the data is already interweaved with
top/bottom field lines.
The CSI will capture whole frames when the source specifies alternate
field mode. So the CSI also enables interweave at the IDMAC output pad
for alternate input field type.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 ++
drivers/staging/media/imx/imx-media-csi.c | 22 ++
2 files changed, 36 insertions(+), 8 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index ae453fd..894db21 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -132,6 +132,18 @@ static inline struct prp_priv *sd_to_priv(struct
v4l2_subdev *sd)
return ic_priv->task_priv;
}
+/*
+ * If the field type at IDMAC output pad is interlaced, and
+ * the input is sequential fields, the IDMAC output channel
+ * can accommodate by interweaving.
+ */
+static inline bool idmac_interweave(enum v4l2_field outfield,
+ enum v4l2_field infield)
+{
+ return V4L2_FIELD_IS_INTERLACED(outfield) &&
+ V4L2_FIELD_IS_SEQUENTIAL(infield);
+}
+
static void prp_put_ipu_resources(struct prp_priv *priv)
{
if (priv->ic)
@@ -353,6 +365,7 @@ static int prp_setup_channel(struct prp_priv *priv,
struct v4l2_mbus_framefmt *infmt;
unsigned int burst_size;
struct ipu_image image;
+ bool interweave;
int ret;
infmt = >format_mbus[PRPENCVF_SINK_PAD];
@@ -365,6 +378,9 @@ static int prp_setup_channel(struct prp_priv *priv,
image.rect.width = image.pix.width;
image.rect.height = image.pix.height;
+ interweave = (idmac_interweave(image.pix.field, infmt->field) &&
+ channel == priv->out_ch);
+
if (rot_swap_width_height) {
swap(image.pix.width, image.pix.height);
swap(image.rect.width, image.rect.height);
@@ -405,9 +421,7 @@ static int prp_setup_channel(struct prp_priv *priv,
if (rot_mode)
ipu_cpmem_set_rotation(channel, rot_mode);
- if (image.pix.field == V4L2_FIELD_NONE &&
- V4L2_FIELD_HAS_BOTH(infmt->field) &&
- channel == priv->out_ch)
+ if (interweave)
ipu_cpmem_interlaced_scan(channel, image.pix.bytesperline);
ret = ipu_ic_task_idma_init(priv->ic, channel,
@@ -833,7 +847,7 @@ static void prp_try_fmt(struct prp_priv *priv,
infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD, sdformat->which);
if (sdformat->pad == PRPENCVF_SRC_PAD) {
- if (sdformat->format.field != V4L2_FIELD_NONE)
+ if (!V4L2_FIELD_IS_INTERLACED(sdformat->format.field))
sdformat->format.field = infmt->field;
prp_bound_align_output(>format, infmt,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 9bc555c..2c77ef9 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -128,6 +128,19 @@ static inline bool is_parallel_16bit_bus(struct
v4l2_fwnode_endpoint *ep)
}
/*
+ * If the field type at IDMAC output pad is interlaced, and
+ * the input is sequential or alternating fields, the IDMAC
+ * output channel can accommodate by interweaving.
+ */
+static inline bool idmac_interweave(enum v4l2_field outfield,
+ enum v4l2_field infield)
+{
+ return V4L2_FIELD_IS_INTERLACED(outfield) &&
+ (V4L2_FIELD_IS_SEQUENTIAL(infield) ||
+infield == V4L2_FIELD_ALTERNATE);
+}
+
+/*
* Parses the fwnode endpoint from the source pad of the entity
* connected to this CSI. This will either be the entity directly
* upstream from the CSI-2 receiver, or directly upstream from the
@@ -368,10 +381,10 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
{
struct imx_media_video_dev *vdev = priv->vdev;
struct v4l2_mbus_framefmt *infmt;
+ bool passthrough, interweave;
struct ipu_image image;
u32 passthrough_bits;
dma_addr_t phys[2];
- bool passthrough;
u32 burst_size;
int ret;
@@ -389,6 +402,8 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
image.phys0 = phys[0];
image.phys1 = phys[1];
+ int
[PATCH v2 03/10] media: videodev2.h: Add macros V4L2_FIELD_IS_{INTERLACED|SEQUENTIAL}
Adds two helper macros:
Add a macro that returns true if the given field type is 'sequential',
that is a full frame is transmitted, or exists in memory, as all top
field lines followed by all bottom field lines, or vice-versa.
Add a macro that returns true if the given field type is 'interlaced',
that is a full frame is transmitted, or exists in memory, as top field
lines interlaced with bottom field lines.
Signed-off-by: Steve Longerbeam
---
Changes since v1:
- add the complement macro V4L2_FIELD_IS_INTERLACED
- remove V4L2_FIELD_ALTERNATE from V4L2_FIELD_IS_SEQUENTIAL macro.
- moved new macros past end of existing V4L2_FIELD_HAS_* macros.
---
include/uapi/linux/videodev2.h | 7 +++
1 file changed, 7 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877b..d4e12f4 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -130,6 +130,13 @@ enum v4l2_field {
((field) == V4L2_FIELD_BOTTOM ||\
(field) == V4L2_FIELD_TOP ||\
(field) == V4L2_FIELD_ALTERNATE)
+#define V4L2_FIELD_IS_INTERLACED(field) \
+ ((field) == V4L2_FIELD_INTERLACED ||\
+(field) == V4L2_FIELD_INTERLACED_TB ||\
+(field) == V4L2_FIELD_INTERLACED_BT)
+#define V4L2_FIELD_IS_SEQUENTIAL(field) \
+ ((field) == V4L2_FIELD_SEQ_TB ||\
+(field) == V4L2_FIELD_SEQ_BT)
enum v4l2_buf_type {
V4L2_BUF_TYPE_VIDEO_CAPTURE= 1,
--
2.7.4
[PATCH v2 01/10] media: imx-csi: Pass sink pad field to ipu_csi_init_interface
The output pad's field type was being passed to ipu_csi_init_interface(),
in order to deal with field type 'alternate' at the sink pad, which
is not understood by ipu_csi_init_interface().
Remove that code and pass the sink pad field to ipu_csi_init_interface().
The latter function will have to explicity deal with field type 'alternate'
when setting up the CSI interface for BT.656 busses.
Reported-by: Krzysztof Hałasa
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 13 ++---
1 file changed, 2 insertions(+), 11 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 95d7805..9bc555c 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -629,12 +629,10 @@ static void csi_idmac_stop(struct csi_priv *priv)
/* Update the CSI whole sensor and active windows */
static int csi_setup(struct csi_priv *priv)
{
- struct v4l2_mbus_framefmt *infmt, *outfmt;
+ struct v4l2_mbus_framefmt *infmt;
struct v4l2_mbus_config mbus_cfg;
- struct v4l2_mbus_framefmt if_fmt;
infmt = >format_mbus[CSI_SINK_PAD];
- outfmt = >format_mbus[priv->active_output_pad];
/* compose mbus_config from the upstream endpoint */
mbus_cfg.type = priv->upstream_ep.bus_type;
@@ -642,20 +640,13 @@ static int csi_setup(struct csi_priv *priv)
priv->upstream_ep.bus.mipi_csi2.flags :
priv->upstream_ep.bus.parallel.flags;
- /*
-* we need to pass input frame to CSI interface, but
-* with translated field type from output format
-*/
- if_fmt = *infmt;
- if_fmt.field = outfmt->field;
-
ipu_csi_set_window(priv->csi, >crop);
ipu_csi_set_downsize(priv->csi,
priv->crop.width == 2 * priv->compose.width,
priv->crop.height == 2 * priv->compose.height);
- ipu_csi_init_interface(priv->csi, _cfg, _fmt);
+ ipu_csi_init_interface(priv->csi, _cfg, infmt);
ipu_csi_set_dest(priv->csi, priv->dest);
--
2.7.4
[PATCH v2 00/10] imx-media: Fixes for interlaced capture
A set of patches that fixes some bugs with capturing from an
interlaced source, and incompatibilites between IDMAC interlace
interweaving and 4:2:0 data write reduction.
History:
v2:
- update media driver doc.
- enable idmac interweave only if input field is sequential/alternate,
and output field is 'interlaced*'.
- move field try logic out of *try_fmt and into separate function.
- fix bug with resetting crop/compose rectangles.
- add a patch that fixes a field order bug in VDIC indirect mode.
- remove alternate field type from V4L2_FIELD_IS_SEQUENTIAL() macro
Suggested-by: Nicolas Dufresne .
- add macro V4L2_FIELD_IS_INTERLACED().
Steve Longerbeam (10):
media: imx-csi: Pass sink pad field to ipu_csi_init_interface
gpu: ipu-csi: Check for field type alternate
media: videodev2.h: Add macros V4L2_FIELD_IS_{INTERLACED|SEQUENTIAL}
media: imx: interweave only for sequential input/interlaced output
fields
media: imx: interweave and odd-chroma-row skip are incompatible
media: imx: Fix field setting logic in try_fmt
media: imx-csi: Allow skipping odd chroma rows for YVU420
media: imx: vdic: rely on VDIC for correct field order
media: imx-csi: Move crop/compose reset after filling default mbus
fields
media: imx.rst: Update doc to reflect fixes to interlaced capture
Documentation/media/v4l-drivers/imx.rst | 51 +
drivers/gpu/ipu-v3/ipu-csi.c| 3 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 51 +++--
drivers/staging/media/imx/imx-media-csi.c | 114 ++--
drivers/staging/media/imx/imx-media-vdic.c | 12 +--
include/uapi/linux/videodev2.h | 7 ++
6 files changed, 165 insertions(+), 73 deletions(-)
--
2.7.4
Re: i.MX6 IPU CSI analog video input on Ventana
On 05/30/2018 11:46 AM, Krzysztof Hałasa wrote: Steve Longerbeam writes: but it should be possible for the user to explicitly request the field order on CSI output (I can make a patch I guess). If you think that is the correct behavior, I will remove the override code. I suppose it makes sense to allow user to select field order even if that order does not make sense given the input standard. I'm fine either way, Philipp what is your opinion? I'll go with the popular vote :) I think it should be up to the user. Actually, PAL and NTSC aren't valid names in the digital world. Their meaning ends in the ADV7180 (or equivalent). I don't know if PAL and/or NTSC specify the field order in the analog frame (meaningful when someone hooks a camera with progressive sensor and analog, interlaced output), but the digital YUV422 from ADV to CSI isn't NTSC/PAL anymore. It's just WxH @ framerate + possible interlacing, sequential fields, top-bottom or otherwise, etc. The analog standard names could be used here but just as defaults. If we were strict (and we don't want to force it), then we should set NTSC/PAL thing on ADV7180 input, 720x480@29.97i (or 720x576@50i, or 704x... etc) on the input parts of the CSI/IPU (where there are no video frames yet), and 720x480@29.97i B-T or T-B (or default, or separate fields - whatever suits the user) on the output from CSI. I remember the reversed field order was sometimes needed - for example, PAL DV (the casette camcorder thing) produced B-T frames (same as NTSC), and to avoid (slight) additional quality loss one had to process it (up to e.g. .MP4, DVD, and then to HDMI, SCART etc) as B-T. It wasn't a problem in otherwise-PAL-centric environment. I tend to agree, I've found conflicting info out there regarding PAL vs. NTSC field order. And I've never liked having to guess at input analog standard based on input # lines. I will go ahead and remove the field order override code. Steve
Re: i.MX6 IPU CSI analog video input on Ventana
Hi Krzysztof,
On 05/30/2018 01:53 AM, Krzysztof Hałasa wrote:
Steve Longerbeam writes:
Yes, you'll need to patch adv7180.c to select either
'seq-bt/tb' or 'alternate'. The current version will override
any attempt to set field to anything other than 'interlaced'.
This is in anticipation of getting a patch merged for adv7180
that fixes this.
Right. I've applied the patch from your adv718x-v6 branch (just the
"media: adv7180: fix field type" patch) and now it works.
Also, I have changed "seq-bt" to "alternate" (in the examples in
Documentation/media/v4l-drivers/imx.rst) - the data stream from ADV7180
to CSI consists of separate fields which can then be merged into frames
in any order requested by the user (e.g. in accordance with "digital PAL
/ NTSC" requirements).
The following:
media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480 field:alternate]"
media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480]"
media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x480 field:interlaced]"
now produces:
"adv7180 2-0020":0 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1_mux":1 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1_mux":2 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1":0 [fmt:UYVY2X8/720x480 field:alternate]
"ipu2_csi1":2 [fmt:AYUV32/720x480 field:interlaced-bt]
and it works correctly.
The only issue is that I can't:
media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x480 field:interlaced-tb]"
(it remains fixed in -bt mode since NTSC is the default). I think we may
set TB/BT by default (depending on CSI input geometry or TV standard),
Yes, that's what I've implemented. If the user requests an interlaced
field type ('interlaced', 'interlaced-bt', 'interlaced-tb'), but the field
order is not correct given the input height (480=NTSC, 576=PAL),
then the request field type is overridden with the correct field order.
but it should be possible for the user to explicitly request the field
order on CSI output (I can make a patch I guess).
If you think that is the correct behavior, I will remove the override
code. I suppose it makes sense to allow user to select field order even
if that order does not make sense given the input standard. I'm fine
either way, Philipp what is your opinion? I'll go with the popular vote :)
Steve
Re: i.MX6 IPU CSI analog video input on Ventana
Hi Krzysztof, On 05/29/2018 12:26 AM, Krzysztof Hałasa wrote: Hi Steve, Steve Longerbeam writes: Krzysztof, in the meantime the patches are available in my media-tree fork, for testing on the Ventana GW5300: g...@github.com:slongerbeam/mediatree.git, branch 'fix-csi-interlaced' I assume fix-csi-interlaced.2 is a newer version, isn't it? I merged it and I think I can't set the correct config: media-ctl -r -l '"adv7180 2-0020":0->"ipu2_csi1_mux":1[1], "ipu2_csi1_mux":2->"ipu2_csi1":0[1], "ipu2_csi1":2->"ipu2_csi1 capture":0[1]' media-ctl -V "'adv7180 2-0020':0 [fmt:UYVY2X8/720x480 field:seq-bt]" media-ctl -V "'ipu2_csi1_mux':2 [fmt:UYVY2X8/720x480]" media-ctl -V "'ipu2_csi1':2 [fmt:AYUV32/720x480 field:interlaced]" produces: "adv7180 2-0020":0 [fmt:UYVY2X8/720x480 field:interlaced] "ipu2_csi1_mux":1 [fmt:UYVY2X8/720x480 field:interlaced] "ipu2_csi1_mux":2 [fmt:UYVY2X8/720x480 field:interlaced] "ipu2_csi1":0 [fmt:UYVY2X8/720x480 field:interlaced] "ipu2_csi1":2 [fmt:AYUV32/720x480 field:interlaced] Do I need to patch ADV7180 for field type "sequential"? Yes, you'll need to patch adv7180.c to select either 'seq-bt/tb' or 'alternate'. The current version will override any attempt to set field to anything other than 'interlaced'. This is in anticipation of getting a patch merged for adv7180 that fixes this. Steve
Re: [PATCH 4/6] media: imx-csi: Enable interlaced scan for field type alternate
On 05/28/2018 12:59 AM, Ian Arkver wrote: On 28/05/18 08:00, Philipp Zabel wrote: On Fri, 2018-05-25 at 16:53 -0700, Steve Longerbeam wrote: Interlaced scan, a.k.a. interweave, should be enabled at the CSI IDMAC output pad if the input field type is 'alternate' (in addition to field types 'seq-tb' and 'seq-bt'). Which brings up whether V4L2_FIELD_HAS_BOTH() macro should be used to determine enabling interlaced/interweave scan. That macro includes the 'interlaced' field types, and in those cases the data is already interweaved with top/bottom field lines. A heads-up for now that this if statement may need to call V4L2_FIELD_IS_SEQUENTIAL() instead, I have no sensor hardware that sends 'interlaced' data, so can't test. I agree, the check here should be IS_SEQUENTIAL || ALTERNATE, and interlaced_scan should also be enabled if image.pix.field is INTERLACED_TB/BT. Yep, in fact I have implemented that in v2. Interlace_scan will be enabled only if input field type is SEQUENTIAL || ALTERNATE, and output field type is INTERLACED_TB/BT or non-qualified INTERLACED. And for INTERLACED_TB/BT input, the logic should be inverted: then we'd have to enable interlaced_scan whenever image.pix.field is SEQ_BT/TB. Hi Philipp, If your intent here is to de-interweave the two fields back to two sequential fields, I don't believe the IDMAC operation would achieve that. It's basically line stride doubling and a line offset for the lines in the 2nd half of the incoming frame, right? I agree, I'm not sure interlaced_scan will perform the inverse (turn an interlaced buffer into a sequential buffer). If the implementation involves a scan of two lines, second line with a memory offset equal to field size, and doubling the line stride to reach the next set of two lines, then running that operation on an interlaced buffer would not produce a sequential buffer. Steve Signed-off-by: Steve Longerbeam <steve_longerb...@mentor.com> --- drivers/staging/media/imx/imx-media-csi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c index 9bc555c..eef3483 100644 --- a/drivers/staging/media/imx/imx-media-csi.c +++ b/drivers/staging/media/imx/imx-media-csi.c @@ -477,7 +477,8 @@ static int csi_idmac_setup_channel(struct csi_priv *priv) ipu_smfc_set_burstsize(priv->smfc, burst_size); if (image.pix.field == V4L2_FIELD_NONE && - V4L2_FIELD_HAS_BOTH(infmt->field)) + (V4L2_FIELD_HAS_BOTH(infmt->field) || + infmt->field == V4L2_FIELD_ALTERNATE)) ipu_cpmem_interlaced_scan(priv->idmac_ch, image.pix.bytesperline);
Re: [PATCH 3/6] media: videodev2.h: Add macro V4L2_FIELD_IS_SEQUENTIAL
Hi Nicolas,
On 05/25/2018 06:21 PM, Nicolas Dufresne wrote:
Le vendredi 25 mai 2018 à 21:14 -0400, Nicolas Dufresne a écrit :
Le vendredi 25 mai 2018 à 17:19 -0700, Steve Longerbeam a écrit :
On 05/25/2018 05:10 PM, Nicolas Dufresne wrote:
(in text this time, sorry)
Le vendredi 25 mai 2018 à 16:53 -0700, Steve Longerbeam a écrit :
Add a macro that returns true if the given field type is
'sequential',
that is, the data is transmitted, or exists in memory, as all top
field
lines followed by all bottom field lines, or vice-versa.
Signed-off-by: Steve Longerbeam <steve_longerb...@mentor.com>
---
include/uapi/linux/videodev2.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
index 600877b..408ee96 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -126,6 +126,10 @@ enum v4l2_field {
(field) == V4L2_FIELD_INTERLACED_BT ||\
(field) == V4L2_FIELD_SEQ_TB ||\
(field) == V4L2_FIELD_SEQ_BT)
+#define V4L2_FIELD_IS_SEQUENTIAL(field) \
+ ((field) == V4L2_FIELD_SEQ_TB ||\
+(field) == V4L2_FIELD_SEQ_BT ||\
+(field) == V4L2_FIELD_ALTERNATE)
No, alternate has no place here, in alternate mode each buffers have
only one field.
Then I misunderstand what is meant by "alternate". The name implies
to me that a source sends top or bottom field alternately, or top/bottom
fields exist in memory buffers alternately, but with no information about
which field came first. In other words, "alternate" is either seq-tb or
seq-bt,
without any info about field order.
If it is just one field in a memory buffer, why isn't it called
V4L2_FIELD_TOP_OR_BOTTOM, e.g. we don't know which?
I don't see why this could be better then ALTERNATE, were buffers are
only top or bottom fields alternatively. And even if there was another
possible name, this is public API.
V4L2_FIELD_ALTERNATE is a mode, that will only be used with
v4l2_pix_format or v4l2_pix_format_mplane. I should never bet set on
the v4l2_buffer.field, instead the driver indicates the parity of the
field by setting V42_FIELD_TOP/BOTTOM on the v4l2_buffer returned by
DQBUF. This is a very different mode of operation compared to
sequential, hence why I believe it is wrong to make it part of the new
helper. So far, it's the only field value that has this asymmetric
usage and meaning.
I should have put some references. The explanation of the modes, with a
temporal representation of the fields. Small note, in ALTERNATE mode
bottom and top fields will likely not share the same timestamp, it is a
mode used to achieve lower latency.
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/field-order.html#c.v4l2_field
And in this section, you'll see a paragraph that explain the field
values when running in ALTERNATE mode.
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/buffer.html#c.v4l2_buffer
Ok, thanks for clarifying. I suppose it makes sense that this mode
should not
be seen as sequential, since it represents a mode where each userspace
buffer
is a single, self-contained field. I'll remove ALTERNATE from the helper
macro.
Steve
Re: [PATCH 3/6] media: videodev2.h: Add macro V4L2_FIELD_IS_SEQUENTIAL
On 05/25/2018 05:10 PM, Nicolas Dufresne wrote:
(in text this time, sorry)
Le vendredi 25 mai 2018 à 16:53 -0700, Steve Longerbeam a écrit :
Add a macro that returns true if the given field type is
'sequential',
that is, the data is transmitted, or exists in memory, as all top
field
lines followed by all bottom field lines, or vice-versa.
Signed-off-by: Steve Longerbeam <steve_longerb...@mentor.com>
---
include/uapi/linux/videodev2.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
index 600877b..408ee96 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -126,6 +126,10 @@ enum v4l2_field {
(field) == V4L2_FIELD_INTERLACED_BT ||\
(field) == V4L2_FIELD_SEQ_TB ||\
(field) == V4L2_FIELD_SEQ_BT)
+#define V4L2_FIELD_IS_SEQUENTIAL(field) \
+ ((field) == V4L2_FIELD_SEQ_TB ||\
+(field) == V4L2_FIELD_SEQ_BT ||\
+(field) == V4L2_FIELD_ALTERNATE)
No, alternate has no place here, in alternate mode each buffers have
only one field.
Then I misunderstand what is meant by "alternate". The name implies
to me that a source sends top or bottom field alternately, or top/bottom
fields exist in memory buffers alternately, but with no information about
which field came first. In other words, "alternate" is either seq-tb or
seq-bt,
without any info about field order.
If it is just one field in a memory buffer, why isn't it called
V4L2_FIELD_TOP_OR_BOTTOM, e.g. we don't know which?
Steve
[PATCH 3/6] media: videodev2.h: Add macro V4L2_FIELD_IS_SEQUENTIAL
Add a macro that returns true if the given field type is 'sequential',
that is, the data is transmitted, or exists in memory, as all top field
lines followed by all bottom field lines, or vice-versa.
Signed-off-by: Steve Longerbeam <steve_longerb...@mentor.com>
---
include/uapi/linux/videodev2.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877b..408ee96 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -126,6 +126,10 @@ enum v4l2_field {
(field) == V4L2_FIELD_INTERLACED_BT ||\
(field) == V4L2_FIELD_SEQ_TB ||\
(field) == V4L2_FIELD_SEQ_BT)
+#define V4L2_FIELD_IS_SEQUENTIAL(field) \
+ ((field) == V4L2_FIELD_SEQ_TB ||\
+(field) == V4L2_FIELD_SEQ_BT ||\
+(field) == V4L2_FIELD_ALTERNATE)
#define V4L2_FIELD_HAS_T_OR_B(field) \
((field) == V4L2_FIELD_BOTTOM ||\
(field) == V4L2_FIELD_TOP ||\
--
2.7.4
