Re: [PATCH 1/3] v4l2-subdev: Add subdev ioctl support for ENUM/GET/SET INPUT
Hi Laurent, all,
On 6/24/20 7:01 PM, Laurent Pinchart wrote:
Hi Jacopo,
On Wed, Jun 24, 2020 at 09:53:07AM +0200, Jacopo Mondi wrote:
On Tue, Jun 16, 2020 at 12:00:15PM +0200, Ramzi BEN MEFTAH wrote:
From: Steve Longerbeam
+Niklas, +Laurent
Niklas, Laurent, how does this play with CAP_IO_MC ?
I don't think it's related to CAP_IO_MC, but I don't think it's a good
idea either :-) Routing doesn't go through the subdev [gs]_input
operations in MC-based drivers. It should be configured through link
setup instead. This patch goes in the wrong direction, sorry Steve.
That's OK! :) I didn't submit this patch, and as stated in the commit
header, I never recommended this patch be submitted to upstream in the
first place.
Selecting inputs at a subdev should normally make use of media link
setup. But for selecting analog signal inputs, such as the ADV748x AFE
standard definition inputs, that would mean there would need to exist
source "analog" subdevs that connect to the AFE inputs, if only for the
purpose of selecting those inputs, which seems silly IMHO. The ADV748x
AFE subdev defines these inputs as media pads, but have no connections,
so link setup API can't be used to select those inputs.
So a new subdev pad API is clearly needed, not just to get input status
at a subdev pad, but to select/make active such analog inputs without
requiring link setup API.
Steve
This commit enables VIDIOC_ENUMINPUT, VIDIOC_G_INPUT, and VIDIOC_S_INPUT
ioctls for use via v4l2 subdevice node.
This commit should probably not be pushed upstream, because the (old)
idea of video inputs conflicts with the newer concept of establishing
media links between src->sink pads.
However it might make sense for some subdevices to support enum/get/set
inputs. One example would be the analog front end subdevice for the
ADV748x. By providing these ioctls, selecting the ADV748x analog inputs
can be done without requiring the implementation of media entities that
would define the analog source for which to establish a media link.
Signed-off-by: Steve Longerbeam
---
drivers/media/v4l2-core/v4l2-subdev.c | 9 +
include/media/v4l2-subdev.h | 11 +++
2 files changed, 20 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c
b/drivers/media/v4l2-core/v4l2-subdev.c
index 6b989fe..73fbfe9 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -378,6 +378,15 @@ static long subdev_do_ioctl(struct file *file, unsigned
int cmd, void *arg)
return -ENOTTY;
return v4l2_querymenu(vfh->ctrl_handler, arg);
+ case VIDIOC_ENUMINPUT:
+ return v4l2_subdev_call(sd, video, enuminput, arg);
+
+ case VIDIOC_G_INPUT:
+ return v4l2_subdev_call(sd, video, g_input, arg);
+
+ case VIDIOC_S_INPUT:
+ return v4l2_subdev_call(sd, video, s_input, *(u32 *)arg);
+
case VIDIOC_G_CTRL:
if (!vfh->ctrl_handler)
return -ENOTTY;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index f7fe78a..6e1a9cd 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -383,6 +383,14 @@ struct v4l2_mbus_frame_desc {
* @g_input_status: get input status. Same as the status field in the
* _input
*
+ * @enuminput: enumerate inputs. Should return the same input status as
+ * @g_input_status if the passed input index is the currently active
+ * input.
+ *
+ * @g_input: returns the currently active input index.
+ *
+ * @s_input: set the active input.
+ *
* @s_stream: used to notify the driver that a video stream will start or has
*stopped.
*
@@ -423,6 +431,9 @@ struct v4l2_subdev_video_ops {
int (*g_tvnorms)(struct v4l2_subdev *sd, v4l2_std_id *std);
int (*g_tvnorms_output)(struct v4l2_subdev *sd, v4l2_std_id *std);
int (*g_input_status)(struct v4l2_subdev *sd, u32 *status);
+ int (*enuminput)(struct v4l2_subdev *sd, struct v4l2_input *input);
+ int (*g_input)(struct v4l2_subdev *sd, u32 *index);
+ int (*s_input)(struct v4l2_subdev *sd, u32 index);
int (*s_stream)(struct v4l2_subdev *sd, int enable);
int (*g_pixelaspect)(struct v4l2_subdev *sd, struct v4l2_fract *aspect);
int (*g_frame_interval)(struct v4l2_subdev *sd,
Re: [PATCH] media: rcar-vin: Move media_device_register to async completion
Hi Niklas, Michael, On 6/17/20 8:28 AM, Niklas Söderlund wrote: Hi Michael, On 2020-06-17 17:15:37 +0200, Michael Rodin wrote: Hi Niklas and Steve, On Wed, Jun 17, 2020 at 12:56:46PM +0200, Niklas Söderlund wrote: Hi Michael and Steve, On 2020-06-16 19:31:36 +0200, Michael Rodin wrote: From: Steve Longerbeam The media_device is registered during driver probe, before async completion, so it is possible for .link_notify to be called before all devices are bound. Fix this by moving media_device_register() to rvin_group_notify_complete(). This ensures that all devices are now bound (the rcar-csi2 subdevices and and video capture devices) before .link_notify can be called. I'm curious to what situation created the need for this change. I'm currently trying to take the VIN driver in the opposite direction [1] with the end goal of registering video devices at probe time and then allow the media graph to populate as devices becomes available. It looks like almost all platform drivers call media_device_register() in the completion callback. From my understaning it is necessary to ensure that all subdevices are bound and all links are created before the user can enable any link (which would trigger link_notify callback execution) and set formats. If I am not mistaken, Steve could observe an "OOPS" or at least it is theoretically possible. If an OOPS have been observed I would be interested to see it. That way we can fix the OOPS and keep the media graph registration where it is today. It's been a long time since I looked at this on a Salvator-X. But I do remember there was an OOPS if an attempt was made to enable a link before all devices had bound. Unfortunately I no longer have access to RCAR h/w to prove this point. Actually I found that this patch alone is not enough even if it is correct, because we also have to register the media device in rvin_parallel_notify_complete() in case if there is only a parallel video input device attached. My reason for this is that we could have a functional pipeline inside the graph even if it's not complete. This came out of the GMSL work done a while pack where I had a faulty camera that would prevent the other 7 in the system to function. I agree that if a probe of a faulty subdevice fails, this should not affect functionality of the other attached subdevices. The "complete" callback of the async notifier is probably not executed in this case, so I guess, we would have to register the media device in the "bound" callback after the first subdevice has been probed? Otherwise there is not much sense to have video capture devices, which are not connected to any source. Calling it in the bound callback is mostly the same as it is today, as link_notify could then be called when not all entities are in the graph. In fact even if we where tp move the media device registration to the t complete callback we have this problem if any of the subdevices are unbound. Then we are back to the state with a registerd media device where not all entities are present. I think the solution here is to address the issue (if any) in the link_notify callback when the graph is not fully populated. I like the idea of allowing a functional pipeline without all devices bound. So I understand the desire to keep media device registration in probe. So in that case I agree link_notify should be fixed if it still needs fixing to handle non-bound subdevices. Steve (Delayed) population of the media graph after media device registration sounds also like a requirement for device tree overlay support, which would also be a nice feature. 1. [PATCH 0/5] media-device: Report if graph is complete Signed-off-by: Steve Longerbeam Signed-off-by: Michael Rodin --- drivers/media/platform/rcar-vin/rcar-core.c | 14 ++ 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/media/platform/rcar-vin/rcar-core.c b/drivers/media/platform/rcar-vin/rcar-core.c index 7440c89..e70f83b 100644 --- a/drivers/media/platform/rcar-vin/rcar-core.c +++ b/drivers/media/platform/rcar-vin/rcar-core.c @@ -253,7 +253,6 @@ static int rvin_group_init(struct rvin_group *group, struct rvin_dev *vin) struct media_device *mdev = >mdev; const struct of_device_id *match; struct device_node *np; - int ret; mutex_init(>lock); @@ -266,7 +265,6 @@ static int rvin_group_init(struct rvin_group *group, struct rvin_dev *vin) vin_dbg(vin, "found %u enabled VIN's in DT", group->count); mdev->dev = vin->dev; - mdev->ops = _media_ops; match = of_match_node(vin->dev->driver->of_match_table, vin->dev->of_node); @@ -278,11 +276,7 @@ static int rvin_group_init(struct rvin_group *group, struct rvin_dev *vin) media_device_init(mdev); - ret = media_device_register(>mdev); - if (ret) - rvin_group_
Re: [PATCH v8 5/5] media: imx: Try colorimetry at both sink and source pads
Hi Laurent, Rui,
Besides field type ANY, the imx7 CSI should probably be dealing with
other field type conversions as well. I may be mistaken, but like the
imx6, the imx7 does not have the ability to detect whether a captured
field is a top field or a bottom field, so it can't support ALTERNATE
mode. It should probably capture two consecutive fields and in that case
and report seq-tb or seq-bt at its output. Also the imx6 supports
interlacing field lines, if that is the case for imx7 it should also
support converting ALTERNATE to INTERLACED at its output.
Steve
On 10/22/19 6:34 AM, Rui Miguel Silva wrote:
Hi Laurent,
On Tue 22 Oct 2019 at 02:44, Laurent Pinchart wrote:
Hi Steve,
On Tue, May 21, 2019 at 06:03:17PM -0700, Steve Longerbeam wrote:
Retask imx_media_fill_default_mbus_fields() to try colorimetry parameters,
renaming it to to imx_media_try_colorimetry(), and call it at both sink and
source pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.
The IC now supports Rec.709 and BT.601 Y'CbCr encoding, and both limited
and full range quantization for both YUV and RGB space, so allow those
for pipelines that route through the IC.
Signed-off-by: Steve Longerbeam
---
Changes in v7:
- squashed with "media: imx: Allow Rec.709 encoding for IC routes".
- remove the RGB full-range quantization restriction for IC routes.
---
drivers/staging/media/imx/imx-ic-prp.c | 6 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 8 +--
drivers/staging/media/imx/imx-media-csi.c | 19 +++---
drivers/staging/media/imx/imx-media-utils.c | 73 ++---
drivers/staging/media/imx/imx-media-vdic.c | 5 +-
drivers/staging/media/imx/imx-media.h | 5 +-
drivers/staging/media/imx/imx7-media-csi.c | 8 +--
7 files changed, 62 insertions(+), 62 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 10ffe00f1a54..f87fe0203720 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -193,8 +193,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
sdformat->format.code = cc->codes[0];
}
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
@@ -203,6 +203,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
break;
}
+ imx_media_try_colorimetry(>format, true);
+
fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
*fmt = sdformat->format;
out:
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index e8b36a181ccc..f2fe3c11c70e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -907,8 +907,6 @@ static void prp_try_fmt(struct prp_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
@@ -916,9 +914,11 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
}
+
+ imx_media_try_colorimetry(>format, true);
}
static int prp_set_fmt(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 1d248aca40a9..dce4addadff4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1375,9 +1375,15 @@ static void csi_try_field(struct csi_priv *priv,
struct v4l2_mbus_framefmt *infmt =
__csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
- /* no restrictions on sink pad field type */
- if (sdformat->pad == CSI_SINK_PAD)
+ /*
+* no restrictions on sink pad field type except must
+* be initialized.
+*/
+ if (sdformat->pad == CSI_SINK_PAD) {
+ if (sdf
Re: [PATCH v8 5/5] media: imx: Try colorimetry at both sink and source pads
On 10/21/19 8:26 PM, Steve Longerbeam wrote:
Hi Laurent,
On 10/21/19 6:44 PM, Laurent Pinchart wrote:
Hi Steve,
On Tue, May 21, 2019 at 06:03:17PM -0700, Steve Longerbeam wrote:
Retask imx_media_fill_default_mbus_fields() to try colorimetry
parameters,
renaming it to to imx_media_try_colorimetry(), and call it at both
sink and
source pad try_fmt's. The unrelated check for uninitialized field
value is
moved out to appropriate places in each subdev try_fmt.
The IC now supports Rec.709 and BT.601 Y'CbCr encoding, and both
limited
and full range quantization for both YUV and RGB space, so allow those
for pipelines that route through the IC.
Signed-off-by: Steve Longerbeam
---
Changes in v7:
- squashed with "media: imx: Allow Rec.709 encoding for IC routes".
- remove the RGB full-range quantization restriction for IC routes.
---
drivers/staging/media/imx/imx-ic-prp.c | 6 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 8 +--
drivers/staging/media/imx/imx-media-csi.c | 19 +++---
drivers/staging/media/imx/imx-media-utils.c | 73
++---
drivers/staging/media/imx/imx-media-vdic.c | 5 +-
drivers/staging/media/imx/imx-media.h | 5 +-
drivers/staging/media/imx/imx7-media-csi.c | 8 +--
7 files changed, 62 insertions(+), 62 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 10ffe00f1a54..f87fe0203720 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -193,8 +193,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
sdformat->format.code = cc->codes[0];
}
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
@@ -203,6 +203,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
break;
}
+ imx_media_try_colorimetry(>format, true);
+
fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
*fmt = sdformat->format;
out:
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index e8b36a181ccc..f2fe3c11c70e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -907,8 +907,6 @@ static void prp_try_fmt(struct prp_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
@@ -916,9 +914,11 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
}
+
+ imx_media_try_colorimetry(>format, true);
}
static int prp_set_fmt(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 1d248aca40a9..dce4addadff4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1375,9 +1375,15 @@ static void csi_try_field(struct csi_priv *priv,
struct v4l2_mbus_framefmt *infmt =
__csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
- /* no restrictions on sink pad field type */
- if (sdformat->pad == CSI_SINK_PAD)
+ /*
+ * no restrictions on sink pad field type except must
+ * be initialized.
+ */
+ if (sdformat->pad == CSI_SINK_PAD) {
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
return;
+ }
switch (infmt->field) {
case V4L2_FIELD_SEQ_TB:
@@ -1455,8 +1461,6 @@ static void csi_try_fmt(struct csi_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
break;
case CSI_SINK_PAD:
@@ -1476,10 +1480,6 @@ static void csi_try_fmt(struct csi_priv *priv,
csi_try_field(priv, cfg, sdformat);
- imx_media_fill_default_mbus_fields(
- >format, infmt,
- priv->active_output_pad == CSI_SRC_PA
Re: [PATCH v8 5/5] media: imx: Try colorimetry at both sink and source pads
Hi Laurent,
On 10/21/19 6:44 PM, Laurent Pinchart wrote:
Hi Steve,
On Tue, May 21, 2019 at 06:03:17PM -0700, Steve Longerbeam wrote:
Retask imx_media_fill_default_mbus_fields() to try colorimetry parameters,
renaming it to to imx_media_try_colorimetry(), and call it at both sink and
source pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.
The IC now supports Rec.709 and BT.601 Y'CbCr encoding, and both limited
and full range quantization for both YUV and RGB space, so allow those
for pipelines that route through the IC.
Signed-off-by: Steve Longerbeam
---
Changes in v7:
- squashed with "media: imx: Allow Rec.709 encoding for IC routes".
- remove the RGB full-range quantization restriction for IC routes.
---
drivers/staging/media/imx/imx-ic-prp.c | 6 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 8 +--
drivers/staging/media/imx/imx-media-csi.c | 19 +++---
drivers/staging/media/imx/imx-media-utils.c | 73 ++---
drivers/staging/media/imx/imx-media-vdic.c | 5 +-
drivers/staging/media/imx/imx-media.h | 5 +-
drivers/staging/media/imx/imx7-media-csi.c | 8 +--
7 files changed, 62 insertions(+), 62 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 10ffe00f1a54..f87fe0203720 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -193,8 +193,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
sdformat->format.code = cc->codes[0];
}
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
@@ -203,6 +203,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
break;
}
+ imx_media_try_colorimetry(>format, true);
+
fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
*fmt = sdformat->format;
out:
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index e8b36a181ccc..f2fe3c11c70e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -907,8 +907,6 @@ static void prp_try_fmt(struct prp_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
@@ -916,9 +914,11 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
}
+
+ imx_media_try_colorimetry(>format, true);
}
static int prp_set_fmt(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 1d248aca40a9..dce4addadff4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1375,9 +1375,15 @@ static void csi_try_field(struct csi_priv *priv,
struct v4l2_mbus_framefmt *infmt =
__csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
- /* no restrictions on sink pad field type */
- if (sdformat->pad == CSI_SINK_PAD)
+ /*
+* no restrictions on sink pad field type except must
+* be initialized.
+*/
+ if (sdformat->pad == CSI_SINK_PAD) {
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
return;
+ }
switch (infmt->field) {
case V4L2_FIELD_SEQ_TB:
@@ -1455,8 +1461,6 @@ static void csi_try_fmt(struct csi_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
break;
case CSI_SI
[PATCH 1/2] clocksource/drivers/imx: add input capture support
This patch adds support for the input capture function in the
i.MX GPT. Output compare and input capture functions are mixed
in the same register block, so we need to modify the irq ack/enable/
disable primitives to not stomp on the other function.
The input capture API is modelled after request/free irq:
typedef void (*mxc_icap_handler_t)(int, void *, ktime_t);
int mxc_request_input_capture(unsigned int chan,
mxc_icap_handler_t handler,
unsigned long capflags, void *dev_id);
- chan: the channel number being requested (0 or 1).
- handler: a callback when there is an input capture event. The
handler is given the channel number, the dev_id, and a ktime_t
marking the input capture event.
- capflags: IRQF_TRIGGER_RISING and/or IRQF_TRIGGER_FALLING. If
both are specified, events will be triggered on both rising and
falling edges of the input capture signal.
- dev_id: a context pointer given back to the handler.
void mxc_free_input_capture(unsigned int chan, void *dev_id);
This disables the given input capture channel in the GPT.
Signed-off-by: Steve Longerbeam
---
drivers/clocksource/timer-imx-gpt.c | 479 +---
include/linux/mxc_icap.h| 16 +
2 files changed, 445 insertions(+), 50 deletions(-)
create mode 100644 include/linux/mxc_icap.h
diff --git a/drivers/clocksource/timer-imx-gpt.c
b/drivers/clocksource/timer-imx-gpt.c
index 706c0d0ff56c..200777e4f104 100644
--- a/drivers/clocksource/timer-imx-gpt.c
+++ b/drivers/clocksource/timer-imx-gpt.c
@@ -5,9 +5,11 @@
// Copyright (C) 2006-2007 Pavel Pisa (pp...@pikron.com)
// Copyright (C) 2008 Juergen Beisert (ker...@pengutronix.de)
+#include
#include
#include
#include
+#include
#include
#include
#include
@@ -16,6 +18,8 @@
#include
#include
#include
+#include
+#include
#include
/*
@@ -49,16 +53,52 @@
#define V2_TCTL_CLK_PER(2 << 6)
#define V2_TCTL_CLK_OSC_DIV8 (5 << 6)
#define V2_TCTL_FRR(1 << 9)
+#define V2_TCTL_IM1_BIT16
+#define V2_TCTL_IM2_BIT18
+#define V2_IM_DISABLE 0
+#define V2_IM_RISING 1
+#define V2_IM_FALLING 2
+#define V2_IM_BOTH 3
#define V2_TCTL_24MEN (1 << 10)
#define V2_TPRER_PRE24M12
#define V2_IR 0x0c
+#define V2_IR_OF1 (1 << 0)
+#define V2_IR_IF1 (1 << 3)
+#define V2_IR_IF2 (1 << 4)
#define V2_TSTAT 0x08
#define V2_TSTAT_OF1 (1 << 0)
+#define V2_TSTAT_IF1 (1 << 3)
+#define V2_TSTAT_IF2 (1 << 4)
#define V2_TCN 0x24
#define V2_TCMP0x10
+#define V2_TCAP1 0x1c
+#define V2_TCAP2 0x20
#define V2_TIMER_RATE_OSC_DIV8 300
+struct imx_timer;
+
+struct icap_channel {
+ struct imx_timer *imxtm;
+
+ int chan;
+
+ u32 cnt_reg;
+ u32 irqen_bit;
+ u32 status_bit;
+ u32 mode_bit;
+
+ mxc_icap_handler_t handler;
+ void *dev_id;
+
+ struct cyclecounter cc;
+ struct timecounter tc;
+};
+
+/* FIXME, for now can't find icap unless it's statically allocated */
+static struct icap_channel icap_channel[2];
+static DEFINE_SPINLOCK(icap_lock);
+
struct imx_timer {
enum imx_gpt_type type;
void __iomem *base;
@@ -74,12 +114,20 @@ struct imx_gpt_data {
int reg_tstat;
int reg_tcn;
int reg_tcmp;
- void (*gpt_setup_tctl)(struct imx_timer *imxtm);
- void (*gpt_irq_enable)(struct imx_timer *imxtm);
- void (*gpt_irq_disable)(struct imx_timer *imxtm);
- void (*gpt_irq_acknowledge)(struct imx_timer *imxtm);
+ void (*gpt_oc_setup_tctl)(struct imx_timer *imxtm);
+ void (*gpt_oc_irq_enable)(struct imx_timer *imxtm);
+ void (*gpt_oc_irq_disable)(struct imx_timer *imxtm);
+ void (*gpt_oc_irq_acknowledge)(struct imx_timer *imxtm);
+ bool (*gpt_is_oc_irq)(struct imx_timer *imxtm, unsigned int tstat);
int (*set_next_event)(unsigned long evt,
struct clock_event_device *ced);
+
+ void (*gpt_ic_irq_enable)(struct icap_channel *ic);
+ void (*gpt_ic_irq_disable)(struct icap_channel *ic);
+ void (*gpt_ic_irq_acknowledge)(struct icap_channel *ic);
+ bool (*gpt_is_ic_irq)(struct icap_channel *ic, unsigned int tstat);
+ void (*gpt_ic_enable)(struct icap_channel *ic, unsigned int mode);
+ void (*gpt_ic_disable)(struct icap_channel *ic);
};
static inline struct imx_timer *to_imx_timer(struct clock_event_device *ced)
@@ -87,52 +135,144 @@ static inline struct imx_timer *to_imx_timer(struct
clock_event_device *ced)
return container_of(ced, struct imx_timer, ced);
}
-static void imx1_gpt_irq_disable(struct i
[PATCH 2/2] dt-bindings: timer: imx: gpt: Add pin group bindings for input capture
Add pin group bindings to support input capture function of the i.MX
GPT.
Signed-off-by: Steve Longerbeam
---
.../devicetree/bindings/timer/fsl,imxgpt.txt | 28 +++
1 file changed, 28 insertions(+)
diff --git a/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt
b/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt
index 5d8fd5b52598..32797b7b0d02 100644
--- a/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt
+++ b/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt
@@ -33,6 +33,13 @@ Required properties:
an entry for each entry in clock-names.
- clock-names : must include "ipg" entry first, then "per" entry.
+Optional properties:
+
+- pinctrl-0: For the i.MX GPT to support the Input Capture function,
+the input capture channel pin groups must be listed here.
+- pinctrl-names: must be "default".
+
+
Example:
gpt1: timer@10003000 {
@@ -43,3 +50,24 @@ gpt1: timer@10003000 {
< IMX27_CLK_PER1_GATE>;
clock-names = "ipg", "per";
};
+
+
+Example with input capture channel 0 support:
+
+pinctrl_gpt_input_capture0: gptinputcapture0grp {
+ fsl,pins = <
+ MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1 0x1b0b0
+ >;
+};
+
+gpt: gpt@2098000 {
+ compatible = "fsl,imx6q-gpt", "fsl,imx31-gpt";
+ reg = <0x02098000 0x4000>;
+ interrupts = <0 55 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = < IMX6QDL_CLK_GPT_IPG>,
+ < IMX6QDL_CLK_GPT_IPG_PER>,
+ < IMX6QDL_CLK_GPT_3M>;
+ clock-names = "ipg", "per", "osc_per";
+ pinctrl-names = "default";
+ pinctrl-0 = <_gpt_input_capture0>;
+};
--
2.17.1
Re: [PATCH] media: i2c: adv7180: fix adv7280 BT.656-4 compatibility
and of course without knowing who is using these devices we can't tell what would break even if we fix something that may be misconfigured already (or even completely unused). I'm cc'ing Steve Longerbeam as well as at one point he was suggesting adding a 'newavmode' property to the adv7180 bindings and likely recalls the discussions there. I've never understood why the adv7180 driver configures a non-standard V-bit end position (ADV7180_NTSC_V_BIT_END_MANUAL_NVEND), maybe because the driver was introduced along with a new bridge driver that assumes that V-bit position. The parallel CSI interface in imx6 driver is also configuring its crop window to work with this non-standard V-bit position. The most straight-forward approach to decouple these adv7180 dependencies would be to remove the non-standard V-bit setting in the adv7180 driver, and it should output standard bt.656-3 or bt.656-4 SAV/EAV codes for all compatible chips. Then expand on the bus-type DT bindings to distinguish between bt.656-3 and bt.656-4. And finally all bridge drivers that use adv7180 would have to be fixed to work only with standard bt.656-3/4 codes. But I realize that last part isn't so easy, and it's even possible some bridge drivers may not be able to cope with the standard V-bit positions. Steve
Re: [PATCH 15/22] media: imx7-media-csi: Create media links in bound notifier
Hi Rui,
On 9/2/19 2:38 AM, Rui Miguel Silva wrote:
Hi Steve,
On Tue 06 Aug 2019 at 00:34, Steve Longerbeam wrote:
Implement a notifier bound op to register media links from the remote
sub-device's source pad(s) to the CSI sink pad.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx7-media-csi.c | 24 ++
1 file changed, 24 insertions(+)
diff --git a/drivers/staging/media/imx/imx7-media-csi.c
b/drivers/staging/media/imx/imx7-media-csi.c
index a1c96c52a606..f71ac485f780 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -196,6 +196,11 @@ struct imx7_csi {
struct completion last_eof_completion;
};
+static inline struct imx7_csi *notifier_to_dev(struct v4l2_async_notifier *n)
As the other one add the namespace for the function name:
imx7_csi_notifier_to_dev
other than this, looks good to me.
Thanks for the review. I will make those changes in next rev.
Steve
+{
+ return container_of(n, struct imx7_csi, notifier);
+}
+
static u32 imx7_csi_reg_read(struct imx7_csi *csi, unsigned int offset)
{
return readl(csi->regbase + offset);
@@ -1173,6 +1178,23 @@ static int imx7_csi_parse_endpoint(struct device *dev,
return fwnode_device_is_available(asd->match.fwnode) ? 0 : -EINVAL;
}
+static int imx7_csi_notify_bound(struct v4l2_async_notifier *notifier,
+struct v4l2_subdev *sd,
+struct v4l2_async_subdev *asd)
+{
+ struct imx7_csi *csi = notifier_to_dev(notifier);
+ struct media_pad *sink = >sd.entity.pads[IMX7_CSI_PAD_SINK];
+
+ return media_create_fwnode_pad_links(sink,
+dev_fwnode(csi->sd.dev),
+>entity,
+dev_fwnode(sd->dev), 0);
+}
+
+static const struct v4l2_async_notifier_operations imx7_csi_notify_ops = {
+ .bound = imx7_csi_notify_bound,
+};
+
static int imx7_csi_probe(struct platform_device *pdev)
{
struct device *dev = >dev;
@@ -1253,6 +1275,8 @@ static int imx7_csi_probe(struct platform_device *pdev)
v4l2_async_notifier_init(>notifier);
+ csi->notifier.ops = _csi_notify_ops;
+
ret = v4l2_async_register_fwnode_subdev(>sd, >notifier,
sizeof(struct
v4l2_async_subdev),
NULL, 0,
[PATCH v2 2/2] media: imx: Move pads init to probe
If a subdevice is unregistered and then registered again without the
driver being removed and re-probed (which will happen when the media
device is removed and re-probed without also removing/re-probing the
subdevice), media_device_register_entity() is called with a non-zero
entity->num_pads, and then the subdevice's .registered callback calls
media_entity_pads_init(). Thus the subdevice's pad objects are added
to the media device pad list twice, causing list corruption.
One way to fix this would be to create media_entity_pads_destroy(),
and call it in the subdevice's .unregistered callback. But calling
media_entity_pads_init() in the .registered callbacks was done for
legacy reasons and is no longer necessary, so move the call to
media_entity_pads_init() into the subdevice's probe functions. This
fixes the duplicate pad obejcts in the media device pad list.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prp.c| 25 ++---
drivers/staging/media/imx/imx-ic-prpencvf.c | 28 +--
drivers/staging/media/imx/imx-media-capture.c | 15 +-
drivers/staging/media/imx/imx-media-csi.c | 21 +++---
drivers/staging/media/imx/imx-media-vdic.c| 27 --
drivers/staging/media/imx/imx6-mipi-csi2.c| 27 --
drivers/staging/media/imx/imx7-media-csi.c| 18 ++--
drivers/staging/media/imx/imx7-mipi-csis.c| 23 +--
8 files changed, 81 insertions(+), 103 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 35e60a120dc1..2a4f77e83ed3 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -428,32 +428,19 @@ static int prp_s_frame_interval(struct v4l2_subdev *sd,
return 0;
}
-/*
- * retrieve our pads parsed from the OF graph by the media device
- */
static int prp_registered(struct v4l2_subdev *sd)
{
struct prp_priv *priv = sd_to_priv(sd);
- int i, ret;
u32 code;
- for (i = 0; i < PRP_NUM_PADS; i++) {
- priv->pad[i].flags = (i == PRP_SINK_PAD) ?
- MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
- }
-
/* init default frame interval */
priv->frame_interval.numerator = 1;
priv->frame_interval.denominator = 30;
/* set a default mbus format */
imx_media_enum_ipu_format(, 0, CS_SEL_YUV);
- ret = imx_media_init_mbus_fmt(>format_mbus, 640, 480, code,
- V4L2_FIELD_NONE, NULL);
- if (ret)
- return ret;
-
- return media_entity_pads_init(>entity, PRP_NUM_PADS, priv->pad);
+ return imx_media_init_mbus_fmt(>format_mbus, 640, 480, code,
+ V4L2_FIELD_NONE, NULL);
}
static const struct v4l2_subdev_pad_ops prp_pad_ops = {
@@ -487,6 +474,7 @@ static const struct v4l2_subdev_internal_ops
prp_internal_ops = {
static int prp_init(struct imx_ic_priv *ic_priv)
{
struct prp_priv *priv;
+ int i;
priv = devm_kzalloc(ic_priv->ipu_dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
@@ -496,7 +484,12 @@ static int prp_init(struct imx_ic_priv *ic_priv)
ic_priv->task_priv = priv;
priv->ic_priv = ic_priv;
- return 0;
+ for (i = 0; i < PRP_NUM_PADS; i++)
+ priv->pad[i].flags = (i == PRP_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+ return media_entity_pads_init(_priv->sd.entity, PRP_NUM_PADS,
+ priv->pad);
}
static void prp_remove(struct imx_ic_priv *ic_priv)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index f8a0b21fcd02..09c4e3f33807 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1240,9 +1240,6 @@ static int prp_s_frame_interval(struct v4l2_subdev *sd,
return 0;
}
-/*
- * retrieve our pads parsed from the OF graph by the media device
- */
static int prp_registered(struct v4l2_subdev *sd)
{
struct prp_priv *priv = sd_to_priv(sd);
@@ -1250,12 +1247,9 @@ static int prp_registered(struct v4l2_subdev *sd)
int i, ret;
u32 code;
+ /* set a default mbus format */
+ imx_media_enum_ipu_format(, 0, CS_SEL_YUV);
for (i = 0; i < PRPENCVF_NUM_PADS; i++) {
- priv->pad[i].flags = (i == PRPENCVF_SINK_PAD) ?
- MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
-
- /* set a default mbus format */
- imx_media_enum_ipu_format(, 0, CS_SEL_YUV);
ret = imx_media_init_mbus_fmt(>format_mbus[i],
640, 480, code, V4L2_FIELD_NONE,
>cc[i]);
@@ -1267,11 +1261,6 @@ st
[PATCH v2 1/2] media: imx: Move capture device init to registered
If the CSI is unregistered and then registered again without the
driver being removed and re-probed (which will happen when the media
device is removed and re-probed without also removing/re-probing the
CSI), the result is the kobject error and backtrace "tried to init an
initialized object". This is because the video device is left in an
initialized state after being unregistered, thus the video device's
underlying kobject is also left in an initialized state when the device
is registered again.
Fix this by moving imx_media_capture_device_init() and _remove()
into csi_registered() and csi_unregistered(). This will create a new
un-initialized video device when the CSI is re-registered. Do this for
all the subdevices that register a capture device.
Reported-by: Russell King
Signed-off-by: Steve Longerbeam
---
Changes in v2:
Add missing local var ic_priv in prp_registered().
Reported-by: kbuild test robot
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 25 -
drivers/staging/media/imx/imx-media-csi.c | 20 ++---
drivers/staging/media/imx/imx7-media-csi.c | 22 +-
3 files changed, 38 insertions(+), 29 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 67ffa46a8e96..f8a0b21fcd02 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1246,6 +1246,7 @@ static int prp_s_frame_interval(struct v4l2_subdev *sd,
static int prp_registered(struct v4l2_subdev *sd)
{
struct prp_priv *priv = sd_to_priv(sd);
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
int i, ret;
u32 code;
@@ -1271,17 +1272,26 @@ static int prp_registered(struct v4l2_subdev *sd)
if (ret)
return ret;
+ priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
+ _priv->sd,
+ PRPENCVF_SRC_PAD);
+ if (IS_ERR(priv->vdev))
+ return PTR_ERR(priv->vdev);
+
ret = imx_media_capture_device_register(priv->vdev);
if (ret)
- return ret;
+ goto remove_vdev;
ret = prp_init_controls(priv);
if (ret)
- goto unreg;
+ goto unreg_vdev;
return 0;
-unreg:
+
+unreg_vdev:
imx_media_capture_device_unregister(priv->vdev);
+remove_vdev:
+ imx_media_capture_device_remove(priv->vdev);
return ret;
}
@@ -1290,6 +1300,8 @@ static void prp_unregistered(struct v4l2_subdev *sd)
struct prp_priv *priv = sd_to_priv(sd);
imx_media_capture_device_unregister(priv->vdev);
+ imx_media_capture_device_remove(priv->vdev);
+
v4l2_ctrl_handler_free(>ctrl_hdlr);
}
@@ -1336,12 +1348,6 @@ static int prp_init(struct imx_ic_priv *ic_priv)
spin_lock_init(>irqlock);
timer_setup(>eof_timeout_timer, prp_eof_timeout, 0);
- priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
- _priv->sd,
- PRPENCVF_SRC_PAD);
- if (IS_ERR(priv->vdev))
- return PTR_ERR(priv->vdev);
-
mutex_init(>lock);
return 0;
@@ -1352,7 +1358,6 @@ static void prp_remove(struct imx_ic_priv *ic_priv)
struct prp_priv *priv = ic_priv->task_priv;
mutex_destroy(>lock);
- imx_media_capture_device_remove(priv->vdev);
}
struct imx_ic_ops imx_ic_prpencvf_ops = {
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 367e39f5b382..b3f1cf08a102 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1797,12 +1797,22 @@ static int csi_registered(struct v4l2_subdev *sd)
if (ret)
goto free_fim;
+ priv->vdev = imx_media_capture_device_init(priv->sd.dev,
+ >sd,
+ CSI_SRC_PAD_IDMAC);
+ if (IS_ERR(priv->vdev)) {
+ ret = PTR_ERR(priv->vdev);
+ goto free_fim;
+ }
+
ret = imx_media_capture_device_register(priv->vdev);
if (ret)
- goto free_fim;
+ goto remove_vdev;
return 0;
+remove_vdev:
+ imx_media_capture_device_remove(priv->vdev);
free_fim:
if (priv->fim)
imx_media_fim_free(priv->fim);
@@ -1816,6 +1826,7 @@ static void csi_unregistered(struct v4l2_subdev *sd)
struct csi_priv *priv = v4l2_get_subdevdata(sd);
imx_media_capture_device_unregister(priv->vdev);
+ imx_media_capture_device_remove(priv->vdev);
if (priv->fim)
[PATCH 1/2] media: imx: Move capture device init to registered
If the CSI is unregistered and then registered again without the
driver being removed and re-probed (which will happen when the media
device is removed and re-probed without also removing/re-probing the
CSI), the result is the kobject error and backtrace "tried to init an
initialized object". This is because the video device is left in an
initialized state after being unregistered, thus the video device's
underlying kobject is also left in an initialized state when the device
is registered again.
Fix this by moving imx_media_capture_device_init() and _remove()
into csi_registered() and csi_unregistered(). This will create a new
un-initialized video device when the CSI is re-registered. Do this for
all the subdevices that register a capture device.
Reported-by: Russell King
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 24 -
drivers/staging/media/imx/imx-media-csi.c | 20 ++---
drivers/staging/media/imx/imx7-media-csi.c | 22 +--
3 files changed, 37 insertions(+), 29 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 67ffa46a8e96..301f5fce53c0 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1271,17 +1271,26 @@ static int prp_registered(struct v4l2_subdev *sd)
if (ret)
return ret;
+ priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
+ _priv->sd,
+ PRPENCVF_SRC_PAD);
+ if (IS_ERR(priv->vdev))
+ return PTR_ERR(priv->vdev);
+
ret = imx_media_capture_device_register(priv->vdev);
if (ret)
- return ret;
+ goto remove_vdev;
ret = prp_init_controls(priv);
if (ret)
- goto unreg;
+ goto unreg_vdev;
return 0;
-unreg:
+
+unreg_vdev:
imx_media_capture_device_unregister(priv->vdev);
+remove_vdev:
+ imx_media_capture_device_remove(priv->vdev);
return ret;
}
@@ -1290,6 +1299,8 @@ static void prp_unregistered(struct v4l2_subdev *sd)
struct prp_priv *priv = sd_to_priv(sd);
imx_media_capture_device_unregister(priv->vdev);
+ imx_media_capture_device_remove(priv->vdev);
+
v4l2_ctrl_handler_free(>ctrl_hdlr);
}
@@ -1336,12 +1347,6 @@ static int prp_init(struct imx_ic_priv *ic_priv)
spin_lock_init(>irqlock);
timer_setup(>eof_timeout_timer, prp_eof_timeout, 0);
- priv->vdev = imx_media_capture_device_init(ic_priv->ipu_dev,
- _priv->sd,
- PRPENCVF_SRC_PAD);
- if (IS_ERR(priv->vdev))
- return PTR_ERR(priv->vdev);
-
mutex_init(>lock);
return 0;
@@ -1352,7 +1357,6 @@ static void prp_remove(struct imx_ic_priv *ic_priv)
struct prp_priv *priv = ic_priv->task_priv;
mutex_destroy(>lock);
- imx_media_capture_device_remove(priv->vdev);
}
struct imx_ic_ops imx_ic_prpencvf_ops = {
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 367e39f5b382..b3f1cf08a102 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1797,12 +1797,22 @@ static int csi_registered(struct v4l2_subdev *sd)
if (ret)
goto free_fim;
+ priv->vdev = imx_media_capture_device_init(priv->sd.dev,
+ >sd,
+ CSI_SRC_PAD_IDMAC);
+ if (IS_ERR(priv->vdev)) {
+ ret = PTR_ERR(priv->vdev);
+ goto free_fim;
+ }
+
ret = imx_media_capture_device_register(priv->vdev);
if (ret)
- goto free_fim;
+ goto remove_vdev;
return 0;
+remove_vdev:
+ imx_media_capture_device_remove(priv->vdev);
free_fim:
if (priv->fim)
imx_media_fim_free(priv->fim);
@@ -1816,6 +1826,7 @@ static void csi_unregistered(struct v4l2_subdev *sd)
struct csi_priv *priv = v4l2_get_subdevdata(sd);
imx_media_capture_device_unregister(priv->vdev);
+ imx_media_capture_device_remove(priv->vdev);
if (priv->fim)
imx_media_fim_free(priv->fim);
@@ -1963,11 +1974,6 @@ static int imx_csi_probe(struct platform_device *pdev)
imx_media_grp_id_to_sd_name(priv->sd.name, sizeof(priv->sd.name),
priv->sd.grp_id, ipu_get_num(priv->ipu));
- priv->vdev = imx_media_capture_device_init(priv->sd.dev, >sd,
-
[PATCH 2/2] media: imx: Move pads init to probe
If a subdevice is unregistered and then registered again without the
driver being removed and re-probed (which will happen when the media
device is removed and re-probed without also removing/re-probing the
subdevice), media_device_register_entity() is called with a non-zero
entity->num_pads, and then the subdevice's .registered callback calls
media_entity_pads_init(). Thus the subdevice's pad objects are added
to the media device pad list twice, causing list corruption.
One way to fix this would be to create media_entity_pads_destroy(),
and call it in the subdevice's .unregistered callback. But calling
media_entity_pads_init() in the .registered callbacks was done for
legacy reasons and is no longer necessary, so move the call to
media_entity_pads_init() into the subdevice's probe functions. This
fixes the duplicate pad obejcts in the media device pad list.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prp.c| 25 ++--
drivers/staging/media/imx/imx-ic-prpencvf.c | 29 ++-
drivers/staging/media/imx/imx-media-capture.c | 15 +-
drivers/staging/media/imx/imx-media-csi.c | 21 +++---
drivers/staging/media/imx/imx-media-vdic.c| 27 +++--
drivers/staging/media/imx/imx6-mipi-csi2.c| 27 -
drivers/staging/media/imx/imx7-media-csi.c| 18 +++-
drivers/staging/media/imx/imx7-mipi-csis.c| 23 +--
8 files changed, 82 insertions(+), 103 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 35e60a120dc1..2a4f77e83ed3 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -428,32 +428,19 @@ static int prp_s_frame_interval(struct v4l2_subdev *sd,
return 0;
}
-/*
- * retrieve our pads parsed from the OF graph by the media device
- */
static int prp_registered(struct v4l2_subdev *sd)
{
struct prp_priv *priv = sd_to_priv(sd);
- int i, ret;
u32 code;
- for (i = 0; i < PRP_NUM_PADS; i++) {
- priv->pad[i].flags = (i == PRP_SINK_PAD) ?
- MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
- }
-
/* init default frame interval */
priv->frame_interval.numerator = 1;
priv->frame_interval.denominator = 30;
/* set a default mbus format */
imx_media_enum_ipu_format(, 0, CS_SEL_YUV);
- ret = imx_media_init_mbus_fmt(>format_mbus, 640, 480, code,
- V4L2_FIELD_NONE, NULL);
- if (ret)
- return ret;
-
- return media_entity_pads_init(>entity, PRP_NUM_PADS, priv->pad);
+ return imx_media_init_mbus_fmt(>format_mbus, 640, 480, code,
+ V4L2_FIELD_NONE, NULL);
}
static const struct v4l2_subdev_pad_ops prp_pad_ops = {
@@ -487,6 +474,7 @@ static const struct v4l2_subdev_internal_ops
prp_internal_ops = {
static int prp_init(struct imx_ic_priv *ic_priv)
{
struct prp_priv *priv;
+ int i;
priv = devm_kzalloc(ic_priv->ipu_dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
@@ -496,7 +484,12 @@ static int prp_init(struct imx_ic_priv *ic_priv)
ic_priv->task_priv = priv;
priv->ic_priv = ic_priv;
- return 0;
+ for (i = 0; i < PRP_NUM_PADS; i++)
+ priv->pad[i].flags = (i == PRP_SINK_PAD) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+
+ return media_entity_pads_init(_priv->sd.entity, PRP_NUM_PADS,
+ priv->pad);
}
static void prp_remove(struct imx_ic_priv *ic_priv)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 301f5fce53c0..09c4e3f33807 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1240,21 +1240,16 @@ static int prp_s_frame_interval(struct v4l2_subdev *sd,
return 0;
}
-/*
- * retrieve our pads parsed from the OF graph by the media device
- */
static int prp_registered(struct v4l2_subdev *sd)
{
struct prp_priv *priv = sd_to_priv(sd);
+ struct imx_ic_priv *ic_priv = priv->ic_priv;
int i, ret;
u32 code;
+ /* set a default mbus format */
+ imx_media_enum_ipu_format(, 0, CS_SEL_YUV);
for (i = 0; i < PRPENCVF_NUM_PADS; i++) {
- priv->pad[i].flags = (i == PRPENCVF_SINK_PAD) ?
- MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
-
- /* set a default mbus format */
- imx_media_enum_ipu_format(, 0, CS_SEL_YUV);
ret = imx_media_init_mbus_fmt(>format_mbus[i],
640, 480, code, V4L2_FIELD_NONE,
>cc[i]);
@@ -1266,11 +1261,6 @@
Re: [BUG] removing and reinserting imx-media causes kernel to explode
On 8/14/19 2:25 AM, Russell King - ARM Linux admin wrote: I just did this: rmmod imx-media modprobe imx-media and was greeted by the below kernel messages. Yes this needs fixing, the CSI needs to check first that it is already registered before going through the ->registered() steps. Posting a patch. I don't think this has been the first issue I found with the iMX media stuff involving a module unload/reload cycle - may I suggest that this is added to the testing regime for this code? Thanks. I do test module unload/reload cycles, but v4l2-async does not support re-registering subdevices unless the subdevice is basically completely removed and re-probed, so there won't be a working media device if only imx-media is reloaded. So I have always unloaded/reloaded all modules for every entity in the graph, i.e.: rmmod imx6_media imx6_media_csi imx6_mipi_csi2 ov5640 video_mux imx_media_common (replace ov5640 with your imx219 sensor). But I'll make sure to test single module unload/reload cycles in the future. But note after applying the patch mentioned above to CSI ->registered() callback, there are list corruption backtraces, see [1]. The root cause is that both media_device_register_entity() and media_entity_pads_init() add the same graph objects for the entity's pads, so duplicate pad objects are added to the media device pads list. Removing the pad object creation in media_device_register_entity() fixes the list corruption. Sending a patch for that also. This is a problem for any entity that sets its ->num_pads to a non-zero value before media_device_register_entity() is called. For example, the following will produce the same list corruption backtrace: rmmod video-mux modprobe video-mux rmmod video-mux Steve [1] rmmod imx6-media modprobe imx6-media rmmod imx6-media [ 249.387953] WARNING: CPU: 2 PID: 843 at lib/list_debug.c:53 __list_del_entry_valid+0xa0/0xdc [ 249.396442] list_del corruption. prev->next should be e8fb0510, but was e93b5914 [ 249.404076] Modules linked in: imx6_media_csi(C) imx6_media(C-) imx6_mipi_csi2(C) bnep dw_hdmi_ahb_audio dw_hdmi_cec ov5640 mux_mmio video_mux mux_core dw_hdmi_imx dw_hdmi coda_vpu cec imx_vdoa videobuf2_vmalloc imx_media_common(C) v4l2_fwnode imx_ldb imxdrm imx_ipu_v3 [last unloaded: imx6_media_csi] [ 249.430956] CPU: 2 PID: 843 Comm: rmmod Tainted: G C 5.3.0-rc4-01115-g62119fd20fda #5 [ 249.440115] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree) [ 249.446689] [] (unwind_backtrace) from [] (show_stack+0x10/0x14) [ 249.454462] [] (show_stack) from [] (dump_stack+0xd8/0x110) [ 249.461804] [] (dump_stack) from [] (__warn+0xe0/0x10c) [ 249.468789] [] (__warn) from [] (warn_slowpath_fmt+0x44/0x6c) [ 249.476301] [] (warn_slowpath_fmt) from [] (__list_del_entry_valid+0xa0/0xdc) [ 249.485207] [] (__list_del_entry_valid) from [] (media_gobj_destroy.part.4+0x30/0x64) [ 249.494802] [] (media_gobj_destroy.part.4) from [] (__media_device_unregister_entity+0xa8/0xcc) [ 249.505259] [] (__media_device_unregister_entity) from [] (media_device_unregister_entity+0x2c/0x38) [ 249.516157] [] (media_device_unregister_entity) from [] (v4l2_device_unregister_subdev+0x90/0xb4) [ 249.526793] [] (v4l2_device_unregister_subdev) from [] (v4l2_async_cleanup+0x10/0x3c) [ 249.536382] [] (v4l2_async_cleanup) from [] (v4l2_async_notifier_unbind_all_subdevs+0x9c/0x10c) [ 249.546840] [] (v4l2_async_notifier_unbind_all_subdevs) from [] (v4l2_async_notifier_unbind_all_subdevs+0x6c/0x10c) [ 249.559035] [] (v4l2_async_notifier_unbind_all_subdevs) from [] (__v4l2_async_notifier_unregister.part.4+0xc/0x44) [ 249.571140] [] (__v4l2_async_notifier_unregister.part.4) from [] (v4l2_async_notifier_unregister+0x30/0x50) [ 249.582665] [] (v4l2_async_notifier_unregister) from [] (imx_media_remove+0x20/0x54 [imx6_media]) [ 249.593389] [] (imx_media_remove [imx6_media]) from [] (platform_drv_remove+0x20/0x40) [ 249.603068] [] (platform_drv_remove) from [] (device_release_driver_internal+0xdc/0x1ac) [ 249.612917] [] (device_release_driver_internal) from [] (driver_detach+0x44/0x80) [ 249.622164] [] (driver_detach) from [] (bus_remove_driver+0x5c/0xd8) [ 249.630287] [] (bus_remove_driver) from [] (sys_delete_module+0x17c/0x20c) [ 249.638926] [] (sys_delete_module) from [] (ret_fast_syscall+0x0/0x28) [ 249.647205] Exception stack(0xe90d5fa8 to 0xe90d5ff0) [ 249.652278] 5fa0: beed5d6c 0003 01401134 0800 4f13b6f4 2002 [ 249.660475] 5fc0: beed5d6c 0003 beed5b80 0081 beed5e78 0001 014010f8 [ 249.668669] 5fe0: 0003b2c4 beed5b4c 0001f248 4f1012dc [ 249.673859] irq event stamp: 4113 [ 249.677267] hardirqs last enabled at (4131): [] console_unlock+0x408/0x5f8 [ 249.685125] hardirqs last disabled at (4138): [] console_unlock+0x88/0x5f8 [ 249.692970] softirqs last enabled at (4154): [] __do_softirq+0x360/0x524 [ 249.700735] softirqs last disabled
Re: [PATCH 04/22] media: Move v4l2_fwnode_parse_link from v4l2 to driver base
On 8/14/19 4:15 PM, Russell King - ARM Linux admin wrote: On Wed, Aug 14, 2019 at 04:00:30PM -0700, Steve Longerbeam wrote: On 8/14/19 3:04 PM, Russell King - ARM Linux admin wrote: On Wed, Aug 14, 2019 at 12:04:41PM -0700, Steve Longerbeam wrote: On 8/14/19 3:30 AM, Russell King - ARM Linux admin wrote: On Tue, Aug 06, 2019 at 09:53:41AM -0700, Steve Longerbeam wrote: The full patchset doesn't seem to be up yet, but see [1] for the cover letter. Was the entire series copied to the mailing lists, or just selected patches? I only saw 4, 9, 11 and 13-22 via lakml. The whole series was posted to the linux-media ML, see [1]. At the time, none of the linux-media ML archives had the whole series. In the absence of the other patches, will this solve imx-media binding the internal subdevs of sensor devices to the CSI2 interface? "internal subdevs of sensor devices" ?? That doesn't make any sense. Sorry, but it makes complete sense when you consider that sensor devices may have more than one subdev, but there should be only one that is the "output" to whatever the camera is attached to. The other subdevs are internal to the sensor. Ah, thanks for the clarification. Yes, by "internal subdevs" I understand what you mean now. The adv748x and smiapp are examples. subdevs are not purely the remit of SoC drivers. So there is no binding of internal subdevs to the receiver CSI-2. The receiver CSI-2 subdev will create media links to the subdev that has an externally exposed fwnode endpoint that connects with the CSI-2 sink pad. Maybe - with 5.2, I get: - entity 15: imx6-mipi-csi2 (5 pads, 6 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2 pad0: Sink ... <- "imx219 0-0010":0 [] <- "imx219 pixel 0-0010":0 [] Adding some debug in gives: [ 11.963362] imx-media: imx_media_create_of_links() for imx6-mipi-csi2 [ 11.963396] imx-media: create_of_link(): /soc/aips-bus@200/iomuxc-gpr@20e/ipu1_csi0_mux [ 11.963422] imx-media: create_of_link(): /soc/ipu@240 [ 11.963450] imx-media: create_of_link(): /soc/ipu@280 [ 11.963478] imx-media: create_of_link(): /soc/aips-bus@200/iomuxc-gpr@20e/ipu2_csi1_mux [ 11.963489] imx-media: imx6-mipi-csi2:4 -> ipu2_csi1_mux:0 [ 11.963522] imx-media: create_of_link(): /soc/aips-bus@210/i2c@21a/camera@10 [ 11.963533] imx-media: imx219 0-0010:0 -> imx6-mipi-csi2:0 [ 11.963549] imx-media: imx_media_create_of_links() for imx219 pixel 0-0010 [ 11.963577] imx-media: create_of_link(): /soc/aips-bus@210/mipi@21dc000 [ 11.963587] imx-media: imx219 pixel 0-0010:0 -> imx6-mipi-csi2:0 [ 11.963602] imx-media: imx_media_create_of_links() for imx219 0-0010 Note that it's not created by imx6-mipi-csi2, but by imx-media delving around in the imx219 subdevs. From what I can see, smiapp does the same thing that I do in imx219 - sets the subdev->dev member to point at the struct device, which then means that v4l2_device_register_subdev() will associate the same fwnode with both "imx219 pixel 0-0010" and "imx219 0-0010". Ok, understood. I realize imx_media_create_of_link() is a bit intrusive, and that's one of the things I'm trying to get rid of in this patchset. Unfortunately it's there for a reason which is described in patch 0021. But to explain here, the imx6-mipi-csi2 receiver outputs its four virtual channels on four separate source pads, and those connect to four different subdevices (video mux's and CSI's), and the problem is that only the first subdev that adds imx6-mipi-csi2 to its notifier asd list will get a notifier bind() callback (where links can be created to imx6-mipi-csi2) -- the other subdevs don't contain it in their asd lists so they never create the links to imx6-mipi-csi2. So until the requirement in v4l2-async that no notifiers can contain the same asd in its list is relaxed, this function will have to remain, but I can make it less intrusive (only create the missing links from imx6-mipi-csi2). I'm not able to find a cleaner workaround at the moment. Steve
Re: [PATCH 04/22] media: Move v4l2_fwnode_parse_link from v4l2 to driver base
On 8/14/19 3:04 PM, Russell King - ARM Linux admin wrote: On Wed, Aug 14, 2019 at 12:04:41PM -0700, Steve Longerbeam wrote: On 8/14/19 3:30 AM, Russell King - ARM Linux admin wrote: On Tue, Aug 06, 2019 at 09:53:41AM -0700, Steve Longerbeam wrote: The full patchset doesn't seem to be up yet, but see [1] for the cover letter. Was the entire series copied to the mailing lists, or just selected patches? I only saw 4, 9, 11 and 13-22 via lakml. The whole series was posted to the linux-media ML, see [1]. At the time, none of the linux-media ML archives had the whole series. In the absence of the other patches, will this solve imx-media binding the internal subdevs of sensor devices to the CSI2 interface? "internal subdevs of sensor devices" ?? That doesn't make any sense. Sorry, but it makes complete sense when you consider that sensor devices may have more than one subdev, but there should be only one that is the "output" to whatever the camera is attached to. The other subdevs are internal to the sensor. Ah, thanks for the clarification. Yes, by "internal subdevs" I understand what you mean now. The adv748x and smiapp are examples. subdevs are not purely the remit of SoC drivers. So there is no binding of internal subdevs to the receiver CSI-2. The receiver CSI-2 subdev will create media links to the subdev that has an externally exposed fwnode endpoint that connects with the CSI-2 sink pad. Steve
Re: [PATCH 04/22] media: Move v4l2_fwnode_parse_link from v4l2 to driver base
On 8/14/19 3:30 AM, Russell King - ARM Linux admin wrote: On Tue, Aug 06, 2019 at 09:53:41AM -0700, Steve Longerbeam wrote: The full patchset doesn't seem to be up yet, but see [1] for the cover letter. Was the entire series copied to the mailing lists, or just selected patches? I only saw 4, 9, 11 and 13-22 via lakml. The whole series was posted to the linux-media ML, see [1]. At the time, none of the linux-media ML archives had the whole series. In the absence of the other patches, will this solve imx-media binding the internal subdevs of sensor devices to the CSI2 interface? "internal subdevs of sensor devices" ?? That doesn't make any sense. Sensors are external to the SoC, there are no "internal" sensor devices. Not sure what you mean by "binding" either in this context, but external sensors can connect via fwnode endpoint, and later translated to media link, to the receiver CSI-2 sink. Steve [1] https://www.spinics.net/lists/linux-media/msg155160.html
[PATCH v2] media: docs-rst: Clarify duration of LP-11 mode
From: Steve Longerbeam Add a sentence that makes it more clear when the CSI-2 transmitter must, if possible, exit LP-11 mode. That is, maintain LP-11 mode until stream on, at which point the transmitter activates the clock lane and transition to HS mode. Signed-off-by: Steve Longerbeam --- Changes in v2: - change wording "Only until stream on ..." to "Only at stream on ..." --- Documentation/media/kapi/csi2.rst | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/Documentation/media/kapi/csi2.rst b/Documentation/media/kapi/csi2.rst index a7e75e2eba85..f87c413cd433 100644 --- a/Documentation/media/kapi/csi2.rst +++ b/Documentation/media/kapi/csi2.rst @@ -49,9 +49,13 @@ where The transmitter drivers must, if possible, configure the CSI-2 transmitter to *LP-11 mode* whenever the transmitter is powered on but -not active. Some transmitters do this automatically but some have to -be explicitly programmed to do so, and some are unable to do so -altogether due to hardware constraints. +not active, and maintain *LP-11 mode* until stream on. Only at stream +on should the transmitter activate the clock on the clock lane and +transition to *HS mode*. + +Some transmitters do this automatically but some have to be explicitly +programmed to do so, and some are unable to do so altogether due to +hardware constraints. Stopping the transmitter -- 2.17.1
Re: [PATCH] media: docs-rst: Clarify duration of LP-11 mode
Hi Ian, On 8/12/19 1:53 AM, Ian Arkver wrote: Hi Steve, On 11/08/2019 18:36, Steve Longerbeam wrote: Add a sentence that makes it more clear when the CSI-2 transmitter must, if possible, exit LP-11 mode. That is, maintain LP-11 mode until stream on, at which point the transmitter activates the clock lane and transition to HS mode. Signed-off-by: Steve Longerbeam --- Documentation/media/kapi/csi2.rst | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/Documentation/media/kapi/csi2.rst b/Documentation/media/kapi/csi2.rst index a7e75e2eba85..6cd1d4b0df17 100644 --- a/Documentation/media/kapi/csi2.rst +++ b/Documentation/media/kapi/csi2.rst @@ -49,9 +49,13 @@ where The transmitter drivers must, if possible, configure the CSI-2 transmitter to *LP-11 mode* whenever the transmitter is powered on but -not active. Some transmitters do this automatically but some have to -be explicitly programmed to do so, and some are unable to do so -altogether due to hardware constraints. +not active, and maintain *LP-11 mode* until stream on. Only until s/until/at/ perhaps? Sure, that's a bit better language. Steve Regards, Ian +stream on should the transmitter activate the clock on the clock lane +and transition to *HS mode*. + +Some transmitters do this automatically but some have to be explicitly +programmed to do so, and some are unable to do so altogether due to +hardware constraints. Stopping the transmitter
[PATCH] media: docs-rst: Clarify duration of LP-11 mode
Add a sentence that makes it more clear when the CSI-2 transmitter must, if possible, exit LP-11 mode. That is, maintain LP-11 mode until stream on, at which point the transmitter activates the clock lane and transition to HS mode. Signed-off-by: Steve Longerbeam --- Documentation/media/kapi/csi2.rst | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/Documentation/media/kapi/csi2.rst b/Documentation/media/kapi/csi2.rst index a7e75e2eba85..6cd1d4b0df17 100644 --- a/Documentation/media/kapi/csi2.rst +++ b/Documentation/media/kapi/csi2.rst @@ -49,9 +49,13 @@ where The transmitter drivers must, if possible, configure the CSI-2 transmitter to *LP-11 mode* whenever the transmitter is powered on but -not active. Some transmitters do this automatically but some have to -be explicitly programmed to do so, and some are unable to do so -altogether due to hardware constraints. +not active, and maintain *LP-11 mode* until stream on. Only until +stream on should the transmitter activate the clock on the clock lane +and transition to *HS mode*. + +Some transmitters do this automatically but some have to be explicitly +programmed to do so, and some are unable to do so altogether due to +hardware constraints. Stopping the transmitter -- 2.17.1
[PATCH] media: imx: mipi csi-2: Release DPHY reset in s_power
Some CSI-2 transmitters may only set their data lanes to LP-11 state for
a very short time after being powered on, after which they transition
the lanes to high speed mode.
If that occurs, the lanes will likely be in HS mode long before the imx6
DPHY receiver is initialized and brought out of reset at stream on.
According to the imx6 reference manual, the lanes need to be in LP-11
state at least for some period after the DPHY reset is released. This
might mean that the state machine in the DPHY core (a Synopsys DesignWare
core) needs to detect a LP-11 to HS transition on the lanes before it can
proceed to detect a clock on the clock lane and begin to accept packets.
In an attempt to accommodate such sensors, carry out steps 1-5 in the
s_power op (moved out of s_stream op). This moves steps 1-5 closer in
time to after the sensor is powered ON by v4l2_pipeline_pm_use(), and
provides a better chance that the receiver DPHY will catch an early
LP-11 to HS transition.
This works because the graph walk stack used by v4l2_pipeline_pm_use()
is setup such that the transmitter s_power op is called immediately
before the receiver's s_power op.
For sensors that can persist LP-11 state until stream on, then this
should still work fine. The receiver will detect the HS transition
at step 6, when streaming is enabled at the transmitter.
Tested on imx6q SabreSD with OV5640.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx6-mipi-csi2.c | 93 +-
1 file changed, 73 insertions(+), 20 deletions(-)
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c
b/drivers/staging/media/imx/imx6-mipi-csi2.c
index f29e28df36ed..06345de9f060 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -47,6 +47,7 @@ struct csi2_dev {
struct v4l2_mbus_framefmt format_mbus;
+ int power_count;
int stream_count;
struct v4l2_subdev *src_sd;
boolsink_linked[CSI2_NUM_SRC_PADS];
@@ -113,9 +114,10 @@ static inline struct csi2_dev *sd_to_dev(struct
v4l2_subdev *sdev)
* 7. CSI2 Controller programming - Read the PHY status register (PHY_STATE)
*to confirm that the D-PHY is receiving a clock on the D-PHY clock lane.
*
- * All steps 3 through 7 are carried out by csi2_s_stream(ON) here. Step
- * 6 is accomplished by calling the source subdev's s_stream(ON) between
- * steps 5 and 7.
+ * Steps 3 through 5 are carried out by csi2_s_power(ON) here.
+ *
+ * Steps 6 and 7 are carried out by stream(ON) here. Step 6 is accomplished
+ * by calling the source subdev's s_stream(ON).
*/
static void csi2_enable(struct csi2_dev *csi2, bool enable)
@@ -295,7 +297,7 @@ static void csi2ipu_gasket_init(struct csi2_dev *csi2)
writel(reg, csi2->base + CSI2IPU_GASKET);
}
-static int csi2_start(struct csi2_dev *csi2)
+static int csi2_power_on(struct csi2_dev *csi2)
{
int ret;
@@ -320,41 +322,87 @@ static int csi2_start(struct csi2_dev *csi2)
if (ret)
goto err_assert_reset;
+ return 0;
+
+err_assert_reset:
+ csi2_enable(csi2, false);
+err_disable_clk:
+ clk_disable_unprepare(csi2->pix_clk);
+ return ret;
+}
+
+static void csi2_power_off(struct csi2_dev *csi2)
+{
+ csi2_enable(csi2, false);
+ clk_disable_unprepare(csi2->pix_clk);
+}
+
+static int csi2_stream_on(struct csi2_dev *csi2)
+{
+ int ret;
+
/* Step 6 */
ret = v4l2_subdev_call(csi2->src_sd, video, s_stream, 1);
ret = (ret && ret != -ENOIOCTLCMD) ? ret : 0;
if (ret)
- goto err_assert_reset;
+ return ret;
/* Step 7 */
ret = csi2_dphy_wait_clock_lane(csi2);
if (ret)
- goto err_stop_upstream;
-
- return 0;
+ v4l2_subdev_call(csi2->src_sd, video, s_stream, 0);
-err_stop_upstream:
- v4l2_subdev_call(csi2->src_sd, video, s_stream, 0);
-err_assert_reset:
- csi2_enable(csi2, false);
-err_disable_clk:
- clk_disable_unprepare(csi2->pix_clk);
return ret;
}
-static void csi2_stop(struct csi2_dev *csi2)
+static void csi2_stream_off(struct csi2_dev *csi2)
{
/* stop upstream */
v4l2_subdev_call(csi2->src_sd, video, s_stream, 0);
-
- csi2_enable(csi2, false);
- clk_disable_unprepare(csi2->pix_clk);
}
/*
* V4L2 subdev operations.
*/
+static int csi2_s_power(struct v4l2_subdev *sd, int on)
+{
+ struct csi2_dev *csi2 = sd_to_dev(sd);
+ int ret = 0;
+
+ mutex_lock(>lock);
+
+ if (!csi2->src_sd) {
+ ret = -EPIPE;
+ goto out;
+ }
+
+ /*
+* power on/off only if power_count is going from
+* 0 to 1 / 1 to 0.
+*/
+ if (csi2->power_count != !on)
+ goto update_count;
+
+ dev_d
Re: [PATCH 04/22] media: Move v4l2_fwnode_parse_link from v4l2 to driver base
Hi Andy,
On 8/5/19 11:41 PM, Andy Shevchenko wrote:
On Tue, Aug 6, 2019 at 2:37 AM Steve Longerbeam wrote:
There is nothing v4l2-specific about v4l2_fwnode_{parse|put}_link().
Make these functions more generally available by moving them to driver
base, with the appropriate name changes to the functions and struct.
In the process embed a 'struct fwnode_endpoint' in 'struct fwnode_link'
for both sides of the link, and make use of fwnode_graph_parse_endpoint()
to fully parse both endpoints. Rename members local_node and
remote_node to more descriptive local_port_parent and
remote_port_parent.
May I ask if it's going to be used outside of v4l2?
It could be, I can see fwnode_graph_parse_link() being useful in DRM,
for example. But at this, only media core and v4l2 are making use of it.
This patch was created so that fwnode links can be parsed in a new media
core function that forms media links from fwnode links.
The full patchset doesn't seem to be up yet, but see [1] for the cover
letter.
Steve
[1] https://www.mail-archive.com/linux-media@vger.kernel.org/msg148910.html
[PATCH 03/22] media: entity: Convert media_entity_get_fwnode_pad() args to const
The function media_entity_get_fwnode_pad() can be passed the
const local_fwnode member from a struct fwnode_endpoint, so
the fwnode argument should be a const pointer. Change the
direction_flags argument to const in the process.
Signed-off-by: Steve Longerbeam
---
drivers/media/mc/mc-entity.c | 4 ++--
include/media/media-entity.h | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 47a39d9383d8..e9e090244fd4 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -366,8 +366,8 @@ struct media_entity *media_graph_walk_next(struct
media_graph *graph)
EXPORT_SYMBOL_GPL(media_graph_walk_next);
int media_entity_get_fwnode_pad(struct media_entity *entity,
- struct fwnode_handle *fwnode,
- unsigned long direction_flags)
+ const struct fwnode_handle *fwnode,
+ const unsigned long direction_flags)
{
struct fwnode_endpoint endpoint;
int ret;
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index ed00adb4313b..de7fc3676b5a 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -886,8 +886,8 @@ struct media_pad *media_entity_remote_pad(const struct
media_pad *pad);
* Return: returns the pad number on success or a negative error code.
*/
int media_entity_get_fwnode_pad(struct media_entity *entity,
- struct fwnode_handle *fwnode,
- unsigned long direction_flags);
+ const struct fwnode_handle *fwnode,
+ const unsigned long direction_flags);
/**
* media_graph_walk_init - Allocate resources used by graph walk.
--
2.17.1
[PATCH 15/22] media: imx7-media-csi: Create media links in bound notifier
Implement a notifier bound op to register media links from the remote
sub-device's source pad(s) to the CSI sink pad.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx7-media-csi.c | 24 ++
1 file changed, 24 insertions(+)
diff --git a/drivers/staging/media/imx/imx7-media-csi.c
b/drivers/staging/media/imx/imx7-media-csi.c
index a1c96c52a606..f71ac485f780 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -196,6 +196,11 @@ struct imx7_csi {
struct completion last_eof_completion;
};
+static inline struct imx7_csi *notifier_to_dev(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct imx7_csi, notifier);
+}
+
static u32 imx7_csi_reg_read(struct imx7_csi *csi, unsigned int offset)
{
return readl(csi->regbase + offset);
@@ -1173,6 +1178,23 @@ static int imx7_csi_parse_endpoint(struct device *dev,
return fwnode_device_is_available(asd->match.fwnode) ? 0 : -EINVAL;
}
+static int imx7_csi_notify_bound(struct v4l2_async_notifier *notifier,
+struct v4l2_subdev *sd,
+struct v4l2_async_subdev *asd)
+{
+ struct imx7_csi *csi = notifier_to_dev(notifier);
+ struct media_pad *sink = >sd.entity.pads[IMX7_CSI_PAD_SINK];
+
+ return media_create_fwnode_pad_links(sink,
+dev_fwnode(csi->sd.dev),
+>entity,
+dev_fwnode(sd->dev), 0);
+}
+
+static const struct v4l2_async_notifier_operations imx7_csi_notify_ops = {
+ .bound = imx7_csi_notify_bound,
+};
+
static int imx7_csi_probe(struct platform_device *pdev)
{
struct device *dev = >dev;
@@ -1253,6 +1275,8 @@ static int imx7_csi_probe(struct platform_device *pdev)
v4l2_async_notifier_init(>notifier);
+ csi->notifier.ops = _csi_notify_ops;
+
ret = v4l2_async_register_fwnode_subdev(>sd, >notifier,
sizeof(struct
v4l2_async_subdev),
NULL, 0,
--
2.17.1
[PATCH 05/22] media: entity: Add functions to convert fwnode endpoints to media links
Adds two functions:
media_create_fwnode_pad_links(), which converts fwnode endpoints that
connect a local pad to all pads on a remote entity into media links.
media_create_fwnode_links(), which converts fwnode endpoints that
connect all pads from a local entity to all pads on a remote entity into
media links.
Signed-off-by: Steve Longerbeam
---
drivers/media/mc/mc-entity.c | 178 +++
include/media/media-entity.h | 71 ++
2 files changed, 249 insertions(+)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index e9e090244fd4..45bbd6c91019 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -787,6 +787,184 @@ int media_create_pad_links(const struct media_device
*mdev,
}
EXPORT_SYMBOL_GPL(media_create_pad_links);
+static int __media_create_fwnode_pad_link(struct media_pad *local_pad,
+ struct media_entity *remote,
+ const struct fwnode_handle *remote_ep,
+ const u32 flags)
+{
+ struct media_entity *local = local_pad->entity;
+ unsigned long local_dir = local_pad->flags;
+ unsigned long remote_dir = (local_dir & MEDIA_PAD_FL_SOURCE) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+ struct media_entity *src, *sink;
+ int src_pad, sink_pad;
+ int remote_pad;
+ int ret;
+
+ remote_pad = media_entity_get_fwnode_pad(remote, remote_ep,
+remote_dir);
+ if (remote_pad < 0)
+ return 0;
+
+ if (local_dir & MEDIA_PAD_FL_SOURCE) {
+ src = local;
+ src_pad = local_pad->index;
+ sink = remote;
+ sink_pad = remote_pad;
+ } else {
+ src = remote;
+ src_pad = remote_pad;
+ sink = local;
+ sink_pad = local_pad->index;
+ }
+
+ /* make sure link doesn't already exist */
+ if (media_entity_find_link(>pads[src_pad],
+ >pads[sink_pad]))
+ return 0;
+
+ ret = media_create_pad_link(src, src_pad, sink, sink_pad, flags);
+ if (ret) {
+ dev_dbg(sink->graph_obj.mdev->dev,
+ "%s:%d -> %s:%d failed with %d\n",
+ src->name, src_pad, sink->name, sink_pad,
+ ret);
+ return ret;
+ }
+
+ dev_dbg(sink->graph_obj.mdev->dev, "%s:%d -> %s:%d\n",
+ src->name, src_pad, sink->name, sink_pad);
+
+ return 0;
+}
+
+int __media_create_fwnode_pad_links(struct media_pad *local_pad,
+ const struct fwnode_handle *local_fwnode,
+ struct media_entity *remote,
+ const struct fwnode_handle *remote_fwnode,
+ const u32 link_flags)
+{
+ struct fwnode_handle *endpoint;
+
+ fwnode_graph_for_each_endpoint(remote_fwnode, endpoint) {
+ struct fwnode_link link;
+ int ret;
+
+ ret = fwnode_graph_parse_link(endpoint, );
+ if (ret)
+ continue;
+
+ /*
+* if this endpoint does not link to the local fwnode
+* device, ignore it and continue.
+*/
+ if (link.remote_port_parent != local_fwnode) {
+ fwnode_graph_put_link();
+ continue;
+ }
+
+ ret = __media_create_fwnode_pad_link(local_pad,
+remote, endpoint,
+link_flags);
+
+ fwnode_graph_put_link();
+
+ if (ret) {
+ fwnode_handle_put(endpoint);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(__media_create_fwnode_pad_links);
+
+int media_create_fwnode_pad_links(struct media_pad *local_pad,
+ const struct fwnode_handle *local_fwnode,
+ struct media_entity *remote,
+ const struct fwnode_handle *remote_fwnode,
+ const u32 link_flags)
+{
+ struct media_device *mdev = local_pad->entity->graph_obj.mdev;
+ int ret;
+
+ mutex_lock(>graph_mutex);
+ ret = __media_create_fwnode_pad_links(local_pad, local_fwnode,
+ remote, remote_fwnode,
+ link_flags);
+ mutex_unlock(>graph_mutex);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(media_create_fwnode_pad_links);
+
+int __media_c
[PATCH 13/22] media: imx: mipi csi-2: Create media links in bound notifier
Implement a notifier bound op to register media links from the remote
sub-device's source pad(s) to the mipi csi-2 receiver sink pad.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx6-mipi-csi2.c | 24 ++
1 file changed, 24 insertions(+)
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c
b/drivers/staging/media/imx/imx6-mipi-csi2.c
index 408ee2765e77..5b2258156781 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -91,6 +91,11 @@ static inline struct csi2_dev *sd_to_dev(struct v4l2_subdev
*sdev)
return container_of(sdev, struct csi2_dev, sd);
}
+static inline struct csi2_dev *notifier_to_dev(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct csi2_dev, notifier);
+}
+
/*
* The required sequence of MIPI CSI-2 startup as specified in the i.MX6
* reference manual is as follows:
@@ -573,6 +578,23 @@ static int csi2_parse_endpoint(struct device *dev,
return 0;
}
+static int csi2_notify_bound(struct v4l2_async_notifier *notifier,
+struct v4l2_subdev *sd,
+struct v4l2_async_subdev *asd)
+{
+ struct csi2_dev *csi2 = notifier_to_dev(notifier);
+ struct media_pad *sink = >sd.entity.pads[CSI2_SINK_PAD];
+
+ return media_create_fwnode_pad_links(sink,
+dev_fwnode(csi2->dev),
+>entity,
+dev_fwnode(sd->dev), 0);
+}
+
+static const struct v4l2_async_notifier_operations csi2_notify_ops = {
+ .bound = csi2_notify_bound,
+};
+
static int csi2_probe(struct platform_device *pdev)
{
unsigned int sink_port = 0;
@@ -646,6 +668,8 @@ static int csi2_probe(struct platform_device *pdev)
v4l2_async_notifier_init(>notifier);
+ csi2->notifier.ops = _notify_ops;
+
ret = v4l2_async_register_fwnode_subdev(
>sd, >notifier, sizeof(struct v4l2_async_subdev),
_port, 1, csi2_parse_endpoint);
--
2.17.1
[PATCH 14/22] media: imx7-mipi-csis: Create media links in bound notifier
Implement a notifier bound op to register media links from the remote
sub-device's source pad(s) to the mipi csi-2 receiver sink pad.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx7-mipi-csis.c | 25 ++
1 file changed, 25 insertions(+)
diff --git a/drivers/staging/media/imx/imx7-mipi-csis.c
b/drivers/staging/media/imx/imx7-mipi-csis.c
index f71d9183cad2..e03b2317a1ac 100644
--- a/drivers/staging/media/imx/imx7-mipi-csis.c
+++ b/drivers/staging/media/imx/imx7-mipi-csis.c
@@ -259,6 +259,12 @@ struct csi_state {
bool sink_linked;
};
+static inline struct csi_state *
+notifier_to_csis_state(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct csi_state, notifier);
+}
+
struct csis_pix_format {
unsigned int pix_width_alignment;
u32 code;
@@ -863,6 +869,23 @@ static int mipi_csis_parse_endpoint(struct device *dev,
return 0;
}
+static int mipi_csis_notify_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ struct csi_state *state = notifier_to_csis_state(notifier);
+ struct media_pad *sink = >mipi_sd.entity.pads[CSIS_PAD_SINK];
+
+ return media_create_fwnode_pad_links(sink,
+dev_fwnode(state->mipi_sd.dev),
+>entity,
+dev_fwnode(sd->dev), 0);
+}
+
+static const struct v4l2_async_notifier_operations mipi_csis_notify_ops = {
+ .bound = mipi_csis_notify_bound,
+};
+
static int mipi_csis_subdev_init(struct v4l2_subdev *mipi_sd,
struct platform_device *pdev,
const struct v4l2_subdev_ops *ops)
@@ -895,6 +918,8 @@ static int mipi_csis_subdev_init(struct v4l2_subdev
*mipi_sd,
v4l2_async_notifier_init(>notifier);
+ state->notifier.ops = _csis_notify_ops;
+
ret = v4l2_async_register_fwnode_subdev(mipi_sd, >notifier,
sizeof(struct
v4l2_async_subdev),
_port, 1,
--
2.17.1
[PATCH 10/22] media: st-mipid02: Fix fwnode media link creation
mipid02_async_bound() passes the bound subdev's sd->fwnode to
media_entity_get_fwnode_pad(). This is likely not an endpoint
fwnode as required by media_entity_get_fwnode_pad(), for most
subdevices it is the port parent of endpoint fwnode(s). This has only
worked before because no entities have implemented the .get_fwnode_pad()
op yet, and the default behavior of media_entity_get_fwnode_pad()
was to ignore the passed fwnode and return the first pad that matches
the given direction flags.
Fix this by replacing the calls to media_entity_get_fwnode_pad() and
media_create_pad_link() with a call to media_create_fwnode_pad_links().
Fixes: 642bb5e88fed8 ("media: st-mipid02: MIPID02 CSI-2 to PARALLEL
bridge driver")
Signed-off-by: Steve Longerbeam
---
drivers/media/i2c/st-mipid02.c | 20 ++--
1 file changed, 6 insertions(+), 14 deletions(-)
diff --git a/drivers/media/i2c/st-mipid02.c b/drivers/media/i2c/st-mipid02.c
index 81285b8d5cfb..b26d5550874f 100644
--- a/drivers/media/i2c/st-mipid02.c
+++ b/drivers/media/i2c/st-mipid02.c
@@ -798,24 +798,16 @@ static int mipid02_async_bound(struct v4l2_async_notifier
*notifier,
{
struct mipid02_dev *bridge = to_mipid02_dev(notifier->sd);
struct i2c_client *client = bridge->i2c_client;
- int source_pad;
int ret;
dev_dbg(>dev, "sensor_async_bound call %p", s_subdev);
- source_pad = media_entity_get_fwnode_pad(_subdev->entity,
-s_subdev->fwnode,
-MEDIA_PAD_FL_SOURCE);
- if (source_pad < 0) {
- dev_err(>dev, "Couldn't find output pad for subdev
%s\n",
- s_subdev->name);
- return source_pad;
- }
-
- ret = media_create_pad_link(_subdev->entity, source_pad,
- >sd.entity, 0,
- MEDIA_LNK_FL_ENABLED |
- MEDIA_LNK_FL_IMMUTABLE);
+ ret = media_create_fwnode_pad_links(>sd.entity.pads[0],
+ dev_fwnode(>dev),
+ _subdev->entity,
+ dev_fwnode(s_subdev->dev),
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
if (ret) {
dev_err(>dev, "Couldn't create media link %d", ret);
return ret;
--
2.17.1
[PATCH 09/22] media: sun6i: Fix fwnode media link creation
sun6i_csi_link_entity() passes the bound subdev's sd->fwnode to
media_entity_get_fwnode_pad(). This is likely not an endpoint
fwnode as required by media_entity_get_fwnode_pad(), for most
subdevices it is the port parent of endpoint fwnode(s). This has only
worked before because no entities have implemented the .get_fwnode_pad()
op yet, and the default behavior of media_entity_get_fwnode_pad()
was to ignore the passed fwnode and return the first pad that matches
the given direction flags.
Fix this by replacing the calls to media_entity_get_fwnode_pad() and
media_create_pad_link() with a call to media_create_fwnode_pad_links().
Fixes: 5cc7522d89655 ("media: sun6i: Add support for Allwinner CSI V3s")
Signed-off-by: Steve Longerbeam
---
.../platform/sunxi/sun6i-csi/sun6i_csi.c | 36 +++
1 file changed, 5 insertions(+), 31 deletions(-)
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 6e0e894154f4..9cf726b5c25a 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -630,37 +630,11 @@ static int sun6i_csi_link_entity(struct sun6i_csi *csi,
struct media_entity *entity,
struct fwnode_handle *fwnode)
{
- struct media_entity *sink;
- struct media_pad *sink_pad;
- int src_pad_index;
- int ret;
-
- ret = media_entity_get_fwnode_pad(entity, fwnode, MEDIA_PAD_FL_SOURCE);
- if (ret < 0) {
- dev_err(csi->dev, "%s: no source pad in external entity %s\n",
- __func__, entity->name);
- return -EINVAL;
- }
-
- src_pad_index = ret;
-
- sink = >video.vdev.entity;
- sink_pad = >video.pad;
-
- dev_dbg(csi->dev, "creating %s:%u -> %s:%u link\n",
- entity->name, src_pad_index, sink->name, sink_pad->index);
- ret = media_create_pad_link(entity, src_pad_index, sink,
- sink_pad->index,
- MEDIA_LNK_FL_ENABLED |
- MEDIA_LNK_FL_IMMUTABLE);
- if (ret < 0) {
- dev_err(csi->dev, "failed to create %s:%u -> %s:%u link\n",
- entity->name, src_pad_index,
- sink->name, sink_pad->index);
- return ret;
- }
-
- return 0;
+ return media_create_fwnode_pad_links(>video.pad,
+dev_fwnode(csi->dev),
+entity, fwnode,
+MEDIA_LNK_FL_ENABLED |
+MEDIA_LNK_FL_IMMUTABLE);
}
static int sun6i_subdev_notify_complete(struct v4l2_async_notifier *notifier)
--
2.17.1
[PATCH 19/22] media: imx: csi: Lookup endpoint fwnode with media_entity_get_fwnode_pad
Fix the 1:1 port-id:pad-index assumption for the upstream subdevice
connected to a CSI via one of the subdevice's pads, by searching the
upstream subdevice's endpoints for one that maps to the pad's index,
using media_entity_get_fwnode_pad(). This is carried out by a new
reverse mapping function imx_media_get_pad_fwnode().
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 22 --
drivers/staging/media/imx/imx-media-utils.c | 33 +
drivers/staging/media/imx/imx-media.h | 1 +
drivers/staging/media/imx/imx7-media-csi.c | 25 +---
4 files changed, 47 insertions(+), 34 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 37b10cbf3c1a..38e70082ff2d 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -164,7 +164,7 @@ static inline bool requires_passthrough(struct
v4l2_fwnode_endpoint *ep,
static int csi_get_upstream_endpoint(struct csi_priv *priv,
struct v4l2_fwnode_endpoint *ep)
{
- struct device_node *endpoint, *port;
+ struct fwnode_handle *endpoint;
struct media_entity *src;
struct v4l2_subdev *sd;
struct media_pad *pad;
@@ -203,23 +203,13 @@ static int csi_get_upstream_endpoint(struct csi_priv
*priv,
if (!pad)
return -ENODEV;
- sd = media_entity_to_v4l2_subdev(pad->entity);
+ endpoint = imx_media_get_pad_fwnode(pad);
+ if (IS_ERR(endpoint))
+ return PTR_ERR(endpoint);
- /*
-* NOTE: this assumes an OF-graph port id is the same as a
-* media pad index.
-*/
- port = of_graph_get_port_by_id(sd->dev->of_node, pad->index);
- if (!port)
- return -ENODEV;
-
- endpoint = of_get_next_child(port, NULL);
- of_node_put(port);
- if (!endpoint)
- return -ENODEV;
+ v4l2_fwnode_endpoint_parse(endpoint, ep);
- v4l2_fwnode_endpoint_parse(of_fwnode_handle(endpoint), ep);
- of_node_put(endpoint);
+ fwnode_handle_put(endpoint);
return 0;
}
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index 4cc6a7462ae2..2e4f27c214c4 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -908,6 +908,39 @@ imx_media_pipeline_video_device(struct media_entity
*start_entity,
}
EXPORT_SYMBOL_GPL(imx_media_pipeline_video_device);
+/*
+ * Find a fwnode endpoint that maps to the given subdevice's pad.
+ * If there are multiple endpoints that map to the pad, only the
+ * first endpoint encountered is returned.
+ *
+ * On success the refcount of the returned fwnode endpoint is
+ * incremented.
+ */
+struct fwnode_handle *imx_media_get_pad_fwnode(struct media_pad *pad)
+{
+ struct fwnode_handle *endpoint;
+ struct v4l2_subdev *sd;
+
+ if (!is_media_entity_v4l2_subdev(pad->entity))
+ return ERR_PTR(-ENODEV);
+
+ sd = media_entity_to_v4l2_subdev(pad->entity);
+
+ fwnode_graph_for_each_endpoint(dev_fwnode(sd->dev), endpoint) {
+ int pad_idx = media_entity_get_fwnode_pad(>entity,
+ endpoint,
+ pad->flags);
+ if (pad_idx < 0)
+ continue;
+
+ if (pad_idx == pad->index)
+ return endpoint;
+ }
+
+ return ERR_PTR(-ENODEV);
+}
+EXPORT_SYMBOL_GPL(imx_media_get_pad_fwnode);
+
/*
* Turn current pipeline streaming on/off starting from entity.
*/
diff --git a/drivers/staging/media/imx/imx-media.h
b/drivers/staging/media/imx/imx-media.h
index 4d124a86b358..d75b3c80101a 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -195,6 +195,7 @@ imx_media_pipeline_subdev(struct media_entity
*start_entity, u32 grp_id,
struct video_device *
imx_media_pipeline_video_device(struct media_entity *start_entity,
enum v4l2_buf_type buftype, bool upstream);
+struct fwnode_handle *imx_media_get_pad_fwnode(struct media_pad *pad);
struct imx_media_dma_buf {
void *virt;
diff --git a/drivers/staging/media/imx/imx7-media-csi.c
b/drivers/staging/media/imx/imx7-media-csi.c
index f71ac485f780..50826e637cb5 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -438,9 +438,8 @@ static int imx7_csi_get_upstream_endpoint(struct imx7_csi
*csi,
struct v4l2_fwnode_endpoint *ep,
bool skip_mux)
{
- struct device_node *endpoint, *port;
+ struct fwnode_handle *endpoint;
struct media_entity
[PATCH 02/22] media: entity: Modify default behavior of media_entity_get_fwnode_pad
Modify the default behavior of media_entity_get_fwnode_pad() (when the
entity does not provide the get_fwnode_pad op) to first assume the entity
implements a 1:1 mapping between fwnode port number and media pad index.
If the 1:1 mapping is not valid, e.g. the port number falls outside the
entity's pad index range, or the pad at that port number doesn't match the
given direction_flags, fall-back to the previous behavior that searches
the entity for the first pad that matches the given direction_flags.
The previous default behavior can choose the wrong pad for entities with
multiple sink or source pads. With this change the function will choose
the correct pad index if the entity implements a 1:1 port to pad mapping
at that port.
Add some comments to the @get_fwnode_pad operation to make it more clear
under what conditions entities must implement the operation.
Signed-off-by: Steve Longerbeam
---
drivers/media/mc/mc-entity.c | 25 -
include/media/media-entity.h | 21 +++--
2 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index c20f790a..47a39d9383d8 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -370,22 +370,37 @@ int media_entity_get_fwnode_pad(struct media_entity
*entity,
unsigned long direction_flags)
{
struct fwnode_endpoint endpoint;
- unsigned int i;
int ret;
+ ret = fwnode_graph_parse_endpoint(fwnode, );
+ if (ret)
+ return ret;
+
if (!entity->ops || !entity->ops->get_fwnode_pad) {
+ unsigned int i;
+
+ /*
+* for the default case, first try a 1:1 mapping between
+* fwnode port number and pad index.
+*/
+ ret = endpoint.port;
+ if (ret < entity->num_pads &&
+ (entity->pads[ret].flags & direction_flags))
+ return ret;
+
+ /*
+* if that didn't work search the entity for the first
+* pad that matches the @direction_flags.
+*/
for (i = 0; i < entity->num_pads; i++) {
if (entity->pads[i].flags & direction_flags)
return i;
}
+ /* else fail */
return -ENXIO;
}
- ret = fwnode_graph_parse_endpoint(fwnode, );
- if (ret)
- return ret;
-
ret = entity->ops->get_fwnode_pad(entity, );
if (ret < 0)
return ret;
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index cde80ad029b7..ed00adb4313b 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -199,6 +199,12 @@ struct media_pad {
* @get_fwnode_pad:Return the pad number based on a fwnode endpoint or
* a negative value on error. This operation can be used
* to map a fwnode to a media pad number. Optional.
+ * Entities do not need to implement this operation
+ * unless two conditions are met:
+ * - the entity has more than one sink and/or source
+ * pad, _and_
+ * - the entity does not implement a 1:1 mapping of
+ * fwnode port numbers to pad indexes.
* @link_setup:Notify the entity of link changes. The
operation can
* return an error, in which case link setup will be
* cancelled. Optional.
@@ -858,21 +864,24 @@ struct media_link *media_entity_find_link(struct
media_pad *source,
struct media_pad *media_entity_remote_pad(const struct media_pad *pad);
/**
- * media_entity_get_fwnode_pad - Get pad number from fwnode
+ * media_entity_get_fwnode_pad - Get pad number from an endpoint fwnode
*
* @entity: The entity
- * @fwnode: Pointer to the fwnode_handle which should be used to find the pad
+ * @fwnode: Pointer to the endpoint fwnode_handle which should be used to
+ * find the pad
* @direction_flags: Expected direction of the pad, as defined in
* :ref:`include/uapi/linux/media.h `
* (seek for ``MEDIA_PAD_FL_*``)
*
* This function can be used to resolve the media pad number from
- * a fwnode. This is useful for devices which use more complex
- * mappings of media pads.
+ * an endpoint fwnode. This is useful for devices which use more
+ * complex mappings of media pads.
*
* If the entity does not implement the get_fwnode_pad() operation
- * then this function searches the entity for the first pad that
- * matches the @direction_flags.
+ * then this function first assumes the entity implements a 1:1 mapping
+ * between fwnode port number and
[PATCH 18/22] media: imx: csi: Create media links in bound notifier
Implement a notifier bound op to register media links from the remote
sub-device's source pad(s) to the CSI sink pad.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 24 +++
1 file changed, 24 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index b39d79e63ac2..37b10cbf3c1a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -120,6 +120,11 @@ static inline struct csi_priv *sd_to_dev(struct
v4l2_subdev *sdev)
return container_of(sdev, struct csi_priv, sd);
}
+static inline struct csi_priv *notifier_to_dev(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct csi_priv, notifier);
+}
+
static inline bool is_parallel_bus(struct v4l2_fwnode_endpoint *ep)
{
return ep->bus_type != V4L2_MBUS_CSI2_DPHY;
@@ -1895,6 +1900,23 @@ static int imx_csi_parse_endpoint(struct device *dev,
return fwnode_device_is_available(asd->match.fwnode) ? 0 : -ENOTCONN;
}
+static int imx_csi_notify_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ struct csi_priv *priv = notifier_to_dev(notifier);
+ struct media_pad *sink = >sd.entity.pads[CSI_SINK_PAD];
+
+ return media_create_fwnode_pad_links(sink,
+dev_fwnode(priv->dev->parent),
+>entity,
+dev_fwnode(sd->dev), 0);
+}
+
+static const struct v4l2_async_notifier_operations csi_notify_ops = {
+ .bound = imx_csi_notify_bound,
+};
+
static int imx_csi_async_register(struct csi_priv *priv)
{
struct fwnode_handle *fwnode;
@@ -1903,6 +1925,8 @@ static int imx_csi_async_register(struct csi_priv *priv)
v4l2_async_notifier_init(>notifier);
+ priv->notifier.ops = _notify_ops;
+
fwnode = dev_fwnode(priv->dev);
/* get this CSI's port id */
--
2.17.1
[PATCH 21/22] media: imx: Use media_create_fwnode_links for external links
The entities external to the i.MX6 IPU and i.MX7 now create the links
to their fwnode-endpoint connected entities in their notifier bound
callbacks.
This should mean that there is no longer a need for the imx-media driver
to create these external fwnode-endpoint based links. But the v4l2-async
framework does not allow multiple subdevice notifiers to contain a
duplicate subdevice in their asd_list. Only the first subdev notifier that
discovers and adds that one subdevice to its asd_list will receive a bound
callback for it. Other subdevices that also have firmware endpoint
connections to this duplicate subdevice will not have it in their asd_list,
and thus will never receive a bound callback for it.
Until there is a solution to that problem, imx_media_create_links() is
still needed to create media links for subdevices that never received a
bound callback.
But media_create_fwnode_links() can now be used to do that work, so
imx_media_create_of_links() and imx_media_create_csi_of_links() are
removed.
Signed-off-by: Steve Longerbeam
---
.../staging/media/imx/imx-media-dev-common.c | 49
drivers/staging/media/imx/imx-media-of.c | 113 --
drivers/staging/media/imx/imx-media.h | 4 -
3 files changed, 27 insertions(+), 139 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-dev-common.c
b/drivers/staging/media/imx/imx-media-dev-common.c
index 66b505f7e8df..4dbe983af030 100644
--- a/drivers/staging/media/imx/imx-media-dev-common.c
+++ b/drivers/staging/media/imx/imx-media-dev-common.c
@@ -36,31 +36,36 @@ static int imx_media_subdev_bound(struct
v4l2_async_notifier *notifier,
static int imx_media_create_links(struct v4l2_async_notifier *notifier)
{
struct imx_media_dev *imxmd = notifier2dev(notifier);
- struct v4l2_subdev *sd;
+ struct fwnode_handle *sda_fwnode, *sdb_fwnode;
+ struct v4l2_subdev *sda, *sdb;
+
+ list_for_each_entry(sda, >v4l2_dev.subdevs, list) {
+ /* links have already been created for the IPU subdevs */
+ if (is_ipu_internal_subdev(sda))
+ continue;
+
+ sda_fwnode = (sda->grp_id & IMX_MEDIA_GRP_ID_IPU_CSI) ?
+ dev_fwnode(sda->dev->parent) :
+ dev_fwnode(sda->dev);
+
+ list_for_each_entry(sdb, >v4l2_dev.subdevs, list) {
+ if (sda == sdb)
+ continue;
+ if (is_ipu_internal_subdev(sdb))
+ continue;
+
+ sdb_fwnode = (sdb->grp_id & IMX_MEDIA_GRP_ID_IPU_CSI) ?
+ dev_fwnode(sdb->dev->parent) :
+ dev_fwnode(sdb->dev);
- list_for_each_entry(sd, >v4l2_dev.subdevs, list) {
- switch (sd->grp_id) {
- case IMX_MEDIA_GRP_ID_IPU_VDIC:
- case IMX_MEDIA_GRP_ID_IPU_IC_PRP:
- case IMX_MEDIA_GRP_ID_IPU_IC_PRPENC:
- case IMX_MEDIA_GRP_ID_IPU_IC_PRPVF:
- /*
-* links have already been created for the
-* sync-registered subdevs.
-*/
- break;
- case IMX_MEDIA_GRP_ID_IPU_CSI0:
- case IMX_MEDIA_GRP_ID_IPU_CSI1:
- case IMX_MEDIA_GRP_ID_CSI:
- imx_media_create_csi_of_links(imxmd, sd);
- break;
- default:
/*
-* if this subdev has fwnode links, create media
-* links for them.
+* if there are fwnode endpoint connections between
+* these subdevs, create media links for them.
*/
- imx_media_create_of_links(imxmd, sd);
- break;
+ __media_create_fwnode_links(>entity,
+ sda_fwnode,
+ >entity,
+ sdb_fwnode, 0);
}
}
diff --git a/drivers/staging/media/imx/imx-media-of.c
b/drivers/staging/media/imx/imx-media-of.c
index 736c954a8ff5..82e13e972e23 100644
--- a/drivers/staging/media/imx/imx-media-of.c
+++ b/drivers/staging/media/imx/imx-media-of.c
@@ -74,116 +74,3 @@ int imx_media_add_of_subdevs(struct imx_media_dev *imxmd,
return ret;
}
EXPORT_SYMBOL_GPL(imx_media_add_of_subdevs);
-
-/*
- * Create a single media link to/from sd using a fwnode link.
- *
- * NOTE: this function assumes an OF port node is equivalent to
- * a media pad (port id equal to media pad index), and that an
- * OF endpoint node is equivalent to a media link.
- */
-static int create_of_link(struct imx_media_dev *imxmd
[PATCH 01/22] media: entity: Pass entity to get_fwnode_pad operation
Add a missing pointer to the entity in the media_entity operation
get_fwnode_pad.
Fixes: ae45cd5efc120 ("[media] media: entity: Add get_fwnode_pad entity
operation")
Signed-off-by: Steve Longerbeam
---
drivers/media/mc/mc-entity.c | 2 +-
include/media/media-entity.h | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 7c429ce98bae..c20f790a 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -386,7 +386,7 @@ int media_entity_get_fwnode_pad(struct media_entity *entity,
if (ret)
return ret;
- ret = entity->ops->get_fwnode_pad();
+ ret = entity->ops->get_fwnode_pad(entity, );
if (ret < 0)
return ret;
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 8cb2c504a05c..cde80ad029b7 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -212,7 +212,8 @@ struct media_pad {
*mutex held.
*/
struct media_entity_operations {
- int (*get_fwnode_pad)(struct fwnode_endpoint *endpoint);
+ int (*get_fwnode_pad)(struct media_entity *entity,
+ struct fwnode_endpoint *endpoint);
int (*link_setup)(struct media_entity *entity,
const struct media_pad *local,
const struct media_pad *remote, u32 flags);
--
2.17.1
[PATCH 12/22] media: video-mux: Create media links in bound notifier
Implement a notifier bound op to register media links from the remote
sub-device's source pads to all of the video-mux sink pads.
Signed-off-by: Steve Longerbeam
---
drivers/media/platform/video-mux.c | 24
1 file changed, 24 insertions(+)
diff --git a/drivers/media/platform/video-mux.c
b/drivers/media/platform/video-mux.c
index ca1cef783646..de5b0698fc1d 100644
--- a/drivers/media/platform/video-mux.c
+++ b/drivers/media/platform/video-mux.c
@@ -36,6 +36,12 @@ static const struct v4l2_mbus_framefmt
video_mux_format_mbus_default = {
.field = V4L2_FIELD_NONE,
};
+static inline struct video_mux *
+notifier_to_video_mux(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct video_mux, notifier);
+}
+
static inline struct video_mux *v4l2_subdev_to_video_mux(struct v4l2_subdev
*sd)
{
return container_of(sd, struct video_mux, subdev);
@@ -343,6 +349,22 @@ static int video_mux_parse_endpoint(struct device *dev,
return fwnode_device_is_available(asd->match.fwnode) ? 0 : -ENOTCONN;
}
+static int video_mux_notify_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ struct video_mux *vmux = notifier_to_video_mux(notifier);
+
+ return media_create_fwnode_links(>subdev.entity,
+dev_fwnode(vmux->subdev.dev),
+>entity,
+dev_fwnode(sd->dev), 0);
+}
+
+static const struct v4l2_async_notifier_operations video_mux_notify_ops = {
+ .bound = video_mux_notify_bound,
+};
+
static int video_mux_async_register(struct video_mux *vmux,
unsigned int num_input_pads)
{
@@ -357,6 +379,8 @@ static int video_mux_async_register(struct video_mux *vmux,
v4l2_async_notifier_init(>notifier);
+ vmux->notifier.ops = _mux_notify_ops;
+
ret = v4l2_async_register_fwnode_subdev(
>subdev, >notifier,
sizeof(struct v4l2_async_subdev),
--
2.17.1
[PATCH 16/22] media: imx: csi: Implement get_fwnode_pad
The CSI does not have a 1:1 relationship between fwnode port numbers and
pad indexes. In fact the CSI fwnode device is itself a port which is the
sink, containing only a single fwnode endpoint. Implement media_entity
operation get_fwnode_pad to first verify the given endpoint is the CSI's
sink endpoint, and if so return the CSI sink pad index.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 23 +++
1 file changed, 23 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 367e39f5b382..e193f66fa230 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1824,9 +1824,32 @@ static void csi_unregistered(struct v4l2_subdev *sd)
ipu_csi_put(priv->csi);
}
+/*
+ * The CSI has only one fwnode endpoint, at the sink pad. Verify the
+ * endpoint belongs to us, and return CSI_SINK_PAD.
+ */
+static int csi_get_fwnode_pad(struct media_entity *entity,
+ struct fwnode_endpoint *endpoint)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct csi_priv *priv = v4l2_get_subdevdata(sd);
+ struct fwnode_handle *csi_port = dev_fwnode(priv->dev);
+ struct fwnode_handle *csi_ep;
+ int ret;
+
+ csi_ep = fwnode_get_next_child_node(csi_port, NULL);
+
+ ret = endpoint->local_fwnode == csi_ep ? CSI_SINK_PAD : -ENXIO;
+
+ fwnode_handle_put(csi_ep);
+
+ return ret;
+}
+
static const struct media_entity_operations csi_entity_ops = {
.link_setup = csi_link_setup,
.link_validate = v4l2_subdev_link_validate,
+ .get_fwnode_pad = csi_get_fwnode_pad,
};
static const struct v4l2_subdev_core_ops csi_core_ops = {
--
2.17.1
[PATCH 07/22] media: rcar-csi2: Fix fwnode media link creation
rcsi2_notify_bound() passes the bound subdev's match fwnode to
media_entity_get_fwnode_pad() to determine the subdev's source
pad for creating the media link to it. When the bound subdev is
the adv748x-csi2 transmitter, this is in fact correctly the endpoint
fwnode. For other subdevices this likely will not be the case, the
asd match fwnode is usually not an endpoint fwnode but rather the
port parent fwnode. So rcar-csi2 will fail to get the correct source
pad for bound subdev's other than the adv748x.
To fix and make rcar-csi2 connect more generally to other subdevices,
replace the calls to media_entity_get_fwnode_pad() and
media_create_pad_link() with a call to media_create_fwnode_pad_links().
Fixes: 769afd212b160 ("media: rcar-csi2: add Renesas R-Car MIPI CSI-2 receiver
driver")
Signed-off-by: Steve Longerbeam
---
drivers/media/platform/rcar-vin/rcar-csi2.c | 23 +
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/drivers/media/platform/rcar-vin/rcar-csi2.c
b/drivers/media/platform/rcar-vin/rcar-csi2.c
index c14af1b929df..13242cbad8d3 100644
--- a/drivers/media/platform/rcar-vin/rcar-csi2.c
+++ b/drivers/media/platform/rcar-vin/rcar-csi2.c
@@ -738,23 +738,20 @@ static int rcsi2_notify_bound(struct v4l2_async_notifier
*notifier,
struct v4l2_async_subdev *asd)
{
struct rcar_csi2 *priv = notifier_to_csi2(notifier);
- int pad;
+ int ret;
- pad = media_entity_get_fwnode_pad(>entity, asd->match.fwnode,
- MEDIA_PAD_FL_SOURCE);
- if (pad < 0) {
- dev_err(priv->dev, "Failed to find pad for %s\n", subdev->name);
- return pad;
- }
+ ret = media_create_fwnode_pad_links(>subdev.entity.pads[0],
+ dev_fwnode(priv->dev),
+ >entity,
+ dev_fwnode(subdev->dev),
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret)
+ return ret;
priv->remote = subdev;
- dev_dbg(priv->dev, "Bound %s pad: %d\n", subdev->name, pad);
-
- return media_create_pad_link(>entity, pad,
->subdev.entity, 0,
-MEDIA_LNK_FL_ENABLED |
-MEDIA_LNK_FL_IMMUTABLE);
+ return 0;
}
static void rcsi2_notify_unbind(struct v4l2_async_notifier *notifier,
--
2.17.1
[PATCH 20/22] media: imx: Add is_ipu_internal_subdev()
Add a convenience inline function that returns true if the given
subdevice is one of the i.MX5/6 IPU-internal subdevices.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media.h | 8
1 file changed, 8 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media.h
b/drivers/staging/media/imx/imx-media.h
index d75b3c80101a..c42a2cafb8fb 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -282,4 +282,12 @@ void imx_media_capture_device_error(struct
imx_media_video_dev *vdev);
#define IMX_MEDIA_GRP_ID_IPU_IC_PRPENC BIT(14)
#define IMX_MEDIA_GRP_ID_IPU_IC_PRPVF BIT(15)
+static inline bool is_ipu_internal_subdev(struct v4l2_subdev *sd)
+{
+ return sd->grp_id == IMX_MEDIA_GRP_ID_IPU_VDIC ||
+ sd->grp_id == IMX_MEDIA_GRP_ID_IPU_IC_PRP ||
+ sd->grp_id == IMX_MEDIA_GRP_ID_IPU_IC_PRPENC ||
+ sd->grp_id == IMX_MEDIA_GRP_ID_IPU_IC_PRPVF;
+}
+
#endif
--
2.17.1
[PATCH 04/22] media: Move v4l2_fwnode_parse_link from v4l2 to driver base
There is nothing v4l2-specific about v4l2_fwnode_{parse|put}_link().
Make these functions more generally available by moving them to driver
base, with the appropriate name changes to the functions and struct.
In the process embed a 'struct fwnode_endpoint' in 'struct fwnode_link'
for both sides of the link, and make use of fwnode_graph_parse_endpoint()
to fully parse both endpoints. Rename members local_node and
remote_node to more descriptive local_port_parent and
remote_port_parent.
Signed-off-by: Steve Longerbeam
---
drivers/base/property.c | 63 +++
drivers/media/platform/xilinx/xilinx-vipp.c | 69 +++--
drivers/media/v4l2-core/v4l2-fwnode.c | 39
drivers/staging/media/imx/imx-media-of.c| 49 +++
include/linux/fwnode.h | 14 +
include/linux/property.h| 5 ++
include/media/v4l2-fwnode.h | 44 -
7 files changed, 141 insertions(+), 142 deletions(-)
diff --git a/drivers/base/property.c b/drivers/base/property.c
index 81bd01ed4042..dd82cd150d84 100644
--- a/drivers/base/property.c
+++ b/drivers/base/property.c
@@ -1100,6 +1100,69 @@ int fwnode_graph_parse_endpoint(const struct
fwnode_handle *fwnode,
}
EXPORT_SYMBOL(fwnode_graph_parse_endpoint);
+/**
+ * fwnode_graph_parse_link() - parse a link between two endpoints
+ * @local_endpoint: the endpoint's fwnode at the local end of the link
+ * @link: pointer to the fwnode link data structure
+ *
+ * Fill the link structure with the parsed local and remote endpoint info
+ * and the local and remote port parent nodes.
+ *
+ * A reference is taken to both the local and remote port parent nodes,
+ * the caller must use fwnode_graph_put_link() to drop the references
+ * when done with the link.
+ *
+ * Return: 0 on success, or -ENOLINK if the remote endpoint fwnode
+ * can't be found.
+ */
+int fwnode_graph_parse_link(struct fwnode_handle *local_endpoint,
+ struct fwnode_link *link)
+{
+ struct fwnode_handle *remote_endpoint;
+ int ret;
+
+ memset(link, 0, sizeof(*link));
+
+ ret = fwnode_graph_parse_endpoint(local_endpoint, >local);
+ if (ret < 0)
+ return ret;
+
+ remote_endpoint = fwnode_graph_get_remote_endpoint(local_endpoint);
+ if (!remote_endpoint)
+ return -ENOLINK;
+
+ ret = fwnode_graph_parse_endpoint(remote_endpoint, >remote);
+ if (ret < 0) {
+ fwnode_handle_put(remote_endpoint);
+ return ret;
+ }
+
+ link->local_port_parent =
+ fwnode_graph_get_port_parent(local_endpoint);
+ link->remote_port_parent =
+ fwnode_graph_get_port_parent(remote_endpoint);
+
+ fwnode_handle_put(remote_endpoint);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(fwnode_graph_parse_link);
+
+/**
+ * fwnode_graph_put_link() - drop references to port parent nodes in a link
+ * @link: pointer to the fwnode link data structure
+ *
+ * Drop references to the local and remote port parent nodes in the link.
+ * This function must be called on every link parsed with
+ * fwnode_graph_parse_link().
+ */
+void fwnode_graph_put_link(struct fwnode_link *link)
+{
+ fwnode_handle_put(link->local_port_parent);
+ fwnode_handle_put(link->remote_port_parent);
+}
+EXPORT_SYMBOL_GPL(fwnode_graph_put_link);
+
const void *device_get_match_data(struct device *dev)
{
return fwnode_call_ptr_op(dev_fwnode(dev), device_get_match_data, dev);
diff --git a/drivers/media/platform/xilinx/xilinx-vipp.c
b/drivers/media/platform/xilinx/xilinx-vipp.c
index cc2856efea59..9c0dfc694478 100644
--- a/drivers/media/platform/xilinx/xilinx-vipp.c
+++ b/drivers/media/platform/xilinx/xilinx-vipp.c
@@ -74,7 +74,7 @@ static int xvip_graph_build_one(struct xvip_composite_device
*xdev,
struct media_pad *local_pad;
struct media_pad *remote_pad;
struct xvip_graph_entity *ent;
- struct v4l2_fwnode_link link;
+ struct fwnode_link link;
struct fwnode_handle *ep = NULL;
int ret = 0;
@@ -89,7 +89,7 @@ static int xvip_graph_build_one(struct xvip_composite_device
*xdev,
dev_dbg(xdev->dev, "processing endpoint %p\n", ep);
- ret = v4l2_fwnode_parse_link(ep, );
+ ret = fwnode_graph_parse_link(ep, );
if (ret < 0) {
dev_err(xdev->dev, "failed to parse link for %p\n",
ep);
@@ -99,54 +99,55 @@ static int xvip_graph_build_one(struct
xvip_composite_device *xdev,
/* Skip sink ports, they will be processed from the other end of
* the link.
*/
- if (link.local_port >= local->num_pads) {
+ if (link.local.port >= local->num_pads) {
[PATCH 06/22] media: adv748x: csi2: Implement get_fwnode_pad
If the given endpoint fwnode passed to the .get_fwnode_pad() op is
the adv748x-csi2 TXA/TXB source endpoint, return the associated media
pad index ADV748X_CSI2_SOURCE. The adv748x-csi2 has no other media pads
that are associated with fwnode endpoints.
Signed-off-by: Steve Longerbeam
---
drivers/media/i2c/adv748x/adv748x-csi2.c | 21 +
1 file changed, 21 insertions(+)
diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c
b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 2091cda50935..810085a1f2f0 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -228,6 +228,24 @@ static const struct v4l2_subdev_ops adv748x_csi2_ops = {
.pad = _csi2_pad_ops,
};
+/*
-
+ * media_entity_operations
+ */
+
+static int adv748x_csi2_get_fwnode_pad(struct media_entity *entity,
+ struct fwnode_endpoint *endpoint)
+{
+ struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+ struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
+
+ return endpoint->local_fwnode == tx->sd.fwnode ?
+ ADV748X_CSI2_SOURCE : -ENXIO;
+}
+
+static const struct media_entity_operations adv748x_csi2_entity_ops = {
+ .get_fwnode_pad = adv748x_csi2_get_fwnode_pad,
+};
+
/*
-
* Subdev module and controls
*/
@@ -295,6 +313,9 @@ int adv748x_csi2_init(struct adv748x_state *state, struct
adv748x_csi2 *tx)
/* Register internal ops for incremental subdev registration */
tx->sd.internal_ops = _csi2_internal_ops;
+ /* Register media_entity ops */
+ tx->sd.entity.ops = _csi2_entity_ops;
+
tx->pads[ADV748X_CSI2_SINK].flags = MEDIA_PAD_FL_SINK;
tx->pads[ADV748X_CSI2_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
--
2.17.1
[PATCH 08/22] media: cadence: csi2rx: Fix fwnode media link creation
csi2rx_async_bound() passes the bound subdev's sd->fwnode to
media_entity_get_fwnode_pad(). This is likely not an endpoint
fwnode as required by media_entity_get_fwnode_pad(), for most
subdevices it is the port parent of endpoint fwnode(s). This has only
worked before because no entities have implemented the .get_fwnode_pad()
op yet, and the default behavior of media_entity_get_fwnode_pad()
was to ignore the passed fwnode and return the first pad that matches
the given direction flags.
Fix this by replacing the calls to media_entity_get_fwnode_pad() and
media_create_pad_link() with a call to media_create_fwnode_pad_links().
Fixes: 1fc3b37f34f69 ("media: v4l: cadence: Add Cadence MIPI-CSI2 RX driver")
Signed-off-by: Steve Longerbeam
---
drivers/media/platform/cadence/cdns-csi2rx.c | 27
1 file changed, 10 insertions(+), 17 deletions(-)
diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c
b/drivers/media/platform/cadence/cdns-csi2rx.c
index 31ace114eda1..d478800a3859 100644
--- a/drivers/media/platform/cadence/cdns-csi2rx.c
+++ b/drivers/media/platform/cadence/cdns-csi2rx.c
@@ -83,7 +83,6 @@ struct csi2rx_priv {
/* Remote source */
struct v4l2_async_subdevasd;
struct v4l2_subdev *source_subdev;
- int source_pad;
};
static inline
@@ -251,26 +250,20 @@ static int csi2rx_async_bound(struct v4l2_async_notifier
*notifier,
{
struct v4l2_subdev *subdev = notifier->sd;
struct csi2rx_priv *csi2rx = v4l2_subdev_to_csi2rx(subdev);
+ int ret;
- csi2rx->source_pad = media_entity_get_fwnode_pad(_subdev->entity,
-s_subdev->fwnode,
-MEDIA_PAD_FL_SOURCE);
- if (csi2rx->source_pad < 0) {
- dev_err(csi2rx->dev, "Couldn't find output pad for subdev %s\n",
- s_subdev->name);
- return csi2rx->source_pad;
- }
+ ret = media_create_fwnode_pad_links(>subdev.entity.pads[0],
+ dev_fwnode(csi2rx->dev),
+ _subdev->entity,
+ dev_fwnode(s_subdev->dev),
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret)
+ return ret;
csi2rx->source_subdev = s_subdev;
- dev_dbg(csi2rx->dev, "Bound %s pad: %d\n", s_subdev->name,
- csi2rx->source_pad);
-
- return media_create_pad_link(>source_subdev->entity,
-csi2rx->source_pad,
->subdev.entity, 0,
-MEDIA_LNK_FL_ENABLED |
-MEDIA_LNK_FL_IMMUTABLE);
+ return 0;
}
static const struct v4l2_async_notifier_operations csi2rx_notifier_ops = {
--
2.17.1
[PATCH 11/22] media: v4l2-fwnode: Pass notifier to v4l2_async_register_fwnode_subdev()
Instead of allocating a notifier in v4l2_async_register_fwnode_subdev(),
have the caller provide one. This allows the caller to implement
notifier ops (bind, unbind).
The caller is now responsible for first initializing its notifier with a
call to v4l2_async_notifier_init().
Signed-off-by: Steve Longerbeam
---
drivers/media/platform/video-mux.c | 6 +-
drivers/media/v4l2-core/v4l2-fwnode.c | 11 +--
drivers/staging/media/imx/imx6-mipi-csi2.c | 5 -
drivers/staging/media/imx/imx7-media-csi.c | 5 -
drivers/staging/media/imx/imx7-mipi-csis.c | 5 -
include/media/v4l2-fwnode.h| 12
6 files changed, 26 insertions(+), 18 deletions(-)
diff --git a/drivers/media/platform/video-mux.c
b/drivers/media/platform/video-mux.c
index ddd0e338f9e4..ca1cef783646 100644
--- a/drivers/media/platform/video-mux.c
+++ b/drivers/media/platform/video-mux.c
@@ -21,6 +21,7 @@
struct video_mux {
struct v4l2_subdev subdev;
+ struct v4l2_async_notifier notifier;
struct media_pad *pads;
struct v4l2_mbus_framefmt *format_mbus;
struct mux_control *mux;
@@ -354,8 +355,11 @@ static int video_mux_async_register(struct video_mux *vmux,
for (i = 0; i < num_input_pads; i++)
ports[i] = i;
+ v4l2_async_notifier_init(>notifier);
+
ret = v4l2_async_register_fwnode_subdev(
- >subdev, sizeof(struct v4l2_async_subdev),
+ >subdev, >notifier,
+ sizeof(struct v4l2_async_subdev),
ports, num_input_pads, video_mux_parse_endpoint);
kfree(ports);
diff --git a/drivers/media/v4l2-core/v4l2-fwnode.c
b/drivers/media/v4l2-core/v4l2-fwnode.c
index 5d4ce4aa3fdc..8f5b3fdea061 100644
--- a/drivers/media/v4l2-core/v4l2-fwnode.c
+++ b/drivers/media/v4l2-core/v4l2-fwnode.c
@@ -1124,12 +1124,12 @@ int v4l2_async_register_subdev_sensor_common(struct
v4l2_subdev *sd)
EXPORT_SYMBOL_GPL(v4l2_async_register_subdev_sensor_common);
int v4l2_async_register_fwnode_subdev(struct v4l2_subdev *sd,
+ struct v4l2_async_notifier *notifier,
size_t asd_struct_size,
unsigned int *ports,
unsigned int num_ports,
parse_endpoint_func parse_endpoint)
{
- struct v4l2_async_notifier *notifier;
struct device *dev = sd->dev;
struct fwnode_handle *fwnode;
int ret;
@@ -1141,12 +1141,6 @@ int v4l2_async_register_fwnode_subdev(struct v4l2_subdev
*sd,
if (!fwnode_device_is_available(fwnode))
return -ENODEV;
- notifier = kzalloc(sizeof(*notifier), GFP_KERNEL);
- if (!notifier)
- return -ENOMEM;
-
- v4l2_async_notifier_init(notifier);
-
if (!ports) {
ret = v4l2_async_notifier_parse_fwnode_endpoints(dev, notifier,
asd_struct_size,
@@ -1171,15 +1165,12 @@ int v4l2_async_register_fwnode_subdev(struct
v4l2_subdev *sd,
if (ret < 0)
goto out_unregister;
- sd->subdev_notifier = notifier;
-
return 0;
out_unregister:
v4l2_async_notifier_unregister(notifier);
out_cleanup:
v4l2_async_notifier_cleanup(notifier);
- kfree(notifier);
return ret;
}
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c
b/drivers/staging/media/imx/imx6-mipi-csi2.c
index f29e28df36ed..408ee2765e77 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -35,6 +35,7 @@
struct csi2_dev {
struct device *dev;
struct v4l2_subdev sd;
+ struct v4l2_async_notifier notifier;
struct media_pad pad[CSI2_NUM_PADS];
struct clk *dphy_clk;
struct clk *pllref_clk;
@@ -643,8 +644,10 @@ static int csi2_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, >sd);
+ v4l2_async_notifier_init(>notifier);
+
ret = v4l2_async_register_fwnode_subdev(
- >sd, sizeof(struct v4l2_async_subdev),
+ >sd, >notifier, sizeof(struct v4l2_async_subdev),
_port, 1, csi2_parse_endpoint);
if (ret)
goto dphy_off;
diff --git a/drivers/staging/media/imx/imx7-media-csi.c
b/drivers/staging/media/imx/imx7-media-csi.c
index 500b4c08d967..a1c96c52a606 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -155,6 +155,7 @@
struct imx7_csi {
struct device *dev;
struct v4l2_subdev sd;
+ struct v4l2_async_notifier notifier;
struct imx_media_video_dev *vdev;
struct imx_media_dev *imxmd;
struct media_pad pad[IMX7_CSI_PADS_NUM];
@@ -1250,7 +125
[PATCH 22/22] media: imx: TODO: Remove issues regarding media link creation
Remove the TODO items regarding media link creation, these issues are resolved after exporting media link creation to individual entity bound callbacks and the use of media_create_fwnode_links(), media_create_fwnode_pad_links(), and media_entity_get_fwnode_pad(). Signed-off-by: Steve Longerbeam --- drivers/staging/media/imx/TODO | 29 - 1 file changed, 29 deletions(-) diff --git a/drivers/staging/media/imx/TODO b/drivers/staging/media/imx/TODO index 6f29b5ca5324..a371cdedcdb0 100644 --- a/drivers/staging/media/imx/TODO +++ b/drivers/staging/media/imx/TODO @@ -17,35 +17,6 @@ decided whether this feature is useful enough to make it generally available by exporting to v4l2-core. -- After all async subdevices have been bound, v4l2_fwnode_parse_link() - is used to form the media links between the devices discovered in - the OF graph. - - While this approach allows support for arbitrary OF graphs, there - are some assumptions for this to work: - - 1. If a port owned by a device in the graph has endpoint nodes, the - port is treated as a media pad. - - This presents problems for devices that don't make this port = pad - assumption. Examples are SMIAPP compatible cameras which define only - a single output port node, but which define multiple pads owned - by multiple subdevices (pixel-array, binner, scaler). Or video - decoders (entity function MEDIA_ENT_F_ATV_DECODER), which also define - only a single output port node, but define multiple pads for video, - VBI, and audio out. - - A workaround at present is to set the port reg properties to - correspond to the media pad index that the port represents. A - possible long-term solution is to implement a subdev API that - maps a port id to a media pad index. - - 2. Every endpoint of a port owned by a device in the graph is treated - as a media link. - - Which means a port must not contain mixed-use endpoints, they - must all refer to media links between V4L2 subdevices. - - i.MX7: all of the above, since it uses the imx media core - i.MX7: use Frame Interval Monitor -- 2.17.1
[PATCH 17/22] media: imx: csi: Embed notifier in struct csi_priv
Embed the notifier in 'struct csi_priv', instead of dynamically allocating
it, to make it possible to retrieve csi_priv in a notifier callback op.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-csi.c | 25 +--
1 file changed, 10 insertions(+), 15 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index e193f66fa230..b39d79e63ac2 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -58,6 +58,8 @@ struct csi_priv {
struct ipu_soc *ipu;
struct v4l2_subdev sd;
struct media_pad pad[CSI_NUM_PADS];
+ struct v4l2_async_notifier notifier;
+
/* the video device at IDMAC output pad */
struct imx_media_video_dev *vdev;
struct imx_media_fim *fim;
@@ -1895,31 +1897,28 @@ static int imx_csi_parse_endpoint(struct device *dev,
static int imx_csi_async_register(struct csi_priv *priv)
{
- struct v4l2_async_notifier *notifier;
struct fwnode_handle *fwnode;
unsigned int port;
int ret;
- notifier = kzalloc(sizeof(*notifier), GFP_KERNEL);
- if (!notifier)
- return -ENOMEM;
-
- v4l2_async_notifier_init(notifier);
+ v4l2_async_notifier_init(>notifier);
fwnode = dev_fwnode(priv->dev);
/* get this CSI's port id */
ret = fwnode_property_read_u32(fwnode, "reg", );
if (ret < 0)
- goto out_free;
+ return ret;
ret = v4l2_async_notifier_parse_fwnode_endpoints_by_port(
- priv->dev->parent, notifier, sizeof(struct v4l2_async_subdev),
+ priv->dev->parent, >notifier,
+ sizeof(struct v4l2_async_subdev),
port, imx_csi_parse_endpoint);
if (ret < 0)
goto out_cleanup;
- ret = v4l2_async_subdev_notifier_register(>sd, notifier);
+ ret = v4l2_async_subdev_notifier_register(>sd,
+ >notifier);
if (ret < 0)
goto out_cleanup;
@@ -1927,16 +1926,12 @@ static int imx_csi_async_register(struct csi_priv *priv)
if (ret < 0)
goto out_unregister;
- priv->sd.subdev_notifier = notifier;
-
return 0;
out_unregister:
- v4l2_async_notifier_unregister(notifier);
+ v4l2_async_notifier_unregister(>notifier);
out_cleanup:
- v4l2_async_notifier_cleanup(notifier);
-out_free:
- kfree(notifier);
+ v4l2_async_notifier_cleanup(>notifier);
return ret;
}
--
2.17.1
[PATCH] media: staging/imx: Fix NULL deref in find_pipeline_entity()
Fix a cut error in find_pipeline_entity(). The start entity must be
passed to media_entity_to_video_device() in find_pipeline_entity(), not
pad->entity. The pad is only put to use later, after determining the start
entity is not the entity being searched for.
Fixes: 3ef46bc97ca2 ("media: staging/imx: Improve pipeline searching")
Reported-by: Colin Ian King
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-utils.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index b5b8a3b7730a..6fb88c22ee27 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -842,7 +842,7 @@ find_pipeline_entity(struct media_entity *start, u32 grp_id,
if (sd->grp_id & grp_id)
return >entity;
} else if (buftype && is_media_entity_v4l2_video_device(start)) {
- vfd = media_entity_to_video_device(pad->entity);
+ vfd = media_entity_to_video_device(start);
if (buftype == vfd->queue->type)
return >entity;
}
--
2.17.1
Re: media: staging/imx: Improve pipeline searching (bug report)
Thanks for catching,
On 6/26/19 11:27 AM, Colin Ian King wrote:
Hi,
Static analysis with Coverity on Linux next has found a potential issue
with the following commit:
commit 3ef46bc97ca2c918b7657a08220c7340a9bb07a2
Author: Steve Longerbeam
Date: Fri May 10 17:50:11 2019 -0400
media: staging/imx: Improve pipeline searching
The issue is in drivers/staging/media/imx/imx-media-utils.c in function
find_pipeline_entity:
struct media_pad *pad = NULL;
pad is assigned a NULL
struct video_device *vfd;
struct v4l2_subdev *sd;
if (grp_id && is_media_entity_v4l2_subdev(start)) {
sd = media_entity_to_v4l2_subdev(start);
if (sd->grp_id & grp_id)
return >entity;
} else if (buftype && is_media_entity_v4l2_video_device(start)) {
vfd = media_entity_to_video_device(pad->entity);
..and above the null pad is being dereferenced causing a kernel oops.
yes, this is a typo and should be:
vfd = media_entity_to_video_device(start);
Sending a patch...
Steve
[PATCH v8 5/5] media: imx: Try colorimetry at both sink and source pads
Retask imx_media_fill_default_mbus_fields() to try colorimetry parameters,
renaming it to to imx_media_try_colorimetry(), and call it at both sink and
source pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.
The IC now supports Rec.709 and BT.601 Y'CbCr encoding, and both limited
and full range quantization for both YUV and RGB space, so allow those
for pipelines that route through the IC.
Signed-off-by: Steve Longerbeam
---
Changes in v7:
- squashed with "media: imx: Allow Rec.709 encoding for IC routes".
- remove the RGB full-range quantization restriction for IC routes.
---
drivers/staging/media/imx/imx-ic-prp.c | 6 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 8 +--
drivers/staging/media/imx/imx-media-csi.c | 19 +++---
drivers/staging/media/imx/imx-media-utils.c | 73 ++---
drivers/staging/media/imx/imx-media-vdic.c | 5 +-
drivers/staging/media/imx/imx-media.h | 5 +-
drivers/staging/media/imx/imx7-media-csi.c | 8 +--
7 files changed, 62 insertions(+), 62 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 10ffe00f1a54..f87fe0203720 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -193,8 +193,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
sdformat->format.code = cc->codes[0];
}
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
@@ -203,6 +203,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
break;
}
+ imx_media_try_colorimetry(>format, true);
+
fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
*fmt = sdformat->format;
out:
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index e8b36a181ccc..f2fe3c11c70e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -907,8 +907,6 @@ static void prp_try_fmt(struct prp_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
@@ -916,9 +914,11 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
}
+
+ imx_media_try_colorimetry(>format, true);
}
static int prp_set_fmt(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 1d248aca40a9..dce4addadff4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1375,9 +1375,15 @@ static void csi_try_field(struct csi_priv *priv,
struct v4l2_mbus_framefmt *infmt =
__csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
- /* no restrictions on sink pad field type */
- if (sdformat->pad == CSI_SINK_PAD)
+ /*
+* no restrictions on sink pad field type except must
+* be initialized.
+*/
+ if (sdformat->pad == CSI_SINK_PAD) {
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
return;
+ }
switch (infmt->field) {
case V4L2_FIELD_SEQ_TB:
@@ -1455,8 +1461,6 @@ static void csi_try_fmt(struct csi_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
break;
case CSI_SINK_PAD:
@@ -1476,10 +1480,6 @@ static void csi_try_fmt(struct csi_priv *priv,
csi_try_fi
[PATCH v5 1/9] Revert "media: staging/imx: add media device to capture register"
The imx6-specific subdevs that register a capture device will no
longer hold a reference to the media device, so this commit must be
reverted.
This reverts commit 16204b8a1c1af77725533b77936e6c73953486ae.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 2 +-
drivers/staging/media/imx/imx-media-capture.c | 6 +++---
drivers/staging/media/imx/imx-media-csi.c | 2 +-
drivers/staging/media/imx/imx-media.h | 3 +--
drivers/staging/media/imx/imx7-media-csi.c| 2 +-
5 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 3ca1422f6154..5c8e6ad8c025 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1270,7 +1270,7 @@ static int prp_registered(struct v4l2_subdev *sd)
if (ret)
return ret;
- ret = imx_media_capture_device_register(priv->md, priv->vdev);
+ ret = imx_media_capture_device_register(priv->vdev);
if (ret)
return ret;
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 7688238a3396..9703c85b19c4 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -706,8 +706,7 @@ void imx_media_capture_device_error(struct
imx_media_video_dev *vdev)
}
EXPORT_SYMBOL_GPL(imx_media_capture_device_error);
-int imx_media_capture_device_register(struct imx_media_dev *md,
- struct imx_media_video_dev *vdev)
+int imx_media_capture_device_register(struct imx_media_video_dev *vdev)
{
struct capture_priv *priv = to_capture_priv(vdev);
struct v4l2_subdev *sd = priv->src_sd;
@@ -716,7 +715,8 @@ int imx_media_capture_device_register(struct imx_media_dev
*md,
struct v4l2_subdev_format fmt_src;
int ret;
- priv->md = md;
+ /* get media device */
+ priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
vfd->v4l2_dev = sd->v4l2_dev;
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index c33d714ed953..41965d8b56c4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1816,7 +1816,7 @@ static int csi_registered(struct v4l2_subdev *sd)
if (ret)
goto free_fim;
- ret = imx_media_capture_device_register(priv->md, priv->vdev);
+ ret = imx_media_capture_device_register(priv->vdev);
if (ret)
goto free_fim;
diff --git a/drivers/staging/media/imx/imx-media.h
b/drivers/staging/media/imx/imx-media.h
index fc5d969ded79..dd603a6b3a70 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -272,8 +272,7 @@ int imx_media_of_add_csi(struct imx_media_dev *imxmd,
struct imx_media_video_dev *
imx_media_capture_device_init(struct v4l2_subdev *src_sd, int pad);
void imx_media_capture_device_remove(struct imx_media_video_dev *vdev);
-int imx_media_capture_device_register(struct imx_media_dev *md,
- struct imx_media_video_dev *vdev);
+int imx_media_capture_device_register(struct imx_media_video_dev *vdev);
void imx_media_capture_device_unregister(struct imx_media_video_dev *vdev);
struct imx_media_buffer *
imx_media_capture_device_next_buf(struct imx_media_video_dev *vdev);
diff --git a/drivers/staging/media/imx/imx7-media-csi.c
b/drivers/staging/media/imx/imx7-media-csi.c
index a708a0340eb1..18eb5d3ecf10 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -1126,7 +1126,7 @@ static int imx7_csi_registered(struct v4l2_subdev *sd)
if (ret < 0)
return ret;
- ret = imx_media_capture_device_register(csi->imxmd, csi->vdev);
+ ret = imx_media_capture_device_register(csi->vdev);
if (ret < 0)
return ret;
--
2.17.1
[PATCH v5 5/9] Revert "media: imx: Set capture compose rectangle in capture_device_set_format"
Rvert this commit, as imx_media_capture_device_set_format() will be
removed. The arguments to mx_media_mbus_fmt_to_pix_fmt() and
imx_media_capture_device_set_format() in imx7_csi_set_fmt() are also
reverted.
This reverts commit 5964cbd8692252615370b77eb96764dd70c2f837.
Signed-off-by: Steve Longerbeam
---
Chnges in v3:
- revert to previous args in imx7_csi_set_fmt().
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 5 ++--
drivers/staging/media/imx/imx-media-capture.c | 24 +--
drivers/staging/media/imx/imx-media-csi.c | 5 ++--
drivers/staging/media/imx/imx-media-utils.c | 20
drivers/staging/media/imx/imx-media.h | 6 ++---
drivers/staging/media/imx/imx7-media-csi.c| 5 ++--
6 files changed, 25 insertions(+), 40 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 8242d88dfb82..afaa3a8b15e9 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -910,7 +910,6 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
const struct imx_media_pixfmt *cc;
struct v4l2_pix_format vdev_fmt;
struct v4l2_mbus_framefmt *fmt;
- struct v4l2_rect vdev_compose;
int ret = 0;
if (sdformat->pad >= PRPENCVF_NUM_PADS)
@@ -952,11 +951,11 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
priv->cc[sdformat->pad] = cc;
/* propagate output pad format to capture device */
- imx_media_mbus_fmt_to_pix_fmt(_fmt, _compose,
+ imx_media_mbus_fmt_to_pix_fmt(_fmt,
>format_mbus[PRPENCVF_SRC_PAD],
priv->cc[PRPENCVF_SRC_PAD]);
mutex_unlock(>lock);
- imx_media_capture_device_set_format(vdev, _fmt, _compose);
+ imx_media_capture_device_set_format(vdev, _fmt);
return 0;
out:
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 335084a6b0cd..555f6204660b 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -205,8 +205,7 @@ static int capture_g_fmt_vid_cap(struct file *file, void
*fh,
static int __capture_try_fmt_vid_cap(struct capture_priv *priv,
struct v4l2_subdev_format *fmt_src,
-struct v4l2_format *f,
-struct v4l2_rect *compose)
+struct v4l2_format *f)
{
const struct imx_media_pixfmt *cc, *cc_src;
@@ -246,8 +245,7 @@ static int __capture_try_fmt_vid_cap(struct capture_priv
*priv,
}
}
- imx_media_mbus_fmt_to_pix_fmt(>fmt.pix, compose,
- _src->format, cc);
+ imx_media_mbus_fmt_to_pix_fmt(>fmt.pix, _src->format, cc);
return 0;
}
@@ -265,7 +263,7 @@ static int capture_try_fmt_vid_cap(struct file *file, void
*fh,
if (ret)
return ret;
- return __capture_try_fmt_vid_cap(priv, _src, f, NULL);
+ return __capture_try_fmt_vid_cap(priv, _src, f);
}
static int capture_s_fmt_vid_cap(struct file *file, void *fh,
@@ -273,7 +271,6 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
{
struct capture_priv *priv = video_drvdata(file);
struct v4l2_subdev_format fmt_src;
- struct v4l2_rect compose;
int ret;
if (vb2_is_busy(>q)) {
@@ -287,14 +284,17 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
if (ret)
return ret;
- ret = __capture_try_fmt_vid_cap(priv, _src, f, );
+ ret = __capture_try_fmt_vid_cap(priv, _src, f);
if (ret)
return ret;
priv->vdev.fmt.fmt.pix = f->fmt.pix;
priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat,
CS_SEL_ANY, true);
- priv->vdev.compose = compose;
+ priv->vdev.compose.left = 0;
+ priv->vdev.compose.top = 0;
+ priv->vdev.compose.width = fmt_src.format.width;
+ priv->vdev.compose.height = fmt_src.format.height;
return 0;
}
@@ -655,8 +655,7 @@ static struct video_device capture_videodev = {
};
void imx_media_capture_device_set_format(struct imx_media_video_dev *vdev,
-const struct v4l2_pix_format *pix,
-const struct v4l2_rect *compose)
+struct v4l2_pix_format *pix)
{
struct capture_priv *priv = to_capture_priv(vdev);
@@ -664,7 +663,6 @@ void imx_media_capture_device_set_format(struct
imx_media_video_dev *vdev,
priv->vdev.fmt.fmt.pix = *pix;
priv->vdev.cc = imx_media_find_format(pix->pixelformat, CS_SE
[PATCH v5 4/9] media: staging/imx: Move add_video_device into capture_device_register
Move imx_media_add_video_device() into imx_media_capture_device_register().
Also the former has no error conditions to convert to void.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 5 -
drivers/staging/media/imx/imx-media-capture.c | 3 +++
drivers/staging/media/imx/imx-media-csi.c | 7 +--
drivers/staging/media/imx/imx-media-utils.c | 9 -
drivers/staging/media/imx/imx-media.h | 4 ++--
drivers/staging/media/imx/imx7-media-csi.c| 12 +---
6 files changed, 11 insertions(+), 29 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index ddcd87a17c71..8242d88dfb82 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1241,7 +1241,6 @@ static int prp_s_frame_interval(struct v4l2_subdev *sd,
static int prp_registered(struct v4l2_subdev *sd)
{
struct prp_priv *priv = sd_to_priv(sd);
- struct imx_ic_priv *ic_priv = priv->ic_priv;
int i, ret;
u32 code;
@@ -1271,10 +1270,6 @@ static int prp_registered(struct v4l2_subdev *sd)
if (ret)
return ret;
- ret = imx_media_add_video_device(ic_priv->md, priv->vdev);
- if (ret)
- goto unreg;
-
ret = prp_init_controls(priv);
if (ret)
goto unreg;
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 211ec4df2066..335084a6b0cd 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -780,6 +780,9 @@ int imx_media_capture_device_register(struct
imx_media_video_dev *vdev)
vfd->ctrl_handler = >ctrl_hdlr;
+ /* add vdev to the video device list */
+ imx_media_add_video_device(priv->md, vdev);
+
return 0;
unreg:
video_unregister_device(vfd);
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index ea3d13103c91..c70fa6b509ae 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1820,13 +1820,8 @@ static int csi_registered(struct v4l2_subdev *sd)
if (ret)
goto free_fim;
- ret = imx_media_add_video_device(priv->md, priv->vdev);
- if (ret)
- goto unreg;
-
return 0;
-unreg:
- imx_media_capture_device_unregister(priv->vdev);
+
free_fim:
if (priv->fim)
imx_media_fim_free(priv->fim);
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index c52aa59acd05..8a6e57652402 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -767,18 +767,17 @@ imx_media_find_subdev_by_devname(struct imx_media_dev
*imxmd,
EXPORT_SYMBOL_GPL(imx_media_find_subdev_by_devname);
/*
- * Adds a video device to the master video device list. This is called by
- * an async subdev that owns a video device when it is registered.
+ * Adds a video device to the master video device list. This is called
+ * when a video device is registered.
*/
-int imx_media_add_video_device(struct imx_media_dev *imxmd,
- struct imx_media_video_dev *vdev)
+void imx_media_add_video_device(struct imx_media_dev *imxmd,
+ struct imx_media_video_dev *vdev)
{
mutex_lock(>mutex);
list_add_tail(>list, >vdev_list);
mutex_unlock(>mutex);
- return 0;
}
EXPORT_SYMBOL_GPL(imx_media_add_video_device);
diff --git a/drivers/staging/media/imx/imx-media.h
b/drivers/staging/media/imx/imx-media.h
index ba2d75bcc4c9..71e20f53ed7b 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -189,8 +189,8 @@ imx_media_find_subdev_by_fwnode(struct imx_media_dev *imxmd,
struct v4l2_subdev *
imx_media_find_subdev_by_devname(struct imx_media_dev *imxmd,
const char *devname);
-int imx_media_add_video_device(struct imx_media_dev *imxmd,
- struct imx_media_video_dev *vdev);
+void imx_media_add_video_device(struct imx_media_dev *imxmd,
+ struct imx_media_video_dev *vdev);
int imx_media_find_mipi_csi2_channel(struct imx_media_dev *imxmd,
struct media_entity *start_entity);
struct media_pad *
diff --git a/drivers/staging/media/imx/imx7-media-csi.c
b/drivers/staging/media/imx/imx7-media-csi.c
index 96d01d8af874..f2037aba6e0e 100644
--- a/drivers/staging/media/imx/imx7-media-csi.c
+++ b/drivers/staging/media/imx/imx7-media-csi.c
@@ -1126,17 +1126,7 @@ static int imx7_csi_registered(struct v4l2_subdev *sd)
if (ret < 0)
return ret;
- ret = imx_media_capture
[PATCH v5 9/9] media: staging/imx: Don't set driver data for v4l2_dev
The media device is already available via multiple methods, there is no
need to set driver data for v4l2_dev to the media device.
In imx_media_link_notify(), get media device from link->graph_obj.mdev.
In imx_media_capture_device_register(), get media device from
v4l2_dev->mdev.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-capture.c| 5 +++--
drivers/staging/media/imx/imx-media-dev-common.c | 7 ++-
2 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 8a908c3e5e60..ea7f2decfc16 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -735,15 +735,16 @@ int imx_media_capture_device_register(struct
imx_media_video_dev *vdev)
{
struct capture_priv *priv = to_capture_priv(vdev);
struct v4l2_subdev *sd = priv->src_sd;
+ struct v4l2_device *v4l2_dev = sd->v4l2_dev;
struct video_device *vfd = vdev->vfd;
struct vb2_queue *vq = >q;
struct v4l2_subdev_format fmt_src;
int ret;
/* get media device */
- priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+ priv->md = container_of(v4l2_dev->mdev, struct imx_media_dev, md);
- vfd->v4l2_dev = sd->v4l2_dev;
+ vfd->v4l2_dev = v4l2_dev;
ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
if (ret) {
diff --git a/drivers/staging/media/imx/imx-media-dev-common.c
b/drivers/staging/media/imx/imx-media-dev-common.c
index 89dc4ec8dadb..66b505f7e8df 100644
--- a/drivers/staging/media/imx/imx-media-dev-common.c
+++ b/drivers/staging/media/imx/imx-media-dev-common.c
@@ -260,10 +260,11 @@ static int imx_media_inherit_controls(struct
imx_media_dev *imxmd,
static int imx_media_link_notify(struct media_link *link, u32 flags,
unsigned int notification)
{
+ struct imx_media_dev *imxmd = container_of(link->graph_obj.mdev,
+ struct imx_media_dev, md);
struct media_entity *source = link->source->entity;
struct imx_media_pad_vdev *pad_vdev;
struct list_head *pad_vdev_list;
- struct imx_media_dev *imxmd;
struct video_device *vfd;
struct v4l2_subdev *sd;
int pad_idx, ret;
@@ -279,8 +280,6 @@ static int imx_media_link_notify(struct media_link *link,
u32 flags,
sd = media_entity_to_v4l2_subdev(source);
pad_idx = link->source->index;
- imxmd = dev_get_drvdata(sd->v4l2_dev->dev);
-
pad_vdev_list = to_pad_vdev_list(sd, pad_idx);
if (!pad_vdev_list) {
/* nothing to do if source sd has no pad vdev list */
@@ -384,8 +383,6 @@ struct imx_media_dev *imx_media_dev_init(struct device *dev,
goto cleanup;
}
- dev_set_drvdata(imxmd->v4l2_dev.dev, imxmd);
-
INIT_LIST_HEAD(>vdev_list);
v4l2_async_notifier_init(>notifier);
--
2.17.1
[PATCH v5 2/9] media: staging/imx: Switch to sync registration for IPU subdevs
Because the IPU sub-devices VDIC and IC are not present in the
device-tree, platform devices were created for them instead. This
allowed these sub-devices to be added to the media device's async
notifier and registered asynchronously along with the other
sub-devices that do have a device-tree presence (CSI and devices
external to the IPU and SoC).
But that approach isn't really necessary. The IPU sub-devices don't
actually require a backing device (sd->dev is allowed to be NULL).
And that approach can't get around the fact that the IPU sub-devices
are not part of a device hierarchy, which makes it awkward to retrieve
the parent IPU of these devices.
By registering them synchronously, they can be registered from the CSI
async bound notifier, so the init function for them can be given the CSI
subdev, who's dev->parent is the IPU. That is a somewhat cleaner way
to retrieve the parent IPU.
So convert to synchronous registration for the VDIC and IC task
sub-devices, at the time a CSI sub-device is bound. There is no longer
a backing device for them (sd->dev is NULL), but that's ok. Also
set the VDIC/IC sub-device owner as the IPU, so that a reference can
be taken on the IPU module.
Since the VDIC and IC task drivers are no longer platform drivers,
they are now statically linked to imx-media module.
Signed-off-by: Steve Longerbeam
---
Changes in v3:
- statically link VDIC and IC task objects to imx-media module in
Makefile.
---
drivers/staging/media/imx/Makefile| 6 +-
drivers/staging/media/imx/imx-ic-common.c | 70 ++--
drivers/staging/media/imx/imx-ic-prp.c| 34 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 70 ++--
drivers/staging/media/imx/imx-ic.h| 7 +-
drivers/staging/media/imx/imx-media-capture.c | 7 +-
drivers/staging/media/imx/imx-media-csi.c | 2 +-
drivers/staging/media/imx/imx-media-dev.c | 121 +-
.../staging/media/imx/imx-media-internal-sd.c | 356 --
drivers/staging/media/imx/imx-media-of.c | 38 +-
drivers/staging/media/imx/imx-media-vdic.c| 85 ++---
drivers/staging/media/imx/imx-media.h | 67 ++--
drivers/staging/media/imx/imx7-media-csi.c| 3 +-
13 files changed, 327 insertions(+), 539 deletions(-)
diff --git a/drivers/staging/media/imx/Makefile
b/drivers/staging/media/imx/Makefile
index d2d909a36239..86f0c81b6a3b 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -1,14 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
-imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o
+imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o \
+ imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o imx-media-vdic.o
imx-media-objs += imx-media-dev-common.o
imx-media-common-objs := imx-media-utils.o imx-media-fim.o
-imx-media-ic-objs := imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
-obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
-obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx6-mipi-csi2.o
diff --git a/drivers/staging/media/imx/imx-ic-common.c
b/drivers/staging/media/imx/imx-ic-common.c
index 1addb0893c57..ad0c291db03c 100644
--- a/drivers/staging/media/imx/imx-ic-common.c
+++ b/drivers/staging/media/imx/imx-ic-common.c
@@ -8,8 +8,6 @@
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*/
-#include
-#include
#include
#include
#include "imx-media.h"
@@ -24,23 +22,25 @@ static struct imx_ic_ops *ic_ops[IC_NUM_OPS] = {
[IC_TASK_VIEWFINDER] = _ic_prpencvf_ops,
};
-static int imx_ic_probe(struct platform_device *pdev)
+struct v4l2_subdev *imx_media_ic_register(struct imx_media_dev *imxmd,
+ struct device *ipu_dev,
+ struct ipu_soc *ipu,
+ u32 grp_id)
{
- struct imx_media_ipu_internal_sd_pdata *pdata;
+ struct v4l2_device *v4l2_dev = >v4l2_dev;
struct imx_ic_priv *priv;
int ret;
- priv = devm_kzalloc(>dev, sizeof(*priv), GFP_KERNEL);
+ priv = devm_kzalloc(ipu_dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
- platform_set_drvdata(pdev, >sd);
- priv->dev = >dev;
+ priv->ipu_dev = ipu_dev;
+ priv->ipu = ipu;
+ priv->md = imxmd;
- /* get our ipu_id, grp_id and IC task id */
- pdata = priv->dev->platform_data;
- priv->ipu_id = pdata->ipu_id;
- switch (pdata->grp_id) {
+ /* get our IC task id */
+ switch (grp_id) {
case IMX_MEDIA_GRP_
[PATCH v5 8/9] media: staging/imx: Improve pipeline searching
Export find_pipeline_pad(), renaming to imx_media_pipeline_pad(), and
extend its functionality to allow searching for video devices in the
enabled pipeline in addition to sub-devices.
As part of this:
- Rename imx_media_find_mipi_csi2_channel() to
imx_media_pipeline_csi2_channel().
- Remove imx_media_find_upstream_pad(), it is redundant now.
- Rename imx_media_find_upstream_subdev() to imx_media_pipeline_subdev()
with an additional boolean argument for searching upstream or downstream.
- Add imx_media_pipeline_video_device() which is analogous to
imx_media_pipeline_subdev() but searches for video devices.
- Remove imxmd pointer arg from all of the functions above, it was
never used in those functions. With that change the i.MX5/6 CSI,
VDIC, and IC sub-devices no longer require the media_device.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-common.c | 4 +-
drivers/staging/media/imx/imx-ic-prp.c| 4 +-
drivers/staging/media/imx/imx-ic.h| 1 -
drivers/staging/media/imx/imx-media-csi.c | 13 +-
drivers/staging/media/imx/imx-media-fim.c | 4 -
.../staging/media/imx/imx-media-internal-sd.c | 5 +-
drivers/staging/media/imx/imx-media-utils.c | 128 ++
drivers/staging/media/imx/imx-media-vdic.c| 5 +-
drivers/staging/media/imx/imx-media.h | 20 +--
drivers/staging/media/imx/imx7-media-csi.c| 2 +-
10 files changed, 93 insertions(+), 93 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-common.c
b/drivers/staging/media/imx/imx-ic-common.c
index ad0c291db03c..37734984beb4 100644
--- a/drivers/staging/media/imx/imx-ic-common.c
+++ b/drivers/staging/media/imx/imx-ic-common.c
@@ -22,12 +22,11 @@ static struct imx_ic_ops *ic_ops[IC_NUM_OPS] = {
[IC_TASK_VIEWFINDER] = _ic_prpencvf_ops,
};
-struct v4l2_subdev *imx_media_ic_register(struct imx_media_dev *imxmd,
+struct v4l2_subdev *imx_media_ic_register(struct v4l2_device *v4l2_dev,
struct device *ipu_dev,
struct ipu_soc *ipu,
u32 grp_id)
{
- struct v4l2_device *v4l2_dev = >v4l2_dev;
struct imx_ic_priv *priv;
int ret;
@@ -37,7 +36,6 @@ struct v4l2_subdev *imx_media_ic_register(struct
imx_media_dev *imxmd,
priv->ipu_dev = ipu_dev;
priv->ipu = ipu;
- priv->md = imxmd;
/* get our IC task id */
switch (grp_id) {
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 663db200e594..1432776a33f9 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -302,8 +302,8 @@ static int prp_link_validate(struct v4l2_subdev *sd,
if (ret)
return ret;
- csi = imx_media_find_upstream_subdev(ic_priv->md, _priv->sd.entity,
-IMX_MEDIA_GRP_ID_IPU_CSI);
+ csi = imx_media_pipeline_subdev(_priv->sd.entity,
+ IMX_MEDIA_GRP_ID_IPU_CSI, true);
if (IS_ERR(csi))
csi = NULL;
diff --git a/drivers/staging/media/imx/imx-ic.h
b/drivers/staging/media/imx/imx-ic.h
index 1dcbb37aeada..ff2f66f11982 100644
--- a/drivers/staging/media/imx/imx-ic.h
+++ b/drivers/staging/media/imx/imx-ic.h
@@ -16,7 +16,6 @@
struct imx_ic_priv {
struct device *ipu_dev;
struct ipu_soc *ipu;
- struct imx_media_dev *md;
struct v4l2_subdev sd;
inttask_id;
void *task_priv;
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 68c2b1a3066a..555904759078 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -60,7 +60,6 @@ struct csi_skip_desc {
struct csi_priv {
struct device *dev;
struct ipu_soc *ipu;
- struct imx_media_dev *md;
struct v4l2_subdev sd;
struct media_pad pad[CSI_NUM_PADS];
/* the video device at IDMAC output pad */
@@ -182,8 +181,8 @@ static int csi_get_upstream_endpoint(struct csi_priv *priv,
* CSI-2 receiver if it is in the path, otherwise stay
* with video mux.
*/
- sd = imx_media_find_upstream_subdev(priv->md, src,
- IMX_MEDIA_GRP_ID_CSI2);
+ sd = imx_media_pipeline_subdev(src, IMX_MEDIA_GRP_ID_CSI2,
+ true);
if (!IS_ERR(sd))
src = >entity;
}
@@ -197,7 +196,7 @@ static int csi_get_upstream_endpoint(struct csi_priv *priv,
src = >sd.entity;
/* get source pad of entity directly upstream from src */
- pad = imx_media_find_upstream_pad(priv->md, src, 0);
+ pad = imx_m
[PATCH v5 3/9] media: staging/imx: Pass device to alloc/free_dma_buf
Allocate and free a DMA coherent buffer in imx_media_alloc/free_dma_buf()
from the given device. This allows DMA alloc and free using a device
that is backed by real hardware, which for the imx5/6/7 CSI is the CSI
unit, and for the internal IPU sub-devices, is the parent IPU.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 18 +-
drivers/staging/media/imx/imx-media-csi.c | 6 +++---
drivers/staging/media/imx/imx-media-utils.c | 13 ++---
drivers/staging/media/imx/imx-media.h | 4 ++--
drivers/staging/media/imx/imx7-media-csi.c | 4 ++--
5 files changed, 22 insertions(+), 23 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 069cce512280..ddcd87a17c71 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -464,13 +464,13 @@ static int prp_setup_rotation(struct prp_priv *priv)
incc = priv->cc[PRPENCVF_SINK_PAD];
outcc = vdev->cc;
- ret = imx_media_alloc_dma_buf(ic_priv->md, >rot_buf[0],
+ ret = imx_media_alloc_dma_buf(ic_priv->ipu_dev, >rot_buf[0],
outfmt->sizeimage);
if (ret) {
v4l2_err(_priv->sd, "failed to alloc rot_buf[0], %d\n", ret);
return ret;
}
- ret = imx_media_alloc_dma_buf(ic_priv->md, >rot_buf[1],
+ ret = imx_media_alloc_dma_buf(ic_priv->ipu_dev, >rot_buf[1],
outfmt->sizeimage);
if (ret) {
v4l2_err(_priv->sd, "failed to alloc rot_buf[1], %d\n", ret);
@@ -543,9 +543,9 @@ static int prp_setup_rotation(struct prp_priv *priv)
unsetup_vb2:
prp_unsetup_vb2_buf(priv, VB2_BUF_STATE_QUEUED);
free_rot1:
- imx_media_free_dma_buf(ic_priv->md, >rot_buf[1]);
+ imx_media_free_dma_buf(ic_priv->ipu_dev, >rot_buf[1]);
free_rot0:
- imx_media_free_dma_buf(ic_priv->md, >rot_buf[0]);
+ imx_media_free_dma_buf(ic_priv->ipu_dev, >rot_buf[0]);
return ret;
}
@@ -563,8 +563,8 @@ static void prp_unsetup_rotation(struct prp_priv *priv)
ipu_ic_disable(priv->ic);
- imx_media_free_dma_buf(ic_priv->md, >rot_buf[0]);
- imx_media_free_dma_buf(ic_priv->md, >rot_buf[1]);
+ imx_media_free_dma_buf(ic_priv->ipu_dev, >rot_buf[0]);
+ imx_media_free_dma_buf(ic_priv->ipu_dev, >rot_buf[1]);
}
static int prp_setup_norotation(struct prp_priv *priv)
@@ -656,7 +656,7 @@ static int prp_start(struct prp_priv *priv)
outfmt = >fmt.fmt.pix;
- ret = imx_media_alloc_dma_buf(ic_priv->md, >underrun_buf,
+ ret = imx_media_alloc_dma_buf(ic_priv->ipu_dev, >underrun_buf,
outfmt->sizeimage);
if (ret)
goto out_put_ipu;
@@ -726,7 +726,7 @@ static int prp_start(struct prp_priv *priv)
out_unsetup:
prp_unsetup(priv, VB2_BUF_STATE_QUEUED);
out_free_underrun:
- imx_media_free_dma_buf(ic_priv->md, >underrun_buf);
+ imx_media_free_dma_buf(ic_priv->ipu_dev, >underrun_buf);
out_put_ipu:
prp_put_ipu_resources(priv);
return ret;
@@ -763,7 +763,7 @@ static void prp_stop(struct prp_priv *priv)
prp_unsetup(priv, VB2_BUF_STATE_ERROR);
- imx_media_free_dma_buf(ic_priv->md, >underrun_buf);
+ imx_media_free_dma_buf(ic_priv->ipu_dev, >underrun_buf);
/* cancel the EOF timeout timer */
del_timer_sync(>eof_timeout_timer);
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 93b107eab5f5..ea3d13103c91 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -612,7 +612,7 @@ static int csi_idmac_start(struct csi_priv *priv)
outfmt = >fmt.fmt.pix;
- ret = imx_media_alloc_dma_buf(priv->md, >underrun_buf,
+ ret = imx_media_alloc_dma_buf(priv->dev, >underrun_buf,
outfmt->sizeimage);
if (ret)
goto out_put_ipu;
@@ -666,7 +666,7 @@ static int csi_idmac_start(struct csi_priv *priv)
out_unsetup:
csi_idmac_unsetup(priv, VB2_BUF_STATE_QUEUED);
out_free_dma_buf:
- imx_media_free_dma_buf(priv->md, >underrun_buf);
+ imx_media_free_dma_buf(priv->dev, >underrun_buf);
out_put_ipu:
csi_idmac_put_ipu_resources(priv);
return ret;
@@ -698,7 +698,7 @@ static void csi_idmac_stop(struct csi_priv *priv)
csi_idmac_unsetup(priv, VB2_BUF_STATE_ERROR);
- imx_media_free_dma_buf(priv->md, >underrun_buf);
+ imx_media_free_dma_buf(priv->dev, >underrun_buf);
/* cancel the EOF timeout timer */
del_timer_sync(>eof_timeou
[PATCH v5 6/9] media: staging/imx: Remove capture_device_set_format
Don't propagate the source pad format to the connected capture device.
It's now the responsibility of userspace to call VIDIOC_S_FMT on the
capture device to ensure the capture format and compose rectangle
are compatible with the connected source. To check this, validate
the capture format with the source before streaming starts.
Signed-off-by: Steve Longerbeam
---
Changes in v4:
- add **cc arg to __capture_try_fmt_vid_cap() to validate colorspace,
instead of calling ipu_pixelformat_to_colorspace().
- add error message if capture format validation failed.
---
drivers/staging/media/imx/imx-ic-prpencvf.c | 16 +
drivers/staging/media/imx/imx-media-capture.c | 71 +--
drivers/staging/media/imx/imx-media-csi.c | 16 +
drivers/staging/media/imx/imx-media.h | 2 -
drivers/staging/media/imx/imx7-media-csi.c| 17 +
5 files changed, 55 insertions(+), 67 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index afaa3a8b15e9..63334fd61492 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -906,9 +906,7 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_format *sdformat)
{
struct prp_priv *priv = sd_to_priv(sd);
- struct imx_media_video_dev *vdev = priv->vdev;
const struct imx_media_pixfmt *cc;
- struct v4l2_pix_format vdev_fmt;
struct v4l2_mbus_framefmt *fmt;
int ret = 0;
@@ -945,19 +943,9 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
priv->cc[PRPENCVF_SRC_PAD] = outcc;
}
- if (sdformat->which == V4L2_SUBDEV_FORMAT_TRY)
- goto out;
-
- priv->cc[sdformat->pad] = cc;
+ if (sdformat->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+ priv->cc[sdformat->pad] = cc;
- /* propagate output pad format to capture device */
- imx_media_mbus_fmt_to_pix_fmt(_fmt,
- >format_mbus[PRPENCVF_SRC_PAD],
- priv->cc[PRPENCVF_SRC_PAD]);
- mutex_unlock(>lock);
- imx_media_capture_device_set_format(vdev, _fmt);
-
- return 0;
out:
mutex_unlock(>lock);
return ret;
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index 555f6204660b..8a908c3e5e60 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -205,7 +205,9 @@ static int capture_g_fmt_vid_cap(struct file *file, void
*fh,
static int __capture_try_fmt_vid_cap(struct capture_priv *priv,
struct v4l2_subdev_format *fmt_src,
-struct v4l2_format *f)
+struct v4l2_format *f,
+const struct imx_media_pixfmt **retcc,
+struct v4l2_rect *compose)
{
const struct imx_media_pixfmt *cc, *cc_src;
@@ -247,6 +249,16 @@ static int __capture_try_fmt_vid_cap(struct capture_priv
*priv,
imx_media_mbus_fmt_to_pix_fmt(>fmt.pix, _src->format, cc);
+ if (retcc)
+ *retcc = cc;
+
+ if (compose) {
+ compose->left = 0;
+ compose->top = 0;
+ compose->width = fmt_src->format.width;
+ compose->height = fmt_src->format.height;
+ }
+
return 0;
}
@@ -263,7 +275,7 @@ static int capture_try_fmt_vid_cap(struct file *file, void
*fh,
if (ret)
return ret;
- return __capture_try_fmt_vid_cap(priv, _src, f);
+ return __capture_try_fmt_vid_cap(priv, _src, f, NULL, NULL);
}
static int capture_s_fmt_vid_cap(struct file *file, void *fh,
@@ -284,17 +296,12 @@ static int capture_s_fmt_vid_cap(struct file *file, void
*fh,
if (ret)
return ret;
- ret = __capture_try_fmt_vid_cap(priv, _src, f);
+ ret = __capture_try_fmt_vid_cap(priv, _src, f, >vdev.cc,
+ >vdev.compose);
if (ret)
return ret;
priv->vdev.fmt.fmt.pix = f->fmt.pix;
- priv->vdev.cc = imx_media_find_format(f->fmt.pix.pixelformat,
- CS_SEL_ANY, true);
- priv->vdev.compose.left = 0;
- priv->vdev.compose.top = 0;
- priv->vdev.compose.width = fmt_src.format.width;
- priv->vdev.compose.height = fmt_src.format.height;
return 0;
}
@@ -524,6 +531,33 @@ static void capture_buf_queue(struct vb2_buffer *vb)
spin_unlock_irqrestore(>q_lock, flags);
}
+static int capture_validate_fmt(struct capture_priv *priv)
+{
+ struct v4l2_subdev_format fmt_src;
+ const struct imx_medi
[PATCH v5 7/9] media: staging/imx: Re-organize modules
Re-organize modules, and which objects are linked into those modules, so
that:
- imx6-media (renamed from imx-media) is the media driver module for
imx5/6 only, and has no symbol exports.
- imx6-media-csi (renamed from imx-media-csi) is the subdev driver
module for imx5/6 CSI. It is now linked direcly with imx-media-fim,
since only the imx5/6 CSI makes use of the frame interval monitor.
- imx-media-common now only contains common code between imx5/6 and imx7
media drivers. It contains imx-media-utils, imx-media-of,
imx-media-dev-common, and imx-media-capture. In order to acheive that,
some functions common to imx5/6 and imx7 have been moved out of
imx-media-dev.c and into imx-media-dev-common.c.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/Makefile| 14 +-
.../staging/media/imx/imx-media-dev-common.c | 345 +-
drivers/staging/media/imx/imx-media-dev.c | 340 +
drivers/staging/media/imx/imx-media-fim.c | 5 -
drivers/staging/media/imx/imx-media-of.c | 3 +
drivers/staging/media/imx/imx-media.h | 16 +-
drivers/staging/media/imx/imx7-media-csi.c| 4 +-
7 files changed, 369 insertions(+), 358 deletions(-)
diff --git a/drivers/staging/media/imx/Makefile
b/drivers/staging/media/imx/Makefile
index 86f0c81b6a3b..aa6c4b4ad37e 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -1,14 +1,16 @@
# SPDX-License-Identifier: GPL-2.0
-imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o \
+imx6-media-objs := imx-media-dev.o imx-media-internal-sd.o \
imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o imx-media-vdic.o
-imx-media-objs += imx-media-dev-common.o
-imx-media-common-objs := imx-media-utils.o imx-media-fim.o
-obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
+imx-media-common-objs := imx-media-capture.o imx-media-dev-common.o \
+ imx-media-of.o imx-media-utils.o
+
+imx6-media-csi-objs := imx-media-csi.o imx-media-fim.o
+
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx6-media.o
obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
-obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
-obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
+obj-$(CONFIG_VIDEO_IMX_CSI) += imx6-media-csi.o
obj-$(CONFIG_VIDEO_IMX_CSI) += imx6-mipi-csi2.o
obj-$(CONFIG_VIDEO_IMX7_CSI) += imx7-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-dev-common.c
b/drivers/staging/media/imx/imx-media-dev-common.c
index 6cd93419b81d..89dc4ec8dadb 100644
--- a/drivers/staging/media/imx/imx-media-dev-common.c
+++ b/drivers/staging/media/imx/imx-media-dev-common.c
@@ -8,9 +8,342 @@
#include
#include
+#include
+#include
+#include
+#include
#include "imx-media.h"
-static const struct v4l2_async_notifier_operations imx_media_subdev_ops = {
+static inline struct imx_media_dev *notifier2dev(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct imx_media_dev, notifier);
+}
+
+/* async subdev bound notifier */
+static int imx_media_subdev_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ v4l2_info(sd->v4l2_dev, "subdev %s bound\n", sd->name);
+
+ return 0;
+}
+
+/*
+ * Create the media links for all subdevs that registered.
+ * Called after all async subdevs have bound.
+ */
+static int imx_media_create_links(struct v4l2_async_notifier *notifier)
+{
+ struct imx_media_dev *imxmd = notifier2dev(notifier);
+ struct v4l2_subdev *sd;
+
+ list_for_each_entry(sd, >v4l2_dev.subdevs, list) {
+ switch (sd->grp_id) {
+ case IMX_MEDIA_GRP_ID_IPU_VDIC:
+ case IMX_MEDIA_GRP_ID_IPU_IC_PRP:
+ case IMX_MEDIA_GRP_ID_IPU_IC_PRPENC:
+ case IMX_MEDIA_GRP_ID_IPU_IC_PRPVF:
+ /*
+* links have already been created for the
+* sync-registered subdevs.
+*/
+ break;
+ case IMX_MEDIA_GRP_ID_IPU_CSI0:
+ case IMX_MEDIA_GRP_ID_IPU_CSI1:
+ case IMX_MEDIA_GRP_ID_CSI:
+ imx_media_create_csi_of_links(imxmd, sd);
+ break;
+ default:
+ /*
+* if this subdev has fwnode links, create media
+* links for them.
+*/
+ imx_media_create_of_links(imxmd, sd);
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * adds given video device to given imx-media source pad vdev list.
+ * Continues upstream from the pad entity's sink pads.
+ */
+static int imx_media_add_vdev_to_pad(struct imx_media_dev *imxmd,
+
[PATCH v3 8/8] media: staging/imx: Don't set driver data for v4l2_dev
The media device is already available via multiple methods, there is no
need to set driver data for v4l2_dev to the media device.
In imx_media_link_notify(), get media device from link->graph_obj.mdev.
In imx_media_capture_device_register(), get media device from
v4l2_dev->mdev.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-capture.c| 5 +++--
drivers/staging/media/imx/imx-media-dev-common.c | 7 ++-
2 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-capture.c
b/drivers/staging/media/imx/imx-media-capture.c
index b77a67bda47c..565d21f169d8 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -732,15 +732,16 @@ int imx_media_capture_device_register(struct
imx_media_video_dev *vdev)
{
struct capture_priv *priv = to_capture_priv(vdev);
struct v4l2_subdev *sd = priv->src_sd;
+ struct v4l2_device *v4l2_dev = sd->v4l2_dev;
struct video_device *vfd = vdev->vfd;
struct vb2_queue *vq = >q;
struct v4l2_subdev_format fmt_src;
int ret;
/* get media device */
- priv->md = dev_get_drvdata(sd->v4l2_dev->dev);
+ priv->md = container_of(v4l2_dev->mdev, struct imx_media_dev, md);
- vfd->v4l2_dev = sd->v4l2_dev;
+ vfd->v4l2_dev = v4l2_dev;
ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
if (ret) {
diff --git a/drivers/staging/media/imx/imx-media-dev-common.c
b/drivers/staging/media/imx/imx-media-dev-common.c
index 89dc4ec8dadb..66b505f7e8df 100644
--- a/drivers/staging/media/imx/imx-media-dev-common.c
+++ b/drivers/staging/media/imx/imx-media-dev-common.c
@@ -260,10 +260,11 @@ static int imx_media_inherit_controls(struct
imx_media_dev *imxmd,
static int imx_media_link_notify(struct media_link *link, u32 flags,
unsigned int notification)
{
+ struct imx_media_dev *imxmd = container_of(link->graph_obj.mdev,
+ struct imx_media_dev, md);
struct media_entity *source = link->source->entity;
struct imx_media_pad_vdev *pad_vdev;
struct list_head *pad_vdev_list;
- struct imx_media_dev *imxmd;
struct video_device *vfd;
struct v4l2_subdev *sd;
int pad_idx, ret;
@@ -279,8 +280,6 @@ static int imx_media_link_notify(struct media_link *link,
u32 flags,
sd = media_entity_to_v4l2_subdev(source);
pad_idx = link->source->index;
- imxmd = dev_get_drvdata(sd->v4l2_dev->dev);
-
pad_vdev_list = to_pad_vdev_list(sd, pad_idx);
if (!pad_vdev_list) {
/* nothing to do if source sd has no pad vdev list */
@@ -384,8 +383,6 @@ struct imx_media_dev *imx_media_dev_init(struct device *dev,
goto cleanup;
}
- dev_set_drvdata(imxmd->v4l2_dev.dev, imxmd);
-
INIT_LIST_HEAD(>vdev_list);
v4l2_async_notifier_init(>notifier);
--
2.17.1
Re: [PATCH v6 5/7] gpu: ipu-v3: ipu-ic: Add support for limited range encoding
On 3/8/19 3:57 AM, Philipp Zabel wrote:
On Thu, 2019-03-07 at 15:33 -0800, Steve Longerbeam wrote:
Add support for the following conversions:
- YUV full-range to YUV limited-range
- YUV limited-range to YUV full-range
- YUV limited-range to RGB full-range
- RGB full-range to YUV limited-range
The last two conversions require operating on the YUV full-range
encoding and inverse encoding coefficients, with the YUV-to-YUV
limited<->full coefficients. The formula to convert is
M_c = M_a * M_b
O_c = M_a * O_b + O_a
For calculating the RGB full-range to YUV limited-range coefficients:
[M_a, O_a] = YUV full-range to YUV limited-range coefficients.
[M_b, O_b] = RGB full-range to YUV full-range coefficients.
For calculating the YUV limited-range to RGB full-range coefficients:
[M_a, O_a] = YUV full-range to RGB full-range coefficients.
[M_b, O_b] = YUV limited-range to YUV full-range coefficients.
The calculation of [M_c, O_c] is carried out by the function
transform_coeffs().
In the future if RGB limited range encoding is required, the same
function can be used. And cascaded to create all combinations of
encoding for YUV limited/full range <-> RGB limited/full range,
passing the output coefficients from one call as the input for the
next.
For example, to create YUV full-range to RGB limited-range coefficients:
[M_a, O_a] = RGB full-range to RGB limited-range coefficients.
[M_b, O_b] = YUV full-range to RGB full-range coefficients.
and that output sent as input to create YUV limited-range to RGB
limited-range coefficients:
[M_a, O_a] = YUV full-range to RGB limited-range coefficients.
[M_b, O_b] = YUV limited-range to YUV full-range coefficients.
Signed-off-by: Steve Longerbeam
I'm not a big fan of this. Wouldn't it be much easier to compute all
necessary task parameter sets offline with high precision, and store the
precomputed sets in the compact representation?
I am thinking of when support might be added for the other encoding
standards. With this transform function, only two new task parameter
structs need to be added, one for yuv-full-to-rgb-full, and one for
rgb-full-to-yuv-full. Without transform_coeffs(), four structs would
have to be added (adding encoding to and from yuv-limited). And if
rgb-limited support is added, it would mean a total of eight new structs
for a new encoding standard. But with transform_coeffs(), still only the
two structs above are needed, and the function would compute the others
automatically in runtime.
Steve
---
drivers/gpu/ipu-v3/ipu-ic.c | 281 +---
1 file changed, 263 insertions(+), 18 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 1460901af9b5..a7dd85f8d832 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -178,10 +178,10 @@ static inline void ipu_ic_write(struct ipu_ic *ic, u32
value, unsigned offset)
}
struct ic_encode_coeff {
- s16 coeff[3][3];/* signed 9-bit integer coefficients */
- s16 offset[3]; /* signed 11+2-bit fixed point offset */
- u8 scale:2; /* scale coefficients * 2^(scale-1) */
- bool sat:1; /* saturate to (16, 235(Y) / 240(U, V)) */
+ int coeff[3][3];/* signed 9-bit integer coefficients */
+ int offset[3]; /* signed 13-bit integer offset */
+ int scale; /* scale coefficients * 2^(scale-1) */
+ bool sat; /* saturate to (16, 235(Y) / 240(U, V)) */
};
/*
@@ -277,6 +277,231 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_709 = {
.scale = 2,
};
+/*
+ * YUV full range to YUV limited range:
+ *
+ * Y_lim = 0.8588 * Y_full + 16
+ * Cb_lim = 0.8784 * (Cb_full - 128) + 128
+ * Cr_lim = 0.8784 * (Cr_full - 128) + 128
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr_full2lim = {
+ .coeff = {
+ { 219, 0, 0 },
+ { 0, 224, 0 },
+ { 0, 0, 224 },
+ },
+ .offset = { 64, 62, 62 },
+ .scale = 1,
+};
+
+/*
+ * YUV limited range to YUV full range:
+ *
+ * Y_full = 1.1644 * (Y_lim - 16)
+ * Cb_full = 1.1384 * (Cb_lim - 128) + 128
+ * Cr_full = 1.1384 * (Cr_lim - 128) + 128
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr_lim2full = {
+ .coeff = {
+ { 149, 0, 0 },
+ { 0, 145, 0 },
+ { 0, 0, 145 },
+ },
+ .offset = { -37, -35, -35 },
+ .scale = 2,
+};
+
+/*
+ * RGB full range to RGB limited range:
+ *
+ * R_lim = 0.8588 * R_full + 16
+ * G_lim = 0.8588 * G_full + 16
+ * B_lim = 0.8588 * B_full + 16
+ */
+static const struct ic_encode_coeff
+ic_encode_rgb_full2lim __maybe_unused = {
+ .coeff = {
+ { 220, 0, 0 },
+ { 0, 220, 0 },
+ { 0, 0, 220 },
+ },
+ .offset = { 64, 64, 64 },
+ .scale = 1,
+};
+
+/*
+ * RGB limited range to RG
Re: [PATCH v6 3/7] gpu: ipu-v3: ipu-ic: Fully describe colorspace conversions
On 3/8/19 3:46 AM, Philipp Zabel wrote:
On Thu, 2019-03-07 at 15:33 -0800, Steve Longerbeam wrote:
Only providing the input and output RGB/YUV space to the IC task init
functions is not sufficient. To fully characterize a colorspace
conversion, the colorspace (chromaticities), Y'CbCr encoding standard,
and quantization also need to be specified.
Define a 'struct ipu_ic_colorspace' that includes all the above, and pass
the input and output ipu_ic_colorspace to the IC task init functions.
This allows to actually enforce the fact that the IC:
- can only encode to/from YUV full range (follow-up patch will remove
this restriction).
- can only encode to/from RGB full range.
- can only encode using BT.601 standard (follow-up patch will add
Rec.709 encoding support).
- cannot convert colorspaces from input to output, the
input and output colorspace chromaticities must be the same.
The determination of the CSC coefficients based on the input/output
colorspace parameters are moved to a new function calc_csc_coeffs(),
called by init_csc().
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 136 +---
drivers/gpu/ipu-v3/ipu-image-convert.c | 27 ++--
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 +++-
include/video/imx-ipu-v3.h | 37 +-
4 files changed, 154 insertions(+), 68 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index b63a2826b629..c4048c921801 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -146,8 +146,10 @@ struct ipu_ic {
const struct ic_task_regoffs *reg;
const struct ic_task_bitfields *bit;
- enum ipu_color_space in_cs, g_in_cs;
- enum ipu_color_space out_cs;
+ struct ipu_ic_colorspace in_cs;
+ struct ipu_ic_colorspace g_in_cs;
+ struct ipu_ic_colorspace out_cs;
+
bool graphics;
bool rotation;
bool in_use;
@@ -235,42 +237,83 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_601 = {
.scale = 2,
};
+static int calc_csc_coeffs(struct ipu_ic_priv *priv,
+ struct ic_encode_coeff *coeff_out,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out)
+{
+ bool inverse_encode;
+
+ if (in->colorspace != out->colorspace) {
+ dev_err(priv->ipu->dev, "Cannot convert colorspaces\n");
+ return -ENOTSUPP;
+ }
I don't think this is useful enough to warrant having the colorspace
field in ipu_ic_colorspace. Let the caller make sure of this, same as
for xfer_func.
Ok, for xfer_func it is implicit that the gamma function must be the
same for input and output, so I agree it might as well be implicit for
chromaticities too.
+ if (out->enc != V4L2_YCBCR_ENC_601) {
+ dev_err(priv->ipu->dev, "Only BT.601 encoding supported\n");
+ return -ENOTSUPP;
+ }
This is only important if out->cs is IPUV3_COLORSPACE_YUV, right? If the
output is RGB this field shouldn't matter.
It matters for encoding YUV to RGB, or the inverse RGB to YUV. The
encoding standard doesn't matter only if no encoding/inverse encoding is
requested (YUV to YUV or RGB to RGB).
+
+ if ((in->cs == IPUV3_COLORSPACE_YUV &&
+in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
+ (out->cs == IPUV3_COLORSPACE_YUV &&
+out->quant != V4L2_QUANTIZATION_FULL_RANGE)) {
+ dev_err(priv->ipu->dev, "Limited range YUV not supported\n");
+ return -ENOTSUPP;
+ }
+
+ if ((in->cs == IPUV3_COLORSPACE_RGB &&
+in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
+ (out->cs == IPUV3_COLORSPACE_RGB &&
+out->quant != V4L2_QUANTIZATION_FULL_RANGE)) {
+ dev_err(priv->ipu->dev, "Limited range RGB not supported\n");
+ return -ENOTSUPP;
+ }
+
+ if (in->cs == out->cs) {
+ *coeff_out = ic_encode_identity;
+
+ return 0;
+ }
+
+ inverse_encode = (in->cs == IPUV3_COLORSPACE_YUV);
What does inverse_encode mean in this context?
It means YUV to RGB. At this point in the function it is determined that
encoding or inverse encoding is requested.
+
+ *coeff_out = inverse_encode ?
+ ic_encode_ycbcr2rgb_601 : ic_encode_rgb2ycbcr_601;
+
+ return 0;
+}
+
static int init_csc(struct ipu_ic *ic,
- enum ipu_color_space inf,
- enum ipu_color_space outf,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
- const stru
Re: [PATCH v6 2/7] gpu: ipu-v3: ipu-ic: Fix BT.601 coefficients
On 3/8/19 2:23 AM, Philipp Zabel wrote:
Hi Steve,
On Thu, 2019-03-07 at 15:33 -0800, Steve Longerbeam wrote:
The ycbcr2rgb and inverse rgb2ycbcr tables define the BT.601 Y'CbCr
encoding coefficients.
The rgb2ycbcr table specifically describes the BT.601 encoding from
full range RGB to full range YUV. Add table comments to make this more
clear.
The ycbcr2rgb inverse table describes encoding YUV limited range to RGB
full range. To be consistent with the rgb2ycbcr table, convert this to
YUV full range to RGB full range, and adjust/expand on the comments.
The ic_csc_rgb2rgb table is just an identity matrix, so rename to
ic_encode_identity.
Fixes: 1aa8ea0d2bd5d ("gpu: ipu-v3: Add Image Converter unit")
Suggested-by: Philipp Zabel
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/gpu/ipu-v3/ipu-ic.c | 61 ++---
1 file changed, 37 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 18816ccf600e..b63a2826b629 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -175,7 +175,7 @@ static inline void ipu_ic_write(struct ipu_ic *ic, u32
value, unsigned offset)
writel(value, ic->priv->base + offset);
}
-struct ic_csc_params {
+struct ic_encode_coeff {
This less accurate. These are called IC (Task) Parameters in the
reference manual, the 64-bit aligned words are called CSC words. Beside
the coefficients, this structure also contains the coefficient scale,
the offsets, and the saturation mode flag.
It seemed to me the renaming was more clear, but I agree the former name
conforms better to the manual nomenclature. I will revert this renaming.
s16 coeff[3][3];/* signed 9-bit integer coefficients */
s16 offset[3]; /* signed 11+2-bit fixed point offset */
u8 scale:2; /* scale coefficients * 2^(scale-1) */
@@ -183,13 +183,15 @@ struct ic_csc_params {
};
/*
- * Y = R * .299 + G * .587 + B * .114;
- * U = R * -.169 + G * -.332 + B * .500 + 128.;
- * V = R * .500 + G * -.419 + B * -.0813 + 128.;
+ * BT.601 encoding from RGB full range to YUV full range:
+ *
+ * Y = .2990 * R + .5870 * G + .1140 * B
+ * U = -.1687 * R - .3313 * G + .5000 * B + 128
+ * V = .5000 * R - .4187 * G - .0813 * B + 128
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_encode_coeff ic_encode_rgb2ycbcr_601 = {
.coeff = {
- { 77, 150, 29 },
+ { 77, 150, 29 },
{ 469, 427, 128 },
{ 128, 405, 491 },
We could subtract 512 from the negative values, to improve readability.
Agreed.
},
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_encode_coeff ic_encode_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,17 +213,25 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
- * R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
- * G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
- * B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
+ * Inverse BT.601 encoding from YUV full range to RGB full range:
+ *
+ * R = 1. * Y + 0 * (Cb - 128) + 1.4020 * (Cr - 128)
+ * G = 1. * Y - .3442 * (Cb - 128) - 0.7142 * (Cr - 128)
Should that be ^ .3441 and ^ .7141 ?
The coefficients and offsets after rounding should end up the same.
Ok.
Also, let's consistently either add the leading zero, or leave it out.
Yes.
+ * B = 1. * Y + 1.7720 * (Cb - 128) + 0 * (Cr - 128)
+ *
+ * equivalently (factoring out the offsets):
+ *
+ * R = 1. * Y + 0 * Cb + 1.4020 * Cr - 179.456
+ * G = 1. * Y - .3442 * Cb - 0.7142 * Cr + 135.475
+ * B = 1. * Y + 1.7720 * Cb + 0 * Cr - 226.816
*/
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_encode_coeff ic_encode_ycbcr2rgb_601 = {
.coeff = {
- { 149, 0, 204 },
- { 149, 462, 408 },
- { 149, 255, 0 },
+ { 128, 0, 179 },
+ { 128, 468, 421 },
+ { 128, 227, 0 },
},
- .offset = { -446, 266, -554 },
+ .offset = { -359, 271, -454 },
These seem to be correct. Again, I think this would be easier to read if
the negative coefficients were written with a sign as well.
.scale = 2,
};
@@ -228,7 +241,7 @@ static int init_csc(struct ipu_ic *ic,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
- const struct ic_csc_params *params;
+ const struct ic_encode_coeff *coeff;
u32 __i
[PATCH v6 6/7] media: imx: Try colorimetry at both sink and source pads
The colorimetry parameters need to be tested at both sink and source
pads. Specifically, for prpencvf, the IC only supports RGB full-range
quantization at input and output.
Fix this by cleaning up imx_media_fill_default_mbus_fields(), renaming
to imx_media_try_colorimetry(), and call it at both sink and source
pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prp.c | 6 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 8 +--
drivers/staging/media/imx/imx-media-csi.c | 19 +++---
drivers/staging/media/imx/imx-media-utils.c | 68 +++--
drivers/staging/media/imx/imx-media-vdic.c | 5 +-
drivers/staging/media/imx/imx-media.h | 5 +-
drivers/staging/media/imx/imx7-media-csi.c | 8 +--
7 files changed, 62 insertions(+), 57 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 3d43cdcb4bb9..8010ee706164 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -197,8 +197,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
sdformat->format.code = cc->codes[0];
}
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
@@ -207,6 +207,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
break;
}
+ imx_media_try_colorimetry(>format, true);
+
fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
*fmt = sdformat->format;
out:
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 10f2c7684727..b1886a4e362e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -899,8 +899,6 @@ static void prp_try_fmt(struct prp_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
@@ -908,9 +906,11 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
}
+
+ imx_media_try_colorimetry(>format, true);
}
static int prp_set_fmt(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 3b7517348666..cc3e0086b08a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1369,9 +1369,15 @@ static void csi_try_field(struct csi_priv *priv,
struct v4l2_mbus_framefmt *infmt =
__csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
- /* no restrictions on sink pad field type */
- if (sdformat->pad == CSI_SINK_PAD)
+ /*
+* no restrictions on sink pad field type except must
+* be initialized.
+*/
+ if (sdformat->pad == CSI_SINK_PAD) {
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
return;
+ }
switch (infmt->field) {
case V4L2_FIELD_SEQ_TB:
@@ -1449,8 +1455,6 @@ static void csi_try_fmt(struct csi_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
break;
case CSI_SINK_PAD:
@@ -1470,10 +1474,6 @@ static void csi_try_fmt(struct csi_priv *priv,
csi_try_field(priv, cfg, sdformat);
- imx_media_fill_default_mbus_fields(
- >format, infmt,
- priv->active_output_pad == CSI_SRC_PAD_DIRECT);
-
[PATCH v6 5/7] gpu: ipu-v3: ipu-ic: Add support for limited range encoding
Add support for the following conversions:
- YUV full-range to YUV limited-range
- YUV limited-range to YUV full-range
- YUV limited-range to RGB full-range
- RGB full-range to YUV limited-range
The last two conversions require operating on the YUV full-range
encoding and inverse encoding coefficients, with the YUV-to-YUV
limited<->full coefficients. The formula to convert is
M_c = M_a * M_b
O_c = M_a * O_b + O_a
For calculating the RGB full-range to YUV limited-range coefficients:
[M_a, O_a] = YUV full-range to YUV limited-range coefficients.
[M_b, O_b] = RGB full-range to YUV full-range coefficients.
For calculating the YUV limited-range to RGB full-range coefficients:
[M_a, O_a] = YUV full-range to RGB full-range coefficients.
[M_b, O_b] = YUV limited-range to YUV full-range coefficients.
The calculation of [M_c, O_c] is carried out by the function
transform_coeffs().
In the future if RGB limited range encoding is required, the same
function can be used. And cascaded to create all combinations of
encoding for YUV limited/full range <-> RGB limited/full range,
passing the output coefficients from one call as the input for the
next.
For example, to create YUV full-range to RGB limited-range coefficients:
[M_a, O_a] = RGB full-range to RGB limited-range coefficients.
[M_b, O_b] = YUV full-range to RGB full-range coefficients.
and that output sent as input to create YUV limited-range to RGB
limited-range coefficients:
[M_a, O_a] = YUV full-range to RGB limited-range coefficients.
[M_b, O_b] = YUV limited-range to YUV full-range coefficients.
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 281 +---
1 file changed, 263 insertions(+), 18 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 1460901af9b5..a7dd85f8d832 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -178,10 +178,10 @@ static inline void ipu_ic_write(struct ipu_ic *ic, u32
value, unsigned offset)
}
struct ic_encode_coeff {
- s16 coeff[3][3];/* signed 9-bit integer coefficients */
- s16 offset[3]; /* signed 11+2-bit fixed point offset */
- u8 scale:2; /* scale coefficients * 2^(scale-1) */
- bool sat:1; /* saturate to (16, 235(Y) / 240(U, V)) */
+ int coeff[3][3];/* signed 9-bit integer coefficients */
+ int offset[3]; /* signed 13-bit integer offset */
+ int scale; /* scale coefficients * 2^(scale-1) */
+ bool sat; /* saturate to (16, 235(Y) / 240(U, V)) */
};
/*
@@ -277,6 +277,231 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_709 = {
.scale = 2,
};
+/*
+ * YUV full range to YUV limited range:
+ *
+ * Y_lim = 0.8588 * Y_full + 16
+ * Cb_lim = 0.8784 * (Cb_full - 128) + 128
+ * Cr_lim = 0.8784 * (Cr_full - 128) + 128
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr_full2lim = {
+ .coeff = {
+ { 219, 0, 0 },
+ { 0, 224, 0 },
+ { 0, 0, 224 },
+ },
+ .offset = { 64, 62, 62 },
+ .scale = 1,
+};
+
+/*
+ * YUV limited range to YUV full range:
+ *
+ * Y_full = 1.1644 * (Y_lim - 16)
+ * Cb_full = 1.1384 * (Cb_lim - 128) + 128
+ * Cr_full = 1.1384 * (Cr_lim - 128) + 128
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr_lim2full = {
+ .coeff = {
+ { 149, 0, 0 },
+ { 0, 145, 0 },
+ { 0, 0, 145 },
+ },
+ .offset = { -37, -35, -35 },
+ .scale = 2,
+};
+
+/*
+ * RGB full range to RGB limited range:
+ *
+ * R_lim = 0.8588 * R_full + 16
+ * G_lim = 0.8588 * G_full + 16
+ * B_lim = 0.8588 * B_full + 16
+ */
+static const struct ic_encode_coeff
+ic_encode_rgb_full2lim __maybe_unused = {
+ .coeff = {
+ { 220, 0, 0 },
+ { 0, 220, 0 },
+ { 0, 0, 220 },
+ },
+ .offset = { 64, 64, 64 },
+ .scale = 1,
+};
+
+/*
+ * RGB limited range to RGB full range:
+ *
+ * R_full = 1.1644 * (R_lim - 16)
+ * G_full = 1.1644 * (G_lim - 16)
+ * B_full = 1.1644 * (B_lim - 16)
+ */
+static const struct ic_encode_coeff
+ic_encode_rgb_lim2full __maybe_unused = {
+ .coeff = {
+ { 149, 0, 0 },
+ { 0, 149, 0 },
+ { 0, 0, 149 },
+ },
+ .offset = { -37, -37, -37 },
+ .scale = 2,
+};
+
+/*
+ * Convert a coefficient and scale value in TPMEM register format
+ * to a signed int times 256 (fix the radix point). The TPMEM register
+ * coefficient format is a signed 9-bit value (sign bit at bit 8,
+ * mantissa = coeff * 2 ^ (8 - scale - 1)).
+ */
+static int coeff_fix(int coeff, int scale)
+{
+ if (coeff >= 256)
+ coeff -= 512;
+ if (scale == 0)
+ return DIV_ROUND_CLOSEST(coeff, 2);
+ return coeff << (scale - 1);
+}
+
+/*
+ * Convert a signed int coefficie
[PATCH v6 4/7] gpu: ipu-v3: ipu-ic: Add support for Rec.709 encoding
Add support for Rec.709 encoding and inverse encoding.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v5:
- moved API changes to a previous patch.
- moved CSC coeff calc to new function calc_csc_coeffs().
Changes in v4:
- fix compile error.
Chnges in v3:
- none.
Changes in v2:
- only return "Unsupported YCbCr encoding" error if inf != outf,
since if inf == outf, the identity matrix can be used. Reported
by Tim Harvey.
---
drivers/gpu/ipu-v3/ipu-ic.c | 63 -
1 file changed, 56 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index c4048c921801..1460901af9b5 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -214,6 +214,23 @@ static const struct ic_encode_coeff ic_encode_identity = {
.scale = 2,
};
+/*
+ * REC.709 encoding from RGB full range to YUV full range:
+ *
+ * Y = .2126 * R + .7152 * G + .0722 * B
+ * U = -.1146 * R - .3854 * G + .5000 * B + 128
+ * V = .5000 * R - .4542 * G - .0458 * B + 128
+ */
+static const struct ic_encode_coeff ic_encode_rgb2ycbcr_709 = {
+ .coeff = {
+ { 54, 183, 19 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/*
* Inverse BT.601 encoding from YUV full range to RGB full range:
*
@@ -237,11 +254,35 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_601 = {
.scale = 2,
};
+/*
+ * Inverse REC.709 encoding from YUV full range to RGB full range:
+ *
+ * R = 1. * Y + 0 * (Cb - 128) + 1.5748 * (Cr - 128)
+ * G = 1. * Y - .1873 * (Cb - 128) - .4681 * (Cr - 128)
+ * B = 1. * Y + 1.8556 * (Cb - 128) + 0 * (Cr - 128)
+ *
+ * equivalently (factoring out the offsets):
+ *
+ * R = 1. * Y + 0 * Cb + 1.5748 * Cr - 201.574
+ * G = 1. * Y - .1873 * Cb - .4681 * Cr + 83.891
+ * B = 1. * Y + 1.8556 * Cb + 0 * Cr - 237.517
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr2rgb_709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -403, 168, -475 },
+ .scale = 2,
+};
+
static int calc_csc_coeffs(struct ipu_ic_priv *priv,
struct ic_encode_coeff *coeff_out,
const struct ipu_ic_colorspace *in,
const struct ipu_ic_colorspace *out)
{
+ const struct ic_encode_coeff *encode_coeff;
bool inverse_encode;
if (in->colorspace != out->colorspace) {
@@ -249,11 +290,6 @@ static int calc_csc_coeffs(struct ipu_ic_priv *priv,
return -ENOTSUPP;
}
- if (out->enc != V4L2_YCBCR_ENC_601) {
- dev_err(priv->ipu->dev, "Only BT.601 encoding supported\n");
- return -ENOTSUPP;
- }
-
if ((in->cs == IPUV3_COLORSPACE_YUV &&
in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
(out->cs == IPUV3_COLORSPACE_YUV &&
@@ -278,8 +314,21 @@ static int calc_csc_coeffs(struct ipu_ic_priv *priv,
inverse_encode = (in->cs == IPUV3_COLORSPACE_YUV);
- *coeff_out = inverse_encode ?
- ic_encode_ycbcr2rgb_601 : ic_encode_rgb2ycbcr_601;
+ switch (out->enc) {
+ case V4L2_YCBCR_ENC_601:
+ encode_coeff = inverse_encode ?
+ _encode_ycbcr2rgb_601 : _encode_rgb2ycbcr_601;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ encode_coeff = inverse_encode ?
+ _encode_ycbcr2rgb_709 : _encode_rgb2ycbcr_709;
+ break;
+ default:
+ dev_err(priv->ipu->dev, "Unsupported YCbCr encoding\n");
+ return -ENOTSUPP;
+ }
+
+ *coeff_out = *encode_coeff;
return 0;
}
--
2.17.1
[PATCH v6 1/7] gpu: ipu-v3: ipu-ic: Fix saturation bit offset in TPMEM
The saturation bit was being set at bit 9 in the second 32-bit word
of the TPMEM CSC. This isn't correct, the saturation bit is bit 42,
which is bit 10 of the second word.
Fixes: 1aa8ea0d2bd5d ("gpu: ipu-v3: Add Image Converter unit")
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/gpu/ipu-v3/ipu-ic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..18816ccf600e 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -257,7 +257,7 @@ static int init_csc(struct ipu_ic *ic,
writel(param, base++);
param = ((a[0] & 0x1fe0) >> 5) | (params->scale << 8) |
- (params->sat << 9);
+ (params->sat << 10);
writel(param, base++);
param = ((a[1] & 0x1f) << 27) | ((c[0][1] & 0x1ff) << 18) |
--
2.17.1
[PATCH v6 7/7] media: imx: Allow BT.709 encoding for IC routes
The IC now supports BT.709 Y'CbCr encoding, in addition to existing BT.601
encoding, so allow both, for pipelines that route through the IC.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v5:
- rebased this patch on top of repurposing the function to
imx_media_try_colorimetry().
Changes in v2:
- move ic_route check above default colorimetry checks, and fill default
colorimetry for ic_route, otherwise it's not possible to set BT.709
encoding for ic routes.
---
drivers/staging/media/imx/imx-media-utils.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index aa7d4be77a7e..12967d6d7e1a 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -524,7 +524,7 @@ EXPORT_SYMBOL_GPL(imx_media_init_cfg);
* If this format is destined to be routed through the Image Converter,
* quantization and Y`CbCr encoding must be fixed. The IC supports only
* full-range quantization for RGB at its input and output, and only
- * BT.601 Y`CbCr encoding.
+ * BT.601 or Rec.709 Y`CbCr encoding.
*/
void imx_media_try_colorimetry(struct v4l2_mbus_framefmt *tryfmt,
bool ic_route)
@@ -563,7 +563,9 @@ void imx_media_try_colorimetry(struct v4l2_mbus_framefmt
*tryfmt,
tryfmt->quantization == V4L2_QUANTIZATION_DEFAULT)
tryfmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
- tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
+ if (tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_601 &&
+ tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_709)
+ tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
} else {
if (tryfmt->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT) {
tryfmt->ycbcr_enc =
--
2.17.1
[PATCH v6 3/7] gpu: ipu-v3: ipu-ic: Fully describe colorspace conversions
Only providing the input and output RGB/YUV space to the IC task init
functions is not sufficient. To fully characterize a colorspace
conversion, the colorspace (chromaticities), Y'CbCr encoding standard,
and quantization also need to be specified.
Define a 'struct ipu_ic_colorspace' that includes all the above, and pass
the input and output ipu_ic_colorspace to the IC task init functions.
This allows to actually enforce the fact that the IC:
- can only encode to/from YUV full range (follow-up patch will remove
this restriction).
- can only encode to/from RGB full range.
- can only encode using BT.601 standard (follow-up patch will add
Rec.709 encoding support).
- cannot convert colorspaces from input to output, the
input and output colorspace chromaticities must be the same.
The determination of the CSC coefficients based on the input/output
colorspace parameters are moved to a new function calc_csc_coeffs(),
called by init_csc().
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 136 +---
drivers/gpu/ipu-v3/ipu-image-convert.c | 27 ++--
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 +++-
include/video/imx-ipu-v3.h | 37 +-
4 files changed, 154 insertions(+), 68 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index b63a2826b629..c4048c921801 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -146,8 +146,10 @@ struct ipu_ic {
const struct ic_task_regoffs *reg;
const struct ic_task_bitfields *bit;
- enum ipu_color_space in_cs, g_in_cs;
- enum ipu_color_space out_cs;
+ struct ipu_ic_colorspace in_cs;
+ struct ipu_ic_colorspace g_in_cs;
+ struct ipu_ic_colorspace out_cs;
+
bool graphics;
bool rotation;
bool in_use;
@@ -235,42 +237,83 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_601 = {
.scale = 2,
};
+static int calc_csc_coeffs(struct ipu_ic_priv *priv,
+ struct ic_encode_coeff *coeff_out,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out)
+{
+ bool inverse_encode;
+
+ if (in->colorspace != out->colorspace) {
+ dev_err(priv->ipu->dev, "Cannot convert colorspaces\n");
+ return -ENOTSUPP;
+ }
+
+ if (out->enc != V4L2_YCBCR_ENC_601) {
+ dev_err(priv->ipu->dev, "Only BT.601 encoding supported\n");
+ return -ENOTSUPP;
+ }
+
+ if ((in->cs == IPUV3_COLORSPACE_YUV &&
+in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
+ (out->cs == IPUV3_COLORSPACE_YUV &&
+out->quant != V4L2_QUANTIZATION_FULL_RANGE)) {
+ dev_err(priv->ipu->dev, "Limited range YUV not supported\n");
+ return -ENOTSUPP;
+ }
+
+ if ((in->cs == IPUV3_COLORSPACE_RGB &&
+in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
+ (out->cs == IPUV3_COLORSPACE_RGB &&
+out->quant != V4L2_QUANTIZATION_FULL_RANGE)) {
+ dev_err(priv->ipu->dev, "Limited range RGB not supported\n");
+ return -ENOTSUPP;
+ }
+
+ if (in->cs == out->cs) {
+ *coeff_out = ic_encode_identity;
+
+ return 0;
+ }
+
+ inverse_encode = (in->cs == IPUV3_COLORSPACE_YUV);
+
+ *coeff_out = inverse_encode ?
+ ic_encode_ycbcr2rgb_601 : ic_encode_rgb2ycbcr_601;
+
+ return 0;
+}
+
static int init_csc(struct ipu_ic *ic,
- enum ipu_color_space inf,
- enum ipu_color_space outf,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
- const struct ic_encode_coeff *coeff;
+ struct ic_encode_coeff coeff;
u32 __iomem *base;
const u16 (*c)[3];
const u16 *a;
u32 param;
+ int ret;
+
+ ret = calc_csc_coeffs(priv, , in, out);
+ if (ret)
+ return ret;
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
- if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- coeff = _encode_ycbcr2rgb_601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- coeff = _encode_rgb2ycbcr_601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- coeff = _encode_identity;
- else {
- dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
- return -EINVAL;
-
[PATCH v6 2/7] gpu: ipu-v3: ipu-ic: Fix BT.601 coefficients
The ycbcr2rgb and inverse rgb2ycbcr tables define the BT.601 Y'CbCr
encoding coefficients.
The rgb2ycbcr table specifically describes the BT.601 encoding from
full range RGB to full range YUV. Add table comments to make this more
clear.
The ycbcr2rgb inverse table describes encoding YUV limited range to RGB
full range. To be consistent with the rgb2ycbcr table, convert this to
YUV full range to RGB full range, and adjust/expand on the comments.
The ic_csc_rgb2rgb table is just an identity matrix, so rename to
ic_encode_identity.
Fixes: 1aa8ea0d2bd5d ("gpu: ipu-v3: Add Image Converter unit")
Suggested-by: Philipp Zabel
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/gpu/ipu-v3/ipu-ic.c | 61 ++---
1 file changed, 37 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 18816ccf600e..b63a2826b629 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -175,7 +175,7 @@ static inline void ipu_ic_write(struct ipu_ic *ic, u32
value, unsigned offset)
writel(value, ic->priv->base + offset);
}
-struct ic_csc_params {
+struct ic_encode_coeff {
s16 coeff[3][3];/* signed 9-bit integer coefficients */
s16 offset[3]; /* signed 11+2-bit fixed point offset */
u8 scale:2; /* scale coefficients * 2^(scale-1) */
@@ -183,13 +183,15 @@ struct ic_csc_params {
};
/*
- * Y = R * .299 + G * .587 + B * .114;
- * U = R * -.169 + G * -.332 + B * .500 + 128.;
- * V = R * .500 + G * -.419 + B * -.0813 + 128.;
+ * BT.601 encoding from RGB full range to YUV full range:
+ *
+ * Y = .2990 * R + .5870 * G + .1140 * B
+ * U = -.1687 * R - .3313 * G + .5000 * B + 128
+ * V = .5000 * R - .4187 * G - .0813 * B + 128
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_encode_coeff ic_encode_rgb2ycbcr_601 = {
.coeff = {
- { 77, 150, 29 },
+ { 77, 150, 29 },
{ 469, 427, 128 },
{ 128, 405, 491 },
},
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_encode_coeff ic_encode_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,17 +213,25 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
- * R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
- * G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
- * B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
+ * Inverse BT.601 encoding from YUV full range to RGB full range:
+ *
+ * R = 1. * Y + 0 * (Cb - 128) + 1.4020 * (Cr - 128)
+ * G = 1. * Y - .3442 * (Cb - 128) - 0.7142 * (Cr - 128)
+ * B = 1. * Y + 1.7720 * (Cb - 128) + 0 * (Cr - 128)
+ *
+ * equivalently (factoring out the offsets):
+ *
+ * R = 1. * Y + 0 * Cb + 1.4020 * Cr - 179.456
+ * G = 1. * Y - .3442 * Cb - 0.7142 * Cr + 135.475
+ * B = 1. * Y + 1.7720 * Cb + 0 * Cr - 226.816
*/
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_encode_coeff ic_encode_ycbcr2rgb_601 = {
.coeff = {
- { 149, 0, 204 },
- { 149, 462, 408 },
- { 149, 255, 0 },
+ { 128, 0, 179 },
+ { 128, 468, 421 },
+ { 128, 227, 0 },
},
- .offset = { -446, 266, -554 },
+ .offset = { -359, 271, -454 },
.scale = 2,
};
@@ -228,7 +241,7 @@ static int init_csc(struct ipu_ic *ic,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
- const struct ic_csc_params *params;
+ const struct ic_encode_coeff *coeff;
u32 __iomem *base;
const u16 (*c)[3];
const u16 *a;
@@ -238,26 +251,26 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ coeff = _encode_ycbcr2rgb_601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ coeff = _encode_rgb2ycbcr_601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ coeff = _encode_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
}
/* Cast to unsigned */
- c = (const u16 (*)[3])params->coeff;
- a = (const u16 *)param
[PATCH] media: imx: vdic: Fix wrong CSI group ID
The i.MX7 capture support forgot to change the group ID for the CSI
to the IPU CSI in VDIC sub-device, it was left at the i.MX7 CSI
group ID.
Fixes: 67673ed55084 ("media: staging/imx: rearrange group id to take in account
IPU")
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-vdic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/media/imx/imx-media-vdic.c
b/drivers/staging/media/imx/imx-media-vdic.c
index 2808662e2597..d36f6936c365 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -744,7 +744,7 @@ static int vdic_link_setup(struct media_entity *entity,
remote_sd = media_entity_to_v4l2_subdev(remote->entity);
/* direct pad must connect to a CSI */
- if (!(remote_sd->grp_id & IMX_MEDIA_GRP_ID_CSI) ||
+ if (!(remote_sd->grp_id & IMX_MEDIA_GRP_ID_IPU_CSI) ||
remote->index != CSI_SRC_PAD_DIRECT) {
ret = -EINVAL;
goto out;
--
2.17.1
[PATCH v2 4/4] media: imx: Don't register IPU subdevs/links if CSI port missing
The second IPU internal sub-devices were being registered and links
to them created even when the second IPU is not present. This is wrong
for i.MX6 S/DL and i.MX53 which have only a single IPU.
Fixes: e130291212df5 ("[media] media: Add i.MX media core driver")
Signed-off-by: Steve Longerbeam
Reviewed-by: Philipp Zabel
Cc: sta...@vger.kernel.org
---
drivers/staging/media/imx/imx-media-dev.c | 7 ---
.../staging/media/imx/imx-media-internal-sd.c | 22 ++-
drivers/staging/media/imx/imx-media-of.c | 58 +--
drivers/staging/media/imx/imx-media.h | 3 +-
drivers/staging/media/imx/imx7-media-csi.c| 2 +-
5 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-dev.c
b/drivers/staging/media/imx/imx-media-dev.c
index fc35508d9396..10a63a4fa90b 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -477,13 +477,6 @@ static int imx_media_probe(struct platform_device *pdev)
goto cleanup;
}
- ret = imx_media_add_ipu_internal_subdevs(imxmd);
- if (ret) {
- v4l2_err(>v4l2_dev,
-"add_ipu_internal_subdevs failed with %d\n", ret);
- goto cleanup;
- }
-
ret = imx_media_dev_notifier_register(imxmd);
if (ret)
goto del_int;
diff --git a/drivers/staging/media/imx/imx-media-internal-sd.c
b/drivers/staging/media/imx/imx-media-internal-sd.c
index e620f4adb755..dc510dcfe160 100644
--- a/drivers/staging/media/imx/imx-media-internal-sd.c
+++ b/drivers/staging/media/imx/imx-media-internal-sd.c
@@ -298,13 +298,14 @@ static int add_internal_subdev(struct imx_media_dev
*imxmd,
}
/* adds the internal subdevs in one ipu */
-static int add_ipu_internal_subdevs(struct imx_media_dev *imxmd, int ipu_id)
+int imx_media_add_ipu_internal_subdevs(struct imx_media_dev *imxmd,
+ int ipu_id)
{
enum isd_enum i;
+ int ret;
for (i = 0; i < num_isd; i++) {
const struct internal_subdev *isd = _subdev[i];
- int ret;
/*
* the CSIs are represented in the device-tree, so those
@@ -322,25 +323,10 @@ static int add_ipu_internal_subdevs(struct imx_media_dev
*imxmd, int ipu_id)
}
if (ret)
- return ret;
+ goto remove;
}
return 0;
-}
-
-int imx_media_add_ipu_internal_subdevs(struct imx_media_dev *imxmd)
-{
- int ret;
-
- ret = add_ipu_internal_subdevs(imxmd, 0);
- if (ret)
- goto remove;
-
- ret = add_ipu_internal_subdevs(imxmd, 1);
- if (ret)
- goto remove;
-
- return 0;
remove:
imx_media_remove_ipu_internal_subdevs(imxmd);
diff --git a/drivers/staging/media/imx/imx-media-of.c
b/drivers/staging/media/imx/imx-media-of.c
index a26bdeb1af34..12383f4785ad 100644
--- a/drivers/staging/media/imx/imx-media-of.c
+++ b/drivers/staging/media/imx/imx-media-of.c
@@ -23,36 +23,25 @@
int imx_media_of_add_csi(struct imx_media_dev *imxmd,
struct device_node *csi_np)
{
- int ret;
-
if (!of_device_is_available(csi_np)) {
dev_dbg(imxmd->md.dev, "%s: %pOFn not enabled\n", __func__,
csi_np);
- /* unavailable is not an error */
- return 0;
+ return -ENODEV;
}
/* add CSI fwnode to async notifier */
- ret = imx_media_add_async_subdev(imxmd, of_fwnode_handle(csi_np), NULL);
- if (ret) {
- if (ret == -EEXIST) {
- /* already added, everything is fine */
- return 0;
- }
-
- /* other error, can't continue */
- return ret;
- }
-
- return 0;
+ return imx_media_add_async_subdev(imxmd, of_fwnode_handle(csi_np),
+ NULL);
}
EXPORT_SYMBOL_GPL(imx_media_of_add_csi);
int imx_media_add_of_subdevs(struct imx_media_dev *imxmd,
struct device_node *np)
{
+ bool ipu_found[2] = {false, false};
struct device_node *csi_np;
int i, ret;
+ u32 ipu_id;
for (i = 0; ; i++) {
csi_np = of_parse_phandle(np, "ports", i);
@@ -60,12 +49,43 @@ int imx_media_add_of_subdevs(struct imx_media_dev *imxmd,
break;
ret = imx_media_of_add_csi(imxmd, csi_np);
- of_node_put(csi_np);
- if (ret)
- return ret;
+ if (ret) {
+ /* unavailable or already added is not an error */
+ if (ret == -ENODEV || ret == -EEXIST) {
+ of_node_put(
[PATCH v2 3/4] media: imx: Rename functions that add IPU-internal subdevs
For the functions that add and remove the internal IPU subdevice
descriptors, rename them to make clear they are the subdevs internal
to the IPU. Also rename the platform data structure for the internal
IPU subdevices. No functional changes.
Signed-off-by: Steve Longerbeam
Acked-by: Philipp Zabel
Cc: sta...@vger.kernel.org
---
drivers/staging/media/imx/imx-ic-common.c | 2 +-
drivers/staging/media/imx/imx-media-dev.c | 8
drivers/staging/media/imx/imx-media-internal-sd.c | 12 ++--
drivers/staging/media/imx/imx-media-vdic.c| 2 +-
drivers/staging/media/imx/imx-media.h | 6 +++---
5 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-common.c
b/drivers/staging/media/imx/imx-ic-common.c
index 765919487a73..90a926891eb9 100644
--- a/drivers/staging/media/imx/imx-ic-common.c
+++ b/drivers/staging/media/imx/imx-ic-common.c
@@ -26,7 +26,7 @@ static struct imx_ic_ops *ic_ops[IC_NUM_OPS] = {
static int imx_ic_probe(struct platform_device *pdev)
{
- struct imx_media_internal_sd_platformdata *pdata;
+ struct imx_media_ipu_internal_sd_pdata *pdata;
struct imx_ic_priv *priv;
int ret;
diff --git a/drivers/staging/media/imx/imx-media-dev.c
b/drivers/staging/media/imx/imx-media-dev.c
index 28a3d23aad5b..fc35508d9396 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -477,10 +477,10 @@ static int imx_media_probe(struct platform_device *pdev)
goto cleanup;
}
- ret = imx_media_add_internal_subdevs(imxmd);
+ ret = imx_media_add_ipu_internal_subdevs(imxmd);
if (ret) {
v4l2_err(>v4l2_dev,
-"add_internal_subdevs failed with %d\n", ret);
+"add_ipu_internal_subdevs failed with %d\n", ret);
goto cleanup;
}
@@ -491,7 +491,7 @@ static int imx_media_probe(struct platform_device *pdev)
return 0;
del_int:
- imx_media_remove_internal_subdevs(imxmd);
+ imx_media_remove_ipu_internal_subdevs(imxmd);
cleanup:
v4l2_async_notifier_cleanup(>notifier);
v4l2_device_unregister(>v4l2_dev);
@@ -508,7 +508,7 @@ static int imx_media_remove(struct platform_device *pdev)
v4l2_info(>v4l2_dev, "Removing imx-media\n");
v4l2_async_notifier_unregister(>notifier);
- imx_media_remove_internal_subdevs(imxmd);
+ imx_media_remove_ipu_internal_subdevs(imxmd);
v4l2_async_notifier_cleanup(>notifier);
media_device_unregister(>md);
v4l2_device_unregister(>v4l2_dev);
diff --git a/drivers/staging/media/imx/imx-media-internal-sd.c
b/drivers/staging/media/imx/imx-media-internal-sd.c
index 5e10d95e5529..e620f4adb755 100644
--- a/drivers/staging/media/imx/imx-media-internal-sd.c
+++ b/drivers/staging/media/imx/imx-media-internal-sd.c
@@ -1,7 +1,7 @@
/*
* Media driver for Freescale i.MX5/6 SOC
*
- * Adds the internal subdevices and the media links between them.
+ * Adds the IPU internal subdevices and the media links between them.
*
* Copyright (c) 2016 Mentor Graphics Inc.
*
@@ -192,7 +192,7 @@ static struct v4l2_subdev *find_sink(struct imx_media_dev
*imxmd,
/*
* retrieve IPU id from subdev name, note: can't get this from
-* struct imx_media_internal_sd_platformdata because if src is
+* struct imx_media_ipu_internal_sd_pdata because if src is
* a CSI, it has different struct ipu_client_platformdata which
* does not contain IPU id.
*/
@@ -270,7 +270,7 @@ static int add_internal_subdev(struct imx_media_dev *imxmd,
const struct internal_subdev *isd,
int ipu_id)
{
- struct imx_media_internal_sd_platformdata pdata;
+ struct imx_media_ipu_internal_sd_pdata pdata;
struct platform_device_info pdevinfo = {};
struct platform_device *pdev;
@@ -328,7 +328,7 @@ static int add_ipu_internal_subdevs(struct imx_media_dev
*imxmd, int ipu_id)
return 0;
}
-int imx_media_add_internal_subdevs(struct imx_media_dev *imxmd)
+int imx_media_add_ipu_internal_subdevs(struct imx_media_dev *imxmd)
{
int ret;
@@ -343,11 +343,11 @@ int imx_media_add_internal_subdevs(struct imx_media_dev
*imxmd)
return 0;
remove:
- imx_media_remove_internal_subdevs(imxmd);
+ imx_media_remove_ipu_internal_subdevs(imxmd);
return ret;
}
-void imx_media_remove_internal_subdevs(struct imx_media_dev *imxmd)
+void imx_media_remove_ipu_internal_subdevs(struct imx_media_dev *imxmd)
{
struct imx_media_async_subdev *imxasd;
struct v4l2_async_subdev *asd;
diff --git a/drivers/staging/media/imx/imx-media-vdic.c
b/drivers/staging/media/imx/imx-media-vdic.c
index 2808662e2597..8a9af4688fd4 100644
--- a/drivers/staging/medi
[PATCH v2 2/4] media: imx: Clear fwnode link struct for each endpoint iteration
In imx_media_create_csi_of_links(), the 'struct v4l2_fwnode_link' must
be cleared for each endpoint iteration, otherwise if the remote port
has no "reg" property, link.remote_port will not be reset to zero.
This was discovered on the i.MX53 SMD board, since the OV5642 connects
directly to ipu1_csi0 and has a single source port with no "reg"
property.
Fixes: 621b08eabcddb ("media: staging/imx: remove static media link arrays")
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/staging/media/imx/imx-media-of.c | 15 +--
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-of.c
b/drivers/staging/media/imx/imx-media-of.c
index 03446335ac03..a26bdeb1af34 100644
--- a/drivers/staging/media/imx/imx-media-of.c
+++ b/drivers/staging/media/imx/imx-media-of.c
@@ -145,15 +145,18 @@ int imx_media_create_csi_of_links(struct imx_media_dev
*imxmd,
struct v4l2_subdev *csi)
{
struct device_node *csi_np = csi->dev->of_node;
- struct fwnode_handle *fwnode, *csi_ep;
- struct v4l2_fwnode_link link;
struct device_node *ep;
- int ret;
-
- link.local_node = of_fwnode_handle(csi_np);
- link.local_port = CSI_SINK_PAD;
for_each_child_of_node(csi_np, ep) {
+ struct fwnode_handle *fwnode, *csi_ep;
+ struct v4l2_fwnode_link link;
+ int ret;
+
+ memset(, 0, sizeof(link));
+
+ link.local_node = of_fwnode_handle(csi_np);
+ link.local_port = CSI_SINK_PAD;
+
csi_ep = of_fwnode_handle(ep);
fwnode = fwnode_graph_get_remote_endpoint(csi_ep);
--
2.17.1
[PATCH v2 1/4] media: imx: csi: Allow unknown nearest upstream entities
On i.MX6, the nearest upstream entity to the CSI can only be the
CSI video muxes or the Synopsys DW MIPI CSI-2 receiver.
However the i.MX53 has no CSI video muxes or a MIPI CSI-2 receiver.
So allow for the nearest upstream entity to the CSI to be something
other than those.
Fixes: bf3cfaa712e5c ("media: staging/imx: get CSI bus type from nearest
upstream entity")
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/staging/media/imx/imx-media-csi.c | 18 ++
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 3b7517348666..41965d8b56c4 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -154,9 +154,10 @@ static inline bool requires_passthrough(struct
v4l2_fwnode_endpoint *ep,
/*
* Parses the fwnode endpoint from the source pad of the entity
* connected to this CSI. This will either be the entity directly
- * upstream from the CSI-2 receiver, or directly upstream from the
- * video mux. The endpoint is needed to determine the bus type and
- * bus config coming into the CSI.
+ * upstream from the CSI-2 receiver, directly upstream from the
+ * video mux, or directly upstream from the CSI itself. The endpoint
+ * is needed to determine the bus type and bus config coming into
+ * the CSI.
*/
static int csi_get_upstream_endpoint(struct csi_priv *priv,
struct v4l2_fwnode_endpoint *ep)
@@ -172,7 +173,8 @@ static int csi_get_upstream_endpoint(struct csi_priv *priv,
if (!priv->src_sd)
return -EPIPE;
- src = >src_sd->entity;
+ sd = priv->src_sd;
+ src = >entity;
if (src->function == MEDIA_ENT_F_VID_MUX) {
/*
@@ -186,6 +188,14 @@ static int csi_get_upstream_endpoint(struct csi_priv *priv,
src = >entity;
}
+ /*
+* If the source is neither the video mux nor the CSI-2 receiver,
+* get the source pad directly upstream from CSI itself.
+*/
+ if (src->function != MEDIA_ENT_F_VID_MUX &&
+ sd->grp_id != IMX_MEDIA_GRP_ID_CSI2)
+ src = >sd.entity;
+
/* get source pad of entity directly upstream from src */
pad = imx_media_find_upstream_pad(priv->md, src, 0);
if (IS_ERR(pad))
--
2.17.1
[PATCH] media: imx: vdic: Restore default case to prepare_vdi_in_buffers()
Restore a default case to prepare_vdi_in_buffers() to fix the following
smatch errors:
drivers/staging/media/imx/imx-media-vdic.c:236 prepare_vdi_in_buffers() error:
uninitialized symbol 'prev_phys'.
drivers/staging/media/imx/imx-media-vdic.c:237 prepare_vdi_in_buffers() error:
uninitialized symbol 'curr_phys'.
drivers/staging/media/imx/imx-media-vdic.c:238 prepare_vdi_in_buffers() error:
uninitialized symbol 'next_phys'.
Fixes: 6e537b58de772 ("media: imx: vdic: rely on VDIC for correct field order")
Reported-by: Hans Verkuil
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-media-vdic.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/drivers/staging/media/imx/imx-media-vdic.c
b/drivers/staging/media/imx/imx-media-vdic.c
index 2808662e2597..37bfbd4a1c39 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -231,6 +231,12 @@ static void __maybe_unused prepare_vdi_in_buffers(struct
vdic_priv *priv,
curr_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0);
next_phys = vb2_dma_contig_plane_dma_addr(curr_vb, 0) + is;
break;
+ default:
+ /*
+* can't get here, priv->fieldtype can only be one of
+* the above. This is to quiet smatch errors.
+*/
+ return;
}
ipu_cpmem_set_buffer(priv->vdi_in_ch_p, 0, prev_phys);
--
2.17.1
[PATCH v5 6/7] media: imx: Try colorimetry at both sink and source pads
The colorimetry parameters need to be tested at both sink and source
pads. Specifically, for prpencvf, the IC only supports RGB full-range
quantization at input and output.
Fix this by cleaning up imx_media_fill_default_mbus_fields(), renaming
to imx_media_try_colorimetry(), and call it at both sink and source
pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.
Signed-off-by: Steve Longerbeam
---
drivers/staging/media/imx/imx-ic-prp.c | 6 +-
drivers/staging/media/imx/imx-ic-prpencvf.c | 8 +--
drivers/staging/media/imx/imx-media-csi.c | 19 +++---
drivers/staging/media/imx/imx-media-utils.c | 68 +++--
drivers/staging/media/imx/imx-media-vdic.c | 5 +-
drivers/staging/media/imx/imx-media.h | 5 +-
drivers/staging/media/imx/imx7-media-csi.c | 8 +--
7 files changed, 62 insertions(+), 57 deletions(-)
diff --git a/drivers/staging/media/imx/imx-ic-prp.c
b/drivers/staging/media/imx/imx-ic-prp.c
index 3d43cdcb4bb9..8010ee706164 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -197,8 +197,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
sdformat->format.code = cc->codes[0];
}
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
@@ -207,6 +207,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
break;
}
+ imx_media_try_colorimetry(>format, true);
+
fmt = __prp_get_fmt(priv, cfg, sdformat->pad, sdformat->which);
*fmt = sdformat->format;
out:
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 10f2c7684727..b1886a4e362e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -899,8 +899,6 @@ static void prp_try_fmt(struct prp_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
@@ -908,9 +906,11 @@ static void prp_try_fmt(struct prp_priv *priv,
MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
S_ALIGN);
- imx_media_fill_default_mbus_fields(>format, infmt,
- true);
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
}
+
+ imx_media_try_colorimetry(>format, true);
}
static int prp_set_fmt(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/imx/imx-media-csi.c
b/drivers/staging/media/imx/imx-media-csi.c
index 3b7517348666..cc3e0086b08a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1369,9 +1369,15 @@ static void csi_try_field(struct csi_priv *priv,
struct v4l2_mbus_framefmt *infmt =
__csi_get_fmt(priv, cfg, CSI_SINK_PAD, sdformat->which);
- /* no restrictions on sink pad field type */
- if (sdformat->pad == CSI_SINK_PAD)
+ /*
+* no restrictions on sink pad field type except must
+* be initialized.
+*/
+ if (sdformat->pad == CSI_SINK_PAD) {
+ if (sdformat->format.field == V4L2_FIELD_ANY)
+ sdformat->format.field = V4L2_FIELD_NONE;
return;
+ }
switch (infmt->field) {
case V4L2_FIELD_SEQ_TB:
@@ -1449,8 +1455,6 @@ static void csi_try_fmt(struct csi_priv *priv,
/* propagate colorimetry from sink */
sdformat->format.colorspace = infmt->colorspace;
sdformat->format.xfer_func = infmt->xfer_func;
- sdformat->format.quantization = infmt->quantization;
- sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
break;
case CSI_SINK_PAD:
@@ -1470,10 +1474,6 @@ static void csi_try_fmt(struct csi_priv *priv,
csi_try_field(priv, cfg, sdformat);
- imx_media_fill_default_mbus_fields(
- >format, infmt,
- priv->active_output_pad == CSI_SRC_PAD_DIRECT);
-
[PATCH v5 7/7] media: imx: Allow BT.709 encoding for IC routes
The IC now supports BT.709 Y'CbCr encoding, in addition to existing BT.601
encoding, so allow both, for pipelines that route through the IC.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v5:
- rebased this patch on top of repurposing the function to
imx_media_try_colorimetry().
Changes in v2:
- move ic_route check above default colorimetry checks, and fill default
colorimetry for ic_route, otherwise it's not possible to set BT.709
encoding for ic routes.
---
drivers/staging/media/imx/imx-media-utils.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index aa7d4be77a7e..12967d6d7e1a 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -524,7 +524,7 @@ EXPORT_SYMBOL_GPL(imx_media_init_cfg);
* If this format is destined to be routed through the Image Converter,
* quantization and Y`CbCr encoding must be fixed. The IC supports only
* full-range quantization for RGB at its input and output, and only
- * BT.601 Y`CbCr encoding.
+ * BT.601 or Rec.709 Y`CbCr encoding.
*/
void imx_media_try_colorimetry(struct v4l2_mbus_framefmt *tryfmt,
bool ic_route)
@@ -563,7 +563,9 @@ void imx_media_try_colorimetry(struct v4l2_mbus_framefmt
*tryfmt,
tryfmt->quantization == V4L2_QUANTIZATION_DEFAULT)
tryfmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
- tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
+ if (tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_601 &&
+ tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_709)
+ tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
} else {
if (tryfmt->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT) {
tryfmt->ycbcr_enc =
--
2.17.1
[PATCH v5 1/7] gpu: ipu-v3: ipu-ic: Fix saturation bit offset in TPMEM
The saturation bit was being set at bit 9 in the second 32-bit word
of the TPMEM CSC. This isn't correct, the saturation bit is bit 42,
which is bit 10 of the second word.
Fixes: 1aa8ea0d2bd5d ("gpu: ipu-v3: Add Image Converter unit")
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/gpu/ipu-v3/ipu-ic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..18816ccf600e 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -257,7 +257,7 @@ static int init_csc(struct ipu_ic *ic,
writel(param, base++);
param = ((a[0] & 0x1fe0) >> 5) | (params->scale << 8) |
- (params->sat << 9);
+ (params->sat << 10);
writel(param, base++);
param = ((a[1] & 0x1f) << 27) | ((c[0][1] & 0x1ff) << 18) |
--
2.17.1
[PATCH v5 5/7] gpu: ipu-v3: ipu-ic: Add support for limited range encoding
Add support for the following conversions:
- YUV full-range to YUV limited-range
- YUV limited-range to YUV full-range
- YUV limited-range to RGB full-range
- RGB full-range to YUV limited-range
The last two conversions require operating on the YUV full-range
encoding and inverse encoding coefficients, with the YUV-to-YUV
limited<->full coefficients. The formula to convert is
M_c = M_a * M_b
O_c = M_a * O_b + O_a
For calculating the RGB full-range to YUV limited-range coefficients:
[M_a, O_a] = YUV full-range to YUV limited-range coefficients.
[M_b, O_b] = RGB full-range to YUV full-range coefficients.
For calculating the YUV limited-range to RGB full-range coefficients:
[M_a, O_a] = YUV full-range to RGB full-range coefficients.
[M_b, O_b] = YUV limited-range to YUV full-range coefficients.
The calculation of [M_c, O_c] is carried out by the function
transform_coeffs().
In the future if RGB limited range encoding is required, the same
function can be used. And cascaded to create all combinations of
encoding for YUV limited/full range <-> RGB limited/full range,
passing the output coefficients from one call as the input for the
next.
For example, to create YUV full-range to RGB limited-range coefficients:
[M_a, O_a] = RGB full-range to RGB limited-range coefficients.
[M_b, O_b] = YUV full-range to RGB full-range coefficients.
and that output sent as input to create YUV limited-range to RGB
limited-range coefficients:
[M_a, O_a] = YUV full-range to RGB limited-range coefficients.
[M_b, O_b] = YUV limited-range to YUV full-range coefficients.
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 281 +---
1 file changed, 263 insertions(+), 18 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 012ea2239e97..861f43556df4 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -178,10 +178,10 @@ static inline void ipu_ic_write(struct ipu_ic *ic, u32
value, unsigned offset)
}
struct ic_encode_coeff {
- s16 coeff[3][3];/* signed 9-bit integer coefficients */
- s16 offset[3]; /* signed 11+2-bit fixed point offset */
- u8 scale:2; /* scale coefficients * 2^(scale-1) */
- bool sat:1; /* saturate to (16, 235(Y) / 240(U, V)) */
+ int coeff[3][3];/* signed 9-bit integer coefficients */
+ int offset[3]; /* signed 13-bit integer offset */
+ int scale; /* scale coefficients * 2^(scale-1) */
+ bool sat; /* saturate to (16, 235(Y) / 240(U, V)) */
};
/*
@@ -277,6 +277,231 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_709 = {
.scale = 2,
};
+/*
+ * YUV full range to YUV limited range:
+ *
+ * Y_lim = 0.8588 * Y_full + 16
+ * Cb_lim = 0.8784 * (Cb_full - 128) + 128
+ * Cr_lim = 0.8784 * (Cr_full - 128) + 128
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr_full2lim = {
+ .coeff = {
+ { 219, 0, 0 },
+ { 0, 224, 0 },
+ { 0, 0, 224 },
+ },
+ .offset = { 64, 62, 62 },
+ .scale = 1,
+};
+
+/*
+ * YUV limited range to YUV full range:
+ *
+ * Y_full = 1.1644 * (Y_lim - 16)
+ * Cb_full = 1.1384 * (Cb_lim - 128) + 128
+ * Cr_full = 1.1384 * (Cr_lim - 128) + 128
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr_lim2full = {
+ .coeff = {
+ { 149, 0, 0 },
+ { 0, 145, 0 },
+ { 0, 0, 145 },
+ },
+ .offset = { -37, -35, -35 },
+ .scale = 2,
+};
+
+/*
+ * RGB full range to RGB limited range:
+ *
+ * R_lim = 0.8588 * R_full + 16
+ * G_lim = 0.8588 * G_full + 16
+ * B_lim = 0.8588 * B_full + 16
+ */
+static const struct ic_encode_coeff
+ic_encode_rgb_full2lim __maybe_unused = {
+ .coeff = {
+ { 219, 0, 0 },
+ { 0, 219, 0 },
+ { 0, 0, 219 },
+ },
+ .offset = { 64, 64, 64 },
+ .scale = 1,
+};
+
+/*
+ * RGB limited range to RGB full range:
+ *
+ * R_full = 1.1644 * (R_lim - 16)
+ * G_full = 1.1644 * (G_lim - 16)
+ * B_full = 1.1644 * (B_lim - 16)
+ */
+static const struct ic_encode_coeff
+ic_encode_rgb_lim2full __maybe_unused = {
+ .coeff = {
+ { 149, 0, 0 },
+ { 0, 149, 0 },
+ { 0, 0, 149 },
+ },
+ .offset = { -37, -37, -37 },
+ .scale = 2,
+};
+
+/*
+ * Convert a coefficient and scale value in TPMEM register format
+ * to a signed int times 256 (fix the radix point). The TPMEM register
+ * coefficient format is a signed 9-bit value (sign bit at bit 8,
+ * mantissa = coeff * 2 ^ (8 - scale - 1)).
+ */
+static int coeff_fix(int coeff, int scale)
+{
+ if (coeff >= 256)
+ coeff -= 512;
+ if (scale == 0)
+ return DIV_ROUND_CLOSEST(coeff, 2);
+ return coeff << (scale - 1);
+}
+
+/*
+ * Convert a signed int coefficie
[PATCH v5 3/7] gpu: ipu-v3: ipu-ic: Fully describe colorspace conversions
Only providing the input and output RGB/YUV space to the IC task init
functions is not sufficient. To fully characterize a colorspace
conversion, the colorspace (chromaticities), Y'CbCr encoding standard,
and quantization also need to be specified.
Define a 'struct ipu_ic_colorspace' that includes all the above, and pass
the input and output ipu_ic_colorspace to the IC task init functions.
This allows to actually enforce the fact that the IC:
- can only encode to/from YUV full range (follow-up patch will remove
this restriction).
- can only encode to/from RGB full range.
- can only encode using BT.601 standard (follow-up patch will add
Rec.709 encoding support).
- cannot convert colorspaces from input to output, the
input and output colorspace chromaticities must be the same.
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 101
drivers/gpu/ipu-v3/ipu-image-convert.c | 27 --
drivers/staging/media/imx/imx-ic-prpencvf.c | 22 +++--
include/video/imx-ipu-v3.h | 37 +--
4 files changed, 127 insertions(+), 60 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 71a0409093e6..02043f23f411 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -146,8 +146,10 @@ struct ipu_ic {
const struct ic_task_regoffs *reg;
const struct ic_task_bitfields *bit;
- enum ipu_color_space in_cs, g_in_cs;
- enum ipu_color_space out_cs;
+ struct ipu_ic_colorspace in_cs;
+ struct ipu_ic_colorspace g_in_cs;
+ struct ipu_ic_colorspace out_cs;
+
bool graphics;
bool rotation;
bool in_use;
@@ -236,8 +238,8 @@ static const struct ic_encode_coeff ic_encode_ycbcr2rgb_601
= {
};
static int init_csc(struct ipu_ic *ic,
- enum ipu_color_space inf,
- enum ipu_color_space outf,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
@@ -247,19 +249,41 @@ static int init_csc(struct ipu_ic *ic,
const u16 *a;
u32 param;
+ if (in->colorspace != out->colorspace) {
+ dev_err(priv->ipu->dev, "Cannot convert colorspaces\n");
+ return -ENOTSUPP;
+ }
+
+ if (out->enc != V4L2_YCBCR_ENC_601) {
+ dev_err(priv->ipu->dev, "Only BT.601 encoding supported\n");
+ return -ENOTSUPP;
+ }
+
+ if ((in->cs == IPUV3_COLORSPACE_YUV &&
+in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
+ (out->cs == IPUV3_COLORSPACE_YUV &&
+out->quant != V4L2_QUANTIZATION_FULL_RANGE)) {
+ dev_err(priv->ipu->dev, "Limited range YUV not supported\n");
+ return -ENOTSUPP;
+ }
+
+ if ((in->cs == IPUV3_COLORSPACE_RGB &&
+in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
+ (out->cs == IPUV3_COLORSPACE_RGB &&
+out->quant != V4L2_QUANTIZATION_FULL_RANGE)) {
+ dev_err(priv->ipu->dev, "Limited range RGB not supported\n");
+ return -ENOTSUPP;
+ }
+
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
- if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
+ if (in->cs == out->cs)
+ coeff = _encode_identity;
+ else if (in->cs == IPUV3_COLORSPACE_YUV)
coeff = _encode_ycbcr2rgb_601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
+ else
coeff = _encode_rgb2ycbcr_601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- coeff = _encode_identity;
- else {
- dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
- return -EINVAL;
- }
/* Cast to unsigned */
c = (const u16 (*)[3])coeff->coeff;
@@ -357,14 +381,14 @@ void ipu_ic_task_enable(struct ipu_ic *ic)
if (ic->rotation)
ic_conf |= ic->bit->ic_conf_rot_en;
- if (ic->in_cs != ic->out_cs)
+ if (ic->in_cs.cs != ic->out_cs.cs)
ic_conf |= ic->bit->ic_conf_csc1_en;
if (ic->graphics) {
ic_conf |= ic->bit->ic_conf_cmb_en;
ic_conf |= ic->bit->ic_conf_csc1_en;
- if (ic->g_in_cs != ic->out_cs)
+ if (ic->g_in_cs.cs != ic->out_cs.cs)
ic_conf |= ic->bit->ic_conf_csc2_en;
}
@@ -399,7 +423,7 @@ void ipu_ic_task_disable(struct ipu_ic *ic)
EXPORT_SYMBOL_GPL(i
[PATCH v5 2/7] gpu: ipu-v3: ipu-ic: Fix BT.601 coefficients
The ycbcr2rgb and inverse rgb2ycbcr tables define the BT.601 Y'CbCr
encoding coefficients.
The rgb2ycbcr table specifically describes the BT.601 encoding from
full range RGB to full range YUV. Add table comments to make this more
clear.
The ycbcr2rgb inverse table describes encoding YUV limited range to RGB
full range. To be consistent with the rgb2ycbcr table, convert this to
YUV full range to RGB full range, and adjust/expand on the comments.
The ic_csc_rgb2rgb table is just an identity matrix, so rename to
ic_encode_identity.
Fixes: 1aa8ea0d2bd5d ("gpu: ipu-v3: Add Image Converter unit")
Suggested-by: Philipp Zabel
Signed-off-by: Steve Longerbeam
Cc: sta...@vger.kernel.org
---
drivers/gpu/ipu-v3/ipu-ic.c | 63 ++---
1 file changed, 38 insertions(+), 25 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 18816ccf600e..71a0409093e6 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -175,7 +175,7 @@ static inline void ipu_ic_write(struct ipu_ic *ic, u32
value, unsigned offset)
writel(value, ic->priv->base + offset);
}
-struct ic_csc_params {
+struct ic_encode_coeff {
s16 coeff[3][3];/* signed 9-bit integer coefficients */
s16 offset[3]; /* signed 11+2-bit fixed point offset */
u8 scale:2; /* scale coefficients * 2^(scale-1) */
@@ -183,22 +183,27 @@ struct ic_csc_params {
};
/*
- * Y = R * .299 + G * .587 + B * .114;
- * U = R * -.169 + G * -.332 + B * .500 + 128.;
- * V = R * .500 + G * -.419 + B * -.0813 + 128.;
+ * BT.601 encoding from RGB full range to YUV full range:
+ *
+ * Y = .2990 * R + .5870 * G + .1140 * B
+ * U = -.1687 * R - .3313 * G + .5000 * B + 128
+ * V = .5000 * R - .4187 * G - .0813 * B + 128
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_encode_coeff ic_encode_rgb2ycbcr_601 = {
.coeff = {
- { 77, 150, 29 },
- { 469, 427, 128 },
+ { 76, 150, 29 },
+ { 469, 428, 128 },
{ 128, 405, 491 },
},
.offset = { 0, 512, 512 },
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_encode_coeff ic_encode_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,17 +213,25 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
- * R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
- * G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
- * B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
+ * Inverse BT.601 encoding from YUV full range to RGB full range:
+ *
+ * R = 1. * Y + 0 * (Cb - 128) + 1.4020 * (Cr - 128)
+ * G = 1. * Y - .3442 * (Cb - 128) - 0.7142 * (Cr - 128)
+ * B = 1. * Y + 1.7720 * (Cb - 128) + 0 * (Cr - 128)
+ *
+ * equivalently (factoring out the offsets):
+ *
+ * R = 1. * Y + 0 * Cb + 1.4020 * Cr - 179.456
+ * G = 1. * Y - .3442 * Cb - 0.7142 * Cr + 135.475
+ * B = 1. * Y + 1.7720 * Cb + 0 * Cr - 226.816
*/
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_encode_coeff ic_encode_ycbcr2rgb_601 = {
.coeff = {
- { 149, 0, 204 },
- { 149, 462, 408 },
- { 149, 255, 0 },
+ { 128, 0, 179 },
+ { 128, 468, 421 },
+ { 128, 226, 0 },
},
- .offset = { -446, 266, -554 },
+ .offset = { -359, 271, -454 },
.scale = 2,
};
@@ -228,7 +241,7 @@ static int init_csc(struct ipu_ic *ic,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
- const struct ic_csc_params *params;
+ const struct ic_encode_coeff *coeff;
u32 __iomem *base;
const u16 (*c)[3];
const u16 *a;
@@ -238,26 +251,26 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ coeff = _encode_ycbcr2rgb_601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ coeff = _encode_rgb2ycbcr_601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ coeff = _encode_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
}
/* Cast to unsigned */
- c = (const u16 (*)[3])params->coeff;
- a = (const u16 *)params->of
[PATCH v5 4/7] gpu: ipu-v3: ipu-ic: Add support for Rec.709 encoding
Add support for Rec.709 encoding and inverse encoding.
The determination of the CSC coefficients based on the input/output
colorspace parameters are moved to a new function calc_csc_coeffs().
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v5:
- moved API changes to a previous patch.
- moved CSC coeff calc to new function calc_csc_coeffs().
Changes in v4:
- fix compile error.
Chnges in v3:
- none.
Changes in v2:
- only return "Unsupported YCbCr encoding" error if inf != outf,
since if inf == outf, the identity matrix can be used. Reported
by Tim Harvey.
---
drivers/gpu/ipu-v3/ipu-ic.c | 120
1 file changed, 94 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 02043f23f411..012ea2239e97 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -214,6 +214,23 @@ static const struct ic_encode_coeff ic_encode_identity = {
.scale = 2,
};
+/*
+ * REC.709 encoding from RGB full range to YUV full range:
+ *
+ * Y = .2126 * R + .7152 * G + .0722 * B
+ * U = -.1146 * R - .3854 * G + .5000 * B + 128
+ * V = .5000 * R - .4542 * G - .0458 * B + 128
+ */
+static const struct ic_encode_coeff ic_encode_rgb2ycbcr_709 = {
+ .coeff = {
+ { 54, 183, 19 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/*
* Inverse BT.601 encoding from YUV full range to RGB full range:
*
@@ -237,28 +254,42 @@ static const struct ic_encode_coeff
ic_encode_ycbcr2rgb_601 = {
.scale = 2,
};
-static int init_csc(struct ipu_ic *ic,
- const struct ipu_ic_colorspace *in,
- const struct ipu_ic_colorspace *out,
- int csc_index)
+/*
+ * Inverse REC.709 encoding from YUV full range to RGB full range:
+ *
+ * R = 1. * Y + 0 * (Cb - 128) + 1.5748 * (Cr - 128)
+ * G = 1. * Y - .1873 * (Cb - 128) - .4681 * (Cr - 128)
+ * B = 1. * Y + 1.8556 * (Cb - 128) + 0 * (Cr - 128)
+ *
+ * equivalently (factoring out the offsets):
+ *
+ * R = 1. * Y + 0 * Cb + 1.5748 * Cr - 201.574
+ * G = 1. * Y - .1873 * Cb - .4681 * Cr + 83.891
+ * B = 1. * Y + 1.8556 * Cb + 0 * Cr - 237.517
+ */
+static const struct ic_encode_coeff ic_encode_ycbcr2rgb_709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -403, 168, -475 },
+ .scale = 2,
+};
+
+static int calc_csc_coeffs(struct ipu_ic_priv *priv,
+ struct ic_encode_coeff *coeff_out,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out)
{
- struct ipu_ic_priv *priv = ic->priv;
- const struct ic_encode_coeff *coeff;
- u32 __iomem *base;
- const u16 (*c)[3];
- const u16 *a;
- u32 param;
+ const struct ic_encode_coeff *encode_coeff;
+ bool inverse_encode;
if (in->colorspace != out->colorspace) {
dev_err(priv->ipu->dev, "Cannot convert colorspaces\n");
return -ENOTSUPP;
}
- if (out->enc != V4L2_YCBCR_ENC_601) {
- dev_err(priv->ipu->dev, "Only BT.601 encoding supported\n");
- return -ENOTSUPP;
- }
-
if ((in->cs == IPUV3_COLORSPACE_YUV &&
in->quant != V4L2_QUANTIZATION_FULL_RANGE) ||
(out->cs == IPUV3_COLORSPACE_YUV &&
@@ -275,26 +306,63 @@ static int init_csc(struct ipu_ic *ic,
return -ENOTSUPP;
}
+ if (in->cs == out->cs) {
+ *coeff_out = ic_encode_identity;
+
+ return 0;
+ }
+
+ inverse_encode = (in->cs == IPUV3_COLORSPACE_YUV);
+
+ switch (out->enc) {
+ case V4L2_YCBCR_ENC_601:
+ encode_coeff = inverse_encode ?
+ _encode_ycbcr2rgb_601 : _encode_rgb2ycbcr_601;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ encode_coeff = inverse_encode ?
+ _encode_ycbcr2rgb_709 : _encode_rgb2ycbcr_709;
+ break;
+ default:
+ dev_err(priv->ipu->dev, "Unsupported YCbCr encoding\n");
+ return -ENOTSUPP;
+ }
+
+ *coeff_out = *encode_coeff;
+
+ return 0;
+}
+
+static int init_csc(struct ipu_ic *ic,
+ const struct ipu_ic_colorspace *in,
+ const struct ipu_ic_colorspace *out,
+ int csc_index)
+{
+ struct ipu_ic_priv *priv = ic->priv;
+ struct ic_encode_coeff coeff;
+ u32 __iomem *base;
+ const u16 (*c)[3];
+ const u16 *a;
+ u32 param;
+ int ret;
+
+ ret
Re: [PATCH v4 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
On 2/13/19 2:35 AM, Philipp Zabel wrote: On Tue, 2019-02-12 at 09:42 -0800, Steve Longerbeam wrote: [...] But what about this "SAT_MODE" field in the IC task parameter memory? That just controls the saturation. The result after the matrix multiplication is either saturated to [0..255] or to [16..235]/[16..240] when converting from the internal representation to the 8 bit output. By saturation I think you mean clipped to those ranges? Yes, thanks. I didn't realize it sounds weird to use saturated this way. See:https://en.wikipedia.org/wiki/Saturation_arithmetic Ok, saturation can mean the same thing as clipping/clamping. Thanks for the article. I tested a RGB->YUV pipeline with the .sat bit set in the BT.601 rgb2yuv table, with the following pipeline on the SabreSD: 'ov5640 1-003c':0 [fmt:RGB565_2X8_LE/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'imx6-mipi-csi2':0 [fmt:RGB565_2X8_LE/1024x768 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'imx6-mipi-csi2':2 [fmt:RGB565_2X8_LE/1024x768 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'ipu1_csi1':0 [fmt:RGB565_2X8_LE/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(0,0)/1024x768 crop:(0,0)/1024x768 compose.bounds:(0,0)/1024x768 compose:(0,0)/1024x768] 'ipu1_csi1':1 [fmt:ARGB_1X32/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'ipu1_ic_prp':0 [fmt:ARGB_1X32/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'ipu1_ic_prp':1 [fmt:ARGB_1X32/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'ipu1_ic_prpenc':0 [fmt:ARGB_1X32/1024x768@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] 'ipu1_ic_prpenc':1 [fmt:AYUV8_1X32/800x600@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] /dev/video0:0 Format Video Capture: Width/Height : 800/600 Pixel Format : 'YV12' Field : None Bytes per Line : 800 Size Image : 72 Colorspace : sRGB Transfer Function : sRGB YCbCr/HSV Encoding: ITU-R 601 Quantization : Limited Range Flags : The result being that the captured image colors are all off (there's a bright pink shade to the images). But I discovered the init_csc() function was not setting the saturation bit at the correct bit offset within the TPMEM. The saturation bit is bit 42, or bit 10 of the second 32-bit word. But the code was writing to bit 9 of the second word. After correcting this, saturation is working fine. I have added another patch that fixes this for v5 series. SAT_MODE should be set for conversions to YUV limited range so that the coefficients can be rounded to the closest value. Well, we have already rounded the coefficients to the nearest int in the tables. Do you mean the final result (coeff * color component + offset) is rounded? The manual says so: "The final calculation result is limited according to the SAT_MODE parameter and rounded to 8 bits", but that's not what I meant. Still, I might have been mistaken. I think due to the fact that the coefficients are multiplied by up to 255 (max pixel value) and then effectively divided by 256 when converting to 8 bit, the only way to overflow limited range is if two coefficients are rounded away from zero in the calculation of a single component. This doesn't seem to happen in practice. A constructed example, conversion to YUV limited range with carefully chosen coefficients. Y = R * .1817 + G * .6153 + B * .0618 + 16; Note that .1817 + .6153 + .0618 < 219/255. With rounded coefficients though: Y = (R * 47 + G * 158 + B * 16 + (64 << 6)) / 256 = 236.136 Yes, for a rec.709 conversion and max/worst-case RGB signal = (255,255,255). But the rec.709 coefficients for Y are actually Y = (R * 47 + G * 157 + B * 16 + (16 << 8)) / 256 which for RGB = (255,255,255), Y = 235.14, which doesn't overflow limited range. Steve
Re: [PATCH v4 3/4] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
On 2/12/19 3:34 AM, Philipp Zabel wrote: Hi Steve, On Mon, 2019-02-11 at 17:20 -0800, Steve Longerbeam wrote: [...] Should we support YUV BT.601 <-> YUV REC.709 conversions? That would require separate encodings for input and output. How about if we pass the input and output encodings to the init ic task functions, but for now require they be the same? We can support transcoding in a later series. [...] Again, I think for now, just include input/output quantization but require full range for RGB and limited range for YUV. Yes, that is fine. I'd just like to avoid unnecessary interface changes between ipu-v3 and imx-media. So if we have to change it right now, why not plan ahead. Agreed! But that really balloons the arguments to ipu_ic_task_init_*(). Should we create an ipu_ic_task_init structure? I wonder if we should just expose struct ic_csc_params I had basically the same idea. I wasn't thinking of creating a helper to fill in the params but sure, I'll add that. Steve and provide a helper to fill it given colorspace and V4L2 encoding/quantization parameters. Something like: struct ipu_ic_csc_params csc; imx_media_init_ic_csc_params(, in_cs, in_encoding, in_quantization, out_cs, out_encoding, out_quantization); ipu_ic_task_init(ic, in_width, in_height, out_width, out_height, ); // or ipu_ic_task_init_rsc(ic, rsc, ); regards Philipp
Re: [PATCH v4 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
On 2/12/19 2:17 AM, Philipp Zabel wrote: Hi Steve, On Mon, 2019-02-11 at 10:24 -0800, Steve Longerbeam wrote: [...] Looking more closely at these coefficients now, I see you are right, they are the BT.601 YUV full-range coefficients (Y range 0 to 1, U and V range -0.5 to 0.5). Well, not even that -- the coefficients are not being scaled to the limited ranges, but the 0.5 offset (128) _is_ being added to U/V, but no offset for Y. So it is even more messed up. Your corrected coefficients and offsets look correct to me: Y coefficients scaled to (235 - 16) / 255 and U/V coefficients scaled to (240 - 16) / 255, and add the offsets for both Y and U/V. But what about this "SAT_MODE" field in the IC task parameter memory? That just controls the saturation. The result after the matrix multiplication is either saturated to [0..255] or to [16..235]/[16..240] when converting from the internal representation to the 8 bit output. By saturation I think you mean clipped to those ranges? According to the manual the hardware will automatically convert the written coefficients to the correct limited ranges. Where did you get that from? "The final calculation result is limited according to the SAT_MODE parameter and rounded to 8 bits." I see no mention of coefficients being modified. Well, as is often the case with this manual, I was interpreting based on poorly written information. By "final calculation result is limited according to the SAT_MODE parameter" I interpreted that to mean the hardware enables scaling from full range to limited range. But I concede that it more likely means it clips the output to those ranges. I see there is a "sat" field defined in the struct but is not being set in the tables. So what should we do, define the full range coefficients, and make use of SAT_MODE h/w feature, or scale/offset the coefficients ourselves and not use SAT_MODE? I'm inclined to do the former. SAT_MODE should be set for conversions to YUV limited range so that the coefficients can be rounded to the closest value. Well, we have already rounded the coefficients to the nearest int in the tables. Do you mean the final result (coeff * color component + offset) is rounded? Otherwise we'd have to round towards zero, possibly with a larger error, to make sure the results are inside the valid ranges. Makes sense, I will turn on that bit for limited range YUV output for v5, so that the final color component result is clipped to limited range and rounded. Steve
Re: [PATCH v4 3/4] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
On 2/11/19 2:12 AM, Philipp Zabel wrote:
On Fri, 2019-02-08 at 17:47 -0800, Steve Longerbeam wrote:
Pass v4l2 encoding enum to the ipu_ic task init functions, and add
support for the BT.709 encoding and inverse encoding matrices.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v4:
- fix compile error.
Chnges in v3:
- none.
Changes in v2:
- only return "Unsupported YCbCr encoding" error if inf != outf,
since if inf == outf, the identity matrix can be used. Reported
by Tim Harvey.
---
drivers/gpu/ipu-v3/ipu-ic.c | 71 +++--
drivers/gpu/ipu-v3/ipu-image-convert.c | 1 +
drivers/staging/media/imx/imx-ic-prpencvf.c | 4 +-
include/video/imx-ipu-v3.h | 5 +-
4 files changed, 71 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index e459615a49a1..c5f83d7e357f 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -212,6 +212,23 @@ static const struct ic_csc_params ic_csc_identity = {
.scale = 2,
};
+/*
+ * BT.709 encoding from RGB full range to YUV limited range:
+ *
+ * Y = R * .2126 + G * .7152 + B * .0722;
+ * U = R * -.1146 + G * -.3854 + B * .5000 + 128.;
+ * V = R * .5000 + G * -.4542 + B * -.0458 + 128.;
This is a conversion to YUV full range. Limited range should be:
Y R * .1826 + G * .6142 + B * .0620 + 16;
U = R * -.1007 + G * -.3385 + B * .4392 + 128;
V R * .4392 + G * -.3990 + B * -.0402 + 128;
Yep, I fixed these to encode to limited range YUV, and ...
+ */
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt709 = {
+ .coeff = {
+ { 54, 183, 18 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/*
* Inverse BT.601 encoding from YUV limited range to RGB full range:
*
@@ -229,12 +246,31 @@ static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601
= {
.scale = 2,
};
+/*
+ * Inverse BT.709 encoding from YUV limited range to RGB full range:
+ *
+ * R = (1. * (Y - 16)) + (1.5748 * (Cr - 128));
+ * G = (1. * (Y - 16)) - (0.1873 * (Cb - 128)) - (0.4681 * (Cr - 128));
+ * B = (1. * (Y - 16)) + (1.8556 * (Cb - 128);
The coefficients look like full range again, conversion from limited
range YUV should look like:
R = (1.1644 * (Y - 16)) + (1.7927 * (Cr - 128));
G = (1.1644 * (Y - 16)) - (0.2132 * (Cb - 128)) - (0.5329 * (Cr - 128));
B = (1.1644 * (Y - 16)) + (2.1124 * (Cb - 128);
fixed these to inverse encode from limited range YUV.
+ */
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -435, 136, -507 },
+ .scale = 2,
+};
+
static int init_csc(struct ipu_ic *ic,
enum ipu_color_space inf,
enum ipu_color_space outf,
+ enum v4l2_ycbcr_encoding encoding,
Should we support YUV BT.601 <-> YUV REC.709 conversions? That would
require separate encodings for input and output.
How about if we pass the input and output encodings to the init ic task
functions, but for now require they be the same? We can support
transcoding in a later series.
Also, this might be a
good time to think about adding quantization range parameters as well.
Again, I think for now, just include input/output quantization but
require full range for RGB and limited range for YUV.
But that really balloons the arguments to ipu_ic_task_init_*(). Should
we create an ipu_ic_task_init structure?
Steve
Re: [PATCH v4 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
On 2/11/19 1:58 AM, Philipp Zabel wrote:
On Fri, 2019-02-08 at 17:47 -0800, Steve Longerbeam wrote:
The ycbcr2rgb and inverse rgb2ycbcr matrices define the BT.601 encoding
coefficients, so rename them to indicate that. And add some comments
to make clear these are BT.601 coefficients encoding between YUV limited
range and RGB full range. The ic_csc_rgb2rgb matrix is just an identity
matrix, so rename to ic_csc_identity. No functional changes.
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- rename ic_csc_rgb2rgb matrix to ic_csc_identity.
---
drivers/gpu/ipu-v3/ipu-ic.c | 21 ++---
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..3ef61f0b509b 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -183,11 +183,13 @@ struct ic_csc_params {
};
/*
+ * BT.601 encoding from RGB full range to YUV limited range:
+ *
* Y = R * .299 + G * .587 + B * .114;
* U = R * -.169 + G * -.332 + B * .500 + 128.;
* V = R * .500 + G * -.419 + B * -.0813 + 128.;
Hm, this is a conversion to full range BT.601. For limited range, the
matrix coefficients
0.2990 0.5870 0.1140
-0.1687 -0.3313 0.5000
0.5000 -0.4187 -0.0813
should be multiplied with 219/255 (Y) and 224/255 (U,V), respectively:
Y = R * .2568 + G * .5041 + B * .0979 + 16;
U = R * -.1482 + G * -.2910 + B * .4392 + 128;
V = R * .4392 + G * -.3678 + B * -.0714 + 128;
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt601 = {
.coeff = {
{ 77, 150, 29 },
{ 469, 427, 128 },
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_csc_params ic_csc_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,11 +213,13 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
+ * Inverse BT.601 encoding from YUV limited range to RGB full range:
+ *
* R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
* G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
* B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
*/
This looks correct.
Agreed, the coefficients in the comments are correct, but the actual
table values were a bit off. I will correct them for v5 (Green offset
should be 272 in the table, not 266).
Steve
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601 = {
.coeff = {
{ 149, 0, 204 },
{ 149, 462, 408 },
@@ -238,11 +245,11 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ params = _csc_ycbcr2rgb_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ params = _csc_rgb2ycbcr_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ params = _csc_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
regards
Philipp
Re: [PATCH v4 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
Hi Philipp,
On 2/11/19 1:58 AM, Philipp Zabel wrote:
On Fri, 2019-02-08 at 17:47 -0800, Steve Longerbeam wrote:
The ycbcr2rgb and inverse rgb2ycbcr matrices define the BT.601 encoding
coefficients, so rename them to indicate that. And add some comments
to make clear these are BT.601 coefficients encoding between YUV limited
range and RGB full range. The ic_csc_rgb2rgb matrix is just an identity
matrix, so rename to ic_csc_identity. No functional changes.
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- rename ic_csc_rgb2rgb matrix to ic_csc_identity.
---
drivers/gpu/ipu-v3/ipu-ic.c | 21 ++---
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..3ef61f0b509b 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -183,11 +183,13 @@ struct ic_csc_params {
};
/*
+ * BT.601 encoding from RGB full range to YUV limited range:
+ *
* Y = R * .299 + G * .587 + B * .114;
* U = R * -.169 + G * -.332 + B * .500 + 128.;
* V = R * .500 + G * -.419 + B * -.0813 + 128.;
Hm, this is a conversion to full range BT.601. For limited range, the
matrix coefficients
0.2990 0.5870 0.1140
-0.1687 -0.3313 0.5000
0.5000 -0.4187 -0.0813
should be multiplied with 219/255 (Y) and 224/255 (U,V), respectively:
Y = R * .2568 + G * .5041 + B * .0979 + 16;
U = R * -.1482 + G * -.2910 + B * .4392 + 128;
V = R * .4392 + G * -.3678 + B * -.0714 + 128;
Looking more closely at these coefficients now, I see you are right,
they are the BT.601 YUV full-range coefficients (Y range 0 to 1, U and V
range -0.5 to 0.5). Well, not even that -- the coefficients are not
being scaled to the limited ranges, but the 0.5 offset (128) _is_ being
added to U/V, but no offset for Y. So it is even more messed up.
Your corrected coefficients and offsets look correct to me: Y
coefficients scaled to (235 - 16) / 255 and U/V coefficients scaled to
(240 - 16) / 255, and add the offsets for both Y and U/V.
But what about this "SAT_MODE" field in the IC task parameter memory?
According to the manual the hardware will automatically convert the
written coefficients to the correct limited ranges. I see there is a
"sat" field defined in the struct but is not being set in the tables.
So what should we do, define the full range coefficients, and make use
of SAT_MODE h/w feature, or scale/offset the coefficients ourselves and
not use SAT_MODE? I'm inclined to do the former.
Steve
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt601 = {
.coeff = {
{ 77, 150, 29 },
{ 469, 427, 128 },
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_csc_params ic_csc_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,11 +213,13 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
+ * Inverse BT.601 encoding from YUV limited range to RGB full range:
+ *
* R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
* G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
* B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
*/
This looks correct.
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601 = {
.coeff = {
{ 149, 0, 204 },
{ 149, 462, 408 },
@@ -238,11 +245,11 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ params = _csc_ycbcr2rgb_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ params = _csc_rgb2ycbcr_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ params = _csc_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
regards
Philipp
[PATCH v4 2/4] gpu: ipu-v3: ipu-ic: Simplify selection of encoding matrix
Simplify the selection of the Y'CbCr encoding matrices in init_csc().
A side-effect of this change is that init_csc() now allows YUV->YUV
using the identity matrix, intead of returning error.
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 12
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 3ef61f0b509b..e459615a49a1 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -244,16 +244,12 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
- if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
+ if (inf == outf)
+ params = _csc_identity;
+ else if (inf == IPUV3_COLORSPACE_YUV)
params = _csc_ycbcr2rgb_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
+ else
params = _csc_rgb2ycbcr_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_identity;
- else {
- dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
- return -EINVAL;
- }
/* Cast to unsigned */
c = (const u16 (*)[3])params->coeff;
--
2.17.1
[PATCH v4 4/4] media: imx: Allow BT.709 encoding for IC routes
The IC now supports BT.709 Y'CbCr encoding, in addition to existing BT.601
encoding, so allow both, for pipelines that route through the IC.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- move ic_route check above default colorimetry checks, and fill default
colorimetry for ic_route, otherwise it's not possible to set BT.709
encoding for ic routes.
---
drivers/staging/media/imx/imx-media-utils.c | 20 +---
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index 5f110d90a4ef..dde0e47550d7 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -544,6 +544,19 @@ void imx_media_fill_default_mbus_fields(struct
v4l2_mbus_framefmt *tryfmt,
if (tryfmt->field == V4L2_FIELD_ANY)
tryfmt->field = fmt->field;
+ if (ic_route) {
+ if (tryfmt->colorspace == V4L2_COLORSPACE_DEFAULT)
+ tryfmt->colorspace = fmt->colorspace;
+
+ tryfmt->quantization = is_rgb ?
+ V4L2_QUANTIZATION_FULL_RANGE :
+ V4L2_QUANTIZATION_LIM_RANGE;
+
+ if (tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_601 &&
+ tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_709)
+ tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
+ }
+
/* fill colorimetry if necessary */
if (tryfmt->colorspace == V4L2_COLORSPACE_DEFAULT) {
tryfmt->colorspace = fmt->colorspace;
@@ -566,13 +579,6 @@ void imx_media_fill_default_mbus_fields(struct
v4l2_mbus_framefmt *tryfmt,
tryfmt->ycbcr_enc);
}
}
-
- if (ic_route) {
- tryfmt->quantization = is_rgb ?
- V4L2_QUANTIZATION_FULL_RANGE :
- V4L2_QUANTIZATION_LIM_RANGE;
- tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
- }
}
EXPORT_SYMBOL_GPL(imx_media_fill_default_mbus_fields);
--
2.17.1
[PATCH v4 3/4] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
Pass v4l2 encoding enum to the ipu_ic task init functions, and add
support for the BT.709 encoding and inverse encoding matrices.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v4:
- fix compile error.
Chnges in v3:
- none.
Changes in v2:
- only return "Unsupported YCbCr encoding" error if inf != outf,
since if inf == outf, the identity matrix can be used. Reported
by Tim Harvey.
---
drivers/gpu/ipu-v3/ipu-ic.c | 71 +++--
drivers/gpu/ipu-v3/ipu-image-convert.c | 1 +
drivers/staging/media/imx/imx-ic-prpencvf.c | 4 +-
include/video/imx-ipu-v3.h | 5 +-
4 files changed, 71 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index e459615a49a1..c5f83d7e357f 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -212,6 +212,23 @@ static const struct ic_csc_params ic_csc_identity = {
.scale = 2,
};
+/*
+ * BT.709 encoding from RGB full range to YUV limited range:
+ *
+ * Y = R * .2126 + G * .7152 + B * .0722;
+ * U = R * -.1146 + G * -.3854 + B * .5000 + 128.;
+ * V = R * .5000 + G * -.4542 + B * -.0458 + 128.;
+ */
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt709 = {
+ .coeff = {
+ { 54, 183, 18 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/*
* Inverse BT.601 encoding from YUV limited range to RGB full range:
*
@@ -229,12 +246,31 @@ static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601
= {
.scale = 2,
};
+/*
+ * Inverse BT.709 encoding from YUV limited range to RGB full range:
+ *
+ * R = (1. * (Y - 16)) + (1.5748 * (Cr - 128));
+ * G = (1. * (Y - 16)) - (0.1873 * (Cb - 128)) - (0.4681 * (Cr - 128));
+ * B = (1. * (Y - 16)) + (1.8556 * (Cb - 128);
+ */
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -435, 136, -507 },
+ .scale = 2,
+};
+
static int init_csc(struct ipu_ic *ic,
enum ipu_color_space inf,
enum ipu_color_space outf,
+ enum v4l2_ycbcr_encoding encoding,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
+ const struct ic_csc_params *params_rgb2yuv, *params_yuv2rgb;
const struct ic_csc_params *params;
u32 __iomem *base;
const u16 (*c)[3];
@@ -244,12 +280,30 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
+ switch (encoding) {
+ case V4L2_YCBCR_ENC_601:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt601;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt601;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt709;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt709;
+ break;
+ default:
+ if (inf != outf) {
+ dev_err(priv->ipu->dev,
+ "Unsupported YCbCr encoding\n");
+ return -EINVAL;
+ }
+ break;
+ }
+
if (inf == outf)
params = _csc_identity;
else if (inf == IPUV3_COLORSPACE_YUV)
- params = _csc_ycbcr2rgb_bt601;
+ params = params_yuv2rgb;
else
- params = _csc_rgb2ycbcr_bt601;
+ params = params_rgb2yuv;
/* Cast to unsigned */
c = (const u16 (*)[3])params->coeff;
@@ -390,6 +444,7 @@ EXPORT_SYMBOL_GPL(ipu_ic_task_disable);
int ipu_ic_task_graphics_init(struct ipu_ic *ic,
enum ipu_color_space in_g_cs,
+ enum v4l2_ycbcr_encoding encoding,
bool galpha_en, u32 galpha,
bool colorkey_en, u32 colorkey)
{
@@ -408,7 +463,7 @@ int ipu_ic_task_graphics_init(struct ipu_ic *ic,
if (!(ic_conf & ic->bit->ic_conf_csc1_en)) {
/* need transparent CSC1 conversion */
ret = init_csc(ic, IPUV3_COLORSPACE_RGB,
- IPUV3_COLORSPACE_RGB, 0);
+ IPUV3_COLORSPACE_RGB, encoding, 0);
if (ret)
goto unlock;
}
@@ -416,7 +471,7 @@ int ipu_ic_task_graphics_init(struct ipu_ic *ic,
ic->g_in_cs = in_g_cs;
if (ic->g_in_cs != ic->out_cs) {
- ret = init_csc(ic, ic->g_in_cs, ic->out_cs, 1);
+ ret = init_csc(ic, ic->g_in_cs, ic->out_cs, encoding, 1);
if (ret)
goto unlock;
}
@@ -450,6 +505,7 @@
[PATCH v4 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
The ycbcr2rgb and inverse rgb2ycbcr matrices define the BT.601 encoding
coefficients, so rename them to indicate that. And add some comments
to make clear these are BT.601 coefficients encoding between YUV limited
range and RGB full range. The ic_csc_rgb2rgb matrix is just an identity
matrix, so rename to ic_csc_identity. No functional changes.
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- rename ic_csc_rgb2rgb matrix to ic_csc_identity.
---
drivers/gpu/ipu-v3/ipu-ic.c | 21 ++---
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..3ef61f0b509b 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -183,11 +183,13 @@ struct ic_csc_params {
};
/*
+ * BT.601 encoding from RGB full range to YUV limited range:
+ *
* Y = R * .299 + G * .587 + B * .114;
* U = R * -.169 + G * -.332 + B * .500 + 128.;
* V = R * .500 + G * -.419 + B * -.0813 + 128.;
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt601 = {
.coeff = {
{ 77, 150, 29 },
{ 469, 427, 128 },
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_csc_params ic_csc_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,11 +213,13 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
+ * Inverse BT.601 encoding from YUV limited range to RGB full range:
+ *
* R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
* G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
* B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
*/
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601 = {
.coeff = {
{ 149, 0, 204 },
{ 149, 462, 408 },
@@ -238,11 +245,11 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ params = _csc_ycbcr2rgb_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ params = _csc_rgb2ycbcr_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ params = _csc_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
--
2.17.1
Re: [PATCH v3 3/4] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
On 2/8/19 4:20 PM, Tim Harvey wrote: On Fri, Feb 8, 2019 at 11:28 AM Steve Longerbeam wrote: if (inf == outf) params = _csc_identity; else if (inf == IPUV3_COLORSPACE_YUV) - params = _csc_ycbcr2rgb_bt601; + params = _csc_ycbcr2rgb; Steve, compile issue... params = params_yuv2rgb; else - params = _csc_rgb2ycbcr_bt601; + params = _csc_rgb2ycbcr; params = params_rgb2yuv; Wow, did I not even compile test that? Must be my head cold :-/ Sending v4. But, I'm still failing when using the mem2mem element (gst-launch-1.0 v4l2src device=/dev/video4 ! v4l2video8convert output-io-mode=dmabuf-import ! fbdevsink) with 'Unsupported YCbCr encoding' because of inf=IPU_COLORSPACE_YCBCR outf=IPU_COLORSPACE_RGB and a seemingly unset encoding being passed in. It looks like maybe something in the mem2mem driver isn't defaulting encoding. The call path is (v4l2_m2m_streamon -> device_run -> ipu_image_convert_queue -> convert_start -> ipu_ic_task_init_rsc -> init_csc). Looking at v7 of the mem2mem driver, it will set ycbcr_enc at the output side to V4L2_YCBCR_ENC_DEFAULT if colorspace is default. So colorspace will need to be set to something non-default in addition to setting ycbcr_enc, at the output side. I don't know whether gstreamer v4l2videoNconvertelement will do this, but you could hack the driver for now to get around it, and let Philipp know this may need a workaround in mem2mem for v8. Steve
[PATCH v3 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
The ycbcr2rgb and inverse rgb2ycbcr matrices define the BT.601 encoding
coefficients, so rename them to indicate that. And add some comments
to make clear these are BT.601 coefficients encoding between YUV limited
range and RGB full range. The ic_csc_rgb2rgb matrix is just an identity
matrix, so rename to ic_csc_identity. No functional changes.
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- rename ic_csc_rgb2rgb matrix to ic_csc_identity.
---
drivers/gpu/ipu-v3/ipu-ic.c | 21 ++---
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..3ef61f0b509b 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -183,11 +183,13 @@ struct ic_csc_params {
};
/*
+ * BT.601 encoding from RGB full range to YUV limited range:
+ *
* Y = R * .299 + G * .587 + B * .114;
* U = R * -.169 + G * -.332 + B * .500 + 128.;
* V = R * .500 + G * -.419 + B * -.0813 + 128.;
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt601 = {
.coeff = {
{ 77, 150, 29 },
{ 469, 427, 128 },
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_csc_params ic_csc_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,11 +213,13 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
+ * Inverse BT.601 encoding from YUV limited range to RGB full range:
+ *
* R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
* G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
* B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
*/
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601 = {
.coeff = {
{ 149, 0, 204 },
{ 149, 462, 408 },
@@ -238,11 +245,11 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ params = _csc_ycbcr2rgb_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ params = _csc_rgb2ycbcr_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ params = _csc_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
--
2.17.1
[PATCH v3 3/4] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
Pass v4l2 encoding enum to the ipu_ic task init functions, and add
support for the BT.709 encoding and inverse encoding matrices.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- only return "Unsupported YCbCr encoding" error if inf != outf,
since if inf == outf, the identity matrix can be used. Reported
by Tim Harvey.
---
drivers/gpu/ipu-v3/ipu-ic.c | 71 +++--
drivers/gpu/ipu-v3/ipu-image-convert.c | 1 +
drivers/staging/media/imx/imx-ic-prpencvf.c | 4 +-
include/video/imx-ipu-v3.h | 5 +-
4 files changed, 71 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index e459615a49a1..0d57ca7ba18e 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -212,6 +212,23 @@ static const struct ic_csc_params ic_csc_identity = {
.scale = 2,
};
+/*
+ * BT.709 encoding from RGB full range to YUV limited range:
+ *
+ * Y = R * .2126 + G * .7152 + B * .0722;
+ * U = R * -.1146 + G * -.3854 + B * .5000 + 128.;
+ * V = R * .5000 + G * -.4542 + B * -.0458 + 128.;
+ */
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt709 = {
+ .coeff = {
+ { 54, 183, 18 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/*
* Inverse BT.601 encoding from YUV limited range to RGB full range:
*
@@ -229,12 +246,31 @@ static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601
= {
.scale = 2,
};
+/*
+ * Inverse BT.709 encoding from YUV limited range to RGB full range:
+ *
+ * R = (1. * (Y - 16)) + (1.5748 * (Cr - 128));
+ * G = (1. * (Y - 16)) - (0.1873 * (Cb - 128)) - (0.4681 * (Cr - 128));
+ * B = (1. * (Y - 16)) + (1.8556 * (Cb - 128);
+ */
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -435, 136, -507 },
+ .scale = 2,
+};
+
static int init_csc(struct ipu_ic *ic,
enum ipu_color_space inf,
enum ipu_color_space outf,
+ enum v4l2_ycbcr_encoding encoding,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
+ const struct ic_csc_params *params_rgb2yuv, *params_yuv2rgb;
const struct ic_csc_params *params;
u32 __iomem *base;
const u16 (*c)[3];
@@ -244,12 +280,30 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
+ switch (encoding) {
+ case V4L2_YCBCR_ENC_601:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt601;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt601;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt709;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt709;
+ break;
+ default:
+ if (inf != outf) {
+ dev_err(priv->ipu->dev,
+ "Unsupported YCbCr encoding\n");
+ return -EINVAL;
+ }
+ break;
+ }
+
if (inf == outf)
params = _csc_identity;
else if (inf == IPUV3_COLORSPACE_YUV)
- params = _csc_ycbcr2rgb_bt601;
+ params = _csc_ycbcr2rgb;
else
- params = _csc_rgb2ycbcr_bt601;
+ params = _csc_rgb2ycbcr;
/* Cast to unsigned */
c = (const u16 (*)[3])params->coeff;
@@ -390,6 +444,7 @@ EXPORT_SYMBOL_GPL(ipu_ic_task_disable);
int ipu_ic_task_graphics_init(struct ipu_ic *ic,
enum ipu_color_space in_g_cs,
+ enum v4l2_ycbcr_encoding encoding,
bool galpha_en, u32 galpha,
bool colorkey_en, u32 colorkey)
{
@@ -408,7 +463,7 @@ int ipu_ic_task_graphics_init(struct ipu_ic *ic,
if (!(ic_conf & ic->bit->ic_conf_csc1_en)) {
/* need transparent CSC1 conversion */
ret = init_csc(ic, IPUV3_COLORSPACE_RGB,
- IPUV3_COLORSPACE_RGB, 0);
+ IPUV3_COLORSPACE_RGB, encoding, 0);
if (ret)
goto unlock;
}
@@ -416,7 +471,7 @@ int ipu_ic_task_graphics_init(struct ipu_ic *ic,
ic->g_in_cs = in_g_cs;
if (ic->g_in_cs != ic->out_cs) {
- ret = init_csc(ic, ic->g_in_cs, ic->out_cs, 1);
+ ret = init_csc(ic, ic->g_in_cs, ic->out_cs, encoding, 1);
if (ret)
goto unlock;
}
@@ -450,6 +505,7 @@ int ipu_ic_task_init_rsc(struct ipu_ic *ic,
[PATCH v3 4/4] media: imx: Allow BT.709 encoding for IC routes
The IC now supports BT.709 Y'CbCr encoding, in addition to existing BT.601
encoding, so allow both, for pipelines that route through the IC.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- move ic_route check above default colorimetry checks, and fill default
colorimetry for ic_route, otherwise it's not possible to set BT.709
encoding for ic routes.
---
drivers/staging/media/imx/imx-media-utils.c | 20 +---
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index 5f110d90a4ef..dde0e47550d7 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -544,6 +544,19 @@ void imx_media_fill_default_mbus_fields(struct
v4l2_mbus_framefmt *tryfmt,
if (tryfmt->field == V4L2_FIELD_ANY)
tryfmt->field = fmt->field;
+ if (ic_route) {
+ if (tryfmt->colorspace == V4L2_COLORSPACE_DEFAULT)
+ tryfmt->colorspace = fmt->colorspace;
+
+ tryfmt->quantization = is_rgb ?
+ V4L2_QUANTIZATION_FULL_RANGE :
+ V4L2_QUANTIZATION_LIM_RANGE;
+
+ if (tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_601 &&
+ tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_709)
+ tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
+ }
+
/* fill colorimetry if necessary */
if (tryfmt->colorspace == V4L2_COLORSPACE_DEFAULT) {
tryfmt->colorspace = fmt->colorspace;
@@ -566,13 +579,6 @@ void imx_media_fill_default_mbus_fields(struct
v4l2_mbus_framefmt *tryfmt,
tryfmt->ycbcr_enc);
}
}
-
- if (ic_route) {
- tryfmt->quantization = is_rgb ?
- V4L2_QUANTIZATION_FULL_RANGE :
- V4L2_QUANTIZATION_LIM_RANGE;
- tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
- }
}
EXPORT_SYMBOL_GPL(imx_media_fill_default_mbus_fields);
--
2.17.1
[PATCH v3 2/4] gpu: ipu-v3: ipu-ic: Simplify selection of encoding matrix
Simplify the selection of the Y'CbCr encoding matrices in init_csc().
A side-effect of this change is that init_csc() now allows YUV->YUV
using the identity matrix, intead of returning error.
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 12
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 3ef61f0b509b..e459615a49a1 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -244,16 +244,12 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
- if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
+ if (inf == outf)
+ params = _csc_identity;
+ else if (inf == IPUV3_COLORSPACE_YUV)
params = _csc_ycbcr2rgb_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
+ else
params = _csc_rgb2ycbcr_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_identity;
- else {
- dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
- return -EINVAL;
- }
/* Cast to unsigned */
c = (const u16 (*)[3])params->coeff;
--
2.17.1
[PATCH v2 4/4] media: imx: Allow BT.709 encoding for IC routes
From: Steve Longerbeam
The IC now supports BT.709 Y'CbCr encoding, in addition to existing BT.601
encoding, so allow both, for pipelines that route through the IC.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- move ic_route check above default colorimetry checks, and fill default
colorimetry for ic_route, otherwise it's not possible to set BT.709
encoding for ic routes.
---
drivers/staging/media/imx/imx-media-utils.c | 20 +---
1 file changed, 13 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/imx/imx-media-utils.c
b/drivers/staging/media/imx/imx-media-utils.c
index 5f110d90a4ef..dde0e47550d7 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -544,6 +544,19 @@ void imx_media_fill_default_mbus_fields(struct
v4l2_mbus_framefmt *tryfmt,
if (tryfmt->field == V4L2_FIELD_ANY)
tryfmt->field = fmt->field;
+ if (ic_route) {
+ if (tryfmt->colorspace == V4L2_COLORSPACE_DEFAULT)
+ tryfmt->colorspace = fmt->colorspace;
+
+ tryfmt->quantization = is_rgb ?
+ V4L2_QUANTIZATION_FULL_RANGE :
+ V4L2_QUANTIZATION_LIM_RANGE;
+
+ if (tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_601 &&
+ tryfmt->ycbcr_enc != V4L2_YCBCR_ENC_709)
+ tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
+ }
+
/* fill colorimetry if necessary */
if (tryfmt->colorspace == V4L2_COLORSPACE_DEFAULT) {
tryfmt->colorspace = fmt->colorspace;
@@ -566,13 +579,6 @@ void imx_media_fill_default_mbus_fields(struct
v4l2_mbus_framefmt *tryfmt,
tryfmt->ycbcr_enc);
}
}
-
- if (ic_route) {
- tryfmt->quantization = is_rgb ?
- V4L2_QUANTIZATION_FULL_RANGE :
- V4L2_QUANTIZATION_LIM_RANGE;
- tryfmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
- }
}
EXPORT_SYMBOL_GPL(imx_media_fill_default_mbus_fields);
--
2.17.1
[PATCH v2 1/4] gpu: ipu-v3: ipu-ic: Rename yuv2rgb encoding matrices
From: Steve Longerbeam
The ycbcr2rgb and inverse rgb2ycbcr matrices define the BT.601 encoding
coefficients, so rename them to indicate that. And add some comments
to make clear these are BT.601 coefficients encoding between YUV limited
range and RGB full range. The ic_csc_rgb2rgb matrix is just an identity
matrix, so rename to ic_csc_identity. No functional changes.
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- rename ic_csc_rgb2rgb matrix to ic_csc_identity.
---
drivers/gpu/ipu-v3/ipu-ic.c | 21 ++---
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 594c3cbc8291..3ef61f0b509b 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -183,11 +183,13 @@ struct ic_csc_params {
};
/*
+ * BT.601 encoding from RGB full range to YUV limited range:
+ *
* Y = R * .299 + G * .587 + B * .114;
* U = R * -.169 + G * -.332 + B * .500 + 128.;
* V = R * .500 + G * -.419 + B * -.0813 + 128.;
*/
-static const struct ic_csc_params ic_csc_rgb2ycbcr = {
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt601 = {
.coeff = {
{ 77, 150, 29 },
{ 469, 427, 128 },
@@ -197,8 +199,11 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr = {
.scale = 1,
};
-/* transparent RGB->RGB matrix for graphics combining */
-static const struct ic_csc_params ic_csc_rgb2rgb = {
+/*
+ * identity matrix, used for transparent RGB->RGB graphics
+ * combining.
+ */
+static const struct ic_csc_params ic_csc_identity = {
.coeff = {
{ 128, 0, 0 },
{ 0, 128, 0 },
@@ -208,11 +213,13 @@ static const struct ic_csc_params ic_csc_rgb2rgb = {
};
/*
+ * Inverse BT.601 encoding from YUV limited range to RGB full range:
+ *
* R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128));
* G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128));
* B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128);
*/
-static const struct ic_csc_params ic_csc_ycbcr2rgb = {
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601 = {
.coeff = {
{ 149, 0, 204 },
{ 149, 462, 408 },
@@ -238,11 +245,11 @@ static int init_csc(struct ipu_ic *ic,
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb;
+ params = _csc_ycbcr2rgb_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr;
+ params = _csc_rgb2ycbcr_bt601;
else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_rgb2rgb;
+ params = _csc_identity;
else {
dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
return -EINVAL;
--
2.17.1
[PATCH v2 3/4] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
From: Steve Longerbeam
Pass v4l2 encoding enum to the ipu_ic task init functions, and add
support for the BT.709 encoding and inverse encoding matrices.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
Changes in v2:
- only return "Unsupported YCbCr encoding" error if inf != outf,
since if inf == outf, the identity matrix can be used. Reported
by Tim Harvey.
---
drivers/gpu/ipu-v3/ipu-ic.c | 71 +++--
drivers/gpu/ipu-v3/ipu-image-convert.c | 1 +
drivers/staging/media/imx/imx-ic-prpencvf.c | 4 +-
include/video/imx-ipu-v3.h | 5 +-
4 files changed, 71 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index e459615a49a1..0d57ca7ba18e 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -212,6 +212,23 @@ static const struct ic_csc_params ic_csc_identity = {
.scale = 2,
};
+/*
+ * BT.709 encoding from RGB full range to YUV limited range:
+ *
+ * Y = R * .2126 + G * .7152 + B * .0722;
+ * U = R * -.1146 + G * -.3854 + B * .5000 + 128.;
+ * V = R * .5000 + G * -.4542 + B * -.0458 + 128.;
+ */
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt709 = {
+ .coeff = {
+ { 54, 183, 18 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/*
* Inverse BT.601 encoding from YUV limited range to RGB full range:
*
@@ -229,12 +246,31 @@ static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601
= {
.scale = 2,
};
+/*
+ * Inverse BT.709 encoding from YUV limited range to RGB full range:
+ *
+ * R = (1. * (Y - 16)) + (1.5748 * (Cr - 128));
+ * G = (1. * (Y - 16)) - (0.1873 * (Cb - 128)) - (0.4681 * (Cr - 128));
+ * B = (1. * (Y - 16)) + (1.8556 * (Cb - 128);
+ */
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -435, 136, -507 },
+ .scale = 2,
+};
+
static int init_csc(struct ipu_ic *ic,
enum ipu_color_space inf,
enum ipu_color_space outf,
+ enum v4l2_ycbcr_encoding encoding,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
+ const struct ic_csc_params *params_rgb2yuv, *params_yuv2rgb;
const struct ic_csc_params *params;
u32 __iomem *base;
const u16 (*c)[3];
@@ -244,12 +280,30 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
+ switch (encoding) {
+ case V4L2_YCBCR_ENC_601:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt601;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt601;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt709;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt709;
+ break;
+ default:
+ if (inf != outf) {
+ dev_err(priv->ipu->dev,
+ "Unsupported YCbCr encoding\n");
+ return -EINVAL;
+ }
+ break;
+ }
+
if (inf == outf)
params = _csc_identity;
else if (inf == IPUV3_COLORSPACE_YUV)
- params = _csc_ycbcr2rgb_bt601;
+ params = _csc_ycbcr2rgb;
else
- params = _csc_rgb2ycbcr_bt601;
+ params = _csc_rgb2ycbcr;
/* Cast to unsigned */
c = (const u16 (*)[3])params->coeff;
@@ -390,6 +444,7 @@ EXPORT_SYMBOL_GPL(ipu_ic_task_disable);
int ipu_ic_task_graphics_init(struct ipu_ic *ic,
enum ipu_color_space in_g_cs,
+ enum v4l2_ycbcr_encoding encoding,
bool galpha_en, u32 galpha,
bool colorkey_en, u32 colorkey)
{
@@ -408,7 +463,7 @@ int ipu_ic_task_graphics_init(struct ipu_ic *ic,
if (!(ic_conf & ic->bit->ic_conf_csc1_en)) {
/* need transparent CSC1 conversion */
ret = init_csc(ic, IPUV3_COLORSPACE_RGB,
- IPUV3_COLORSPACE_RGB, 0);
+ IPUV3_COLORSPACE_RGB, encoding, 0);
if (ret)
goto unlock;
}
@@ -416,7 +471,7 @@ int ipu_ic_task_graphics_init(struct ipu_ic *ic,
ic->g_in_cs = in_g_cs;
if (ic->g_in_cs != ic->out_cs) {
- ret = init_csc(ic, ic->g_in_cs, ic->out_cs, 1);
+ ret = init_csc(ic, ic->g_in_cs, ic->out_cs, encoding, 1);
if (ret)
goto unlock;
}
@@ -450,6 +505,7 @@ int ipu_ic_task_init_rsc(s
[PATCH v2 2/4] gpu: ipu-v3: ipu-ic: Simplify selection of encoding matrix
Simplify the selection of the Y'CbCr encoding matrices in init_csc().
A side-effect of this change is that init_csc() now allows YUV->YUV
using the identity matrix, intead of returning error.
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 12
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 3ef61f0b509b..e459615a49a1 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -244,16 +244,12 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
- if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
+ if (inf == outf)
+ params = _csc_identity;
+ else if (inf == IPUV3_COLORSPACE_YUV)
params = _csc_ycbcr2rgb_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
+ else
params = _csc_rgb2ycbcr_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_identity;
- else {
- dev_err(priv->ipu->dev, "Unsupported color space conversion\n");
- return -EINVAL;
- }
/* Cast to unsigned */
c = (const u16 (*)[3])params->coeff;
--
2.17.1
Re: [PATCH 2/3] gpu: ipu-v3: ipu-ic: Add support for BT.709 encoding
On 2/8/19 8:24 AM, Tim Harvey wrote:
On Sun, Feb 3, 2019 at 11:48 AM Steve Longerbeam wrote:
Pass v4l2 encoding enum to the ipu_ic task init functions, and add
support for the BT.709 encoding and inverse encoding matrices.
Reported-by: Tim Harvey
Signed-off-by: Steve Longerbeam
---
drivers/gpu/ipu-v3/ipu-ic.c | 67 ++---
drivers/gpu/ipu-v3/ipu-image-convert.c | 1 +
drivers/staging/media/imx/imx-ic-prpencvf.c | 4 +-
include/video/imx-ipu-v3.h | 5 +-
4 files changed, 67 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
index 35ae86ff0585..63362b4fff81 100644
--- a/drivers/gpu/ipu-v3/ipu-ic.c
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -199,6 +199,23 @@ static const struct ic_csc_params ic_csc_rgb2ycbcr_bt601 =
{
.scale = 1,
};
+/*
+ * BT.709 encoding from RGB full range to YUV limited range:
+ *
+ * Y = R * .2126 + G * .7152 + B * .0722;
+ * U = R * -.1146 + G * -.3854 + B * .5000 + 128.;
+ * V = R * .5000 + G * -.4542 + B * -.0458 + 128.;
+ */
+static const struct ic_csc_params ic_csc_rgb2ycbcr_bt709 = {
+ .coeff = {
+ { 54, 183, 18 },
+ { 483, 413, 128 },
+ { 128, 396, 500 },
+ },
+ .offset = { 0, 512, 512 },
+ .scale = 1,
+};
+
/* transparent RGB->RGB matrix for graphics combining */
static const struct ic_csc_params ic_csc_rgb2rgb = {
.coeff = {
@@ -226,12 +243,31 @@ static const struct ic_csc_params ic_csc_ycbcr2rgb_bt601
= {
.scale = 2,
};
+/*
+ * Inverse BT.709 encoding from YUV limited range to RGB full range:
+ *
+ * R = (1. * (Y - 16)) + (1.5748 * (Cr - 128));
+ * G = (1. * (Y - 16)) - (0.1873 * (Cb - 128)) - (0.4681 * (Cr - 128));
+ * B = (1. * (Y - 16)) + (1.8556 * (Cb - 128);
+ */
+static const struct ic_csc_params ic_csc_ycbcr2rgb_bt709 = {
+ .coeff = {
+ { 128, 0, 202 },
+ { 128, 488, 452 },
+ { 128, 238, 0 },
+ },
+ .offset = { -435, 136, -507 },
+ .scale = 2,
+};
+
static int init_csc(struct ipu_ic *ic,
enum ipu_color_space inf,
enum ipu_color_space outf,
+ enum v4l2_ycbcr_encoding encoding,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
+ const struct ic_csc_params *params_rgb2yuv, *params_yuv2rgb;
const struct ic_csc_params *params;
u32 __iomem *base;
const u16 (*c)[3];
@@ -241,10 +277,24 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
+ switch (encoding) {
+ case V4L2_YCBCR_ENC_601:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt601;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt601;
+ break;
+ case V4L2_YCBCR_ENC_709:
+ params_rgb2yuv = _csc_rgb2ycbcr_bt709;
+ params_yuv2rgb = _csc_ycbcr2rgb_bt709;
+ break;
+ default:
+ dev_err(priv->ipu->dev, "Unsupported YCbCr encoding\n");
+ return -EINVAL;
+ }
+
Steve,
This will fail for RGB to RGB with 'Unsupported YCbCr encoding. We
need to account for the RGB->RGB case.
How about something like:
Thanks for reporting Tim
I rather keep the check for supported encoding, and instead get rid of
"Unsupported color space conversion" error, because that is the YUV->YUV
case which can be allowed using the identity matrix.
Steve
static int init_csc(struct ipu_ic *ic,
enum ipu_color_space inf,
enum ipu_color_space outf,
+ enum v4l2_ycbcr_encoding encoding,
int csc_index)
{
struct ipu_ic_priv *priv = ic->priv;
- const struct ic_csc_params *params;
+ const struct ic_csc_params *params = NULL;
u32 __iomem *base;
const u16 (*c)[3];
const u16 *a;
@@ -241,13 +276,18 @@ static int init_csc(struct ipu_ic *ic,
base = (u32 __iomem *)
(priv->tpmem_base + ic->reg->tpmem_csc[csc_index]);
- if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB)
- params = _csc_ycbcr2rgb_bt601;
- else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV)
- params = _csc_rgb2ycbcr_bt601;
+ if (inf == IPUV3_COLORSPACE_YUV && outf == IPUV3_COLORSPACE_RGB) {
+ params = (encoding == V4L2_YCBCR_ENC_601) ?
+ _csc_ycbcr2rgb_bt601 : _csc_ycbcr2rgb_bt709;
+ }
+ else if (inf == IPUV3_COLORSPACE_RGB && outf == IPUV3_COLORSPACE_YUV) {
+ params = (encoding == V4L2_YCBCR_ENC_601) ?
+ _csc_rgb2ycbcr_bt601 : _csc_rgb2ycbcr_bt709;
+ }
el
