RE: ADV7180 Capture with i.MX53
> -Original Message-
> From: Steve Longerbeam
>
> On 10/9/19 8:40 AM, Tim Harvey wrote:
> > On Tue, Oct 8, 2019 at 4:48 PM Fabio Estevam
> wrote:
> >> Hi Tim,
> >>
> >> On Tue, Oct 8, 2019 at 6:01 PM Tim Harvey
> wrote:
> >>
> >>> Ok that's strange indeed. I did recently test 5.3 on a Gateworks
> >>> IMX6 board with ADV7180 and the one patch to drop the first few
> >>> frames and its stable. What does your testing show on an IMX6 and do
> >>> you know
> >> I will give it a try on a imx6q-sabreauto board for comparison.
> >>
> >>> when it may have broken for IMX53?
> >> i.MX53 capture is relatively new and this is my first time trying to
> >> get it to work with mainline.
> >>
> >> I assume I should do something similar to your
> >> https://raw.githubusercontent.com/Gateworks/media-ctl-
> setup/master/me
> >> dia-ctl-setup script, more especifically the mode 3 setup where you
> >> have:
> >>
> > I struggled with coming up with a way to easily document all the
> > variations with our boards as we have several different boards that
> > have an adv7180 using different CSI ports and then you also have to
> > deal with the differences between IMX6SDL and IMX6Q. The script was
> > the best solution I could come up with but if you read through it its
> > pretty complicated.
> >
> >> case "$SENS" in
> >> adv7180)
> >> fmt "'$SENSOR':0 [fmt:UYVY2X8/$res field:alternate]"
> >> fmt "'ipu${IPU}_csi${CSI}_mux':$((p+1)) [fmt:UYVY2X8/$res]"
> >> # rec709 config at CSI pad 0
> >> fmt "'ipu${IPU}_csi${CSI}':0 [fmt:$fmt field:$field colorspace:rec709
> >> ycbcr:709]"
> >> # CSI src pad output is frame height
> >> h=$((h*2))
> >> res=${w}x${h}
> >> fmt "'ipu${IPU}_csi${CSI}':1 [fmt:AYUV32/$res]"
> >> fmt "'ipu${IPU}_vdic':2 [fmt:AYUV32/$res field:none]"
> >> fmt "'ipu${IPU}_ic_prp':2 [fmt:AYUV32/$res field:none]"
> >> fmt "'$EP':1 [fmt:AYUV32/$res]"
> >> ;;
> >>
> >> Why do you multiple h by 2?
> > The input the the CSI is a field of 240 lines but the vdic will
> > combine these and have 480 lines. I don't recall exactly why but for
> > this to propagate properly you need to set the 480 lines on the csi
> > output.
>
> The reason is because there are is no register status bits that will tell you
> whether a captured field was field 0 or field 1. For this reason the driver
> can't
> support alternate capture mode (which captures individual fields and reports
> to userspace in the returned v4l2_buffer's whether the field is a top field or
> bottom field). So the CSI captures two consecutive fields and reports field
> type seq-bt or seq-tb, and doubles height, at its output.
>
>
> >>> I do have a discussion going on here about NEWAVMODE and BT.656-3
> vs
> >>> BT.656-4. I wonder if its something to do with that as that issue
> >>> causes rolling video on IMX6 with ADV7280:
> >>> https://patchwork.kernel.org/patch/7579/
> >> Tested this patch, but it did not help.
>
> About a year ago Matthew Starr reported similar horizontal rolling issue with
> i.mx53 + adv7180. I didn't have an explanation for the problem, since IPU
> register-level is the same between i.mx53 and i.mx6, and
> adv7180 capture is working fine on i.mx6 SabreAuto. And like now, the
> skipping of initial corrupt frames solved vertical sync but not the horizontal
> rolling. I never heard back whether he was able to track down the issue.
>
> Matthew, were you ever able to solve this?
Steve,
I never had a chance to dig deeper on this issue. The last time it was worked
on the video could never get a proper sync between frames, so the images were
always split between frames.
If there are some new updates over the last year I would be willing to test
them out.
- Matt Starr
>
> > That patch won't affect adv7180 but I wonder if the issues you are
> > having have to do with something like this. The adv7180_init function
> > will set BT.656-4 mode and adjust V bit end position for NTSC... I
> > don't know if that's consistent with the IMX53 CSI needs?
>
> Well, like mentioned above, the IPU CSI is register-level compatible between
> i.mx53 and i.mx6, at least according to the reference manuals, so the non-
> standard V-bit position set by adv7180, and adjusted for by the CSI, should
> work for i.mx53 just like it does for i.mx6. But it's possible there are some
> undocumented differences in the CSI between these SoC's.
>
> Steve
>
> > There are
> > lots of little tweaks that can be made to the EAV/SAV codes and its
> > not clear to me how to deal with compat issues like i have run into
> > with the adv7280 config not being compatible with the IMX6 CSI needs.
Re: ADV7180 Capture with i.MX53
On 10/9/19 8:40 AM, Tim Harvey wrote:
On Tue, Oct 8, 2019 at 4:48 PM Fabio Estevam wrote:
Hi Tim,
On Tue, Oct 8, 2019 at 6:01 PM Tim Harvey wrote:
Ok that's strange indeed. I did recently test 5.3 on a Gateworks IMX6
board with ADV7180 and the one patch to drop the first few frames and
its stable. What does your testing show on an IMX6 and do you know
I will give it a try on a imx6q-sabreauto board for comparison.
when it may have broken for IMX53?
i.MX53 capture is relatively new and this is my first time trying to
get it to work with mainline.
I assume I should do something similar to your
https://raw.githubusercontent.com/Gateworks/media-ctl-setup/master/media-ctl-setup
script, more especifically the mode 3 setup where you have:
I struggled with coming up with a way to easily document all the
variations with our boards as we have several different boards that
have an adv7180 using different CSI ports and then you also have to
deal with the differences between IMX6SDL and IMX6Q. The script was
the best solution I could come up with but if you read through it its
pretty complicated.
case "$SENS" in
adv7180)
fmt "'$SENSOR':0 [fmt:UYVY2X8/$res field:alternate]"
fmt "'ipu${IPU}_csi${CSI}_mux':$((p+1)) [fmt:UYVY2X8/$res]"
# rec709 config at CSI pad 0
fmt "'ipu${IPU}_csi${CSI}':0 [fmt:$fmt field:$field colorspace:rec709
ycbcr:709]"
# CSI src pad output is frame height
h=$((h*2))
res=${w}x${h}
fmt "'ipu${IPU}_csi${CSI}':1 [fmt:AYUV32/$res]"
fmt "'ipu${IPU}_vdic':2 [fmt:AYUV32/$res field:none]"
fmt "'ipu${IPU}_ic_prp':2 [fmt:AYUV32/$res field:none]"
fmt "'$EP':1 [fmt:AYUV32/$res]"
;;
Why do you multiple h by 2?
The input the the CSI is a field of 240 lines but the vdic will
combine these and have 480 lines. I don't recall exactly why but for
this to propagate properly you need to set the 480 lines on the csi
output.
The reason is because there are is no register status bits that will
tell you whether a captured field was field 0 or field 1. For this
reason the driver can't support alternate capture mode (which captures
individual fields and reports to userspace in the returned v4l2_buffer's
whether the field is a top field or bottom field). So the CSI captures
two consecutive fields and reports field type seq-bt or seq-tb, and
doubles height, at its output.
I do have a discussion going on here about NEWAVMODE and BT.656-3 vs
BT.656-4. I wonder if its something to do with that as that issue
causes rolling video on IMX6 with ADV7280:
https://patchwork.kernel.org/patch/7579/
Tested this patch, but it did not help.
About a year ago Matthew Starr reported similar horizontal rolling issue
with i.mx53 + adv7180. I didn't have an explanation for the problem,
since IPU register-level is the same between i.mx53 and i.mx6, and
adv7180 capture is working fine on i.mx6 SabreAuto. And like now, the
skipping of initial corrupt frames solved vertical sync but not the
horizontal rolling. I never heard back whether he was able to track down
the issue.
Matthew, were you ever able to solve this?
That patch won't affect adv7180 but I wonder if the issues you are
having have to do with something like this. The adv7180_init function
will set BT.656-4 mode and adjust V bit end position for NTSC... I
don't know if that's consistent with the IMX53 CSI needs?
Well, like mentioned above, the IPU CSI is register-level compatible
between i.mx53 and i.mx6, at least according to the reference manuals,
so the non-standard V-bit position set by adv7180, and adjusted for by
the CSI, should work for i.mx53 just like it does for i.mx6. But it's
possible there are some undocumented differences in the CSI between
these SoC's.
Steve
There are
lots of little tweaks that can be made to the EAV/SAV codes and its
not clear to me how to deal with compat issues like i have run into
with the adv7280 config not being compatible with the IMX6 CSI needs.
Re: ADV7180 Capture with i.MX53
On Tue, Oct 8, 2019 at 4:48 PM Fabio Estevam wrote:
>
> Hi Tim,
>
> On Tue, Oct 8, 2019 at 6:01 PM Tim Harvey wrote:
>
> > Ok that's strange indeed. I did recently test 5.3 on a Gateworks IMX6
> > board with ADV7180 and the one patch to drop the first few frames and
> > its stable. What does your testing show on an IMX6 and do you know
>
> I will give it a try on a imx6q-sabreauto board for comparison.
>
> > when it may have broken for IMX53?
>
> i.MX53 capture is relatively new and this is my first time trying to
> get it to work with mainline.
>
> I assume I should do something similar to your
> https://raw.githubusercontent.com/Gateworks/media-ctl-setup/master/media-ctl-setup
> script, more especifically the mode 3 setup where you have:
>
I struggled with coming up with a way to easily document all the
variations with our boards as we have several different boards that
have an adv7180 using different CSI ports and then you also have to
deal with the differences between IMX6SDL and IMX6Q. The script was
the best solution I could come up with but if you read through it its
pretty complicated.
> case "$SENS" in
> adv7180)
> fmt "'$SENSOR':0 [fmt:UYVY2X8/$res field:alternate]"
> fmt "'ipu${IPU}_csi${CSI}_mux':$((p+1)) [fmt:UYVY2X8/$res]"
> # rec709 config at CSI pad 0
> fmt "'ipu${IPU}_csi${CSI}':0 [fmt:$fmt field:$field colorspace:rec709
> ycbcr:709]"
> # CSI src pad output is frame height
> h=$((h*2))
> res=${w}x${h}
> fmt "'ipu${IPU}_csi${CSI}':1 [fmt:AYUV32/$res]"
> fmt "'ipu${IPU}_vdic':2 [fmt:AYUV32/$res field:none]"
> fmt "'ipu${IPU}_ic_prp':2 [fmt:AYUV32/$res field:none]"
> fmt "'$EP':1 [fmt:AYUV32/$res]"
> ;;
>
> Why do you multiple h by 2?
The input the the CSI is a field of 240 lines but the vdic will
combine these and have 480 lines. I don't recall exactly why but for
this to propagate properly you need to set the 480 lines on the csi
output.
>
> > I do have a discussion going on here about NEWAVMODE and BT.656-3 vs
> > BT.656-4. I wonder if its something to do with that as that issue
> > causes rolling video on IMX6 with ADV7280:
> > https://patchwork.kernel.org/patch/7579/
>
> Tested this patch, but it did not help.
That patch won't affect adv7180 but I wonder if the issues you are
having have to do with something like this. The adv7180_init function
will set BT.656-4 mode and adjust V bit end position for NTSC... I
don't know if that's consistent with the IMX53 CSI needs? There are
lots of little tweaks that can be made to the EAV/SAV codes and its
not clear to me how to deal with compat issues like i have run into
with the adv7280 config not being compatible with the IMX6 CSI needs.
Tim
Re: ADV7180 Capture with i.MX53
Hi Tim,
On Tue, Oct 8, 2019 at 6:01 PM Tim Harvey wrote:
> Ok that's strange indeed. I did recently test 5.3 on a Gateworks IMX6
> board with ADV7180 and the one patch to drop the first few frames and
> its stable. What does your testing show on an IMX6 and do you know
I will give it a try on a imx6q-sabreauto board for comparison.
> when it may have broken for IMX53?
i.MX53 capture is relatively new and this is my first time trying to
get it to work with mainline.
I assume I should do something similar to your
https://raw.githubusercontent.com/Gateworks/media-ctl-setup/master/media-ctl-setup
script, more especifically the mode 3 setup where you have:
case "$SENS" in
adv7180)
fmt "'$SENSOR':0 [fmt:UYVY2X8/$res field:alternate]"
fmt "'ipu${IPU}_csi${CSI}_mux':$((p+1)) [fmt:UYVY2X8/$res]"
# rec709 config at CSI pad 0
fmt "'ipu${IPU}_csi${CSI}':0 [fmt:$fmt field:$field colorspace:rec709
ycbcr:709]"
# CSI src pad output is frame height
h=$((h*2))
res=${w}x${h}
fmt "'ipu${IPU}_csi${CSI}':1 [fmt:AYUV32/$res]"
fmt "'ipu${IPU}_vdic':2 [fmt:AYUV32/$res field:none]"
fmt "'ipu${IPU}_ic_prp':2 [fmt:AYUV32/$res field:none]"
fmt "'$EP':1 [fmt:AYUV32/$res]"
;;
Why do you multiple h by 2?
> I do have a discussion going on here about NEWAVMODE and BT.656-3 vs
> BT.656-4. I wonder if its something to do with that as that issue
> causes rolling video on IMX6 with ADV7280:
> https://patchwork.kernel.org/patch/7579/
Tested this patch, but it did not help.
Thanks
Re: ADV7180 Capture with i.MX53
On Tue, Oct 8, 2019 at 1:34 PM Fabio Estevam wrote: > > Hi Tim, > > On Tue, Oct 8, 2019 at 1:55 PM Tim Harvey wrote: > > > I still carry around a patch from Steve for imx-csi that drops first > > few frames from BT656 sources: > > https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad > > to deal with this. > > Tried this patch and now I see the scrolling happening horizontally > (from right to left). > > Stil trying to get stable video from ADV7180. Ok that's strange indeed. I did recently test 5.3 on a Gateworks IMX6 board with ADV7180 and the one patch to drop the first few frames and its stable. What does your testing show on an IMX6 and do you know when it may have broken for IMX53? I do have a discussion going on here about NEWAVMODE and BT.656-3 vs BT.656-4. I wonder if its something to do with that as that issue causes rolling video on IMX6 with ADV7280: https://patchwork.kernel.org/patch/7579/ Tim
Re: ADV7180 Capture with i.MX53
Hi Tim, On Tue, Oct 8, 2019 at 1:55 PM Tim Harvey wrote: > I still carry around a patch from Steve for imx-csi that drops first > few frames from BT656 sources: > https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad > to deal with this. Tried this patch and now I see the scrolling happening horizontally (from right to left). Stil trying to get stable video from ADV7180. Thanks
Re: ADV7180 Capture with i.MX53
On 08/10/2019 18:30, Steve Longerbeam wrote: On 10/8/19 10:20 AM, Ian Arkver wrote: On 08/10/2019 18:14, Steve Longerbeam wrote: On 10/8/19 9:55 AM, Tim Harvey wrote: On Tue, Oct 8, 2019 at 4:21 AM Fabio Estevam wrote: On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: So now I need to see if I can get Gstreamer to accept a pipeline like: gst-lauch-1.0 v4l2src ! kmssink Ok, so now I decided use the hardware video deinterlacer: media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:alternate]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x480field:none]" And then Gstreamer can be launched: # gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink --verbose Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800 /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480 Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive Fabio, Yes, you need to use the vdic to capture from adv7180 with gstreamer as it can't handle alternate. However the video looks like a broken old TV scrolling the image horizontally: https://www.dropbox.com/s/2yef8egn6s8z7ff/mx53_adv7180_capture.mp4?dl=0 This would be because of the initial corrupt frames that this and many other decoders produce while waiting for proper sync. I added 'g_skip_frames' support in 9483a3f8e1b58ba1d7cd21687d8d0a63a015c36b but I'm not sure how to get gstreamer to use it? I still carry around a patch from Steve for imx-csi that drops first few frames from BT656 sources: https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad to deal with this. Yes, that's likely the issue, from a look at Fabio's video. The patch referenced by Tim hard-codes the number of frames to skip, instead of calling the adv7180's g_skip_frames op. I still don't have an answer as to how to call the adv7180 from the CSI subdev. Seems to me initial corrupt frames would produce a fixed offset of some kind. A rolling video like that looks more like the number of lines being captured is wrong. Nope, rolling video is one of the symptoms of initial corrupt frames, from my own experience. I don't really have an explanation for it, but IIRC the IPU will insert lines on its own to recover from an initial wrong # lines captured, to regain vertical sync. That should mean the rolling should eventually stop once vertical sync is re-established, but I've seen many instances where rolling video continues, and skipping the initial corrupt frames fixes it. OK, good to know. Thanks Steve. Steve
Re: ADV7180 Capture with i.MX53
On 10/8/19 10:20 AM, Ian Arkver wrote: On 08/10/2019 18:14, Steve Longerbeam wrote: On 10/8/19 9:55 AM, Tim Harvey wrote: On Tue, Oct 8, 2019 at 4:21 AM Fabio Estevam wrote: On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: So now I need to see if I can get Gstreamer to accept a pipeline like: gst-lauch-1.0 v4l2src ! kmssink Ok, so now I decided use the hardware video deinterlacer: media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:alternate]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x480field:none]" And then Gstreamer can be launched: # gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink --verbose Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800 /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480 Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive Fabio, Yes, you need to use the vdic to capture from adv7180 with gstreamer as it can't handle alternate. However the video looks like a broken old TV scrolling the image horizontally: https://www.dropbox.com/s/2yef8egn6s8z7ff/mx53_adv7180_capture.mp4?dl=0 This would be because of the initial corrupt frames that this and many other decoders produce while waiting for proper sync. I added 'g_skip_frames' support in 9483a3f8e1b58ba1d7cd21687d8d0a63a015c36b but I'm not sure how to get gstreamer to use it? I still carry around a patch from Steve for imx-csi that drops first few frames from BT656 sources: https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad to deal with this. Yes, that's likely the issue, from a look at Fabio's video. The patch referenced by Tim hard-codes the number of frames to skip, instead of calling the adv7180's g_skip_frames op. I still don't have an answer as to how to call the adv7180 from the CSI subdev. Seems to me initial corrupt frames would produce a fixed offset of some kind. A rolling video like that looks more like the number of lines being captured is wrong. Nope, rolling video is one of the symptoms of initial corrupt frames, from my own experience. I don't really have an explanation for it, but IIRC the IPU will insert lines on its own to recover from an initial wrong # lines captured, to regain vertical sync. That should mean the rolling should eventually stop once vertical sync is re-established, but I've seen many instances where rolling video continues, and skipping the initial corrupt frames fixes it. Steve
Re: ADV7180 Capture with i.MX53
On 08/10/2019 18:14, Steve Longerbeam wrote: On 10/8/19 9:55 AM, Tim Harvey wrote: On Tue, Oct 8, 2019 at 4:21 AM Fabio Estevam wrote: On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: So now I need to see if I can get Gstreamer to accept a pipeline like: gst-lauch-1.0 v4l2src ! kmssink Ok, so now I decided use the hardware video deinterlacer: media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:alternate]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x480field:none]" And then Gstreamer can be launched: # gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink --verbose Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800 /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480 Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive Fabio, Yes, you need to use the vdic to capture from adv7180 with gstreamer as it can't handle alternate. However the video looks like a broken old TV scrolling the image horizontally: https://www.dropbox.com/s/2yef8egn6s8z7ff/mx53_adv7180_capture.mp4?dl=0 This would be because of the initial corrupt frames that this and many other decoders produce while waiting for proper sync. I added 'g_skip_frames' support in 9483a3f8e1b58ba1d7cd21687d8d0a63a015c36b but I'm not sure how to get gstreamer to use it? I still carry around a patch from Steve for imx-csi that drops first few frames from BT656 sources: https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad to deal with this. Yes, that's likely the issue, from a look at Fabio's video. The patch referenced by Tim hard-codes the number of frames to skip, instead of calling the adv7180's g_skip_frames op. I still don't have an answer as to how to call the adv7180 from the CSI subdev. Seems to me initial corrupt frames would produce a fixed offset of some kind. A rolling video like that looks more like the number of lines being captured is wrong. Regards, Ian. Steve
Re: ADV7180 Capture with i.MX53
On 10/8/19 9:55 AM, Tim Harvey wrote: On Tue, Oct 8, 2019 at 4:21 AM Fabio Estevam wrote: On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: So now I need to see if I can get Gstreamer to accept a pipeline like: gst-lauch-1.0 v4l2src ! kmssink Ok, so now I decided use the hardware video deinterlacer: media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:alternate]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x480field:none]" And then Gstreamer can be launched: # gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink --verbose Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800 /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480 Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive Fabio, Yes, you need to use the vdic to capture from adv7180 with gstreamer as it can't handle alternate. However the video looks like a broken old TV scrolling the image horizontally: https://www.dropbox.com/s/2yef8egn6s8z7ff/mx53_adv7180_capture.mp4?dl=0 This would be because of the initial corrupt frames that this and many other decoders produce while waiting for proper sync. I added 'g_skip_frames' support in 9483a3f8e1b58ba1d7cd21687d8d0a63a015c36b but I'm not sure how to get gstreamer to use it? I still carry around a patch from Steve for imx-csi that drops first few frames from BT656 sources: https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad to deal with this. Yes, that's likely the issue, from a look at Fabio's video. The patch referenced by Tim hard-codes the number of frames to skip, instead of calling the adv7180's g_skip_frames op. I still don't have an answer as to how to call the adv7180 from the CSI subdev. Steve
Re: ADV7180 Capture with i.MX53
On Tue, Oct 8, 2019 at 4:21 AM Fabio Estevam wrote: > > On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: > > > So now I need to see if I can get Gstreamer to accept a pipeline like: > > > > gst-lauch-1.0 v4l2src ! kmssink > > Ok, so now I decided use the hardware video deinterlacer: > > media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':0[1]" > media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" > media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" > media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" > media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" > > media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:alternate]" > media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" > media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" > media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" > media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x480field:none]" > > And then Gstreamer can be launched: > > # gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink --verbose > Setting pipeline to PAUSED ... > Pipeline is live and does not need PREROLL ... > /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800 > /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480 > Setting pipeline to PLAYING ... > New clock: GstSystemClock > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = > video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, > framerate=(fraction)25/1, colorimetry=(string)bt601, > interlace-mode=(string)progressive > /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = > video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, > framerate=(fraction)25/1, colorimetry=(string)bt601, > interlace-mode=(string)progressive > Fabio, Yes, you need to use the vdic to capture from adv7180 with gstreamer as it can't handle alternate. > However the video looks like a broken old TV scrolling the image horizontally: > https://www.dropbox.com/s/2yef8egn6s8z7ff/mx53_adv7180_capture.mp4?dl=0 > This would be because of the initial corrupt frames that this and many other decoders produce while waiting for proper sync. I added 'g_skip_frames' support in 9483a3f8e1b58ba1d7cd21687d8d0a63a015c36b but I'm not sure how to get gstreamer to use it? I still carry around a patch from Steve for imx-csi that drops first few frames from BT656 sources: https://github.com/Gateworks/linux-imx6/commit/959fbd42ee6433f49ef4a04fb1abe8f8c78db5ad to deal with this. Tim
Re: ADV7180 Capture with i.MX53
On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: > So now I need to see if I can get Gstreamer to accept a pipeline like: > > gst-lauch-1.0 v4l2src ! kmssink Ok, so now I decided use the hardware video deinterlacer: media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':0[1]" media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:alternate]" media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480]" media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/720x480field:none]" And then Gstreamer can be launched: # gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink --verbose Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800 /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480 Setting pipeline to PLAYING ... New clock: GstSystemClock /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)720, height=(int)480, framerate=(fraction)25/1, colorimetry=(string)bt601, interlace-mode=(string)progressive However the video looks like a broken old TV scrolling the image horizontally: https://www.dropbox.com/s/2yef8egn6s8z7ff/mx53_adv7180_capture.mp4?dl=0 Any suggestions, please? Thanks
Re: ADV7180 Capture with i.MX53
On Mon, Oct 7, 2019 at 10:07 PM Fabio Estevam wrote: > > On Mon, Oct 7, 2019 at 9:51 PM Steve Longerbeam wrote: > > > Ah progress. Try: > > > > v4l2-ctl -d0 --set-fmt-video=field=interlaced > > Yes, with this hint I can run: > > # v4l2-ctl -d0 --set-fmt-video=field=interlaced > # v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw > --stream-count=1 > > And it seems to accept the capture of a frame. > > Without passing field=interlaced, I used to get: > > # v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw > --stream-count=1 > [ 163.838944] ipu1_csi0: capture format not valid > > So now I need to see if I can get Gstreamer to accept a pipeline like: > > gst-lauch-1.0 v4l2src ! kmssink Even though the one-frame capture via v4l2-ctl seems to work: # v4l2-ctl -d0 --set-fmt-video=field=interlaced # v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw --stream-count=1 , Gstreamer is still not happy: # gst-launch-1.0 v4l2src ! kmssink Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' does not support progressive interlacing Additional debug info: ../../../gst-plugins-good-1.14.4/sys/v4l2/gstv4l2object.c(3813): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device wants interleaved interlacing Execution ended after 0:00:00.022400639 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...
Re: ADV7180 Capture with i.MX53
On Mon, Oct 7, 2019 at 9:51 PM Steve Longerbeam wrote: > Ah progress. Try: > > v4l2-ctl -d0 --set-fmt-video=field=interlaced Yes, with this hint I can run: # v4l2-ctl -d0 --set-fmt-video=field=interlaced # v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw --stream-count=1 And it seems to accept the capture of a frame. Without passing field=interlaced, I used to get: # v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=x.raw --stream-count=1 [ 163.838944] ipu1_csi0: capture format not valid So now I need to see if I can get Gstreamer to accept a pipeline like: gst-lauch-1.0 v4l2src ! kmssink Thanks > Unless you specify interlaced at the video interface, the driver will > just combine alternate fields into seq-bt. I guess the v4l2src plugin > doesn't support seq-bt.
Re: ADV7180 Capture with i.MX53
On 10/7/19 5:46 PM, Fabio Estevam wrote: Hi Steve, On Mon, Oct 7, 2019 at 9:37 PM Steve Longerbeam wrote: The adv7180 driver in 5.3.x doesn't support seq-bt, only alternate. So pad format should be "[fmt:UYVY2X8/720x240 field:alternate]". Thanks for the suggestion. After changing to field:alternate I get: # gst-launch-1.0 v4l2src ! fakesink Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' does not support progressive interlacing Ah progress. Try: v4l2-ctl -d0 --set-fmt-video=field=interlaced Unless you specify interlaced at the video interface, the driver will just combine alternate fields into seq-bt. I guess the v4l2src plugin doesn't support seq-bt. Steve Additional debug info: ../../../gst-plugins-good-1.14.4/sys/v4l2/gstv4l2object.c(3813): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device wants interleaved interlacing Execution ended after 0:00:00.020459489 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...
Re: ADV7180 Capture with i.MX53
Hi Steve, On Mon, Oct 7, 2019 at 9:37 PM Steve Longerbeam wrote: > The adv7180 driver in 5.3.x doesn't support seq-bt, only alternate. So > pad format should be "[fmt:UYVY2X8/720x240 field:alternate]". Thanks for the suggestion. After changing to field:alternate I get: # gst-launch-1.0 v4l2src ! fakesink Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' does not support progressive interlacing Additional debug info: ../../../gst-plugins-good-1.14.4/sys/v4l2/gstv4l2object.c(3813): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device wants interleaved interlacing Execution ended after 0:00:00.020459489 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...
Re: ADV7180 Capture with i.MX53
Hi Fabio, On 10/7/19 5:15 PM, Fabio Estevam wrote: Hi Steve, Are you still able to capture from the camera on the imx53-smd board with kernel 5.3.x? I haven't tried the SMD board in a while, it's possible something broke, but see below... I have a custom board with a ADV7180 and it gets detected like this: [2.970246] ipu1_csi0: Registered ipu1_csi0 capture as /dev/video0 [2.979741] ipu1_ic_prpenc: Registered ipu1_ic_prpenc capture as /dev/video1 [2.988930] ipu1_ic_prpvf: Registered ipu1_ic_prpvf capture as /dev/video2 [2.996264] imx-media: ipu1_csi0:1 -> ipu1_ic_prp:0 [3.001685] mmc0: host does not support reading read-only switch, assuming write-enable [3.009925] imx-media: ipu1_csi0:1 -> ipu1_vdic:0 [3.014659] imx-media: ipu1_vdic:2 -> ipu1_ic_prp:0 [3.019929] imx-media: ipu1_ic_prp:1 -> ipu1_ic_prpenc:0 [3.025305] imx-media: ipu1_ic_prp:2 -> ipu1_ic_prpvf:0 [3.032039] mmc0: new high speed SDHC card at address [3.038252] imx-media: subdev ipu1_csi0 bound ... [ 24.974982] adv7180 1-0021: chip found @ 0x21 (63fc4000.i2c) [ 25.324516] imx-media: adv7180 1-0021:0 -> ipu1_csi0:0 Then I setup the pipelines: # media-ctl -l "'adv7180 1-0021':0 -> 'ipu1_csi0':[1]" # media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" # media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x480 field:seq-bt]" The adv7180 driver in 5.3.x doesn't support seq-bt, only alternate. So pad format should be "[fmt:UYVY2X8/720x240 field:alternate]". # media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/720x480]" # gst-launch-1.0 v4l2src ! fakesink Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock [ 929.317827] ipu1_csi0: pipeline start failed with -32 This probably means there was a pad format mismatch. Try enabling dynamic debug. Steve ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory. Additional debug info: ../../../gst-plugins-good-1.14.4/sys/v4l2/gstv4l2src.c(658): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Buffer pool activation failed Execution ended after 0:00:00.035375819 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... My device tree changes to add the ADV7180 are listed here: http://code.bulix.org/ez8yax-901750 Am I calling the correct media-ctl commands? Any ideas, please? Thanks, Fabio Estevam
