Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset
Hi Rajmohan, On Tuesday, 4 December 2018 18:07:16 EET Mani, Rajmohan wrote: > >> On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > >>> On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > >>>> On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: > >> > >> [snip] > >> > >>>>> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to > >>>>> be enabled prior to the test. > >>>>> 2. Stream tests are not performed since it requires > >>>>> pre-configuration for each case. > >>>> > >>>> And that's a bit of an issue. I've tested the driver with a small > >>>> script based on media-ctl to configure links and yavta to > >>>> interface with the video nodes, and got the following oops: [snip] > >>>> The script can be found at > >>>> https://lists.libcamera.org/pipermail/libcamera-devel/2018-Novembe > >>>> r/40.html. > >>>> > >>>> I may be doing something wrong (and I probably am), but in any > >>>> case, the driver shouldn't crash. Could you please have a look ? > >>> > >>> It looks like the driver doesn't have the default state initialized > >>> correctly somewhere and it ends up using 0 as the divisor in some > >>> calculation? Something to fix indeed. > >> > >> That's probably the case. I'll trust Intel to fix that in v8 :-) > > > > Ack. > > Thanks for catching this. > I was able to reproduce this error and I see that error handling > is missing, leading to the panic. > > https://git.linuxtv.org/sailus/media_tree.git/tree/drivers/media > /pci/intel/ipu3/ipu3-css-params.c?h=ipu3-v7= > 19cee7329ca2d0156043cac6afcde535e93310af#n433 > > is where the -EINVAL is returned. > > Setting the return type as int for the following function and all > its callers to use the return value properly to error out, makes > the panic go away. I assume that I will still not be able to process frames through the pipe then, as I'll get an error :-) Could you tell me why the configuration is incorrect and how I can fix it ? > ipu3_css_osys_calc_frame_and_stripe_params() > > Will include the fix in v8. Thank you. > Thanks for catching this. You're welcome. -- Regards, Laurent Pinchart
[GIT PULL FOR v4.21] uvcvideo updates, part 2
Hi Mauro,
The following changes since commit 9b90dc85c718443a3e573a0ccf55900ff4fa73ae:
media: seco-cec: add missing header file to fix build (2018-12-03 15:11:00
-0500)
are available in the Git repository at:
git://linuxtv.org/pinchartl/media.git uvc/async
for you to fetch changes up to e1b1c84504a76ea48ac0f459c5efb64935dc3dde:
media: uvcvideo: Utilise for_each_uvc_urb iterator (2018-12-04 15:11:15
+0200)
Kieran Bingham (10):
media: uvcvideo: Refactor URB descriptors
media: uvcvideo: Convert decode functions to use new context structure
media: uvcvideo: Protect queue internals with helper
media: uvcvideo: queue: Simplify spin-lock usage
media: uvcvideo: queue: Support asynchronous buffer handling
media: uvcvideo: Abstract streaming object lifetime
media: uvcvideo: Move decode processing to process context
media: uvcvideo: Split uvc_video_enable into two
media: uvcvideo: Rename uvc_{un,}init_video()
media: uvcvideo: Utilise for_each_uvc_urb iterator
drivers/media/usb/uvc/uvc_driver.c | 65 ---
drivers/media/usb/uvc/uvc_isight.c | 6 +-
drivers/media/usb/uvc/uvc_queue.c | 110 ++
drivers/media/usb/uvc/uvc_video.c | 274 ++-
drivers/media/usb/uvc/uvcvideo.h | 65 +--
5 files changed, 369 insertions(+), 151 deletions(-)
--
Regards,
Laurent Pinchart
Re: [PATCH v6 09/10] media: uvcvideo: Rename uvc_{un,}init_video()
Hi Kieran,
Thank you for the patch.
On Friday, 9 November 2018 19:15:42 EET Kieran Bingham wrote:
> Hi Laurent,
>
> I'm sorry - I didn't update the commit message on this one.
>
> On 09/11/2018 17:05, Kieran Bingham wrote:
> > We have both uvc_init_video() and uvc_video_init() calls which can be
> > quite confusing to determine the process for each. Now that video
> > uvc_video_enable() has been renamed to uvc_video_start_streaming(),
> > adapt these calls to suit the new flow.
> >
> > Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
> > uvc_video_stop().
>
> Could you s/uvc_video_start/uvc_video_start_transfer/ and
> s/uvc_video_stop/uvc_video_stop_transfer/ when applying please?
>
> (Assuming you get to apply and don't find something else :D)
Will do.
Reviewed-by: Laurent Pinchart
> > Signed-off-by: Kieran Bingham
> >
> > ---
> >
> > v6:
> > - Append _transfer to {_stop,_start}
> >
> > drivers/media/usb/uvc/uvc_video.c | 20 +++-
> > 1 file changed, 11 insertions(+), 9 deletions(-)
> >
> > diff --git a/drivers/media/usb/uvc/uvc_video.c
> > b/drivers/media/usb/uvc/uvc_video.c index cd67506dc696..a81012c65280
> > 100644
> > --- a/drivers/media/usb/uvc/uvc_video.c
> > +++ b/drivers/media/usb/uvc/uvc_video.c
> > @@ -1641,7 +1641,8 @@ static int uvc_alloc_urb_buffers(struct
> > uvc_streaming *stream,
> > /*
> > * Uninitialize isochronous/bulk URBs and free transfer buffers.
> > */
> > -static void uvc_uninit_video(struct uvc_streaming *stream, int
> > free_buffers)
> > +static void uvc_video_stop_transfer(struct uvc_streaming *stream,
> > + int free_buffers)
> > {
> > struct uvc_urb *uvc_urb;
> >
> > @@ -1718,7 +1719,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
> > *stream,
> >
> > urb = usb_alloc_urb(npackets, gfp_flags);
> > if (urb == NULL) {
> > - uvc_uninit_video(stream, 1);
> > + uvc_video_stop_transfer(stream, 1);
> > return -ENOMEM;
> > }
> >
> > @@ -1786,7 +1787,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
> > *stream,
> >
> > urb = usb_alloc_urb(0, gfp_flags);
> > if (urb == NULL) {
> > - uvc_uninit_video(stream, 1);
> > + uvc_video_stop_transfer(stream, 1);
> > return -ENOMEM;
> > }
> >
> > @@ -1806,7 +1807,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
> > *stream,
> > /*
> > * Initialize isochronous/bulk URBs and allocate transfer buffers.
> > */
> > -static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
> > +static int uvc_video_start_transfer(struct uvc_streaming *stream,
> > + gfp_t gfp_flags)
> {
> > struct usb_interface *intf = stream->intf;
> > struct usb_host_endpoint *ep;
> > @@ -1894,7 +1896,7 @@ static int uvc_init_video(struct uvc_streaming
> > *stream, gfp_t gfp_flags)
> >
> > if (ret < 0) {
> > uvc_printk(KERN_ERR, "Failed to submit URB %u "
> > "(%d).\n", i, ret);
> > - uvc_uninit_video(stream, 1);
> > + uvc_video_stop_transfer(stream, 1);
> > return ret;
> }
> > }
> > @@ -1925,7 +1927,7 @@ int uvc_video_suspend(struct uvc_streaming *stream)
> > return 0;
> >
> > stream->frozen = 1;
> > - uvc_uninit_video(stream, 0);
> > + uvc_video_stop_transfer(stream, 0);
> > usb_set_interface(stream->dev->udev, stream->intfnum, 0);
> > return 0;
> > }
> > @@ -1961,7 +1963,7 @@ int uvc_video_resume(struct uvc_streaming *stream,
> > int reset)
> >
> > if (ret < 0)
> > return ret;
> >
> > - return uvc_init_video(stream, GFP_NOIO);
> > + return uvc_video_start_transfer(stream, GFP_NOIO);
> > }
> >
> > /* --
> > @@ -2089,7 +2091,7 @@ int uvc_video_start_streaming(struct uvc_streaming
> > *stream)
> >
> > if (ret < 0)
> > goto error_commit;
> >
> > - ret = uvc_init_video(stream, GFP_KERNEL);
> > + ret = uvc_video_start_transfer(stream, GFP_KERNEL);
> > if (ret < 0)
> > goto error_video;
> >
> > @@ -2105,7 +2107,7 @@ int uvc_video_start_streaming(struct uvc_streaming
> > *stream)
> >
> > int uvc_video_stop_streaming(struct uvc_streaming *stream)
> > {
> > - uvc_uninit_video(stream, 1);
> > + uvc_video_stop_transfer(stream, 1);
> >
> > if (stream->intf->num_altsetting > 1) {
> > usb_set_interface(stream->dev->udev,
> > stream->intfnum, 0);
--
Regards,
Laurent Pinchart
Re: [PATCH v6 08/10] media: uvcvideo: Split uvc_video_enable into two
->num_altsetting > 1) {
> > + usb_set_interface(stream->dev->udev,
> > + stream->intfnum, 0);
> > + } else {
> > + /* UVC doesn't specify how to inform a bulk-based device
> > +* when the video stream is stopped. Windows sends a
> > +* CLEAR_FEATURE(HALT) request to the video streaming
> > +* bulk endpoint, mimic the same behaviour.
> > +*/
> > + unsigned int epnum = stream->header.bEndpointAddress
> > + & USB_ENDPOINT_NUMBER_MASK;
> > + unsigned int dir = stream->header.bEndpointAddress
> > +& USB_ENDPOINT_DIR_MASK;
> > + unsigned int pipe;
> > +
> > + pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
> > + usb_clear_halt(stream->dev->udev, pipe);
> > + }
> > +
> > + uvc_video_clock_cleanup(stream);
> > + return 0;
> > +}
> > diff --git a/drivers/media/usb/uvc/uvcvideo.h
> > b/drivers/media/usb/uvc/uvcvideo.h index 94accfa3c009..b1b895c67122
> > 100644
> > --- a/drivers/media/usb/uvc/uvcvideo.h
> > +++ b/drivers/media/usb/uvc/uvcvideo.h
> > @@ -786,7 +786,8 @@ void uvc_mc_cleanup_entity(struct uvc_entity *entity);
> >
> > int uvc_video_init(struct uvc_streaming *stream);
> > int uvc_video_suspend(struct uvc_streaming *stream);
> > int uvc_video_resume(struct uvc_streaming *stream, int reset);
> >
> > -int uvc_video_enable(struct uvc_streaming *stream, int enable);
> > +int uvc_video_start_streaming(struct uvc_streaming *stream);
> > +int uvc_video_stop_streaming(struct uvc_streaming *stream);
> >
> > int uvc_probe_video(struct uvc_streaming *stream,
> >
> > struct uvc_streaming_control *probe);
> >
> > int uvc_query_ctrl(struct uvc_device *dev, u8 query, u8 unit,
--
Regards,
Laurent Pinchart
Re: [PATCH v6 07/10] media: uvcvideo: Move decode processing to process context
Hi Kieran,
Thank you for the patch.
On Friday, 9 November 2018 19:05:30 EET Kieran Bingham wrote:
> Newer high definition cameras, and cameras with multiple lenses such as
> the range of stereo-vision cameras now available have ever increasing
> data rates.
>
> The inclusion of a variable length packet header in URB packets mean
> that we must memcpy the frame data out to our destination 'manually'.
> This can result in data rates of up to 2 gigabits per second being
> processed.
>
> To improve efficiency, and maximise throughput, handle the URB decode
> processing through a work queue to move it from interrupt context, and
> allow multiple processors to work on URBs in parallel.
>
> Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
>
> ---
> v2:
> - Lock full critical section of usb_submit_urb()
>
> v3:
> - Fix race on submitting uvc_video_decode_data_work() to work queue.
> - Rename uvc_decode_op -> uvc_copy_op (Generic to encode/decode)
> - Rename decodes -> copy_operations
> - Don't queue work if there is no async task
> - obtain copy op structure directly in uvc_video_decode_data()
> - uvc_video_decode_data_work() -> uvc_video_copy_data_work()
>
> v4:
> - Provide for_each_uvc_urb()
> - Simplify fix for shutdown race to flush queue before freeing URBs
> - Rebase to v4.16-rc4 (linux-media/master) adjusting for metadata
>conflicts.
>
> v5:
> - Rebase to media/v4.20-2
> - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
> - Fix function documentation for uvc_video_copy_data_work()
> - Add periods to the end of sentences
> - Rename 'decode' variable to 'op' in uvc_video_decode_data()
> - Move uvc_urb->async_operations initialisation to before use
> - Move async workqueue to match uvc_streaming lifetime instead of
>streamon/streamoff
>
> v6:
> - Utilise the new streaming object lifetime functions to perform
>allocation and destruction of the async workqueue.
>
> drivers/media/usb/uvc/uvc_driver.c | 11 +++-
> drivers/media/usb/uvc/uvc_video.c | 104 +++---
> drivers/media/usb/uvc/uvcvideo.h | 28 -
> 3 files changed, 119 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index afb44d1c9d04..b62cbd800111
> 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -401,6 +401,9 @@ static struct uvc_streaming *uvc_stream_by_id(struct
> uvc_device *dev, int id)
>
> static void uvc_stream_delete(struct uvc_streaming *stream)
> {
> + if (stream->async_wq)
> + destroy_workqueue(stream->async_wq);
> +
> mutex_destroy(>mutex);
>
> usb_put_intf(stream->intf);
> @@ -425,6 +428,14 @@ static struct uvc_streaming *uvc_stream_new(struct
> uvc_device *dev, stream->intf = usb_get_intf(intf);
> stream->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
>
> + /* Allocate a stream specific work queue for asynchronous tasks. */
> + stream->async_wq = alloc_workqueue("uvcvideo", WQ_UNBOUND | WQ_HIGHPRI,
> +0);
> + if (!stream->async_wq) {
> + uvc_stream_delete(stream);
> + return NULL;
> + }
> +
> return stream;
> }
>
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 7a7779e1b466..e19bdf089cc4 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1094,21 +1094,54 @@ static int uvc_video_decode_start(struct
> uvc_streaming *stream, return data[0];
> }
>
> -static void uvc_video_decode_data(struct uvc_streaming *stream,
> +/*
> + * uvc_video_decode_data_work: Asynchronous memcpy processing
> + *
> + * Copy URB data to video buffers in process context, releasing buffer
> + * references and requeuing the URB when done.
> + */
> +static void uvc_video_copy_data_work(struct work_struct *work)
> +{
> + struct uvc_urb *uvc_urb = container_of(work, struct uvc_urb, work);
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; i < uvc_urb->async_operations; i++) {
> + struct uvc_copy_op *op = _urb->copy_operations[i];
> +
> + memcpy(op->dst, op->src, op->len);
> +
> + /* Release reference taken on this buffer. */
> + uvc_queue_buffer_release(op->buf);
> + }
> +
> + ret = usb_submit_urb(uvc_urb->urb, GFP_KERNEL);
> + if (ret < 0)
> + uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n&quo
Re: [PATCH v6 06/10] media: uvcvideo: Abstract streaming object lifetime
Hi Kieran,
Thank you for the patch.
On Friday, 9 November 2018 19:05:29 EET Kieran Bingham wrote:
> The streaming object is a key part of handling the UVC device. Although
> not critical, we are currently missing a call to destroy the mutex on
> clean up paths, and we are due to extend the objects complexity in the
> near future.
>
> Facilitate easy management of a stream object by creating a pair of
> functions to handle creating and destroying the allocation. The new
> uvc_stream_delete() function also performs the missing mutex_destroy()
> operation.
>
> Previously a failed streaming object allocation would cause
> uvc_parse_streaming() to return -EINVAL, which is inappropriate. If the
> constructor failes, we will instead return -ENOMEM.
>
> While we're here, fix the trivial spelling error in the function banner
> of uvc_delete().
>
> Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
> ---
> drivers/media/usb/uvc/uvc_driver.c | 54 +--
> 1 file changed, 38 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index 67bd58c6f397..afb44d1c9d04
> 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -396,6 +396,39 @@ static struct uvc_streaming *uvc_stream_by_id(struct
> uvc_device *dev, int id) }
>
> /*
> + * Streaming Object Management
> + */
> +
> +static void uvc_stream_delete(struct uvc_streaming *stream)
> +{
> + mutex_destroy(>mutex);
> +
> + usb_put_intf(stream->intf);
> +
> + kfree(stream->format);
> + kfree(stream->header.bmaControls);
> + kfree(stream);
> +}
> +
> +static struct uvc_streaming *uvc_stream_new(struct uvc_device *dev,
> + struct usb_interface *intf)
> +{
> + struct uvc_streaming *stream;
> +
> + stream = kzalloc(sizeof(*stream), GFP_KERNEL);
> + if (stream == NULL)
> + return NULL;
> +
> + mutex_init(>mutex);
> +
> + stream->dev = dev;
> + stream->intf = usb_get_intf(intf);
> + stream->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
> +
> + return stream;
> +}
> +
> +/*
> * Descriptors parsing
> */
>
> @@ -687,17 +720,12 @@ static int uvc_parse_streaming(struct uvc_device *dev,
> return -EINVAL;
> }
>
> - streaming = kzalloc(sizeof(*streaming), GFP_KERNEL);
> + streaming = uvc_stream_new(dev, intf);
> if (streaming == NULL) {
> usb_driver_release_interface(_driver.driver, intf);
> - return -EINVAL;
> + return -ENOMEM;
> }
>
> - mutex_init(>mutex);
> - streaming->dev = dev;
> - streaming->intf = usb_get_intf(intf);
> - streaming->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
> -
> /* The Pico iMage webcam has its class-specific interface descriptors
>* after the endpoint descriptors.
>*/
> @@ -904,10 +932,7 @@ static int uvc_parse_streaming(struct uvc_device *dev,
>
> error:
> usb_driver_release_interface(_driver.driver, intf);
> - usb_put_intf(intf);
> - kfree(streaming->format);
> - kfree(streaming->header.bmaControls);
> - kfree(streaming);
> + uvc_stream_delete(streaming);
> return ret;
> }
>
> @@ -1815,7 +1840,7 @@ static int uvc_scan_device(struct uvc_device *dev)
> * is released.
> *
> * As this function is called after or during disconnect(), all URBs have
> - * already been canceled by the USB core. There is no need to kill the
> + * already been cancelled by the USB core. There is no need to kill the
> * interrupt URB manually.
> */
> static void uvc_delete(struct kref *kref)
> @@ -1853,10 +1878,7 @@ static void uvc_delete(struct kref *kref)
> streaming = list_entry(p, struct uvc_streaming, list);
> usb_driver_release_interface(_driver.driver,
> streaming->intf);
> - usb_put_intf(streaming->intf);
> - kfree(streaming->format);
> - kfree(streaming->header.bmaControls);
> - kfree(streaming);
> + uvc_stream_delete(streaming);
> }
>
> kfree(dev);
--
Regards,
Laurent Pinchart
Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset
Hi Sakari, On Monday, 3 December 2018 11:51:01 EET Sakari Ailus wrote: > On Fri, Nov 30, 2018 at 01:07:53AM +0200, Laurent Pinchart wrote: > > On Wednesday, 7 November 2018 06:16:47 EET Bing Bu Cao wrote: > >> On 11/01/2018 08:03 PM, Sakari Ailus wrote: > >>> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > > > > [snip] > > > >>>> ImgU media topology print: > >>>> > >>>> # media-ctl -d /dev/media0 -p > >>>> Media controller API version 4.19.0 > >>>> > >>>> Media device information > >>>> > >>>> driver ipu3-imgu > >>>> model ipu3-imgu > >>>> serial > >>>> bus infoPCI::00:05.0 > >>>> hw revision 0x80862015 > >>>> driver version 4.19.0 > >>>> > >>>> Device topology > >>>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >>>> type V4L2 subdev subtype Unknown flags 0 > >>>> device node name /dev/v4l-subdev0 > >>>> pad0: Sink > >>>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >>> > >>> This doesn't seem right. Which formats can be enumerated from the pad? > >>> > >>>> crop:(0,0)/1920x1080 > >>>> compose:(0,0)/1920x1080] > >>> > >>> Does the compose rectangle affect the scaling on all outputs? > >> > >> Sakari, driver use crop and compose targets to help set input-feeder and > >> BDS output resolutions which are 2 key block of whole imaging pipeline, > >> not the actual ending output, but they will impact the final output. > >> > >>>> <- "ipu3-imgu 0 input":0 [] > >>> > >>> Are there links that have no useful link configuration? If so, you > >>> should set them enabled and immutable in the driver. > >> > >> The enabled status of input pads is used to get which pipe that user is > >> trying to enable (ipu3_link_setup()), so it could not been set as > >> immutable. > > > > Each pipe needs an input in order to operate, so from that point of view > > the input is mandatory. Why can't we make this link immutable, and use > > the stream state (VIDIOC_STREAMON/VIDIOC_STREAMOFF) to enable/disable the > > pipes ? > > There are only two options (AFAIK) in choosing the firmware, and by > configuring the links this is better visible to the user: the links the > state of which can be changed are not immutable. The driver can also obtain > the explicit pipeline configuration, which makes the implementation more > simple. Do you mean that different firmwares are loaded based on link configuration ? Does it also mean that once we start using the first pipeline the configuration of the second pipeline can't be changed anymore ? If so, what's the reason for such a limitation ? -- Regards, Laurent Pinchart
[GIT PULL FOR v4.21] uvcvideo updates
Hi Mauro, The following changes since commit 708d75fe1c7c6e9abc5381b6fcc32b49830383d0: media: dvb-pll: don't re-validate tuner frequencies (2018-11-23 12:27:18 -0500) are available in the Git repository at: git://linuxtv.org/pinchartl/media.git uvc/next for you to fetch changes up to dd3bd1ac29aa9a0f83a55f100e69091c0567a27c: media: uvcvideo: Refactor teardown of uvc on USB disconnect (2018-11-30 04:12:08 +0200) Daniel Axtens (1): media: uvcvideo: Refactor teardown of uvc on USB disconnect Sergey Dorodnicov (2): media: v4l: Add 4bpp packed depth confidence format CNF4 media: uvcvideo: Add support for the CNF4 format Documentation/media/uapi/v4l/depth-formats.rst | 1 + Documentation/media/uapi/v4l/pixfmt-cnf4.rst | 31 + drivers/media/usb/uvc/uvc_driver.c | 18 ++ drivers/media/usb/uvc/uvc_status.c | 12 drivers/media/usb/uvc/uvcvideo.h | 4 drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 7 files changed, 60 insertions(+), 8 deletions(-) create mode 100644 Documentation/media/uapi/v4l/pixfmt-cnf4.rst -- Regards, Laurent Pinchart
[GIT PULL FOR v4.21] Xilinx media drivers updates
Hi Mauro, The following changes since commit 708d75fe1c7c6e9abc5381b6fcc32b49830383d0: media: dvb-pll: don't re-validate tuner frequencies (2018-11-23 12:27:18 -0500) are available in the Git repository at: git://linuxtv.org/pinchartl/media.git xilinx/next for you to fetch changes up to 672d25bacfce9b50168b0ab7db872d66ae3aa4a6: media: xilinx: fix typo in formats table (2018-11-30 03:37:24 +0200) Andrea Merello (1): media: xilinx: fix typo in formats table Dhaval Shah (1): media: xilinx: Use SPDX-License-Identifier drivers/media/platform/xilinx/Kconfig | 2 ++ drivers/media/platform/xilinx/Makefile | 2 ++ drivers/media/platform/xilinx/xilinx-dma.c | 5 + drivers/media/platform/xilinx/xilinx-dma.h | 5 + drivers/media/platform/xilinx/xilinx-tpg.c | 5 + drivers/media/platform/xilinx/xilinx-vip.c | 7 ++- drivers/media/platform/xilinx/xilinx-vip.h | 5 + drivers/media/platform/xilinx/xilinx-vipp.c | 5 + drivers/media/platform/xilinx/xilinx-vipp.h | 5 + drivers/media/platform/xilinx/xilinx-vtc.c | 5 + drivers/media/platform/xilinx/xilinx-vtc.h | 5 + include/dt-bindings/media/xilinx-vip.h | 5 + 12 files changed, 15 insertions(+), 41 deletions(-) -- Regards, Laurent Pinchart
Re: [yavta PATCH 1/1] Zero dev in main()
Hi Sakari,
Thank you for the patch.
On Thursday, 22 November 2018 17:29:56 EET Sakari Ailus wrote:
> From: Sakari Ailus
>
> This is necessary since video_open() may not be always called soon
Do you mean video_init() ? Isn't it called at the very first line of main() ?
> Signed-off-by: Sakari Ailus
> ---
> yavta.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/yavta.c b/yavta.c
> index c7986bd..de5376d 100644
> --- a/yavta.c
> +++ b/yavta.c
> @@ -342,7 +342,6 @@ static bool video_has_valid_buf_type(struct device *dev)
>
> static void video_init(struct device *dev)
> {
> - memset(dev, 0, sizeof *dev);
> dev->fd = -1;
> dev->memtype = V4L2_MEMORY_MMAP;
> dev->buffers = NULL;
> @@ -1903,7 +1902,7 @@ static struct option opts[] = {
> int main(int argc, char *argv[])
> {
> struct sched_param sched;
> - struct device dev;
> + struct device dev = { 0 };
> int ret;
>
> /* Options parsings */
--
Regards,
Laurent Pinchart
Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset
Hi Sakari, On Friday, 9 November 2018 12:09:54 EET Sakari Ailus wrote: > On Wed, Nov 07, 2018 at 12:16:47PM +0800, Bing Bu Cao wrote: > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > >> On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: [snip] > >>> ImgU media topology print: > >>> > >>> # media-ctl -d /dev/media0 -p > >>> Media controller API version 4.19.0 > >>> > >>> Media device information > >>> > >>> driver ipu3-imgu > >>> model ipu3-imgu > >>> serial > >>> bus infoPCI::00:05.0 > >>> hw revision 0x80862015 > >>> driver version 4.19.0 > >>> > >>> Device topology > >>> - entity 1: ipu3-imgu 0 (5 pads, 5 links) > >>> type V4L2 subdev subtype Unknown flags 0 > >>> device node name /dev/v4l-subdev0 > >>> pad0: Sink > >>> [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown > >> > >> This doesn't seem right. Which formats can be enumerated from the pad? > > Looking at the code, the OUTPUT video nodes have 10-bit GRBG (or a variant) > format whereas the CAPTURE video nodes always have NV12. Can you confirm? > > If the OUTPUT video node format selection has no effect on the rest of the > pipeline (device capabilities, which processing blocks are in use, CAPTURE > video nodes formats etc.), I think you could simply use the FIXED media bus > code for each pad. That would actually make sense: this device always works > from memory to memory, and thus does not really have a pixel data bus > external to the device which is what the media bus codes really are for. Isn't the Bayer variant useful information to configure debayering ? I would expect it to be passed through the format on pad 0. -- Regards, Laurent Pinchart
Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset
have been already > > created? That'd make reading the output easier. > > > >> - entity 17: ipu3-imgu 0 input (1 pad, 1 link) > >> > >> type Node subtype V4L flags 0 > >> device node name /dev/video0 > >> > >>pad0: Source > >>-> "ipu3-imgu 0":0 [] > >> > >> - entity 23: ipu3-imgu 0 parameters (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video1 > >> > >>pad0: Source > >>-> "ipu3-imgu 0":1 [] > >> > >> - entity 29: ipu3-imgu 0 output (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video2 > >> > >>pad0: Sink > >><- "ipu3-imgu 0":2 [] > >> > >> - entity 35: ipu3-imgu 0 viewfinder (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video3 > >> > >>pad0: Sink > >><- "ipu3-imgu 0":3 [] > >> > >> - entity 41: ipu3-imgu 0 3a stat (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video4 > >> > >>pad0: Sink > >><- "ipu3-imgu 0":4 [] > >> > >> - entity 47: ipu3-imgu 1 input (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video5 > >> > >>pad0: Source > >>-> "ipu3-imgu 1":0 [] > >> > >> - entity 53: ipu3-imgu 1 parameters (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video6 > >> > >>pad0: Source > >>-> "ipu3-imgu 1":1 [] > >> > >> - entity 59: ipu3-imgu 1 output (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video7 > >> > >>pad0: Sink > >><- "ipu3-imgu 1":2 [] > >> > >> - entity 65: ipu3-imgu 1 viewfinder (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video8 > >> > >>pad0: Sink > >><- "ipu3-imgu 1":3 [] > >> > >> - entity 71: ipu3-imgu 1 3a stat (1 pad, 1 link) > >> type Node subtype V4L flags 0 > >> device node name /dev/video9 > >> > >>pad0: Sink > >><- "ipu3-imgu 1":4 [] [snip] -- Regards, Laurent Pinchart
Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset
Hi Tomasz, On Thursday, 29 November 2018 21:51:32 EET Tomasz Figa wrote: > On Thu, Nov 29, 2018 at 6:43 AM Laurent Pinchart wrote: > > On Tuesday, 30 October 2018 00:22:54 EET Yong Zhi wrote: [snip] > >> 1. Link pad flag of video nodes (i.e. ipu3-imgu 0 output) need to be > >> enabled prior to the test. > >> 2. Stream tests are not performed since it requires pre-configuration > >> for each case. > > > > And that's a bit of an issue. I've tested the driver with a small script > > based on media-ctl to configure links and yavta to interface with the > > video nodes, and got the following oops: > > > > [ 136.927788] divide error: [#1] PREEMPT SMP PTI > > [ 136.927801] CPU: 2 PID: 2069 Comm: yavta Not tainted 4.20.0-rc1+ #9 > > [ 136.927806] Hardware name: HP Soraka/Soraka, BIOS 08/30/2018 > > [ 136.927820] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] > > [ 136.927825] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 > > fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 > > fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 > > [ 136.927830] RSP: 0018:9af2c0b837c8 EFLAGS: 00010202 > > [ 136.927835] RAX: RBX: RCX: > > 9af2c3e353c0 > > [ 136.927839] RDX: RSI: 9af2c0b838e0 RDI: > > 9af2c3e353c0 > > [ 136.927843] RBP: 0001 R08: R09: > > 9af2c0b83880 > > [ 136.927846] R10: 9af2c3e353c0 R11: 9af2c3e357c0 R12: > > 03a0 > > [ 136.927849] R13: 00025a0a R14: R15: > > > > [ 136.927854] FS: 7f1eca167700() GS:8c19fab0() > > knlGS: > > > > [ 136.927858] CS: 0010 DS: ES: CR0: 80050033 > > [ 136.927862] CR2: 7f1ec776c000 CR3: 0001312a4003 CR4: > > 003606e0 > > [ 136.927865] Call Trace: > > [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a > > [ 136.927892] ? __switch_to_asm+0x40/0x70 > > [ 136.927899] ? alloc_vmap_area+0x78/0x2f6 > > [ 136.927903] ? __switch_to_asm+0x40/0x70 > > [ 136.927907] ? __switch_to_asm+0x34/0x70 > > [ 136.927911] ? __switch_to_asm+0x40/0x70 > > [ 136.927915] ? __switch_to_asm+0x34/0x70 > > [ 136.927923] ? __inc_numa_state+0x28/0x70 > > [ 136.927929] ? preempt_latency_start+0x1e/0x3d > > [ 136.927936] ? get_page_from_freelist+0x821/0xb62 > > [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b > > [ 136.927948] ? kmem_cache_alloc_node_trace+0xf6/0x108 > > [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 > > Is it just me or the backtrace above doesn't seem to make sense? I > don't see any allocations inside ipu3_css_cfg_acc(). I suppose that's why it's prefixed with '?' :-) > > [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] > > [ 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] > > [ 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] > > [ 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] > > [ 136.928022] ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] > > [ 136.928034] vb2_start_streaming+0x6c/0xf2 [videobuf2_common] > > [ 136.928044] vb2_core_streamon+0xcf/0x109 [videobuf2_common] > > [ 136.928061] __video_do_ioctl+0x239/0x388 [videodev] > > [ 136.928081] video_usercopy+0x25d/0x47a [videodev] > > [ 136.928097] ? copy_overflow+0x14/0x14 [videodev] > > [ 136.928115] v4l2_ioctl+0x4d/0x58 [videodev] > > [ 136.928123] vfs_ioctl+0x1b/0x28 > > [ 136.928130] do_vfs_ioctl+0x4de/0x566 > > [ 136.928139] ksys_ioctl+0x50/0x70 > > [ 136.928146] __x64_sys_ioctl+0x16/0x19 > > [ 136.928152] do_syscall_64+0x4d/0x5a > > [ 136.928158] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 136.928164] RIP: 0033:0x7f1ec9a84f47 > > [ 136.928169] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 > > c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 > > <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 > > [ 136.928173] RSP: 002b:7ffe279e6188 EFLAGS: 0246 ORIG_RAX: > > 0010 > > [ 136.928178] RAX: ffda RBX: 0007 RCX: > > 7f1ec9a84f47 > > [ 136.928181] RDX: 7ffe279e6194 RSI: 40045612 RDI: > > 0003 > > [ 136.928184] RBP: R08: 7f1ec776d000 R09: > > > > [ 136.928188] R10: 0020 R11: 0246 R12: > > 7ffe279e6360 > > [ 136.928191] R13: 0004 R1
Re: [PATCH v7 02/16] doc-rst: Add Intel IPU3 documentation
1 for still mode) through the > +control id 0x009819a1 as below. > + > +e.g > + > +v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1 You can use yavta instead of v4l2n. Another option would be v4l2-ctl, but I'd avoid adding a dependency to another tool if yavta is already used in other places. > +RAW bayer frames go through the following ISP pipeline HW blocks to > +have the processed image output to the DDR memory. > + > +RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> Geometric > +Distortion Correction (GDC) -> DDR A more detailed block diagram, with the other blocks included, should be added to the ImgU description above. Each block should have a short description of its purpose. > +The ImgU V4L2 subdev has to be configured with the supported resolutions > +in all the above HW blocks, for a given input resolution. > + > +For a given supported resolution for an input frame, the Input Feeder, > +Bayer Down Scaling and GDC blocks should be configured with the supported > +resolutions. This information can be obtained by looking at the following > +IPU3 ISP configuration table. Does this mean that the ImgU will not operate properly when exercised through the MC and V4L2 only without configuration of the internal blocks through the processing parameters device node ? > +https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/mast > er > + > +Under baseboard-poppy/media-libs/arc-camera3-hal-configs-poppy/files/gcss > +directory, graph_settings_ov5670.xml can be used as an example. The directory name is incorrect. > +The following steps prepare the ImgU ISP pipeline for the image processing. > + > +1. The ImgU V4L2 subdev data format should be set by using the > +VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained > above. > + > +2. The ImgU V4L2 subdev cropping should be set by using the > +VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_CROP as the target, > +using the input feeder height and width. > + > +3. The ImgU V4L2 subdev composing should be set by using the > +VIDIOC_SUBDEV_S_SELECTION on pad 0, with V4L2_SEL_TGT_COMPOSE as the > target, +using the BDS height and width. How the format and selection rectangles related to the internal processing blocks should also be explained in more details. > +For the ov5670 example, for an input frame with a resolution of 2592x1944 > +(which is input to the ImgU subdev pad 0), the corresponding resolutions > +for input feeder, BDS and GDC are 2592x1944, 2592x1944 and 2560x1920 > +respectively. Why ? How is that computed ? If my input resolution was different, how would I compute the other resolutions ? > +Once this is done, the received raw bayer frames can be input to the ImgU > +V4L2 subdev as below, using the open source application v4l2n. > + > +For an image captured with 2592x1944 [#f4]_ resolution, with desired output > +resolution as 2560x1920 and viewfinder resolution as 2560x1920, the > following +v4l2n command can be used. This helps process the raw bayer > frames and +produces the desired results for the main output image and the > viewfinder +output, in NV12 format. > + > +v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4 > +--fmt=type:VIDEO_OUTPUT_MPLANE,width=2592,height=1944,pixelformat=0X4733706 > 9 +--reqbufs=type:VIDEO_OUTPUT_MPLANE,count:1 --pipe=1 > --output=/tmp/frames.out +--open=/dev/video5 > +--fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12 > +--reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=2 > --output=/tmp/frames.vf +--open=/dev/video6 > +--fmt=type:VIDEO_CAPTURE_MPLANE,width=2560,height=1920,pixelformat=NV12 > +--reqbufs=type:VIDEO_CAPTURE_MPLANE,count:1 --pipe=3 --open=/dev/video7 > +--output=/tmp/frames.3A --fmt=type:META_CAPTURE,? > +--reqbufs=count:1,type:META_CAPTURE --pipe=1,2,3,4 --stream=5 You can replace this with four yavta commands. > +where /dev/video4, /dev/video5, /dev/video6 and /dev/video7 devices point > to +input, output, viewfinder and 3A statistics video nodes respectively. + > +Converting the raw bayer image into YUV domain > +-- > + > +The processed images after the above step, can be converted to YUV domain > +as below. > + > +Main output frames > +~~ > + > +raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.out /tmp/frames.out.pnm PNM is an umbrella term that refers to any of the PBM, PGM or PPM format. As the raw2pnm tool outputs PPM files, let's name them .ppm here and below. This helps with some image viewers that use file extensions to identify the format, and have trouble handling .pnm files. > +where 2560x1920 is output resolution, NV12 is the video format, followed > +by input frame and output PNM file. > + > +Viewfinder output frames > + > + > +raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.vf /tmp/frames.vf.pnm > + > +where 2560x1920 is output resolution, NV12 is the video format, followed > +by input frame and output PNM file. > + > +Example user space code for IPU3 > + > + > +User space code that configures and uses IPU3 is available here. > + > +https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/master/ > + > +The source can be located under hal/intel directory. > + > +References > +== > + > +include/uapi/linux/intel-ipu3.h > + > +.. [#f1] https://github.com/intel/nvt > + > +.. [#f2] http://git.ideasonboard.org/yavta.git > + > +.. [#f3] http://git.ideasonboard.org/?p=media-ctl.git;a=summary > + > +.. [#f4] ImgU limitation requires an additional 16x16 for all input > resolutions -- Regards, Laurent Pinchart
Re: [PATCH v7 01/16] v4l: Add Intel IPU3 meta buffer formats
Hello Yong,
Thank you for the patch.
On Tuesday, 30 October 2018 00:22:55 EET Yong Zhi wrote:
> Add IPU3-specific meta formats for parameter
> processing and 3A, DVS statistics:
Unless I'm mistaken DVS support has been removed. You can write this as
Add IPU3-specific meta formats for processing parameters and 3A
statistics.
>
> V4L2_META_FMT_IPU3_PARAMS
> V4L2_META_FMT_IPU3_STAT_3A
>
> Signed-off-by: Yong Zhi
> ---
> drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
> include/uapi/linux/videodev2.h | 4
> 2 files changed, 6 insertions(+)
I would squash this with patch 03/16.
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c index 6489f25..abff64b 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1299,6 +1299,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_META_FMT_VSP1_HGO: descr = "R-Car VSP1 1-D Histogram"; break;
> case V4L2_META_FMT_VSP1_HGT: descr = "R-Car VSP1 2-D Histogram"; break;
> case V4L2_META_FMT_UVC: descr = "UVC payload header metadata";
> break;
> + case V4L2_META_FMT_IPU3_PARAMS: descr = "IPU3 processing parameters";
> break;
> + case V4L2_META_FMT_IPU3_STAT_3A:descr = "IPU3 3A statistics";
> break;
>
> default:
> /* Compressed formats */
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index f0a968a..bdccd7a 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -718,6 +718,10 @@ struct v4l2_pix_format {
> #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC
> Payload Header metadata */ #define V4L2_META_FMT_D4XX
> v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
>
> +/* Vendor specific - used for IPU3 camera sub-system */
> +#define V4L2_META_FMT_IPU3_PARAMSv4l2_fourcc('i', 'p', '3', 'p') /* IPU3
> params */
Maybe "IPU3 processing parameters" in full ?
Apart from that the patch looks good to me.
Reviewed-by: Laurent Pinchart
> +#define V4L2_META_FMT_IPU3_STAT_3A v4l2_fourcc('i', 'p', '3', 's') /* IPU3
> 3A statistics */
> +
> /* priv field value to indicates that subsequent fields are valid. */
> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
Regards,
Laurent Pinchart
Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset
010 DS: ES: CR0: 80050033 [ 136.927862] CR2: 7f1ec776c000 CR3: 0001312a4003 CR4: 003606e0 [ 136.927865] Call Trace: [ 136.927884] ? __accumulate_pelt_segments+0x29/0x3a [ 136.927892] ? __switch_to_asm+0x40/0x70 [ 136.927899] ? alloc_vmap_area+0x78/0x2f6 [ 136.927903] ? __switch_to_asm+0x40/0x70 [ 136.927907] ? __switch_to_asm+0x34/0x70 [ 136.927911] ? __switch_to_asm+0x40/0x70 [ 136.927915] ? __switch_to_asm+0x34/0x70 [ 136.927923] ? __inc_numa_state+0x28/0x70 [ 136.927929] ? preempt_latency_start+0x1e/0x3d [ 136.927936] ? get_page_from_freelist+0x821/0xb62 [ 136.927943] ? slab_pre_alloc_hook+0x12/0x3b [ 136.927948] ? kmem_cache_alloc_node_trace+0xf6/0x108 [ 136.927954] ? alloc_vmap_area+0x78/0x2f6 [ 136.927965] ipu3_css_cfg_acc+0xa0/0x1b5f [ipu3_imgu] [ 136.927981] ipu3_css_set_parameters+0x286/0x6e7 [ipu3_imgu] [ 136.927995] ipu3_css_start_streaming+0x1230/0x130a [ipu3_imgu] [ 136.928010] imgu_s_stream+0x104/0x2f7 [ipu3_imgu] [ 136.928022] ipu3_vb2_start_streaming+0x168/0x1bd [ipu3_imgu] [ 136.928034] vb2_start_streaming+0x6c/0xf2 [videobuf2_common] [ 136.928044] vb2_core_streamon+0xcf/0x109 [videobuf2_common] [ 136.928061] __video_do_ioctl+0x239/0x388 [videodev] [ 136.928081] video_usercopy+0x25d/0x47a [videodev] [ 136.928097] ? copy_overflow+0x14/0x14 [videodev] [ 136.928115] v4l2_ioctl+0x4d/0x58 [videodev] [ 136.928123] vfs_ioctl+0x1b/0x28 [ 136.928130] do_vfs_ioctl+0x4de/0x566 [ 136.928139] ksys_ioctl+0x50/0x70 [ 136.928146] __x64_sys_ioctl+0x16/0x19 [ 136.928152] do_syscall_64+0x4d/0x5a [ 136.928158] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 136.928164] RIP: 0033:0x7f1ec9a84f47 [ 136.928169] Code: 00 00 00 48 8b 05 51 6f 2c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 21 6f 2c 00 f7 d8 64 89 01 48 [ 136.928173] RSP: 002b:7ffe279e6188 EFLAGS: 0246 ORIG_RAX: 0010 [ 136.928178] RAX: ffda RBX: 0007 RCX: 7f1ec9a84f47 [ 136.928181] RDX: 7ffe279e6194 RSI: 40045612 RDI: 0003 [ 136.928184] RBP: R08: 7f1ec776d000 R09: [ 136.928188] R10: 0020 R11: 0246 R12: 7ffe279e6360 [ 136.928191] R13: 0004 R14: 7ffe279e6360 R15: 7ffe279e8826 [ 136.928198] Modules linked in: ccm zram arc4 iwlmvm mac80211 intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp iwlwifi cfg80211 hid_multitouch ipu3_imgu ipu3_cio2 8250_dw videobuf2_dma_sg videobuf2_memops videobuf2_v4l2 processor_thermal_device intel_soc_dts_iosf videobuf2_common ov5670 ov13858 dw9714 v4l2_fwnode v4l2_common videodev media at24 cros_ec_lpcs cros_ec_core int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel chromeos_pstore mac_hid autofs4 usbhid mmc_block hid_generic i915 sdhci_pci video cqhci i2c_algo_bit sdhci drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm drm_panel_orientation_quirks i2c_hid hid [ 136.928273] ---[ end trace 4ec6c2ce09e06d9d ]--- [ 136.928288] RIP: 0010:ipu3_css_osys_calc+0xc54/0xe14 [ipu3_imgu] [ 136.928293] Code: 89 44 24 28 42 8b 44 86 6c f7 54 24 04 81 64 24 28 00 fd ff ff 81 64 24 04 00 03 00 00 8d 44 03 ff 81 44 24 28 80 03 00 00 99 fb 0f af c3 bb 20 00 00 00 99 f7 fb 8b 5c 24 40 83 fd 01 19 d2 [ 136.928297] RSP: 0018:9af2c0b837c8 EFLAGS: 00010202 [ 136.928302] RAX: RBX: RCX: 9af2c3e353c0 [ 136.928307] RDX: RSI: 9af2c0b838e0 RDI: 9af2c3e353c0 [ 136.928311] RBP: 0001 R08: R09: 9af2c0b83880 [ 136.928320] R10: 9af2c3e353c0 R11: 9af2c3e357c0 R12: 03a0 [ 136.928324] R13: 00025a0a R14: R15: [ 136.928330] FS: 7f1eca167700() GS:8c19fab0() knlGS: [ 136.928349] CS: 0010 DS: ES: CR0: 80050033 [ 136.928364] CR2: 7f1ec776c000 CR3: 0001312a4003 CR4: 003606e0 The script can be found at https://lists.libcamera.org/pipermail/libcamera-devel/2018-November/40.html. I may be doing something wrong (and I probably am), but in any case, the driver shouldn't crash. Could you please have a look ? Please note that, as I couldn't get the IMGU to process frames, the part of the script that converts output files to PPM is untested and thus probably incorrect. [snip] -- Regards, Laurent Pinchart
[GIT FIXES FOR v4.20] R-Car VSP1 regression fix
Hi Mauro, The following changes since commit a7c3a0d5f8d8cd5cdb32c06d4d68f5b4e4d2104b: media: mediactl docs: Fix licensing message (2018-11-27 13:52:46 -0500) are available in the Git repository at: git://linuxtv.org/pinchartl/media.git tags/vsp1-fixes-20181128 for you to fetch changes up to cab35a9c77adda8f971a7d1d74b21c0df98ffafe: media: vsp1: Fix LIF buffer thresholds (2018-11-28 20:38:50 +0200) VSP1 regression fixes for v4.20 Laurent Pinchart (1): media: vsp1: Fix LIF buffer thresholds drivers/media/platform/vsp1/vsp1_lif.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- Regards, Laurent Pinchart
Re: [PATCH v5 0/9] Asynchronous UVC
Hi Pavel, On Tuesday, 27 November 2018 22:17:30 EET Pavel Machek wrote: > On Tue 2018-11-06 21:27:11, Kieran Bingham wrote: > > From: Kieran Bingham > > > > The Linux UVC driver has long provided adequate performance capabilities > > for web-cams and low data rate video devices in Linux while resolutions > > were low. > > > > Modern USB cameras are now capable of high data rates thanks to USB3 with > > 1080p, and even 4k capture resolutions supported. > > > > Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO > > (isochronous transfers) can generate more data than an embedded ARM core > > is able to process on a single core, resulting in frame loss. > > > > A large part of this performance impact is from the requirement to > > ‘memcpy’ frames out from URB packets to destination frames. This > > unfortunate requirement is due to the UVC protocol allowing a variable > > length header, and thus it is not possible to provide the target frame > > buffers directly. > > > > Extra throughput is possible by moving the actual memcpy actions to a work > > queue, and moving the memcpy out of interrupt context thus allowing work > > tasks to be scheduled across multiple cores. > > Hmm. Doing memcpy() on many cores is improvement but... not really. > Would it be possible to improve kernel<->user interface, so it says > "data is in this buffer, and it starts here" and so that memcpy in > kernel is not neccessary? Unfortunately not, as the UVC protocol segments the frame in a large number of small packets, each prefixed with a variable-length header. It's a poorly designed protocol from that point of view. -- Regards, Laurent Pinchart
[PATCH 1/2] Update kernel headers to v4.19
From: Jacopo Mondi
Signed-off-by: Jacopo Mondi
Signed-off-by: Laurent Pinchart
---
include/linux/v4l2-common.h | 1 +
include/linux/v4l2-controls.h | 219 ++
include/linux/videodev2.h | 81 -
3 files changed, 221 insertions(+), 80 deletions(-)
diff --git a/include/linux/v4l2-common.h b/include/linux/v4l2-common.h
index 5b3f685a2d50..4f7b892377cd 100644
--- a/include/linux/v4l2-common.h
+++ b/include/linux/v4l2-common.h
@@ -1,3 +1,4 @@
+/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR
BSD-3-Clause) */
/*
* include/linux/v4l2-common.h
*
diff --git a/include/linux/v4l2-controls.h b/include/linux/v4l2-controls.h
index 0d2e1e01fbd5..e4ee10ee917d 100644
--- a/include/linux/v4l2-controls.h
+++ b/include/linux/v4l2-controls.h
@@ -1,3 +1,4 @@
+/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR
BSD-3-Clause) */
/*
* Video for Linux Two controls header file
*
@@ -66,8 +67,8 @@
/* User-class control IDs */
#define V4L2_CID_BASE (V4L2_CTRL_CLASS_USER | 0x900)
-#define V4L2_CID_USER_BASE V4L2_CID_BASE
-#define V4L2_CID_USER_CLASS(V4L2_CTRL_CLASS_USER | 1)
+#define V4L2_CID_USER_BASE V4L2_CID_BASE
+#define V4L2_CID_USER_CLASS(V4L2_CTRL_CLASS_USER | 1)
#define V4L2_CID_BRIGHTNESS(V4L2_CID_BASE+0)
#define V4L2_CID_CONTRAST (V4L2_CID_BASE+1)
#define V4L2_CID_SATURATION(V4L2_CID_BASE+2)
@@ -101,7 +102,7 @@ enum v4l2_power_line_frequency {
#define V4L2_CID_HUE_AUTO (V4L2_CID_BASE+25)
#define V4L2_CID_WHITE_BALANCE_TEMPERATURE (V4L2_CID_BASE+26)
#define V4L2_CID_SHARPNESS (V4L2_CID_BASE+27)
-#define V4L2_CID_BACKLIGHT_COMPENSATION(V4L2_CID_BASE+28)
+#define V4L2_CID_BACKLIGHT_COMPENSATION(V4L2_CID_BASE+28)
#define V4L2_CID_CHROMA_AGC (V4L2_CID_BASE+29)
#define V4L2_CID_COLOR_KILLER (V4L2_CID_BASE+30)
#define V4L2_CID_COLORFX (V4L2_CID_BASE+31)
@@ -180,15 +181,24 @@ enum v4l2_colorfx {
* We reserve 16 controls for this driver. */
#define V4L2_CID_USER_TC358743_BASE(V4L2_CID_USER_BASE + 0x1080)
+/* The base for the max217x driver controls.
+ * We reserve 32 controls for this driver
+ */
+#define V4L2_CID_USER_MAX217X_BASE (V4L2_CID_USER_BASE + 0x1090)
+
+/* The base for the imx driver controls.
+ * We reserve 16 controls for this driver. */
+#define V4L2_CID_USER_IMX_BASE (V4L2_CID_USER_BASE + 0x10b0)
+
/* MPEG-class control IDs */
/* The MPEG controls are applicable to all codec controls
* and the 'MPEG' part of the define is historical */
-#define V4L2_CID_MPEG_BASE (V4L2_CTRL_CLASS_MPEG | 0x900)
-#define V4L2_CID_MPEG_CLASS(V4L2_CTRL_CLASS_MPEG | 1)
+#define V4L2_CID_MPEG_BASE (V4L2_CTRL_CLASS_MPEG | 0x900)
+#define V4L2_CID_MPEG_CLASS(V4L2_CTRL_CLASS_MPEG | 1)
/* MPEG streams, specific to multiplexed streams */
-#define V4L2_CID_MPEG_STREAM_TYPE (V4L2_CID_MPEG_BASE+0)
+#define V4L2_CID_MPEG_STREAM_TYPE (V4L2_CID_MPEG_BASE+0)
enum v4l2_mpeg_stream_type {
V4L2_MPEG_STREAM_TYPE_MPEG2_PS = 0, /* MPEG-2 program stream */
V4L2_MPEG_STREAM_TYPE_MPEG2_TS = 1, /* MPEG-2 transport stream */
@@ -197,26 +207,26 @@ enum v4l2_mpeg_stream_type {
V4L2_MPEG_STREAM_TYPE_MPEG1_VCD = 4, /* MPEG-1 VCD-compatible stream */
V4L2_MPEG_STREAM_TYPE_MPEG2_SVCD = 5, /* MPEG-2 SVCD-compatible stream
*/
};
-#define V4L2_CID_MPEG_STREAM_PID_PMT (V4L2_CID_MPEG_BASE+1)
-#define V4L2_CID_MPEG_STREAM_PID_AUDIO (V4L2_CID_MPEG_BASE+2)
-#define V4L2_CID_MPEG_STREAM_PID_VIDEO (V4L2_CID_MPEG_BASE+3)
-#define V4L2_CID_MPEG_STREAM_PID_PCR (V4L2_CID_MPEG_BASE+4)
-#define V4L2_CID_MPEG_STREAM_PES_ID_AUDIO (V4L2_CID_MPEG_BASE+5)
-#define V4L2_CID_MPEG_STREAM_PES_ID_VIDEO (V4L2_CID_MPEG_BASE+6)
-#define V4L2_CID_MPEG_STREAM_VBI_FMT (V4L2_CID_MPEG_BASE+7)
+#define V4L2_CID_MPEG_STREAM_PID_PMT (V4L2_CID_MPEG_BASE+1)
+#define V4L2_CID_MPEG_STREAM_PID_AUDIO (V4L2_CID_MPEG_BASE+2)
+#define V4L2_CID_MPEG_STREAM_PID_VIDEO (V4L2_CID_MPEG_BASE+3)
+#define V4L2_CID_MPEG_STREAM_PID_PCR (V4L2_CID_MPEG_BASE+4)
+#define V4L2_CID_MPEG_STREAM_PES_ID_AUDIO (V4L2_CID_MPEG_BASE+5)
+#define V4L2_CID_MPEG_STREAM_PES_ID_VIDEO (V4L2_CID_MPEG_BASE+6)
+#define V4L2_CID_MPEG_STREAM_VBI_FMT (V4L2_CID_MPEG_BASE+7)
enum v4l2_mpeg_stream_vbi_fmt {
V4L2_MPEG_STREAM_VBI_FMT_NONE = 0, /* No VBI in the MPEG stream */
V4L2_MPEG_STREAM_VBI_FMT_IVTV = 1, /* VBI in private packets, IVTV
format */
};
/* MPEG audio controls specific to multiplexed streams */
-#define V4L2_CID_MPEG_AUDIO_SAMPLING_FREQ
[PATCH 0/2] yavta: Add support for IPU3 raw 10-bit Bayer packed formats
Update kernel headers to latest mainline release and add support for IPU3 defined formats to yavta. Jacopo Mondi (2): Update kernel headers to v4.19 Add support for IPU3 raw 10-bit Bayer packed formats include/linux/v4l2-common.h | 1 + include/linux/v4l2-controls.h | 219 ++ include/linux/videodev2.h | 81 - yavta.c | 4 + 4 files changed, 225 insertions(+), 80 deletions(-) -- Regards, Laurent Pinchart
[PATCH 2/2] Add support for IPU3 raw 10-bit Bayer packed formats
From: Jacopo Mondi
Signed-off-by: Jacopo Mondi
Signed-off-by: Laurent Pinchart
---
yavta.c | 4
1 file changed, 4 insertions(+)
diff --git a/yavta.c b/yavta.c
index afe96331a520..c7986bd9e031 100644
--- a/yavta.c
+++ b/yavta.c
@@ -224,6 +224,10 @@ static struct v4l2_format_info {
{ "SGBRG12", V4L2_PIX_FMT_SGBRG12, 1 },
{ "SGRBG12", V4L2_PIX_FMT_SGRBG12, 1 },
{ "SRGGB12", V4L2_PIX_FMT_SRGGB12, 1 },
+ { "IPU3_SBGGR10", V4L2_PIX_FMT_IPU3_SBGGR10, 1 },
+ { "IPU3_SGBRG10", V4L2_PIX_FMT_IPU3_SGBRG10, 1 },
+ { "IPU3_SGRBG10", V4L2_PIX_FMT_IPU3_SGRBG10, 1 },
+ { "IPU3_SRGGB10", V4L2_PIX_FMT_IPU3_SRGGB10, 1 },
{ "DV", V4L2_PIX_FMT_DV, 1 },
{ "MJPEG", V4L2_PIX_FMT_MJPEG, 1 },
{ "MPEG", V4L2_PIX_FMT_MPEG, 1 },
--
Regards,
Laurent Pinchart
Re: [PATCH] media: vb2: Allow reqbufs(0) with "in use" MMAP buffers
Hi Hans, On Thursday, 15 November 2018 09:30:55 EET Hans Verkuil wrote: > On 11/14/2018 08:52 PM, Laurent Pinchart wrote: > > On Tuesday, 13 November 2018 17:43:48 EET Hans Verkuil wrote: > >> On 11/13/18 16:06, Philipp Zabel wrote: > >>> From: John Sheu > >>> > >>> Videobuf2 presently does not allow VIDIOC_REQBUFS to destroy outstanding > >>> buffers if the queue is of type V4L2_MEMORY_MMAP, and if the buffers are > >>> considered "in use". This is different behavior than for other memory > >>> types and prevents us from deallocating buffers in following two cases: > >>> > >>> 1) There are outstanding mmap()ed views on the buffer. However even if > >>>we put the buffer in reqbufs(0), there will be remaining references, > >>>due to vma .open/close() adjusting vb2 buffer refcount appropriately. > >>>This means that the buffer will be in fact freed only when the last > >>>mmap()ed view is unmapped. > >>> > >>> 2) Buffer has been exported as a DMABUF. Refcount of the vb2 buffer > >>>is managed properly by VB2 DMABUF ops, i.e. incremented on DMABUF > >>>get and decremented on DMABUF release. This means that the buffer > >>>will be alive until all importers release it. > >>> > >>> Considering both cases above, there does not seem to be any need to > >>> prevent reqbufs(0) operation, because buffer lifetime is already > >>> properly managed by both mmap() and DMABUF code paths. Let's remove it > >>> and allow userspace freeing the queue (and potentially allocating a new > >>> one) even though old buffers might be still in processing. > >>> > >>> To let userspace know that the kernel now supports orphaning buffers > >>> that are still in use, add a new V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS > >>> to be set by reqbufs and create_bufs. > >> > >> Looks good, but I have some questions: > >> > >> 1) does v4l2-compliance together with vivid (easiest to test) still work? > >>I don't think I have a proper test for this in v4l2-compliance, but > >>I'm not 100% certain. If it fails with this patch, then please provide > >>a fix for v4l2-compliance as well. > >> > >> 2) I would like to see a new test in v4l2-compliance for this: i.e. if > >>the capability is set, then check that you can call REQBUFS(0) before > >>unmapping all buffers. Ditto with dmabuffers. > >> > >> I said during the media summit that I wanted to be more strict about > >> requiring compliance tests before adding new features, so you're the > >> unlucky victim of that :-) > > > > Do you have plans to refactor and document the v4l2-compliance internals > > to make it easier ? > > Yes. I hope to be able to set aside one or two days for that in the next two > weeks. That would be great ! Let me know if you would like to discuss how the code base could be refactored. -- Regards, Laurent Pinchart
Re: [PATCH v3] media: vb2: Allow reqbufs(0) with "in use" MMAP buffers
by the means of REQBUFS(0)) call
> - */
> -static bool __buffers_in_use(struct vb2_queue *q)
> -{
> - unsigned int buffer;
> - for (buffer = 0; buffer < q->num_buffers; ++buffer) {
> - if (vb2_buffer_in_use(q, q->bufs[buffer]))
> - return true;
> - }
> - return false;
> -}
> -
> void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb)
> {
> call_void_bufop(q, fill_user_buffer, q->bufs[index], pb);
> @@ -674,23 +660,13 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum
> vb2_memory memory,
>
> if (*count == 0 || q->num_buffers != 0 ||
> (q->memory != VB2_MEMORY_UNKNOWN && q->memory != memory)) {
> - /*
> - * We already have buffers allocated, so first check if they
> - * are not in use and can be freed.
> - */
> - mutex_lock(>mmap_lock);
> - if (q->memory == VB2_MEMORY_MMAP && __buffers_in_use(q)) {
> - mutex_unlock(>mmap_lock);
> - dprintk(1, "memory in use, cannot free\n");
> - return -EBUSY;
> - }
> -
> /*
>* Call queue_cancel to clean up any buffers in the
>* QUEUED state which is possible if buffers were prepared or
>* queued without ever calling STREAMON.
>*/
> __vb2_queue_cancel(q);
> + mutex_lock(>mmap_lock);
This results in __vb2_queue_cancel() called without the mmap_lock held. Is
that OK ?
> ret = __vb2_queue_free(q, q->num_buffers);
> mutex_unlock(>mmap_lock);
> if (ret)
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> b/drivers/media/common/videobuf2/videobuf2-v4l2.c index
> a17033ab2c22..f02d452ceeb9 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -624,7 +624,7 @@ EXPORT_SYMBOL(vb2_querybuf);
>
> static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
> {
> - *caps = 0;
> + *caps = V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS;
> if (q->io_modes & VB2_MMAP)
> *caps |= V4L2_BUF_CAP_SUPPORTS_MMAP;
> if (q->io_modes & VB2_USERPTR)
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index c8e8ff810190..2a223835214c 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -879,6 +879,7 @@ struct v4l2_requestbuffers {
> #define V4L2_BUF_CAP_SUPPORTS_USERPTR(1 << 1)
> #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
> #define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3)
> +#define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
>
> /**
> * struct v4l2_plane - plane info for multi-planar buffers
--
Regards,
Laurent Pinchart
Re: [PATCH] media: vb2: Allow reqbufs(0) with "in use" MMAP buffers
- * __buffers_in_use() - return true if any buffers on the queue are in
> > use and - * the queue cannot be freed (by the means of REQBUFS(0)) call
> > - */
> > -static bool __buffers_in_use(struct vb2_queue *q)
> > -{
> > - unsigned int buffer;
> > - for (buffer = 0; buffer < q->num_buffers; ++buffer) {
> > - if (vb2_buffer_in_use(q, q->bufs[buffer]))
> > - return true;
> > - }
> > - return false;
> > -}
> > -
> >
> > void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb)
> > {
> >
> > call_void_bufop(q, fill_user_buffer, q->bufs[index], pb);
> >
> > @@ -674,23 +660,13 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum
> > vb2_memory memory,>
> > if (*count == 0 || q->num_buffers != 0 ||
> >
> > (q->memory != VB2_MEMORY_UNKNOWN && q->memory != memory)) {
> >
> > - /*
> > -* We already have buffers allocated, so first check if they
> > -* are not in use and can be freed.
> > -*/
> > - mutex_lock(>mmap_lock);
> > - if (q->memory == VB2_MEMORY_MMAP && __buffers_in_use(q)) {
> > - mutex_unlock(>mmap_lock);
> > - dprintk(1, "memory in use, cannot free\n");
> > - return -EBUSY;
> > - }
> > -
> >
> > /*
> >
> > * Call queue_cancel to clean up any buffers in the
> > * QUEUED state which is possible if buffers were prepared or
> > * queued without ever calling STREAMON.
> > */
> >
> > __vb2_queue_cancel(q);
> >
> > + mutex_lock(>mmap_lock);
> >
> > ret = __vb2_queue_free(q, q->num_buffers);
> > mutex_unlock(>mmap_lock);
> > if (ret)
> >
> > diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > b/drivers/media/common/videobuf2/videobuf2-v4l2.c index
> > a17033ab2c22..f02d452ceeb9 100644
> > --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > @@ -624,7 +624,7 @@ EXPORT_SYMBOL(vb2_querybuf);
> >
> > static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
> > {
> >
> > - *caps = 0;
> > + *caps = V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS;
> >
> > if (q->io_modes & VB2_MMAP)
> >
> > *caps |= V4L2_BUF_CAP_SUPPORTS_MMAP;
> >
> > if (q->io_modes & VB2_USERPTR)
> >
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h index c8e8ff810190..2a223835214c 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -879,6 +879,7 @@ struct v4l2_requestbuffers {
> >
> > #define V4L2_BUF_CAP_SUPPORTS_USERPTR (1 << 1)
> > #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
> > #define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3)
> >
> > +#define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
> >
> > /**
> >
> > * struct v4l2_plane - plane info for multi-planar buffers
--
Regards,
Laurent Pinchart
Re: [PATCH v6 05/10] media: uvcvideo: queue: Support asynchronous buffer handling
Hi Kieran,
Thank you for the patch.
On Friday, 9 November 2018 19:05:28 EET Kieran Bingham wrote:
> The buffer queue interface currently operates sequentially, processing
> buffers after they have fully completed.
>
> In preparation for supporting parallel tasks operating on the buffers,
> we will need to support buffers being processed on multiple CPUs.
>
> Adapt the uvc_queue_next_buffer() such that a reference count tracks the
> active use of the buffer, returning the buffer to the VB2 stack at
> completion.
>
> Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
> v5:
> - uvc_queue_requeue() -> uvc_queue_buffer_requeue()
> - Fix comment
> ---
> drivers/media/usb/uvc/uvc_queue.c | 61 ++--
> drivers/media/usb/uvc/uvcvideo.h | 4 ++-
> 2 files changed, 54 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c index fa7059aab49a..2752e386f1e8 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -142,6 +142,7 @@ static void uvc_buffer_queue(struct vb2_buffer *vb)
>
> spin_lock_irqsave(>irqlock, flags);
> if (likely(!(queue->flags & UVC_QUEUE_DISCONNECTED))) {
> + kref_init(>ref);
> list_add_tail(>queue, >irqqueue);
> } else {
> /* If the device is disconnected return the buffer to userspace
> @@ -458,28 +459,66 @@ struct uvc_buffer *uvc_queue_get_current_buffer(struct
> uvc_video_queue *queue) return nextbuf;
> }
>
> -struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
> +/*
> + * uvc_queue_buffer_requeue: Requeue a buffer on our internal irqqueue
> + *
> + * Reuse a buffer through our internal queue without the need to 'prepare'.
> + * The buffer will be returned to userspace through the uvc_buffer_queue
> call if + * the device has been disconnected.
> + */
> +static void uvc_queue_buffer_requeue(struct uvc_video_queue *queue,
> struct uvc_buffer *buf)
> {
> - struct uvc_buffer *nextbuf;
> - unsigned long flags;
> + buf->error = 0;
> + buf->state = UVC_BUF_STATE_QUEUED;
> + buf->bytesused = 0;
> + vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
> +
> + uvc_buffer_queue(>buf.vb2_buf);
> +}
> +
> +static void uvc_queue_buffer_complete(struct kref *ref)
> +{
> + struct uvc_buffer *buf = container_of(ref, struct uvc_buffer, ref);
> + struct vb2_buffer *vb = >buf.vb2_buf;
> + struct uvc_video_queue *queue = vb2_get_drv_priv(vb->vb2_queue);
>
> if ((queue->flags & UVC_QUEUE_DROP_CORRUPTED) && buf->error) {
> - buf->error = 0;
> - buf->state = UVC_BUF_STATE_QUEUED;
> - buf->bytesused = 0;
> - vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
> - return buf;
> + uvc_queue_buffer_requeue(queue, buf);
> + return;
> }
>
> + buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
> + vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
> + vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +/*
> + * Release a reference on the buffer. Complete the buffer when the last
> + * reference is released.
> + */
> +void uvc_queue_buffer_release(struct uvc_buffer *buf)
> +{
> + kref_put(>ref, uvc_queue_buffer_complete);
> +}
> +
> +/*
> + * Remove this buffer from the queue. Lifetime will persist while async
> actions + * are still running (if any), and uvc_queue_buffer_release will
> give the buffer + * back to VB2 when all users have completed.
> + */
> +struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
> + struct uvc_buffer *buf)
> +{
> + struct uvc_buffer *nextbuf;
> + unsigned long flags;
> +
> spin_lock_irqsave(>irqlock, flags);
> list_del(>queue);
> nextbuf = __uvc_queue_get_current_buffer(queue);
> spin_unlock_irqrestore(>irqlock, flags);
>
> - buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
> - vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
> - vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
> + uvc_queue_buffer_release(buf);
>
> return nextbuf;
> }
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index 16b8348f7ff0..7f884c60ae59 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -413,6 +413,9 @@ struct uvc_buffer {
> unsigned i
Re: [PATCH v5 0/9] Asynchronous UVC
Hi Kieran,
On Wednesday, 7 November 2018 01:31:29 EET Laurent Pinchart wrote:
> On Tuesday, 6 November 2018 23:27:11 EET Kieran Bingham wrote:
> > From: Kieran Bingham
> >
> > The Linux UVC driver has long provided adequate performance capabilities
> > for web-cams and low data rate video devices in Linux while resolutions
> > were low.
> >
> > Modern USB cameras are now capable of high data rates thanks to USB3 with
> > 1080p, and even 4k capture resolutions supported.
> >
> > Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO
> > (isochronous transfers) can generate more data than an embedded ARM core
> > is able to process on a single core, resulting in frame loss.
> >
> > A large part of this performance impact is from the requirement to
> > ‘memcpy’ frames out from URB packets to destination frames. This
> > unfortunate requirement is due to the UVC protocol allowing a variable
> > length header, and thus it is not possible to provide the target frame
> > buffers directly.
> >
> > Extra throughput is possible by moving the actual memcpy actions to a work
> > queue, and moving the memcpy out of interrupt context thus allowing work
> > tasks to be scheduled across multiple cores.
> >
> > This series has been tested on both the ZED and BRIO cameras on arm64
> > platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix
> > USB2 Camera, and a built in Toshiba Laptop camera, and with thanks to
> > Philipp Zabel for testing on a Lite-On internal Laptop Webcam, Logitech
> > C910 (USB2 isoc), Oculus Sensor (USB3 isoc), and Microsoft HoloLens
> > Sensors (USB3 bulk).
> >
> > As far as I am aware iSight devices, and devices which use UVC to encode
> > data (output device) have not yet been tested - but should find no ill
> > effect (at least not until they are tested of course :D )
>
> :-D
>
> I'm not sure whether anyone is still using those devices with Linux. I
> wouldn't be surprised if we realized down the road that they already don't
> work.
>
> > Tested-by: Randy Dunlap
> > Tested-by: Philipp Zabel
> >
> > v2:
> > - Fix race reported by Guennadi
> >
> > v3:
> > - Fix similar race reported by Laurent
> > - Only queue work if required (encode/isight do not queue work)
> > - Refactor/Rename variables for clarity
> >
> > v4:
> > - (Yet another) Rework of the uninitialise path.
> >
> >This time to hopefully clean up the shutdown races for good.
> >use usb_poison_urb() to halt all URBs, then flush the work queue
> >before freeing.
> >
> > - Rebase to latest linux-media/master
> >
> > v5:
> > - Provide lockdep validation
> > - rename uvc_queue_requeue -> uvc_queue_buffer_requeue()
> > - Fix comments and periods throughout
> > - Rebase to media/v4.20-2
> > - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
> > - Fix function documentation for uvc_video_copy_data_work()
> > - Add periods to the end of sentences
> > - Rename 'decode' variable to 'op' in uvc_video_decode_data()
> > - Move uvc_urb->async_operations initialisation to before use
> > - Move async workqueue to match uvc_streaming lifetime instead of
> >
> >streamon/streamoff
> >
> > - bracket the for_each_uvc_urb() macro
> >
> > - New patches added to series:
> > media: uvcvideo: Split uvc_video_enable into two
> > media: uvcvideo: Rename uvc_{un,}init_video()
> > media: uvcvideo: Utilise for_each_uvc_urb iterator
> >
> > Kieran Bingham (9):
> > media: uvcvideo: Refactor URB descriptors
> > media: uvcvideo: Convert decode functions to use new context structure
> > media: uvcvideo: Protect queue internals with helper
> > media: uvcvideo: queue: Simplify spin-lock usage
> > media: uvcvideo: queue: Support asynchronous buffer handling
> > media: uvcvideo: Move decode processing to process context
> > media: uvcvideo: Split uvc_video_enable into two
>
> I've taken the above patches in my tree.
I'm afraid I'll have to drop them :-( If I disconnect the camera while in use,
I get the following oops:
[ 237.514625] usb 2-1.4: USB disconnect, device number 5
[ 237.516123] uvcvideo: Failed to resubmit video URB (-19).
[ 237.549470] BUG: unable to handle kernel paging request at 4bec0091
[ 237.549476] PGD 0 P4D 0
[ 237.549481] Oops: [#1] PREEMPT SMP PTI
[ 237.549485] CPU: 3 PID: 5332 Comm: luvcview Tainted: GW O
4.18.16-gentoo #1
[ 237.549487] Hardware name: Dell Inc
Re: [PATCH v5 8/9] media: uvcvideo: Rename uvc_{un,}init_video()
Hi Kieran,
On Wednesday, 7 November 2018 16:30:46 EET Kieran Bingham wrote:
> On 06/11/2018 23:13, Laurent Pinchart wrote:
> > On Tuesday, 6 November 2018 23:27:19 EET Kieran Bingham wrote:
> >> From: Kieran Bingham
> >>
> >> We have both uvc_init_video() and uvc_video_init() calls which can be
> >> quite confusing to determine the process for each. Now that video
> >> uvc_video_enable() has been renamed to uvc_video_start_streaming(),
> >> adapt these calls to suit the new flow.
> >>
> >> Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
> >> uvc_video_stop().
> >
> > I agree that these functions are badly named and should be renamed. We are
> > however entering the nitpicking territory :-) The two functions do more
> > that starting and stopping, they also allocate and free URBs and the
> > associated buffers. It could also be argued that they don't actually
> > start and stop anything, as beyond URB management, they just queue the
> > URBs initially and kill them. I thus wonder if we could come up with
> > better names.
>
> Well the act of killing (poisoning now) the URBs will certainly stop the
> stream, but I guess submitting the URBs isn't necessarily the key act to
> starting the stream.
>
> I believe that needs the interface to be set correctly, and the buffers
> to be available?
>
> Although - I've just double-checked uvc_{video_start,init_video}() and
> that is indeed what it does?
>
> - start stats
> - Initialise endpoints
>- Perform allocations
> - Submit URBs
>
> Am I missing something? Is there another step that is pivotal to
> starting the USB packet/urb stream flow after this point ?
>
>
> Is it not true that the USB stack will start processing data at
> submitting URB completion callbacks after the end of uvc_video_start();
> and will no longer process data at the end of uvc_video_stop() (and thus
> no more completion callbacks)?
>
> (That's a real question to verify my interpretation)
>
> To me - these functions feel like the real 'start' and 'stop' components
> of the data stream - hence my choice in naming.
The other part of the start operation is committing the streaming parameters
(see uvc_video_start_streaming()). For the stop operation it's issuing a
SET_INTERFACE or CLEAR_FEATURE(HALT) request (see uvc_video_stop_streaming()).
> Is your concern that you would like the functions to be more descriptive
> over their other actions such as? :
>
> uvc_video_initialise_start()
> uvc_video_allocate_init_start()
>
> Or something else? (I don't think those two are good names though)
Probably something else :-) A possibly equally bad proposal would be
uvc_video_start_transfer() and uvc_video_stop_transfer().
--
Regards,
Laurent Pinchart
Re: [RFC] Create test script(s?) for regression testing
Hi Mauro, On Wednesday, 7 November 2018 21:53:20 EET Mauro Carvalho Chehab wrote: > Em Wed, 07 Nov 2018 21:35:32 +0200 Laurent Pinchart escreveu: > > On Wednesday, 7 November 2018 21:10:35 EET Mauro Carvalho Chehab wrote: [snip] > >> I'm with Hans on that matter: better to start with an absolute minimum > >> of dependencies (like just: make, autotools, c, c++, bash), > > > > On a site note, for a new project, we might want to move away from > > autotools. cmake and meson are possible alternatives that are way less > > painful. > > Each toolset has advantages or disadvantages. We all know how > autotools can be painful. > > One bad thing with cmake is that they deprecate stuff. A long-live project > usually require several" backward" compat stuff at cmake files in order > to cope with different behaviors that change as cmake evolves. I don't know how much of a problem that would be. My experience with cmake is good so far, but I haven't used it in a large scale project with 10+ years of contributions :-) > I never used mason myself. It's the build system we picked for libcamera, I expect to provide feedback in the not too distant future. [snip] -- Regards, Laurent Pinchart
Re: [RFC] Create test script(s?) for regression testing
Hi Mauro, On Wednesday, 7 November 2018 21:10:35 EET Mauro Carvalho Chehab wrote: > Em Wed, 07 Nov 2018 12:06:55 +0200 Laurent Pinchart escreveu: > > On Wednesday, 7 November 2018 10:05:12 EET Hans Verkuil wrote: > >> On 11/06/2018 08:58 PM, Laurent Pinchart wrote: > >>> On Tuesday, 6 November 2018 15:56:34 EET Hans Verkuil wrote: > >>>> On 11/06/18 14:12, Laurent Pinchart wrote: > >>>>> On Tuesday, 6 November 2018 13:36:55 EET Sakari Ailus wrote: > >>>>>> On Tue, Nov 06, 2018 at 09:37:07AM +0100, Hans Verkuil wrote: > >>>>>>> Hi all, > >>>>>>> > >>>>>>> After the media summit (heavy on test discussions) and the V4L2 > >>>>>>> event regression we just found it is clear we need to do a better > >>>>>>> job with testing. > >>>>>>> > >>>>>>> All the pieces are in place, so what is needed is to combine it > >>>>>>> and create a script that anyone of us as core developers can run to > >>>>>>> check for regressions. The same script can be run as part of the > >>>>>>> kernelci regression testing. > >>>>>> > >>>>>> I'd say that *some* pieces are in place. Of course, the more there > >>>>>> is, the better. > >>>>>> > >>>>>> The more there are tests, the more important it would be they're > >>>>>> automated, preferrably without the developer having to run them on > >>>>>> his/her own machine. > >>>>> > >>>>> From my experience with testing, it's important to have both a core > >>>>> set of tests (a.k.a. smoke tests) that can easily be run on > >>>>> developers' machines, and extended tests that can be offloaded to a > >>>>> shared testing infrastructure (but possibly also run locally if > >>>>> desired). > >>>> > >>>> That was my idea as well for the longer term. First step is to do the > >>>> basic smoke tests (i.e. run compliance tests, do some (limited) > >>>> streaming test). > >>>> > >>>> There are more extensive (and longer running) tests that can be done, > >>>> but that's something to look at later. > >>>> > >>>>>>> We have four virtual drivers: vivid, vim2m, vimc and vicodec. The > >>>>>>> last one is IMHO not quite good enough yet for testing: it is not > >>>>>>> fully compliant to the upcoming stateful codec spec. Work for that > >>>>>>> is planned as part of an Outreachy project. > >>>>>>> > >>>>>>> My idea is to create a script that is maintained as part of > >>>>>>> v4l-utils that loads the drivers and runs v4l2-compliance and > >>>>>>> possibly other tests against the virtual drivers. > > (adding Shuah) > > IMO, the best would be to have something like that as part of Kernel > self test, as this could give a broader covering than just Kernel CI. > > Yeah, I know that one of the concerns is that the *-compliance stuff > we have are written in C++ and it is easier to maintain then at > v4l-utils, but maybe it would be acceptable at kselftest to have a > test bench there with would download the sources from a git tree > and then build just the v4l2-compliance stuff, e. g. having a Kernel > self test target that would do something like: > > git clone --depth 1 git://linuxtv.org/v4l-utils.git tests && \ > cd tests && ./autogen.sh && make tests && ./run_tests.sh Let me make sure I understand this properly. Are you proposing to add to kselftest, which is part of the Linux kernel, and as such benefits from the level of trust of Linus' tree, and which is run by a very large number of machines from developer workstations to automated large-scale test infrastructure, a provision to execute locally code that is downloaded at runtime from the internet, with all the security issues this implies ? > (the actual selftest target would likely be different, as it > should take into account make O=) > > If this would be acceptable upstream, then we'll need to stick with the > output format defined by Kernel Self Test[1]. > > [1] I guess it uses the TAP13 format: > https://testanything.org/tap-version-13-specification.html > > >>>>>> How about spending a little time to pick
Re: [RFC] Create test script(s?) for regression testing
Hi Hans, On Wednesday, 7 November 2018 10:05:12 EET Hans Verkuil wrote: > On 11/06/2018 08:58 PM, Laurent Pinchart wrote: > > On Tuesday, 6 November 2018 15:56:34 EET Hans Verkuil wrote: > >> On 11/06/18 14:12, Laurent Pinchart wrote: > >>> On Tuesday, 6 November 2018 13:36:55 EET Sakari Ailus wrote: > >>>> On Tue, Nov 06, 2018 at 09:37:07AM +0100, Hans Verkuil wrote: > >>>>> Hi all, > >>>>> > >>>>> After the media summit (heavy on test discussions) and the V4L2 event > >>>>> regression we just found it is clear we need to do a better job with > >>>>> testing. > >>>>> > >>>>> All the pieces are in place, so what is needed is to combine it and > >>>>> create a script that anyone of us as core developers can run to check > >>>>> for regressions. The same script can be run as part of the kernelci > >>>>> regression testing. > >>>> > >>>> I'd say that *some* pieces are in place. Of course, the more there is, > >>>> the better. > >>>> > >>>> The more there are tests, the more important it would be they're > >>>> automated, preferrably without the developer having to run them on his/ > >>>> her own machine. > >>> > >>> From my experience with testing, it's important to have both a core set > >>> of tests (a.k.a. smoke tests) that can easily be run on developers' > >>> machines, and extended tests that can be offloaded to a shared testing > >>> infrastructure (but possibly also run locally if desired). > >> > >> That was my idea as well for the longer term. First step is to do the > >> basic smoke tests (i.e. run compliance tests, do some (limited) streaming > >> test). > >> > >> There are more extensive (and longer running) tests that can be done, but > >> that's something to look at later. > >> > >>>>> We have four virtual drivers: vivid, vim2m, vimc and vicodec. The last > >>>>> one is IMHO not quite good enough yet for testing: it is not fully > >>>>> compliant to the upcoming stateful codec spec. Work for that is > >>>>> planned as part of an Outreachy project. > >>>>> > >>>>> My idea is to create a script that is maintained as part of v4l-utils > >>>>> that loads the drivers and runs v4l2-compliance and possibly other > >>>>> tests against the virtual drivers. > >>>> > >>>> How about spending a little time to pick a suitable framework for > >>>> running the tests? It could be useful to get more informative reports > >>>> than just pass / fail. > >>> > >>> We should keep in mind that other tests will be added later, and the > >>> test framework should make that easy. > >> > >> Since we want to be able to run this on kernelci.org, I think it makes > >> sense to let the kernelci folks (Hi Ezequiel!) decide this. > > > > KernelCI isn't the only test infrastructure out there, so let's not forget > > about the other ones. > > True, but they are putting time and money into this, so they get to choose > as far as I am concerned :-) It's still our responsibility to give V4L2 a good test framework, and to drive it in the right direction. We don't accept V4L2 API extensions blindly just because a company happens to put time and money into it (there may have been one exception, but it's not the rule), we instead review all proposals carefully. The same should be true with tests. > If others are interested and willing to put up time and money, they should > let themselves be known. > > I'm not going to work on such an integration, although I happily accept > patches. > > >> As a developer all I need is a script to run smoke tests so I can catch > >> most regressions (you never catch all). > >> > >> I'm happy to work with them to make any changes to compliance tools and > >> scripts so they fit better into their test framework. > >> > >> The one key requirement to all this is that you should be able to run > >> these tests without dependencies to all sorts of external packages/ > >> libraries. > > > > v4l-utils already has a set of dependencies, but those are largely > > manageable. For v4l2-compliance we'll install libv4l, which depends on > > libjpeg. > > That's already too much. You can manually build v4l2-compliance wi
Re: [PATCH v5 0/9] Asynchronous UVC
Hi Kieran,
Thank you for the patches.
On Tuesday, 6 November 2018 23:27:11 EET Kieran Bingham wrote:
> From: Kieran Bingham
>
> The Linux UVC driver has long provided adequate performance capabilities for
> web-cams and low data rate video devices in Linux while resolutions were
> low.
>
> Modern USB cameras are now capable of high data rates thanks to USB3 with
> 1080p, and even 4k capture resolutions supported.
>
> Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO
> (isochronous transfers) can generate more data than an embedded ARM core is
> able to process on a single core, resulting in frame loss.
>
> A large part of this performance impact is from the requirement to
> ‘memcpy’ frames out from URB packets to destination frames. This unfortunate
> requirement is due to the UVC protocol allowing a variable length header,
> and thus it is not possible to provide the target frame buffers directly.
>
> Extra throughput is possible by moving the actual memcpy actions to a work
> queue, and moving the memcpy out of interrupt context thus allowing work
> tasks to be scheduled across multiple cores.
>
> This series has been tested on both the ZED and BRIO cameras on arm64
> platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix
> USB2 Camera, and a built in Toshiba Laptop camera, and with thanks to
> Philipp Zabel for testing on a Lite-On internal Laptop Webcam, Logitech
> C910 (USB2 isoc), Oculus Sensor (USB3 isoc), and Microsoft HoloLens Sensors
> (USB3 bulk).
>
> As far as I am aware iSight devices, and devices which use UVC to encode
> data (output device) have not yet been tested - but should find no ill
> effect (at least not until they are tested of course :D )
:-D
I'm not sure whether anyone is still using those devices with Linux. I
wouldn't be surprised if we realized down the road that they already don't
work.
> Tested-by: Randy Dunlap
> Tested-by: Philipp Zabel
>
> v2:
> - Fix race reported by Guennadi
>
> v3:
> - Fix similar race reported by Laurent
> - Only queue work if required (encode/isight do not queue work)
> - Refactor/Rename variables for clarity
>
> v4:
> - (Yet another) Rework of the uninitialise path.
>This time to hopefully clean up the shutdown races for good.
>use usb_poison_urb() to halt all URBs, then flush the work queue
>before freeing.
> - Rebase to latest linux-media/master
>
> v5:
> - Provide lockdep validation
> - rename uvc_queue_requeue -> uvc_queue_buffer_requeue()
> - Fix comments and periods throughout
> - Rebase to media/v4.20-2
> - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
> - Fix function documentation for uvc_video_copy_data_work()
> - Add periods to the end of sentences
> - Rename 'decode' variable to 'op' in uvc_video_decode_data()
> - Move uvc_urb->async_operations initialisation to before use
> - Move async workqueue to match uvc_streaming lifetime instead of
>streamon/streamoff
> - bracket the for_each_uvc_urb() macro
>
> - New patches added to series:
> media: uvcvideo: Split uvc_video_enable into two
> media: uvcvideo: Rename uvc_{un,}init_video()
> media: uvcvideo: Utilise for_each_uvc_urb iterator
>
> Kieran Bingham (9):
> media: uvcvideo: Refactor URB descriptors
> media: uvcvideo: Convert decode functions to use new context structure
> media: uvcvideo: Protect queue internals with helper
> media: uvcvideo: queue: Simplify spin-lock usage
> media: uvcvideo: queue: Support asynchronous buffer handling
> media: uvcvideo: Move decode processing to process context
> media: uvcvideo: Split uvc_video_enable into two
I've taken the above patches in my tree.
> media: uvcvideo: Rename uvc_{un,}init_video()
> media: uvcvideo: Utilise for_each_uvc_urb iterator
And I've sent review comments for these two.
> drivers/media/usb/uvc/uvc_driver.c | 2 +-
> drivers/media/usb/uvc/uvc_isight.c | 6 +-
> drivers/media/usb/uvc/uvc_queue.c | 110 +---
> drivers/media/usb/uvc/uvc_video.c | 282 +++---
> drivers/media/usb/uvc/uvcvideo.h | 65 ++-
> 5 files changed, 331 insertions(+), 134 deletions(-)
>
> base-commit: dafb7f9aef2fd44991ff1691721ff765a23be27b
--
Regards,
Laurent Pinchart
Re: [PATCH v5 9/9] media: uvcvideo: Utilise for_each_uvc_urb iterator
struct urb *urb;
> - unsigned int npackets, pipe, i;
> + struct uvc_urb *uvc_urb;
> + unsigned int npackets, pipe;
> u16 psize;
> u32 size;
>
> @@ -1781,9 +1785,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
> *stream, if (stream->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
> size = 0;
>
> - for (i = 0; i < UVC_URBS; ++i) {
> - struct uvc_urb *uvc_urb = >uvc_urb[i];
> -
> + for_each_uvc_urb(uvc_urb, stream) {
> urb = usb_alloc_urb(0, gfp_flags);
> if (urb == NULL) {
> uvc_video_stop(stream, 1);
> @@ -1810,6 +1812,7 @@ static int uvc_video_start(struct uvc_streaming
> *stream, gfp_t gfp_flags) {
> struct usb_interface *intf = stream->intf;
> struct usb_host_endpoint *ep;
> + struct uvc_urb *uvc_urb;
> unsigned int i;
> int ret;
>
> @@ -1887,13 +1890,11 @@ static int uvc_video_start(struct uvc_streaming
> *stream, gfp_t gfp_flags) return ret;
>
> /* Submit the URBs. */
> - for (i = 0; i < UVC_URBS; ++i) {
> - struct uvc_urb *uvc_urb = >uvc_urb[i];
> -
> + for_each_uvc_urb(uvc_urb, stream) {
> ret = usb_submit_urb(uvc_urb->urb, gfp_flags);
> if (ret < 0) {
> - uvc_printk(KERN_ERR, "Failed to submit URB %u "
> - "(%d).\n", i, ret);
> + uvc_printk(KERN_ERR, "Failed to submit URB %u (%d).\n",
> +uvc_urb_index(uvc_urb), ret);
> uvc_video_stop(stream, 1);
> return ret;
> }
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index c0a120496a1f..6a0f1b59356c 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -617,6 +617,9 @@ struct uvc_streaming {
>(uvc_urb) < &(uvc_streaming)->uvc_urb[UVC_URBS]; \
>++(uvc_urb))
>
> +#define uvc_urb_index(uvc_urb) \
> + (unsigned int)((uvc_urb) - (&(uvc_urb)->stream->uvc_urb[0]))
> +
> struct uvc_device_info {
> u32 quirks;
> u32 meta_format;
--
Regards,
Laurent Pinchart
Re: [PATCH v5 8/9] media: uvcvideo: Rename uvc_{un,}init_video()
Hi Kieran,
Thank you for the patch.
On Tuesday, 6 November 2018 23:27:19 EET Kieran Bingham wrote:
> From: Kieran Bingham
>
> We have both uvc_init_video() and uvc_video_init() calls which can be
> quite confusing to determine the process for each. Now that video
> uvc_video_enable() has been renamed to uvc_video_start_streaming(),
> adapt these calls to suit the new flow.
>
> Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
> uvc_video_stop().
I agree that these functions are badly named and should be renamed. We are
however entering the nitpicking territory :-) The two functions do more that
starting and stopping, they also allocate and free URBs and the associated
buffers. It could also be argued that they don't actually start and stop
anything, as beyond URB management, they just queue the URBs initially and
kill them. I thus wonder if we could come up with better names.
> Signed-off-by: Kieran Bingham
> ---
> drivers/media/usb/uvc/uvc_video.c | 18 +-
> 1 file changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 0d35e933856a..020022e6ade4 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1641,7 +1641,7 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
> *stream, /*
> * Uninitialize isochronous/bulk URBs and free transfer buffers.
> */
> -static void uvc_uninit_video(struct uvc_streaming *stream, int
> free_buffers) +static void uvc_video_stop(struct uvc_streaming *stream, int
> free_buffers) {
> struct uvc_urb *uvc_urb;
>
> @@ -1718,7 +1718,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
> *stream,
>
> urb = usb_alloc_urb(npackets, gfp_flags);
> if (urb == NULL) {
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop(stream, 1);
> return -ENOMEM;
> }
>
> @@ -1786,7 +1786,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
> *stream,
>
> urb = usb_alloc_urb(0, gfp_flags);
> if (urb == NULL) {
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop(stream, 1);
> return -ENOMEM;
> }
>
> @@ -1806,7 +1806,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
> *stream, /*
> * Initialize isochronous/bulk URBs and allocate transfer buffers.
> */
> -static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
> +static int uvc_video_start(struct uvc_streaming *stream, gfp_t gfp_flags)
> {
> struct usb_interface *intf = stream->intf;
> struct usb_host_endpoint *ep;
> @@ -1894,7 +1894,7 @@ static int uvc_init_video(struct uvc_streaming
> *stream, gfp_t gfp_flags) if (ret < 0) {
> uvc_printk(KERN_ERR, "Failed to submit URB %u "
> "(%d).\n", i, ret);
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop(stream, 1);
> return ret;
> }
> }
> @@ -1925,7 +1925,7 @@ int uvc_video_suspend(struct uvc_streaming *stream)
> return 0;
>
> stream->frozen = 1;
> - uvc_uninit_video(stream, 0);
> + uvc_video_stop(stream, 0);
> usb_set_interface(stream->dev->udev, stream->intfnum, 0);
> return 0;
> }
> @@ -1961,7 +1961,7 @@ int uvc_video_resume(struct uvc_streaming *stream, int
> reset) if (ret < 0)
> return ret;
>
> - return uvc_init_video(stream, GFP_NOIO);
> + return uvc_video_start(stream, GFP_NOIO);
> }
>
> /*
> @@ -2095,7 +2095,7 @@ int uvc_video_start_streaming(struct uvc_streaming
> *stream) if (ret < 0)
> goto error_commit;
>
> - ret = uvc_init_video(stream, GFP_KERNEL);
> + ret = uvc_video_start(stream, GFP_KERNEL);
> if (ret < 0)
> goto error_video;
>
> @@ -2111,7 +2111,7 @@ int uvc_video_start_streaming(struct uvc_streaming
> *stream)
>
> int uvc_video_stop_streaming(struct uvc_streaming *stream)
> {
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop(stream, 1);
> if (stream->intf->num_altsetting > 1) {
> usb_set_interface(stream->dev->udev,
> stream->intfnum, 0);
--
Regards,
Laurent Pinchart
Re: [PATCH v5 7/9] media: uvcvideo: Split uvc_video_enable into two
Hi Kieran,
Thank you for the patch.
On Tuesday, 6 November 2018 23:27:18 EET Kieran Bingham wrote:
> From: Kieran Bingham
>
> uvc_video_enable() is used both to start and stop the video stream
> object, however the single function entry point shares no code between
> the two operations.
>
> Split the function into two distinct calls, and rename to
> uvc_video_start_streaming() and uvc_video_stop_streaming() as
> appropriate.
>
> Signed-off-by: Kieran Bingham
> ---
> drivers/media/usb/uvc/uvc_queue.c | 4 +-
> drivers/media/usb/uvc/uvc_video.c | 56 +++-
> drivers/media/usb/uvc/uvcvideo.h | 3 +-
> 3 files changed, 31 insertions(+), 32 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c index cd8c03341de0..682698ec1118 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -176,7 +176,7 @@ static int uvc_start_streaming(struct vb2_queue *vq,
> unsigned int count)
>
> queue->buf_used = 0;
>
> - ret = uvc_video_enable(stream, 1);
> + ret = uvc_video_start_streaming(stream);
> if (ret == 0)
> return 0;
>
> @@ -194,7 +194,7 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
> lockdep_assert_irqs_enabled();
>
> if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
> - uvc_video_enable(uvc_queue_to_stream(queue), 0);
> + uvc_video_stop_streaming(uvc_queue_to_stream(queue));
>
> spin_lock_irq(>irqlock);
> uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index ce9e40444507..0d35e933856a 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -2082,38 +2082,10 @@ int uvc_video_init(struct uvc_streaming *stream)
> return 0;
> }
>
> -/*
> - * Enable or disable the video stream.
> - */
> -int uvc_video_enable(struct uvc_streaming *stream, int enable)
> +int uvc_video_start_streaming(struct uvc_streaming *stream)
> {
> int ret;
>
> - if (!enable) {
> - uvc_uninit_video(stream, 1);
> - if (stream->intf->num_altsetting > 1) {
> - usb_set_interface(stream->dev->udev,
> - stream->intfnum, 0);
> - } else {
> - /* UVC doesn't specify how to inform a bulk-based device
> - * when the video stream is stopped. Windows sends a
> - * CLEAR_FEATURE(HALT) request to the video streaming
> - * bulk endpoint, mimic the same behaviour.
> - */
> - unsigned int epnum = stream->header.bEndpointAddress
> -& USB_ENDPOINT_NUMBER_MASK;
> - unsigned int dir = stream->header.bEndpointAddress
> - & USB_ENDPOINT_DIR_MASK;
> - unsigned int pipe;
> -
> - pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
> - usb_clear_halt(stream->dev->udev, pipe);
> - }
> -
> - uvc_video_clock_cleanup(stream);
> - return 0;
> - }
> -
> ret = uvc_video_clock_init(stream);
> if (ret < 0)
> return ret;
> @@ -2136,3 +2108,29 @@ int uvc_video_enable(struct uvc_streaming *stream,
> int enable)
>
> return ret;
> }
> +
> +int uvc_video_stop_streaming(struct uvc_streaming *stream)
> +{
> + uvc_uninit_video(stream, 1);
> + if (stream->intf->num_altsetting > 1) {
> + usb_set_interface(stream->dev->udev,
> + stream->intfnum, 0);
This now holds on a single line.
> + } else {
> + /* UVC doesn't specify how to inform a bulk-based device
Let's fix the checkpatch.pl warning here.
> + * when the video stream is stopped. Windows sends a
> + * CLEAR_FEATURE(HALT) request to the video streaming
> + * bulk endpoint, mimic the same behaviour.
> + */
> + unsigned int epnum = stream->header.bEndpointAddress
> +& USB_ENDPOINT_NUMBER_MASK;
> + unsigned int dir = stream->header.bEndpointAddress
> + & USB_ENDPOINT_DIR_MASK;
> + unsigned int pipe;
> +
> + pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
> + usb_clear_halt
Re: [PATCH v5 6/9] media: uvcvideo: Move decode processing to process context
Hi Kieran,
Thank you for the patch.
On Tuesday, 6 November 2018 23:27:17 EET Kieran Bingham wrote:
> From: Kieran Bingham
>
> Newer high definition cameras, and cameras with multiple lenses such as
> the range of stereo-vision cameras now available have ever increasing
> data rates.
>
> The inclusion of a variable length packet header in URB packets mean
> that we must memcpy the frame data out to our destination 'manually'.
> This can result in data rates of up to 2 gigabits per second being
> processed.
>
> To improve efficiency, and maximise throughput, handle the URB decode
> processing through a work queue to move it from interrupt context, and
> allow multiple processors to work on URBs in parallel.
>
> Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
I wonder if we shouldn't, as a future improvement, only queue async work when
the quantity of data to be copied is above a certain threshold.
> ---
> v2:
> - Lock full critical section of usb_submit_urb()
>
> v3:
> - Fix race on submitting uvc_video_decode_data_work() to work queue.
> - Rename uvc_decode_op -> uvc_copy_op (Generic to encode/decode)
> - Rename decodes -> copy_operations
> - Don't queue work if there is no async task
> - obtain copy op structure directly in uvc_video_decode_data()
> - uvc_video_decode_data_work() -> uvc_video_copy_data_work()
>
> v4:
> - Provide for_each_uvc_urb()
> - Simplify fix for shutdown race to flush queue before freeing URBs
> - Rebase to v4.16-rc4 (linux-media/master) adjusting for metadata
>conflicts.
>
> v5:
> - Rebase to media/v4.20-2
> - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
> - Fix function documentation for uvc_video_copy_data_work()
> - Add periods to the end of sentences
> - Rename 'decode' variable to 'op' in uvc_video_decode_data()
> - Move uvc_urb->async_operations initialisation to before use
> - Move async workqueue to match uvc_streaming lifetime instead of
>streamon/streamoff
>
> drivers/media/usb/uvc/uvc_driver.c | 2 +-
> drivers/media/usb/uvc/uvc_video.c | 110 +++---
> drivers/media/usb/uvc/uvcvideo.h | 28 -
> 3 files changed, 116 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index bc369a0934a3..e61a6d26e812
> 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -1883,6 +1883,8 @@ static void uvc_unregister_video(struct uvc_device
> *dev) video_unregister_device(>vdev);
> video_unregister_device(>meta.vdev);
>
> + destroy_workqueue(stream->async_wq);
> +
> uvc_debugfs_cleanup_stream(stream);
> }
> }
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 7a7779e1b466..ce9e40444507 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1094,21 +1094,54 @@ static int uvc_video_decode_start(struct
> uvc_streaming *stream, return data[0];
> }
>
> -static void uvc_video_decode_data(struct uvc_streaming *stream,
> +/*
> + * uvc_video_decode_data_work: Asynchronous memcpy processing
> + *
> + * Copy URB data to video buffers in process context, releasing buffer
> + * references and requeuing the URB when done.
> + */
> +static void uvc_video_copy_data_work(struct work_struct *work)
> +{
> + struct uvc_urb *uvc_urb = container_of(work, struct uvc_urb, work);
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; i < uvc_urb->async_operations; i++) {
> + struct uvc_copy_op *op = _urb->copy_operations[i];
> +
> + memcpy(op->dst, op->src, op->len);
> +
> + /* Release reference taken on this buffer. */
> + uvc_queue_buffer_release(op->buf);
> + }
> +
> + ret = usb_submit_urb(uvc_urb->urb, GFP_KERNEL);
> + if (ret < 0)
> + uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
> +ret);
> +}
> +
> +static void uvc_video_decode_data(struct uvc_urb *uvc_urb,
> struct uvc_buffer *buf, const u8 *data, int len)
> {
> - unsigned int maxlen, nbytes;
> - void *mem;
> + unsigned int active_op = uvc_urb->async_operations;
> + struct uvc_copy_op *op = _urb->copy_operations[active_op];
> + unsigned int maxlen;
>
> if (len <= 0)
> return;
>
> - /* Copy the video data to the buffer. */
> maxlen = buf->length - buf->bytesused;
> - mem = buf->mem + buf->bytesused;
> -
Re: [PATCH v5 5/9] media: uvcvideo: queue: Support asynchronous buffer handling
Hi Kieran,
Thank you for the patch.
On Tuesday, 6 November 2018 23:27:16 EET Kieran Bingham wrote:
> From: Kieran Bingham
>
> The buffer queue interface currently operates sequentially, processing
> buffers after they have fully completed.
>
> In preparation for supporting parallel tasks operating on the buffers,
> we will need to support buffers being processed on multiple CPUs.
>
> Adapt the uvc_queue_next_buffer() such that a reference count tracks the
> active use of the buffer, returning the buffer to the VB2 stack at
> completion.
>
> Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
>
> v5:
> - uvc_queue_requeue() -> uvc_queue_buffer_requeue()
> - Fix comment
> ---
> drivers/media/usb/uvc/uvc_queue.c | 61 ++--
> drivers/media/usb/uvc/uvcvideo.h | 4 ++-
> 2 files changed, 54 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c index bebf2415d9de..cd8c03341de0 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -142,6 +142,7 @@ static void uvc_buffer_queue(struct vb2_buffer *vb)
>
> spin_lock_irqsave(>irqlock, flags);
> if (likely(!(queue->flags & UVC_QUEUE_DISCONNECTED))) {
> + kref_init(>ref);
> list_add_tail(>queue, >irqqueue);
> } else {
> /* If the device is disconnected return the buffer to userspace
> @@ -459,28 +460,66 @@ struct uvc_buffer *uvc_queue_get_current_buffer(struct
> uvc_video_queue *queue) return nextbuf;
> }
>
> -struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
> +/*
> + * uvc_queue_buffer_requeue: Requeue a buffer on our internal irqqueue
> + *
> + * Reuse a buffer through our internal queue without the need to 'prepare'.
> + * The buffer will be returned to userspace through the uvc_buffer_queue
> call if + * the device has been disconnected.
> + */
> +static void uvc_queue_buffer_requeue(struct uvc_video_queue *queue,
> struct uvc_buffer *buf)
> {
> - struct uvc_buffer *nextbuf;
> - unsigned long flags;
> + buf->error = 0;
> + buf->state = UVC_BUF_STATE_QUEUED;
> + buf->bytesused = 0;
> + vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
> +
> + uvc_buffer_queue(>buf.vb2_buf);
> +}
> +
> +static void uvc_queue_buffer_complete(struct kref *ref)
> +{
> + struct uvc_buffer *buf = container_of(ref, struct uvc_buffer, ref);
> + struct vb2_buffer *vb = >buf.vb2_buf;
> + struct uvc_video_queue *queue = vb2_get_drv_priv(vb->vb2_queue);
>
> if ((queue->flags & UVC_QUEUE_DROP_CORRUPTED) && buf->error) {
> - buf->error = 0;
> - buf->state = UVC_BUF_STATE_QUEUED;
> - buf->bytesused = 0;
> - vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
> - return buf;
> + uvc_queue_buffer_requeue(queue, buf);
> + return;
> }
>
> + buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
> + vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
> + vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +/*
> + * Release a reference on the buffer. Complete the buffer when the last
> + * reference is released.
> + */
> +void uvc_queue_buffer_release(struct uvc_buffer *buf)
> +{
> + kref_put(>ref, uvc_queue_buffer_complete);
> +}
> +
> +/*
> + * Remove this buffer from the queue. Lifetime will persist while async
> actions + * are still running (if any), and uvc_queue_buffer_release will
> give the buffer + * back to VB2 when all users have completed.
> + */
> +struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
> + struct uvc_buffer *buf)
> +{
> + struct uvc_buffer *nextbuf;
> + unsigned long flags;
> +
> spin_lock_irqsave(>irqlock, flags);
> list_del(>queue);
> nextbuf = __uvc_queue_get_current_buffer(queue);
> spin_unlock_irqrestore(>irqlock, flags);
>
> - buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
> - vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
> - vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
> + uvc_queue_buffer_release(buf);
>
> return nextbuf;
> }
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index bdb6d8daedab..1bc17da7f3d4 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -410,6 +410,9 @@ struct uvc_buffer {
Re: [RFC] Create test script(s?) for regression testing
Hi Hans, On Tuesday, 6 November 2018 15:56:34 EET Hans Verkuil wrote: > On 11/06/18 14:12, Laurent Pinchart wrote: > > On Tuesday, 6 November 2018 13:36:55 EET Sakari Ailus wrote: > >> On Tue, Nov 06, 2018 at 09:37:07AM +0100, Hans Verkuil wrote: > >>> Hi all, > >>> > >>> After the media summit (heavy on test discussions) and the V4L2 event > >>> regression we just found it is clear we need to do a better job with > >>> testing. > >>> > >>> All the pieces are in place, so what is needed is to combine it and > >>> create a script that anyone of us as core developers can run to check > >>> for regressions. The same script can be run as part of the kernelci > >>> regression testing. > >> > >> I'd say that *some* pieces are in place. Of course, the more there is, > >> the better. > >> > >> The more there are tests, the more important it would be they're > >> automated, preferrably without the developer having to run them on his/ > >> her own machine. > > > > From my experience with testing, it's important to have both a core set of > > tests (a.k.a. smoke tests) that can easily be run on developers' machines, > > and extended tests that can be offloaded to a shared testing > > infrastructure (but possibly also run locally if desired). > > That was my idea as well for the longer term. First step is to do the basic > smoke tests (i.e. run compliance tests, do some (limited) streaming test). > > There are more extensive (and longer running) tests that can be done, but > that's something to look at later. > > >>> We have four virtual drivers: vivid, vim2m, vimc and vicodec. The last > >>> one is IMHO not quite good enough yet for testing: it is not fully > >>> compliant to the upcoming stateful codec spec. Work for that is planned > >>> as part of an Outreachy project. > >>> > >>> My idea is to create a script that is maintained as part of v4l-utils > >>> that loads the drivers and runs v4l2-compliance and possibly other tests > >>> against the virtual drivers. > >> > >> How about spending a little time to pick a suitable framework for running > >> the tests? It could be useful to get more informative reports than just > >> pass / fail. > > > > We should keep in mind that other tests will be added later, and the test > > framework should make that easy. > > Since we want to be able to run this on kernelci.org, I think it makes sense > to let the kernelci folks (Hi Ezequiel!) decide this. KernelCI isn't the only test infrastructure out there, so let's not forget about the other ones. > As a developer all I need is a script to run smoke tests so I can catch most > regressions (you never catch all). > > I'm happy to work with them to make any changes to compliance tools and > scripts so they fit better into their test framework. > > The one key requirement to all this is that you should be able to run these > tests without dependencies to all sorts of external packages/libraries. v4l-utils already has a set of dependencies, but those are largely manageable. For v4l2-compliance we'll install libv4l, which depends on libjpeg. > > Regarding the test output, many formats exist (see > > https://testanything.org/ and > > https://chromium.googlesource.com/chromium/src/+/master/docs/testing/ > > json_test_results_format.md for instance), we should pick one of the > > leading industry standards (what those standards are still needs to be > > researched :-)). > > > >> Do note that for different hardware the tests would be likely different > >> as well although there are classes of devices for which the exact same > >> tests would be applicable. > > > > See http://git.ideasonboard.com/renesas/vsp-tests.git for an example of > > device-specific tests. I think some of that could be generalized. > > > >>> It should be simple to use and require very little in the way of > >>> dependencies. Ideally no dependencies other than what is in v4l-utils so > >>> it can easily be run on an embedded system as well. > >>> > >>> For a 64-bit kernel it should run the tests both with 32-bit and 64-bit > >>> applications. > >>> > >>> It should also test with both single and multiplanar modes where > >>> available. > >>> > >>> Since vivid emulates CEC as well, it should run CEC tests too. > >>> > >>> As core developers we should have an environmen
Re: [RFC] Create test script(s?) for regression testing
Hello, On Tuesday, 6 November 2018 13:36:55 EET Sakari Ailus wrote: > On Tue, Nov 06, 2018 at 09:37:07AM +0100, Hans Verkuil wrote: > > Hi all, > > > > After the media summit (heavy on test discussions) and the V4L2 event > > regression we just found it is clear we need to do a better job with > > testing. > > > > All the pieces are in place, so what is needed is to combine it and create > > a script that anyone of us as core developers can run to check for > > regressions. The same script can be run as part of the kernelci > > regression testing. > > I'd say that *some* pieces are in place. Of course, the more there is, the > better. > > The more there are tests, the more important it would be they're automated, > preferrably without the developer having to run them on his/her own > machine. >From my experience with testing, it's important to have both a core set of tests (a.k.a. smoke tests) that can easily be run on developers' machines, and extended tests that can be offloaded to a shared testing infrastructure (but possibly also run locally if desired). > > We have four virtual drivers: vivid, vim2m, vimc and vicodec. The last one > > is IMHO not quite good enough yet for testing: it is not fully compliant > > to the upcoming stateful codec spec. Work for that is planned as part of > > an Outreachy project. > > > > My idea is to create a script that is maintained as part of v4l-utils that > > loads the drivers and runs v4l2-compliance and possibly other tests > > against the virtual drivers. > > How about spending a little time to pick a suitable framework for running > the tests? It could be useful to get more informative reports than just > pass / fail. We should keep in mind that other tests will be added later, and the test framework should make that easy. Regarding the test output, many formats exist (see https://testanything.org/ and https://chromium.googlesource.com/chromium/src/+/master/docs/testing/ json_test_results_format.md for instance), we should pick one of the leading industry standards (what those standards are still needs to be researched :-)). > Do note that for different hardware the tests would be likely different as > well although there are classes of devices for which the exact same tests > would be applicable. See http://git.ideasonboard.com/renesas/vsp-tests.git for an example of device-specific tests. I think some of that could be generalized. > > It should be simple to use and require very little in the way of > > dependencies. Ideally no dependencies other than what is in v4l-utils so > > it can easily be run on an embedded system as well. > > > > For a 64-bit kernel it should run the tests both with 32-bit and 64-bit > > applications. > > > > It should also test with both single and multiplanar modes where > > available. > > > > Since vivid emulates CEC as well, it should run CEC tests too. > > > > As core developers we should have an environment where we can easily test > > our patches with this script (I use a VM for that). > > > > I think maintaining the script (or perhaps scripts) in v4l-utils is best > > since that keeps it in sync with the latest kernel and v4l-utils > > developments. > > Makes sense --- and that can be always changed later on if there's a need > to. I wonder whether that would be best going forward, especially if we want to add more tests. Wouldn't a v4l-tests project make sense ? -- Regards, Laurent Pinchart
Re: [PATCH v2 2/2] [media] CNF4 pixel format for media subsystem
Hi Sergey,
Thank you for the patch.
As for patch 1/2, the subject line needs a prefix. Furthermore it doesn't
really describe the patch. I propose writing it as
media: uvcvideo: Add support for the CNF4 format
On Wednesday, 12 September 2018 09:42:07 EET dorod...@gmail.com wrote:
> From: Sergey Dorodnicov
>
> Registering new GUID used by Intel RealSense cameras with fourcc CNF4,
> encoding depth sensor confidence information for every pixel.
And there I would write "Register the GUID ...".
Apart from that the patch looks good to me,
Reviewed-by: Laurent Pinchart
If you're fine with the subject line change there's no need to resubmit, I'll
fix it when applying the patch to my tree.
> Signed-off-by: Sergey Dorodnicov
> Signed-off-by: Evgeni Raikhel
> ---
> drivers/media/usb/uvc/uvc_driver.c | 5 +
> drivers/media/usb/uvc/uvcvideo.h | 3 +++
> 2 files changed, 8 insertions(+)
>
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index d46dc43..19f129f 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -214,6 +214,11 @@ static struct uvc_format_desc uvc_fmts[] = {
> .guid = UVC_GUID_FORMAT_INZI,
> .fcc= V4L2_PIX_FMT_INZI,
> },
> + {
> + .name = "4-bit Depth Confidence (Packed)",
> + .guid = UVC_GUID_FORMAT_CNF4,
> + .fcc= V4L2_PIX_FMT_CNF4,
> + },
> };
>
> /*
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index e5f5d84..779bab2 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -154,6 +154,9 @@
> #define UVC_GUID_FORMAT_INVI \
> { 'I', 'N', 'V', 'I', 0xdb, 0x57, 0x49, 0x5e, \
>0x8e, 0x3f, 0xf4, 0x79, 0x53, 0x2b, 0x94, 0x6f}
> +#define UVC_GUID_FORMAT_CNF4 \
> + { 'C', ' ', ' ', ' ', 0x00, 0x00, 0x10, 0x00, \
> + 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
>
> #define UVC_GUID_FORMAT_D3DFMT_L8 \
> {0x32, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, \
--
Regards,
Laurent Pinchart
Re: [PATCH v2 1/2] [media] CNF4 fourcc for 4 bit-per-pixel packed depth confidence information
Hi Sergey,
Thank you for the patch.
The subject line should start with an appropriate prefix. I propose rewriting
it as
media: v4l: Add 4bpp packed depth confidence format CNF4
Apart from that the patch looks good to me,
Reviewed-by: Laurent Pinchart
If you're fine with the subject line change there's no need to resubmit, I'll
fix it when applying the patch to my tree.
On Wednesday, 12 September 2018 09:42:06 EET dorod...@gmail.com wrote:
> From: Sergey Dorodnicov
>
> Adding new fourcc CNF4 for 4 bit-per-pixel packed depth confidence
> information provided by Intel RealSense cameras. Every two consecutive
> pixels are packed into a single byte.
>
> Signed-off-by: Sergey Dorodnicov
> Signed-off-by: Evgeni Raikhel
> ---
> Documentation/media/uapi/v4l/depth-formats.rst | 1 +
> Documentation/media/uapi/v4l/pixfmt-cnf4.rst | 31 +++
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/uapi/linux/videodev2.h | 1 +
> 4 files changed, 34 insertions(+)
> create mode 100644 Documentation/media/uapi/v4l/pixfmt-cnf4.rst
>
> diff --git a/Documentation/media/uapi/v4l/depth-formats.rst
> b/Documentation/media/uapi/v4l/depth-formats.rst index d1641e9..9533348
> 100644
> --- a/Documentation/media/uapi/v4l/depth-formats.rst
> +++ b/Documentation/media/uapi/v4l/depth-formats.rst
> @@ -14,3 +14,4 @@ Depth data provides distance to points, mapped onto the
> image plane
>
> pixfmt-inzi
> pixfmt-z16
> +pixfmt-cnf4
> diff --git a/Documentation/media/uapi/v4l/pixfmt-cnf4.rst
> b/Documentation/media/uapi/v4l/pixfmt-cnf4.rst new file mode 100644
> index 000..8f46929
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-cnf4.rst
> @@ -0,0 +1,31 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _V4L2-PIX-FMT-CNF4:
> +
> +**
> +V4L2_PIX_FMT_CNF4 ('CNF4')
> +**
> +
> +Depth sensor confidence information as a 4 bits per pixel packed array
> +
> +Description
> +===
> +
> +Proprietary format used by Intel RealSense Depth cameras containing depth
> +confidence information in range 0-15 with 0 indicating that the sensor was
> +unable to resolve any signal and 15 indicating maximum level of confidence
> for +the specific sensor (actual error margins might change from sensor to
> sensor). +
> +Every two consecutive pixels are packed into a single byte.
> +Bits 0-3 of byte n refer to confidence value of depth pixel 2*n,
> +bits 4-7 to confidence value of depth pixel 2*n+1.
> +
> +**Bit-packed representation.**
> +
> +.. flat-table::
> +:header-rows: 0
> +:stub-columns: 0
> +:widths: 64 64
> +
> +* - Y'\ :sub:`01[3:0]`\ (bits 7--4) Y'\ :sub:`00[3:0]`\ (bits 3--0)
> + - Y'\ :sub:`03[3:0]`\ (bits 7--4) Y'\ :sub:`02[3:0]`\ (bits 3--0)
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c index 54afc9c..f9aa8bd 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1189,6 +1189,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_PIX_FMT_Y12I: descr = "Interleaved 12-bit Greyscale";
> break;
> case V4L2_PIX_FMT_Z16:descr = "16-bit Depth"; break;
> case V4L2_PIX_FMT_INZI: descr = "Planar 10:16 Greyscale Depth";
> break;
> + case V4L2_PIX_FMT_CNF4: descr = "4-bit Depth Confidence
> (Packed)";
break;
> case V4L2_PIX_FMT_PAL8: descr = "8-bit Palette"; break;
> case V4L2_PIX_FMT_UV8: descr = "8-bit Chrominance UV 4-4";
> break;
> case V4L2_PIX_FMT_YVU410: descr = "Planar YVU 4:1:0"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 622f047..2837c93 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -676,6 +676,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
> 16-bit */ #define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /*
> Mediatek compressed block mode */ #define V4L2_PIX_FMT_INZI
> v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth
> 16-bit */ +#define V4L2_PIX_FMT_CNF4 v4l2_fourcc('C', 'N', 'F', '4') /*
> Intel 4-bit packed depth confidence information */
>
> /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits
> unused */ #define V4L2_PIX_FMT_IPU3_SBGGR10 v4l2_fourcc('i', 'p', '3',
'b')
> /* IPU3 packed 10-bit BGGR bayer */
--
Regards,
Laurent Pinchart
Re: [PATCH v3 2/9] media: tvp5150: fix irq_request error path during probe
Hi Marco,
Thank you for the patch.
On Tuesday, 18 September 2018 16:14:46 EEST Marco Felsch wrote:
> Commit 37c65802e76a ("media: tvp5150: Add sync lock interrupt handling")
> introduced the interrupt handling. But we have to free the
> v4l2_ctrl_handler before we can return the error code.
>
> Fixes: 37c65802e76a ("media: tvp5150: Add sync lock interrupt handling")
> Signed-off-by: Marco Felsch
Reviewed-by: Laurent Pinchart
> ---
> drivers/media/i2c/tvp5150.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/media/i2c/tvp5150.c b/drivers/media/i2c/tvp5150.c
> index 133073518744..40aaa8ca0b63 100644
> --- a/drivers/media/i2c/tvp5150.c
> +++ b/drivers/media/i2c/tvp5150.c
> @@ -1650,7 +1650,7 @@ static int tvp5150_probe(struct i2c_client *c,
> tvp5150_isr, IRQF_TRIGGER_HIGH |
> IRQF_ONESHOT, "tvp5150", core);
> if (res)
> - return res;
> + goto err;
> }
>
> res = v4l2_async_register_subdev(sd);
--
Regards,
Laurent Pinchart
Re: uvcvideo: IR camera lights only every second frame
Hello,
On Tuesday, 30 October 2018 17:48:12 EET Kieran Bingham wrote:
> On 30/10/2018 14:36, Jiri Slaby wrote:
> > Hi,
> >
> > I have a Dell Lattitude 7280 with two webcams. The standard one works
> > fine (/dev/video0). The other one is an IR camera (/dev/video1). The
> > camera proper works fine and produces 340x374 frames. But there is an IR
> > led supposed to light the object. The video is 30fps, but the LED seems
> > to emit light only on half of the frames, i.e. on every second frame (15
> > fps). This makes the video blink a lot. The two consecutive frames look
> > like:
> > https://www.fi.muni.cz/~xslaby/sklad/mpv-shot0002.jpg
> > https://www.fi.muni.cz/~xslaby/sklad/mpv-shot0003.jpg
> >
> > Do you have any ideas what to check/test?
>
> I have an HP Spectre with IR camera, and it also 'flashes' alternate frames.
>
> I assumed this was something to do with controlling the lighting for
> face recognition some how.
>
> I'm fairly sure we don't control the 'IR flash' from the UVC.
>
> I wonder if there is a control parameter for the IR led in the
> extension-units?
[snip]
> > VideoControl Interface Descriptor:
> > bLength18
> > bDescriptorType36
> > bDescriptorSubtype 2 (INPUT_TERMINAL)
> > bTerminalID11
> > wTerminalType 0x0201 Camera Sensor
> > bAssocTerminal 0
> > iTerminal 0
> > wObjectiveFocalLengthMin 0
> > wObjectiveFocalLengthMax 0
> > wOcularFocalLength0
> > bControlSize 3
> > bmControls 0x
> > VideoControl Interface Descriptor:
> > bLength11
> > bDescriptorType36
> > bDescriptorSubtype 5 (PROCESSING_UNIT)
> > Warning: Descriptor too short
> > bUnitID 9
> > bSourceID 11
> > wMaxMultiplier 0
> > bControlSize2
> > bmControls 0x
> > iProcessing 0
> > bmVideoStandards 0x09
> > None
> > SECAM - 625/50
> > VideoControl Interface Descriptor:
> > bLength 9
> > bDescriptorType36
> > bDescriptorSubtype 3 (OUTPUT_TERMINAL)
> > bTerminalID 8
> > wTerminalType 0x0101 USB Streaming
> > bAssocTerminal 0
> > bSourceID 10
> > iTerminal 0
> > VideoControl Interface Descriptor:
> > bLength25
> > bDescriptorType36
> > bDescriptorSubtype 6 (EXTENSION_UNIT)
> > bUnitID12
> > guidExtensionCode {45b5da73-23c1-4a3d-a368-610f078c4397}
> > bNumControl 0
> > bNrPins 1
> > baSourceID( 0) 9
> > bControlSize0
> > iExtension 0
This extension unit is strange, it exposes no control.
> > VideoControl Interface Descriptor:
> > bLength27
> > bDescriptorType36
> > bDescriptorSubtype 6 (EXTENSION_UNIT)
> > bUnitID10
> > guidExtensionCode {1229a78c-47b4-4094-b0ce-db07386fb938}
> > bNumControl 2
> > bNrPins 1
> > baSourceID( 0) 12
> > bControlSize2
> > bmControls( 0) 0x00
> > bmControls( 1) 0x06
> > iExtension 0
This one exposes two controls, which are likely used to control the IR light.
I however suspect that the controls merely expose an indirect way to read/
write internal registers, so we would really need to capture a USB trace when
using the device in Windows (assuming that the machine is shipped with
software that can control the IR light).
[snip]
--
Regards,
Laurent Pinchart
Re: [RFP] Which V4L2 ioctls could be replaced by better versions?
Hi Tomasz, On Friday, 26 October 2018 14:41:26 EEST Tomasz Figa wrote: > On Thu, Sep 20, 2018 at 11:42 PM Hans Verkuil wrote: > > Some parts of the V4L2 API are awkward to use and I think it would be > > a good idea to look at possible candidates for that. > > > > Examples are the ioctls that use struct v4l2_buffer: the multiplanar > > support is really horrible, and writing code to support both single and > > multiplanar is hard. We are also running out of fields and the timeval > > isn't y2038 compliant. > > > > A proof-of-concept is here: > > > > https://git.linuxtv.org/hverkuil/media_tree.git/commit/?h=v4l2-buffer=a > > 95549df06d9900f3559afdbb9da06bd4b22d1f3 > > > > It's a bit old, but it gives a good impression of what I have in mind. > > On a related, but slightly different note, I'm wondering how we should > handle a case where we have an M format (e.g. NV12M with 2 memory > planes), but only 1 DMA-buf with all planes to import. That generally > means that we have to use the same DMA-buf FD with an offset for each > plane. In theory, v4l2_plane::data_offset could be used for this, but > the documentation says that it should be set by the application only > for OUTPUT planes. Moreover, existing drivers tend to just ignore > it... The following patches may be of interest. https://patchwork.linuxtv.org/patch/29177/ https://patchwork.linuxtv.org/patch/29178/ > There is also the opposite problem. Sometimes the application is given > 3 different FDs but pointing to the same buffer. If it has to work > with a video device that only supports non-M formats, it can either > fail, making it unusable, or blindly assume that they all point to the > same buffer and just give the first FD to the video device (we do it > in Chromium, since our allocator is guaranteed to keep all planes of > planar formats in one buffer, if to be used with V4L2). > > Something that we could do is allowing the QBUF semantics of M formats > for non-M formats, where the application would fill the planes[] array > for all planes with all the FDs it has and the kernel could then > figure out if they point to the same buffer (i.e. resolve to the same > dma_buf struct) or fail if not. > > [...] > > > Do we have more ioctls that could use a refresh? S/G/TRY_FMT perhaps, > > again in order to improve single vs multiplanar handling. > > I'd definitely be more than happy to see the plane handling unified > between non-M and M formats, in general. The list of problems with > current interface: > > 1) The userspace has to hardcode the computations of bytesperline for > chroma planes of non-M formats (while they are reported for M > formats). > > 2) Similarly, offsets of the planes in the buffer for non-M formats > must be explicitly calculated in the application, > > 3) Drivers have to explicitly handle both non-M and M formats or > otherwise they would suffer from issues with application compatibility > or sharing buffers with other devices (one supporting only M and the > other only non-M), > > 4) Inconsistency in the meaning of planes[0].sizeimage for non-M > formats and M formats, making it impossible to use planes[0].sizeimage > to set the luma plane size in the hardware for non-M formats (since > it's the total size of all planes). > > I might have probably forgotten about something, but generally fixing > the 4 above, would be a really big step forward. -- Regards, Laurent Pinchart
Re: [RFP] Which V4L2 ioctls could be replaced by better versions?
Hi Hans, On Wednesday, 17 October 2018 12:16:14 EEST Hans Verkuil wrote: > On 10/17/2018 10:57 AM, Laurent Pinchart wrote: > > On Thursday, 20 September 2018 17:42:15 EEST Hans Verkuil wrote: > >> Some parts of the V4L2 API are awkward to use and I think it would be > >> a good idea to look at possible candidates for that. > >> > >> Examples are the ioctls that use struct v4l2_buffer: the multiplanar > >> support is really horrible, and writing code to support both single and > >> multiplanar is hard. We are also running out of fields and the timeval > >> isn't y2038 compliant. > >> > >> A proof-of-concept is here: > >> > >> https://git.linuxtv.org/hverkuil/media_tree.git/commit/?h=v4l2-buffer= > >> a95 549df06d9900f3559afdbb9da06bd4b22d1f3 > >> > >> It's a bit old, but it gives a good impression of what I have in mind. > >> > >> Another candidate is > >> VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL/VIDIOC_ENUM_FRAMEINTERVALS: expressing > >> frame intervals as a fraction is really awkward and so is the fact that > >> the subdev and 'normal' ioctls are not the same. > >> > >> Would using nanoseconds or something along those lines for intervals be > >> better? > >> > >> I have similar concerns with VIDIOC_SUBDEV_ENUM_FRAME_SIZE where there is > >> no stepwise option, making it different from VIDIOC_ENUM_FRAMESIZES. But > >> it should be possible to extend VIDIOC_SUBDEV_ENUM_FRAME_SIZE with > >> stepwise support, I think. > > > > If we refresh the enumeration ioctls, I propose moving away from the one > > syscall per value model, and returning everything in one > > (userspace-allocated) buffer. The same could apply to all enumerations > > (such as controls for instance), even if we don't address them all in one > > go. > > I'm not convinced about this, primarily because I think these enums are done > at configuration time, and rarely if ever while streaming. So does it > really make a difference in practice? Feedback on this would be welcome > during the summit meeting. It's indeed not a hot path in most cases, but if you enumerate formats, frame sizes and frame intervals, you end up with three nested loops with lots of ioctl calls. This makes the code on the userspace side more complex than it should be, and incurs an overhead. If we rework the enumeration ioctls for other reasons, I believe we should fix this as wel. > >> Do we have more ioctls that could use a refresh? S/G/TRY_FMT perhaps, > >> again in order to improve single vs multiplanar handling. > > > > If we refresh the G/S/TRY format ioctls (and I think we should), I would > > propose moving to a G/S model with ACTIVE and TRY formats, as for subdevs. > > This should make it easier to construct full device states internally, in > > order to implement proper request API support for formats. We should then > > also document much better how formats and selection rectangles interact. > > Interesting. I was planning a slide for this. > > >> It is not the intention to come to a full design, it's more to test the > >> waters so to speak. > > > > Another item that we're missing is a way to delete buffers (the > > counterpart of VIDIOC_CREATE_BUFS). As this will introduce holes in the > > buffer indices, we might also need to revamp VIDIOC_CREATE_BUFS (which > > would give us a chance to move away from the format structure passed to > > that ioctl). > > I'm just writing the slide for that :-) Thanks :-) -- Regards, Laurent Pinchart
Re: [RFP] Which V4L2 ioctls could be replaced by better versions?
Hi Hans, On Thursday, 20 September 2018 17:42:15 EEST Hans Verkuil wrote: > Some parts of the V4L2 API are awkward to use and I think it would be > a good idea to look at possible candidates for that. > > Examples are the ioctls that use struct v4l2_buffer: the multiplanar support > is really horrible, and writing code to support both single and multiplanar > is hard. We are also running out of fields and the timeval isn't y2038 > compliant. > > A proof-of-concept is here: > > https://git.linuxtv.org/hverkuil/media_tree.git/commit/?h=v4l2-buffer=a95 > 549df06d9900f3559afdbb9da06bd4b22d1f3 > > It's a bit old, but it gives a good impression of what I have in mind. > > Another candidate is > VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL/VIDIOC_ENUM_FRAMEINTERVALS: expressing > frame intervals as a fraction is really awkward and so is the fact that the > subdev and 'normal' ioctls are not the same. > > Would using nanoseconds or something along those lines for intervals be > better? > > I have similar concerns with VIDIOC_SUBDEV_ENUM_FRAME_SIZE where there is no > stepwise option, making it different from VIDIOC_ENUM_FRAMESIZES. But it > should be possible to extend VIDIOC_SUBDEV_ENUM_FRAME_SIZE with stepwise > support, I think. If we refresh the enumeration ioctls, I propose moving away from the one syscall per value model, and returning everything in one (userspace-allocated) buffer. The same could apply to all enumerations (such as controls for instance), even if we don't address them all in one go. > Do we have more ioctls that could use a refresh? S/G/TRY_FMT perhaps, again > in order to improve single vs multiplanar handling. If we refresh the G/S/TRY format ioctls (and I think we should), I would propose moving to a G/S model with ACTIVE and TRY formats, as for subdevs. This should make it easier to construct full device states internally, in order to implement proper request API support for formats. We should then also document much better how formats and selection rectangles interact. > It is not the intention to come to a full design, it's more to test the > waters so to speak. Another item that we're missing is a way to delete buffers (the counterpart of VIDIOC_CREATE_BUFS). As this will introduce holes in the buffer indices, we might also need to revamp VIDIOC_CREATE_BUFS (which would give us a chance to move away from the format structure passed to that ioctl). -- Regards, Laurent Pinchart
Re: [PATCH 1/4] media: ov5640: fix resolution update
Hi Hugues,
On Monday, 15 October 2018 18:13:12 EEST Hugues FRUCHET wrote:
> Hi Laurent, Jacopo, Sam,
>
> I'm also OK to change to a simpler alternative;
> - drop the "restore" step
> - send the whole init register sequence + mode changes + format changes
> at streamon
>
> is this what you have in mind Laurent ?
Yes, that's pretty much the idea. The init sequence could be sent when
powering the sensor on to save time at streamon. Everything else can be
programmed at streamon time to simplify the implementation.
> On 10/10/2018 02:41 PM, Laurent Pinchart wrote:
> > On Wednesday, 10 October 2018 13:58:04 EEST jacopo mondi wrote:
> >
> >> Hi Sam,
> >>
> >> thanks for the patch, I see the same issue you reported, but I
> >> think this patch can be improved.
> >>
> >> (expanding the Cc list to all people involved in recent ov5640
> >> developemts, not just for this patch, but for the whole series to look
> >> at. Copying names from another series cover letter, hope it is
> >> complete.)
> >>
> >> On Mon, Oct 08, 2018 at 11:47:59PM -0700, Sam Bobrowicz wrote:
> >>
> >>> set_fmt was not properly triggering a mode change when
> >>> a new mode was set that happened to have the same format
> >>> as the previous mode (for example, when only changing the
> >>> frame dimensions). Fix this.
> >>>
> >>> Signed-off-by: Sam Bobrowicz
> >>> ---
> >>>
> >>> drivers/media/i2c/ov5640.c | 8
> >>> 1 file changed, 4 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> >>> index eaefdb5..5031aab 100644
> >>> --- a/drivers/media/i2c/ov5640.c
> >>> +++ b/drivers/media/i2c/ov5640.c
> >>> @@ -2045,12 +2045,12 @@ static int ov5640_set_fmt(struct v4l2_subdev
> >>> *sd,
> >>> goto out;
> >>> }
> >>>
> >>> - if (new_mode != sensor->current_mode) {
> >>> +
> >>> + if (new_mode != sensor->current_mode ||
> >>> + mbus_fmt->code != sensor->fmt.code) {
> >>> + sensor->fmt = *mbus_fmt;
> >>> sensor->current_mode = new_mode;
> >>> sensor->pending_mode_change = true;
> >>> - }
> >>> - if (mbus_fmt->code != sensor->fmt.code) {
> >>> - sensor->fmt = *mbus_fmt;
> >>> sensor->pending_fmt_change = true;
> >>> }
> >>
> >> How I did reproduce the issue:
> >>
> >> # Set 1024x768 on ov5640 without changing the image format
> >> # (default image size at startup is 640x480)
> >> $ media-ctl --set-v4l2 "'ov5640 2-003c':0[fmt:UYVY2X8/1024x768
> >> field:none]"
> >>sensor->pending_mode_change = true; //verified this flag gets set
> >>
> >> # Start streaming, after having configured the whole pipeline to work
> >> # with 1024x768
> >> $ yavta -c10 -n4 -f UYVY -s 1024x768 /dev/video4
> >> Unable to start streaming: Broken pipe (32).
> >>
> >> # Inspect which part of pipeline validation went wrong
> >> # Turns out the sensor->fmt field is not updated, and when get_fmt()
> >> # is called, the old one is returned.
> >> $ media-ctl -e "ov5640 2-003c" -p
> >>...
> >>[fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb
> >>ycbcr:601 quantization:full-range] ^^^ ^^^
> >>
> >> So yes, sensor->fmt is not udapted as it should be when only image
> >> resolution is changed.
> >>
> >> Although I still see value in having two separate flags for the
> >> 'mode_change' (which in ov5640 lingo is resolution) and 'fmt_change'
> >> (which in ov5640 lingo is the image format), and write their
> >> configuration to registers only when they get actually changed.
> >>
> >> For this reasons I would like to propse the following patch which I
> >> have tested by:
> >> 1) changing resolution only
> >> 2) changing format only
> >> 3) change both
> >>
> >> What do you and others think?
> >
> > I think that the format setting code should be completely rewritten, it's
> > pretty much unmaintainable as-is.
> >
> >> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> >> index eaefdb5..e392
Re: [PATCH 1/4] media: ov5640: fix resolution update
Hi Jacopo,
On Wednesday, 10 October 2018 13:58:04 EEST jacopo mondi wrote:
> Hi Sam,
>thanks for the patch, I see the same issue you reported, but I
> think this patch can be improved.
>
> (expanding the Cc list to all people involved in recent ov5640
> developemts, not just for this patch, but for the whole series to look
> at. Copying names from another series cover letter, hope it is
> complete.)
>
> On Mon, Oct 08, 2018 at 11:47:59PM -0700, Sam Bobrowicz wrote:
> > set_fmt was not properly triggering a mode change when
> > a new mode was set that happened to have the same format
> > as the previous mode (for example, when only changing the
> > frame dimensions). Fix this.
> >
> > Signed-off-by: Sam Bobrowicz
> > ---
> >
> > drivers/media/i2c/ov5640.c | 8
> > 1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> > index eaefdb5..5031aab 100644
> > --- a/drivers/media/i2c/ov5640.c
> > +++ b/drivers/media/i2c/ov5640.c
> > @@ -2045,12 +2045,12 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> >
> > goto out;
> >
> > }
> >
> > - if (new_mode != sensor->current_mode) {
> > +
> > + if (new_mode != sensor->current_mode ||
> > + mbus_fmt->code != sensor->fmt.code) {
> > + sensor->fmt = *mbus_fmt;
> >
> > sensor->current_mode = new_mode;
> > sensor->pending_mode_change = true;
> >
> > - }
> > - if (mbus_fmt->code != sensor->fmt.code) {
> > - sensor->fmt = *mbus_fmt;
> >
> > sensor->pending_fmt_change = true;
> >
> > }
>
> How I did reproduce the issue:
>
> # Set 1024x768 on ov5640 without changing the image format
> # (default image size at startup is 640x480)
> $ media-ctl --set-v4l2 "'ov5640 2-003c':0[fmt:UYVY2X8/1024x768 field:none]"
> sensor->pending_mode_change = true; //verified this flag gets set
>
> # Start streaming, after having configured the whole pipeline to work
> # with 1024x768
> $ yavta -c10 -n4 -f UYVY -s 1024x768 /dev/video4
>Unable to start streaming: Broken pipe (32).
>
> # Inspect which part of pipeline validation went wrong
> # Turns out the sensor->fmt field is not updated, and when get_fmt()
> # is called, the old one is returned.
> $ media-ctl -e "ov5640 2-003c" -p
> ...
> [fmt:UYVY8_2X8/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601
> quantization:full-range] ^^^ ^^^
>
> So yes, sensor->fmt is not udapted as it should be when only image
> resolution is changed.
>
> Although I still see value in having two separate flags for the
> 'mode_change' (which in ov5640 lingo is resolution) and 'fmt_change' (which
> in ov5640 lingo is the image format), and write their configuration to
> registers only when they get actually changed.
>
> For this reasons I would like to propse the following patch which I
> have tested by:
> 1) changing resolution only
> 2) changing format only
> 3) change both
>
> What do you and others think?
I think that the format setting code should be completely rewritten, it's
pretty much unmaintainable as-is.
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index eaefdb5..e392b9d 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -2020,6 +2020,7 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> struct ov5640_dev *sensor = to_ov5640_dev(sd);
> const struct ov5640_mode_info *new_mode;
> struct v4l2_mbus_framefmt *mbus_fmt = >format;
> + struct v4l2_mbus_framefmt *fmt;
> int ret;
>
> if (format->pad != 0)
> @@ -2037,22 +2038,19 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
> if (ret)
> goto out;
>
> - if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
> - struct v4l2_mbus_framefmt *fmt =
> - v4l2_subdev_get_try_format(sd, cfg, 0);
> + if (format->which == V4L2_SUBDEV_FORMAT_TRY)
> + fmt = v4l2_subdev_get_try_format(sd, cfg, 0);
> + else
> + fmt = >fmt;
>
> - *fmt = *mbus_fmt;
> - goto out;
> - }
> + *fmt = *mbus_fmt;
>
> if (new_mode != sensor->current_mode) {
> sensor->current_mode = new_mode;
> sensor->pending_mode_change = true;
> }
> - if (mbus_fmt->code != sensor->fmt.code) {
> - sensor->fmt = *mbus_fmt;
> + if (mbus_fmt->code != sensor->fmt.code)
> sensor->pending_fmt_change = true;
> - }
> out:
> mutex_unlock(>lock);
> return ret;
>
> > out:
> > --
> > 2.7.4
--
Regards,
Laurent Pinchart
Re: [RFC] Informal meeting during ELCE to discuss userspace support for stateless codecs
Hi Hans, On Monday, 8 October 2018 14:53:29 EEST Hans Verkuil wrote: > Hi all, > > I would like to meet up somewhere during the ELCE to discuss userspace > support for stateless (and perhaps stateful as well?) codecs. > > It is also planned as a topic during the summit, but I would prefer to > prepare for that in advance, esp. since I myself do not have any experience > writing userspace SW for such devices. > > Nicolas, it would be really great if you can participate in this meeting > since you probably have the most experience with this by far. > > Looking through the ELCE program I found two timeslots that are likely to > work for most of us (because the topics in the program appear to be boring > for us media types!): > > Tuesday from 10:50-15:50 > > or: > > Monday from 15:45 onward On Tuesday I would prefer 12:20 onward. Monday works for me. > My guess is that we need 2-3 hours or so. Hard to predict. > > The basic question that I would like to have answered is what the userspace > component should look like? libv4l-like plugin or a library that userspace > can link with? Do we want more general support for stateful codecs as well > that deals with resolution changes and the more complex parts of the codec > API? > > I've mailed this directly to those that I expect are most interested in > this, but if someone want to join in let me know. > > I want to keep the group small though, so you need to bring relevant > experience to the table. -- Regards, Laurent Pinchart
Re: [PATCH 1/1] omap3isp: Unregister media device as first
Hi Sakari,
Thank you for the patch.
On Tuesday, 9 October 2018 15:03:16 EEST Sakari Ailus wrote:
> While there are issues related to object lifetime management, unregister the
> media device first when the driver is being unbound. This is slightly
> safer.
>
> Signed-off-by: Sakari Ailus
Reviewed-by: Laurent Pinchart
> ---
> drivers/media/platform/omap3isp/isp.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/omap3isp/isp.c
> b/drivers/media/platform/omap3isp/isp.c index 93f032a39470..4194ea82e6c4
> 100644
> --- a/drivers/media/platform/omap3isp/isp.c
> +++ b/drivers/media/platform/omap3isp/isp.c
> @@ -1587,6 +1587,8 @@ static void isp_pm_complete(struct device *dev)
>
> static void isp_unregister_entities(struct isp_device *isp)
> {
> + media_device_unregister(>media_dev);
> +
> omap3isp_csi2_unregister_entities(>isp_csi2a);
> omap3isp_ccp2_unregister_entities(>isp_ccp2);
> omap3isp_ccdc_unregister_entities(>isp_ccdc);
> @@ -1597,7 +1599,6 @@ static void isp_unregister_entities(struct isp_device
> *isp) omap3isp_stat_unregister_entities(>isp_hist);
>
> v4l2_device_unregister(>v4l2_dev);
> - media_device_unregister(>media_dev);
> media_device_cleanup(>media_dev);
> }
--
Regards,
Laurent Pinchart
Re: [PATCH] MAINTAINERS: Remove stale file entry for the Atmel ISI driver
Hi Mauro,
On Thursday, 4 October 2018 21:45:05 EEST Mauro Carvalho Chehab wrote:
> Em Tue, 2 Oct 2018 08:35:47 +0200 Ludovic Desroches escreveu:
> > On Mon, Oct 01, 2018 at 01:51:01PM -0300, Mauro Carvalho Chehab wrote:
> > > Em Sun, 30 Sep 2018 02:40:35 -0700 Joe Perches escreveu:
> > > > On Sun, 2018-09-30 at 06:30 -0300, Mauro Carvalho Chehab wrote:
> > > > > Em Sun, 30 Sep 2018 09:54:48 +0300 Laurent Pinchart escreveu:
> > > > > > include/media/atmel-isi got removed three years ago without the
> > > > > > MAINTAINERS file being updated. Remove the stale entry.
> > > > > >
> > > > > > Fixes: 40a78f36fc92 ("[media] v4l: atmel-isi: Remove support for
> > > > > > platform data") Reported-by: Joe Perches
> > > > > > Signed-off-by: Laurent Pinchart
> > > > > >
> > > > > > ---
> > > > > >
> > > > > > MAINTAINERS | 1 -
> > > > > > 1 file changed, 1 deletion(-)
> > > > > >
> > > > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > >
> > > > []
> > > >
> > > > > > @@ -2497,7 +2497,6 @@ M:Ludovic Desroches
> > > > > > > > > > >
> > > > > > L: linux-media@vger.kernel.org
> > > > > > S: Supported
> > > > > > F: drivers/media/platform/atmel/atmel-isi.c
> > > > > >
> > > > > > -F: include/media/atmel-isi.h
> > > > >
> > > > > I guess the right fix would be to replace it by:
> > > > >
> > > > > F: drivers/media/platform/atmel/atmel-isi.h
> > > >
> > > > Or replace both F entries with:
> > > >
> > > > F: drivers/media/platform/atmel/atmel-isi.*
> > > >
> > > > Or combine the 2 MICROCHIP sections into one
> > > >
> > > > MICROCHIP ISC DRIVER
> > > > M: Eugen Hristev
> > > > L: linux-media@vger.kernel.org
> > > > S: Supported
> > > > F: drivers/media/platform/atmel/atmel-isc.c
> > > > F: drivers/media/platform/atmel/atmel-isc-regs.h
> > > > F: devicetree/bindings/media/atmel-isc.txt
> > > >
> > > > MICROCHIP ISI DRIVER
> > > > M: Eugen Hristev
> > > > L: linux-media@vger.kernel.org
> > > > S: Supported
> > > > F: drivers/media/platform/atmel/atmel-isi.c
> > > > F: include/media/atmel-isi.h
> > > >
> > > > and maybe use something like:
> > > >
> > > > MICROCHIP MEDIA DRIVERS
> > > > M: Eugen Hristev
> > > > L:
> > > > linux-media@vger.kernel.org
> > > > S: Supported
> > > > F: drivers/media/platform/atmel/
> > > > F: devicetree/bindings/media/atmel-isc.txt
> > >
> > > Yeah, combining both of them seems a good alternative to me.
> > >
> > > Eugen/Ludovic/Josh,
> > >
> > > Comments?
> >
> > I have no strong opinion about it. The devices are different but usually
> > there is one person per topic so combining them makes sense.
>
> Hmm... At media tree, currently, MAINTAINERS entry is different:
>
> MICROCHIP / ATMEL ISC DRIVER
> M:Songjun Wu
> L:linux-media@vger.kernel.org
> S: Supported
> F: drivers/media/platform/atmel/atmel-isc.c
> F: drivers/media/platform/atmel/atmel-isc-regs.h
> F:devicetree/bindings/media/atmel-isc.txt
>
> ATMEL ISI DRIVER
> M:Ludovic Desroches
> L:linux-media@vger.kernel.org
> S: Supported
> F: drivers/media/platform/atmel/atmel-isi.c
> F: include/media/atmel-isi.h
>
> Maybe some patch upstream did some recent changes on it via another
> tree.
>
> So, in order to avoid conflicts upstream, for now I would just correct
> the location for the ISI file.
>
> After the merge window, we may revisit and join both entries,
> if the maintainers are the same.
Fine with me.
> -
>
> MAINTAINERS: fix location for atmel-isi.h file
>
> The location of this file got changed by changeset 40a78f36fc92
> ("[media] v4l: atmel-isi: Remove support for platform data"), but
> MAINTAINERS was not updated accordingly.
>
> Fixes: 40a78f36fc92 ("[media] v4l: atmel-isi: Remove support for platform
> data") Reported-by: Joe Perches
> Signed-off-by: Mauro Carvalho Chehab
I don't care about patch count statistics, but the usual practice is to retain
the authorship of the original submitter. This is especially important with
patches submitted by new or less frequent contributors, as recognition of
their work is sometimes the only thing they get (and it doesn't hurt with
regular contributors either).
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 9989925f658d..385ebe9ca0a2 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2496,8 +2496,7 @@ ATMEL ISI DRIVER
> M: Ludovic Desroches
> L: linux-media@vger.kernel.org
> S: Supported
> -F: drivers/media/platform/atmel/atmel-isi.c
> -F: include/media/atmel-isi.h
> +F: drivers/media/platform/atmel/atmel-isi.*
>
> ATMEL LCDFB DRIVER
> M: Nicolas Ferre
--
Regards,
Laurent Pinchart
Re: [PATCH v3] media: docs: add glossary.rst with common terms used at V4L2 spec
someone looks up a term in the glossary because they don't know what it means, they're more likely to search for the abbreviation, not the full term. > + A specialized microprocessor, with its architecture optimized for > + the operational needs of digital signal processing. Stupid question, do we need this entry in the glossary ? The term DSP doesn't seem to be used anywhere in the documentation, and in a video context, I expect the definition of ISP to be more relevant. I also wonder whether we shouldn't drop terms that are defined by the industry (such as DSP, FPGA, I2C, IC, IP core, ISP, Microprocessor, SMBus and SPI), and only focus on terms that have a custom definition in the Linux media subsystem. > +Driver > + Part of the Linux Kernel that implements support for a hardware > + component. > + > +Field-programmable Gate Array - FPGA > + A field-programmable gate array (FPGA) is an integrated circuit > + designed to be configured by a customer or a designer after > + manufacturing. > + > + See https://en.wikipedia.org/wiki/Field-programmable_gate_array. > + > +Inter-Integrated Circuit - I²C > + A multi-master, multi-slave, packet switched, single-ended, > + serial computer bus used to control some hardware components > + like sub-device hardware components. > + > + See http://www.nxp.com/docs/en/user-guide/UM10204.pdf. > + > +Integrated circuit - IC > + A set of electronic circuits on one small flat piece of > + semiconductor material, normally silicon. > + > + Also known as chip. > + > +Intelectual property core - IP block > + In electronic design a semiconductor intellectual property core, > + is a reusable unit of logic, cell, or integrated circuit layout > + design that is the intellectual property of one party. > + IP cores may be licensed to another party or can be owned > + and used by a single party alone. > + > + See > https://en.wikipedia.org/wiki/Semiconductor_intellectual_property_core). > + > +Image Signal Processor - ISP > + A specialised processor that implements a set of algorithms for > + processing image data. ISPs may implement algorithms for lens > + shading correction, demosaic, scaling and pixel format conversion > + as well as produce statistics for the use of the control > + algorithms (e.g. automatic exposure, white balance and focus). > + > +Media API > + A set of userspace APIs used to control a media hardware. How about explcitly listing the APIs that the umbrella term "Media API" covers ? > + See :ref:`media_uapi`. > + > +Media Controller > + An API designed to expose and control devices and sub-devices' > + relationships to applications. What do you mean by "relationships to applications" ? > + See :ref:`media_controller`. > + > +Media Hardware > + Subset of a hardware that is supported by the Linux Media API. > + > + Includes audio and video capture and playback hardware, > + digital and analog TV, camera sensors, ISPs, remote controllers, > + codecs, HDMI Consumer Electronics Control, HDMI capture, etc. > + > + > + See :ref:`media_uapi`. > + > + Extra blank space ? > +Microprocessor > + An electronic circuitry that carries out the instructions > + of a computer program by performing the basic arithmetic, logical, > + control and input/output (I/O) operations specified by the > + instructions on a single integrated circuit. > + > +Remote Controller API > + An API designed to receive and transmit data from remote > + controllers. > + > + See :ref:`remote_controllers`. > + > +SMBus > + A subset of I²C, with defines a stricter usage of the bus. > + > +Serial Peripheral Interface Bus - SPI > + Synchronous serial communication interface specification used for > + short distance communication, primarily in embedded systems. > + > +System on a Chip - SoC > + An integrated circuit that integrates all components of a computer > + or other electronic systems. > + > +Sub-device hardware components > + V4L2 hardware components that aren't controlled by a > + V4L2 main driver. > + > +V4L2 userspace API - V4L2 API > + The userspace API defined at :ref:`v4l2spec`, with is used to s/at/in/ > control > + a V4L2 hardware. > + > +V4L2 hardware > + Part of a media hardware with is supported by the V4L2 > + userspace API. That's kind of a circular definition, isn't it ? > +V4L2 main driver > + A V4L2 device driver that implements the main logic to talk with > + a V4L2 hardware. > + > +V4L2 sub-device > + Part of a media hardware that it is implemented by a device s/it is/is/ > + driver that is not part of the main V4L2 driver. I don't think that's correct a V4L2 subdev is a software object, not a piece of hardware. > + See :ref:`subdev`. -- Regards, Laurent Pinchart
[RESEND] [PATCH] media: v4l: xilinx: Use SPDX-License-Identifier
From: Dhaval Shah SPDX-License-Identifier is used for the Xilinx Video IP and related drivers. Signed-off-by: Dhaval Shah Reviewed-by: Laurent Pinchart [Added drivers/media/platform/xilinx/Kconfig] [Added drivers/media/platform/xilinx/Makefile] [Added include/dt-bindings/media/xilinx-vip.h] Signed-off-by: Laurent Pinchart --- drivers/media/platform/xilinx/Kconfig | 2 ++ drivers/media/platform/xilinx/Makefile | 2 ++ drivers/media/platform/xilinx/xilinx-dma.c | 5 + drivers/media/platform/xilinx/xilinx-dma.h | 5 + drivers/media/platform/xilinx/xilinx-tpg.c | 5 + drivers/media/platform/xilinx/xilinx-vip.c | 5 + drivers/media/platform/xilinx/xilinx-vip.h | 5 + drivers/media/platform/xilinx/xilinx-vipp.c | 5 + drivers/media/platform/xilinx/xilinx-vipp.h | 5 + drivers/media/platform/xilinx/xilinx-vtc.c | 5 + drivers/media/platform/xilinx/xilinx-vtc.h | 5 + include/dt-bindings/media/xilinx-vip.h | 5 + 12 files changed, 14 insertions(+), 40 deletions(-) diff --git a/drivers/media/platform/xilinx/Kconfig b/drivers/media/platform/xilinx/Kconfig index a5d21b7c6e0b..74ec8aaa5ae0 100644 --- a/drivers/media/platform/xilinx/Kconfig +++ b/drivers/media/platform/xilinx/Kconfig @@ -1,3 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + config VIDEO_XILINX tristate "Xilinx Video IP (EXPERIMENTAL)" depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && OF && HAS_DMA diff --git a/drivers/media/platform/xilinx/Makefile b/drivers/media/platform/xilinx/Makefile index e8a0f2a9f733..4cdc0b1ec7a5 100644 --- a/drivers/media/platform/xilinx/Makefile +++ b/drivers/media/platform/xilinx/Makefile @@ -1,3 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + xilinx-video-objs += xilinx-dma.o xilinx-vip.o xilinx-vipp.o obj-$(CONFIG_VIDEO_XILINX) += xilinx-video.o diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c index 4ae9d38c9433..c9d5fdb2d407 100644 --- a/drivers/media/platform/xilinx/xilinx-dma.c +++ b/drivers/media/platform/xilinx/xilinx-dma.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Xilinx Video DMA * @@ -6,10 +7,6 @@ * * Contacts: Hyun Kwon * Laurent Pinchart - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #include diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h index e95d136c153a..5aec4d17eb21 100644 --- a/drivers/media/platform/xilinx/xilinx-dma.h +++ b/drivers/media/platform/xilinx/xilinx-dma.h @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Xilinx Video DMA * @@ -6,10 +7,6 @@ * * Contacts: Hyun Kwon * Laurent Pinchart - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #ifndef __XILINX_VIP_DMA_H__ diff --git a/drivers/media/platform/xilinx/xilinx-tpg.c b/drivers/media/platform/xilinx/xilinx-tpg.c index 851d20dcd550..1399e71d42c2 100644 --- a/drivers/media/platform/xilinx/xilinx-tpg.c +++ b/drivers/media/platform/xilinx/xilinx-tpg.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Xilinx Test Pattern Generator * @@ -6,10 +7,6 @@ * * Contacts: Hyun Kwon * Laurent Pinchart - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #include diff --git a/drivers/media/platform/xilinx/xilinx-vip.c b/drivers/media/platform/xilinx/xilinx-vip.c index 311259129504..5f7efa9f093e 100644 --- a/drivers/media/platform/xilinx/xilinx-vip.c +++ b/drivers/media/platform/xilinx/xilinx-vip.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Xilinx Video IP Core * @@ -6,10 +7,6 @@ * * Contacts: Hyun Kwon * Laurent Pinchart - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #include diff --git a/drivers/media/platform/xilinx/xilinx-vip.h b/drivers/media/platform/xilinx/xilinx-vip.h index 42fee2026815..ba939dd52818 100644 --- a/drivers/media/platform/xilinx/xilinx-vip.h +++ b/drivers/media/platform/xilinx/xilinx-vip.h @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Xilinx Video IP Core * @@ -6,10 +7,6 @@ * * Contacts: Hyun Kwon * Laurent Pinchart - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #ifndef __XILINX_VIP_H__ d
Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG
Hi Hans, On Monday, 1 October 2018 19:28:58 EEST Hans Verkuil wrote: > On 10/01/2018 06:12 PM, Ezequiel Garcia wrote: > > On Mon, 2018-10-01 at 08:42 -0400, Nicolas Dufresne wrote: > >> Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit : > >>> It turns out that we have both JPEG and Motion-JPEG pixel formats > >>> defined. > >>> > >>> Furthermore, some drivers support one, some the other and some both. > >>> > >>> These pixelformats both mean the same. > >>> > >>> I propose that we settle on JPEG (since it seems to be used most often) > >>> and add JPEG support to those drivers that currently only use MJPEG. > >> > >> Thanks for looking into this. As per GStreamer code, I see 3 alias for > >> JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code > >> was written before I knew GStreamer existed. It's possible there is a > >> subtle difference, I have never looked at it, but clearly all our JPEG > >> decoder handle these as being the same. > >> > >> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gst > >> v4l2object.c#n956 > > > > To add more data points on the gstreamer side, there's really no > > difference between gstreamer's types image/jpeg and video/x-jpeg. > > > > Notably, jpegdec element just stuffs a huffman table if one is missing, > > for any jpeg: > > > > https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/ext/jpeg/gstj > > pegdec.c#n584 > > lib/libv4lconvert/libv4lconvert.c also treats JPEG and MJPEG the same. > > It looks like JPEG and MJPEG are randomly used and I don't think you can > assume that one will have a huffman table and not the other. That at least should be fixed. If we decide that whether the frames will contain a Huffman table or not is useful information for userspace, then we should convey it, either through the current mechanism (JPEG vs. MJPEG) or through a different mechanism. Otherwise, we can merge JPEG and MJPEG (as long as it doesn't break userspace). -- Regards, Laurent Pinchart
Re: [PATCH] [media] v4l: xilinx: fix typo in formats table
Hi Michal, On Monday, 1 October 2018 16:28:32 EEST Michal Simek wrote: > On 1.10.2018 15:26, Laurent Pinchart wrote: > > On Monday, 1 October 2018 15:45:49 EEST Michal Simek wrote: > >> On 28.9.2018 14:52, Laurent Pinchart wrote: > >>> On Friday, 28 September 2018 10:32:13 EEST Andrea Merello wrote: > >>>> In formats table the entry for CFA pattern "rggb" has GRBG fourcc. > >>>> This patch fixes it. > >>>> > >>>> Cc: linux-media@vger.kernel.org > >>>> Signed-off-by: Mirco Di Salvo > >>>> Signed-off-by: Andrea Merello > >>> > >>> Reviewed-by: Laurent Pinchart > >>> > >>> Michal, should I take the patch in my tree ? > >> > >> definitely. I am not collecting patches for media tree. > > > > Taken in my tree. > > > > By the way, have we reached any conclusion regarding > > https://lkml.org/lkml/ > > 2017/12/18/112 ? > > Xilinx has started to use SPDX without any issue. It means conversion > should be fine to do. That's good to know, I'll resubmit the patch then, and CC you to get an ack. -- Regards, Laurent Pinchart
Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG
Hello, On Monday, 1 October 2018 15:42:56 EEST Nicolas Dufresne wrote: > Le lundi 01 octobre 2018 à 10:43 +0200, Hans Verkuil a écrit : > > It turns out that we have both JPEG and Motion-JPEG pixel formats defined. > > > > Furthermore, some drivers support one, some the other and some both. > > > > These pixelformats both mean the same. > > > > I propose that we settle on JPEG (since it seems to be used most often) > > and add JPEG support to those drivers that currently only use MJPEG. > > Thanks for looking into this. As per GStreamer code, I see 3 alias for > JPEG. V4L2_PIX_FMT_MJPEG/JPEG/PJPG. I don't know the context, this code > was written before I knew GStreamer existed. It's possible there is a > subtle difference, I have never looked at it, but clearly all our JPEG > decoder handle these as being the same. > > https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l > 2object.c#n956 Some old code to give a bit of context: https://github.com/TimSC/mjpeg/ It should be feasible to implement a decoder that inserts the right Huffman table when none exists, but it has to be explicitly handled. An out-of-band mechanism to convey the information seems potentially useful to me. > > We also need to update the V4L2_PIX_FMT_JPEG documentation since it just > > says TBD: > > > > https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compres > > sed.html > > > > $ git grep -l V4L2_PIX_FMT_MJPEG > > drivers/media/pci/meye/meye.c > > drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c > > drivers/media/platform/sti/delta/delta-cfg.h > > drivers/media/platform/sti/delta/delta-mjpeg-dec.c > > drivers/media/usb/cpia2/cpia2_v4l.c > > drivers/media/usb/go7007/go7007-driver.c > > drivers/media/usb/go7007/go7007-fw.c > > drivers/media/usb/go7007/go7007-v4l2.c > > drivers/media/usb/s2255/s2255drv.c > > drivers/media/usb/uvc/uvc_driver.c > > drivers/staging/media/zoran/zoran_driver.c > > drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c > > drivers/usb/gadget/function/uvc_v4l2.c > > > > It looks like s2255 and cpia2 support both already, so that would leave > > 8 drivers that need to be modified, uvc being the most important of the > > lot. > > > > Any comments? -- Regards, Laurent Pinchart
Re: [PATCH] [media] v4l: xilinx: fix typo in formats table
Hi Michal, On Monday, 1 October 2018 15:45:49 EEST Michal Simek wrote: > On 28.9.2018 14:52, Laurent Pinchart wrote: > > On Friday, 28 September 2018 10:32:13 EEST Andrea Merello wrote: > >> In formats table the entry for CFA pattern "rggb" has GRBG fourcc. > >> This patch fixes it. > >> > >> Cc: linux-media@vger.kernel.org > >> Signed-off-by: Mirco Di Salvo > >> Signed-off-by: Andrea Merello > > > > Reviewed-by: Laurent Pinchart > > > > Michal, should I take the patch in my tree ? > > definitely. I am not collecting patches for media tree. Taken in my tree. By the way, have we reached any conclusion regarding https://lkml.org/lkml/ 2017/12/18/112 ? -- Regards, Laurent Pinchart
Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG
Hi Hans, On Monday, 1 October 2018 14:54:29 EEST Hans Verkuil wrote: > On 10/01/2018 01:48 PM, Laurent Pinchart wrote: > > On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote: > >> It turns out that we have both JPEG and Motion-JPEG pixel formats > >> defined. > >> > >> Furthermore, some drivers support one, some the other and some both. > >> > >> These pixelformats both mean the same. > > > > Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were > > not included in the JPEG headers. > > I'm not aware of any difference. If there is one, then it is certainly not > documented. What I can tell for sure is that many UVC devices don't include Huffman tables in their JPEG headers. > Ezequiel, since you've been working with this recently, do you know anything > about this? -- Regards, Laurent Pinchart
Re: [RFC] V4L2_PIX_FMT_MJPEG vs V4L2_PIX_FMT_JPEG
Hi Hans, On Monday, 1 October 2018 11:43:04 EEST Hans Verkuil wrote: > It turns out that we have both JPEG and Motion-JPEG pixel formats defined. > > Furthermore, some drivers support one, some the other and some both. > > These pixelformats both mean the same. Do they ? I thought MJPEG was JPEG using fixed Huffman tables that were not included in the JPEG headers. > I propose that we settle on JPEG (since it seems to be used most often) and > add JPEG support to those drivers that currently only use MJPEG. > > We also need to update the V4L2_PIX_FMT_JPEG documentation since it just > says TBD: > > https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-compresse > d.html > > $ git grep -l V4L2_PIX_FMT_MJPEG > drivers/media/pci/meye/meye.c > drivers/media/pci/solo6x10/solo6x10-v4l2-enc.c > drivers/media/platform/sti/delta/delta-cfg.h > drivers/media/platform/sti/delta/delta-mjpeg-dec.c > drivers/media/usb/cpia2/cpia2_v4l.c > drivers/media/usb/go7007/go7007-driver.c > drivers/media/usb/go7007/go7007-fw.c > drivers/media/usb/go7007/go7007-v4l2.c > drivers/media/usb/s2255/s2255drv.c > drivers/media/usb/uvc/uvc_driver.c > drivers/staging/media/zoran/zoran_driver.c > drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c > drivers/usb/gadget/function/uvc_v4l2.c > > It looks like s2255 and cpia2 support both already, so that would leave > 8 drivers that need to be modified, uvc being the most important of the > lot. > > Any comments? -- Regards, Laurent Pinchart
[PATCH] media: uvcvideo: Support UVC 1.5 video probe & commit controls
From: ming_qian
commit f620d1d7afc7db57ab59f35000752840c91f67e7 upstream.
The length of UVC 1.5 video control is 48, and it is 34 for UVC 1.1.
Change it to 48 for UVC 1.5 device, and the UVC 1.5 device can be
recognized.
More changes to the driver are needed for full UVC 1.5 compatibility.
However, at least the UVC 1.5 Realtek RTS5847/RTS5852 cameras have been
reported to work well.
[laurent.pinch...@ideasonboard.com: Factor out code to helper function, update
size checks]
Signed-off-by: ming_qian
Signed-off-by: Laurent Pinchart
Tested-by: Kai-Heng Feng
Tested-by: Ana Guerrero Lopez
Signed-off-by: Mauro Carvalho Chehab
---
drivers/media/usb/uvc/uvc_video.c | 24 ++--
1 file changed, 18 insertions(+), 6 deletions(-)
Hello,
This patch was originally marked as a stable candidate, but a driver-wide
switch from __u{8,16,32} to u{8,16,32} created conflicts that prevented
backporting. This version fixes the conflicts and is otherwise not modified.
The decision to mark the patch as a stable candidate came after reports from
distro users that their UVC 1.5 camera was otherwise unusable. A guide has
even been published to tell Debian users how to patch their kernel to fix the
problem. Including the fix in stable will make their life much easier.
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index fb86d6af398d..a6d800291883 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -163,14 +163,27 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming
*stream,
}
}
+static size_t uvc_video_ctrl_size(struct uvc_streaming *stream)
+{
+ /*
+* Return the size of the video probe and commit controls, which depends
+* on the protocol version.
+*/
+ if (stream->dev->uvc_version < 0x0110)
+ return 26;
+ else if (stream->dev->uvc_version < 0x0150)
+ return 34;
+ else
+ return 48;
+}
+
static int uvc_get_video_ctrl(struct uvc_streaming *stream,
struct uvc_streaming_control *ctrl, int probe, __u8 query)
{
+ __u16 size = uvc_video_ctrl_size(stream);
__u8 *data;
- __u16 size;
int ret;
- size = stream->dev->uvc_version >= 0x0110 ? 34 : 26;
if ((stream->dev->quirks & UVC_QUIRK_PROBE_DEF) &&
query == UVC_GET_DEF)
return -EIO;
@@ -225,7 +238,7 @@ static int uvc_get_video_ctrl(struct uvc_streaming *stream,
ctrl->dwMaxVideoFrameSize = get_unaligned_le32([18]);
ctrl->dwMaxPayloadTransferSize = get_unaligned_le32([22]);
- if (size == 34) {
+ if (size >= 34) {
ctrl->dwClockFrequency = get_unaligned_le32([26]);
ctrl->bmFramingInfo = data[30];
ctrl->bPreferedVersion = data[31];
@@ -254,11 +267,10 @@ static int uvc_get_video_ctrl(struct uvc_streaming
*stream,
static int uvc_set_video_ctrl(struct uvc_streaming *stream,
struct uvc_streaming_control *ctrl, int probe)
{
+ __u16 size = uvc_video_ctrl_size(stream);
__u8 *data;
- __u16 size;
int ret;
- size = stream->dev->uvc_version >= 0x0110 ? 34 : 26;
data = kzalloc(size, GFP_KERNEL);
if (data == NULL)
return -ENOMEM;
@@ -275,7 +287,7 @@ static int uvc_set_video_ctrl(struct uvc_streaming *stream,
put_unaligned_le32(ctrl->dwMaxVideoFrameSize, [18]);
put_unaligned_le32(ctrl->dwMaxPayloadTransferSize, [22]);
- if (size == 34) {
+ if (size >= 34) {
put_unaligned_le32(ctrl->dwClockFrequency, [26]);
data[30] = ctrl->bmFramingInfo;
data[31] = ctrl->bPreferedVersion;
--
Regards,
Laurent Pinchart
[PATCH] MAINTAINERS: Remove stale file entry for the Atmel ISI driver
include/media/atmel-isi got removed three years ago without the
MAINTAINERS file being updated. Remove the stale entry.
Fixes: 40a78f36fc92 ("[media] v4l: atmel-isi: Remove support for platform data")
Reported-by: Joe Perches
Signed-off-by: Laurent Pinchart
---
MAINTAINERS | 1 -
1 file changed, 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 9989925f658d..1f5da095aff7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2497,7 +2497,6 @@ M:Ludovic Desroches
L: linux-media@vger.kernel.org
S: Supported
F: drivers/media/platform/atmel/atmel-isi.c
-F: include/media/atmel-isi.h
ATMEL LCDFB DRIVER
M: Nicolas Ferre
--
Regards,
Laurent Pinchart
Re: [PATCH v5] media: imx208: Add imx208 camera sensor driver
Hi Helmut,
On Friday, 21 September 2018 10:23:37 EEST Helmut Grohne wrote:
> On Thu, Sep 20, 2018 at 11:00:26PM +0200, Laurent Pinchart wrote:
> > On Thursday, 20 September 2018 23:16:47 EEST Sylwester Nawrocki wrote:
> >> On 09/20/2018 06:49 PM, Grant Grundler wrote:
> >>> On Thu, Sep 20, 2018 at 1:52 AM Tomasz Figa wrote:
> >>>> We have a problem here. The sensor supports only a discrete range of
> >>>> values here - {1, 2, 4, 8, 16} (multiplied by 256, since the value is
> >>>> fixed point). This makes it possible for the userspace to set values
> >>>> that are not allowed by the sensor specification and also leaves no
> >>>> way to enumerate the supported values.
>
> I'm not sure I understand this correctly. Tomasz appears to imply here
> that the number of actually supported gain values is 5.
>
> >> I'm not sure if it's best approach but I once did something similar for
> >> the ov9650 sensor. The gain was fixed point 10-bits value with 4 bits
> >> for fractional part. The driver reports values multiplied by 16. See
> >> ov965x_set_gain() function in drivers/media/i2c/ov9650.c and "Table 4-1.
> >> Total Gain to Control Bit Correlation" in the OV9650 datasheet for
> >> details. The integer menu control just seemed not suitable for 2^10
> >> values.
>
> As long as values scale linearly (as is the case for fixed point
> numbers) that seems like a reasonable approach to me. That assumption
> appears to not hold for imx208 where the values scale exponentially.
And it's often neither. For instance the MT9P031 has three gain stages.
Quoting the driver:
/* Gain is controlled by 2 analog stages and a digital stage.
* Valid values for the 3 stages are
*
* StageMin Max Step
* --
* First analog stage x1 x2 1
* Second analog stage x1 x4 0.125
* Digital stagex1 x16 0.125
The resulting gain is the product of the three.
> > I've had a similar discussion on IRC recently with Helmut, who posted a
> > nice summary of the problem on the mailing list (see
> > https://www.mail-archive.com/
> > linux-media@vger.kernel.org/msg134502.html). This is a known issue, and
> > while I proposed the same approach, I understand that in some cases
> > userspace may need to know exactly what values are acceptable. In such a
> > case, however, I would expect userspace to have knowledge of the
> > particular sensor model, so the information may not need to come from the
> > kernel.
>
> A big reason to use V4L2 is to abstract hardware. Having to know what
> sensor model you use runs counter that goal. Indeed, gain (whether
> digital or analogue) can be abstracted in principle. I see little reason
> to not do that.
The purpose of V4L2 is to expose the features of the hardware device to
userspace, and we try to do so in an as much abstract as possible way. 100%
abstraction isn't feasible, there will always be small device-specific details
that will break whatever abstraction we design. We should thus strive for a
good balance, abstracting the most common types of features, while avoiding
over-engineering an API that would then become unusable (both because it would
be complex to use by applications, and implemented in subtly differently buggy
ways by drivers).
I don't think we'll reach an agreement here if we don't start talking about
real use cases. Would you have some to share ?
> >> Now the gain control has range 16...1984 out of which only 1024 values
> >> are valid. It might not be best approach for a GUI but at least the
> >> driver exposes mapping of all valid values, which could be enumerated
> >> with VIDIOC_TRY_EXT_CTRLS if required, without a need for a driver-
> >> specific user space code.
>
> If you have very many values, a reasonable compromise could be reducing
> the precision. If you try to represent a floating point number, values
> with higher exponent will have larger "gaps" when represented as
> integers for v4l2. A compromise could be increasing the step size and
> thus removing a few of the gains with lower exponent.
What if an application needs the precision ? Reducing it can cause issues in
the 3A algorithms, including closed-loops stability problems. I think we
should expose the device features with as much precision and coverage as
possible. Hardcoding use case assumptions in the kernel drives us into a wall
sooner or later.
> >>>> I can see two solutions here:
> >>>>
> >>
Re: [PATCH 0/5] Add units to controls
I believe that in the interest of keeping maintenance cost low, this > > should happen rather sooner than later. Just even adding the support to > > smiapp seems wrong when it would be possible to have the framework do > > the work. > > > >> I think I'll merge the undefined and no unit cases. Same for the > >> exponential base actually --- the flag can be removed, too... > > > > I'm not sure I understand. It reads like you are going to revert a > > significant part of the patch. > > A macro or two, it's not a major change. From the user space point of view, > does it make a difference if a control has no unit or when it's not known > what the unit is? > > >> Regarding Ricardo's suggestion --- I was thinking of adding a control > >> flag (yes, there are a few bits available) to tell how to round the > >> value. The user could use the TRY_EXT_CTRLS IOCTL to figure out the next > >> (or previous) control value by incrementing the current value and > >> setting the appropriate flag. This is out of the scope of this set > >> though. > > > > This approach sounds really useful to me. Having control over the > > rounding would allow reading supported control values with reasonable > > effort. That would result in way too many ioctl calls though. If we design an API to report all valid values, we have to do it through a single ioctl call. I'm however not opposed to rounding flags, as there could be a separate use for that, but I'm concerned that drivers will not use them correctly and consistently if we don't implement support for them in the core (and for any integer control that can't be accurately represented by min, max and step, I don't think we can). I'm also concerned about over-engineering a solution for a problem that I'm still not convinced does exist. Applications needing precise information about control will very likely be device-aware in my opinion (due to [1]). I'm open to debating this, in which case I would ask for real use cases, and a real implementation (in something else than a test program). V4L2 is bloated with micro-features that have been badly designed and that are mostly unused, let's try to get it right this time. > > With such an approach, a very sparsely populated control becomes > > feasible and with integer64 controls that'd likely allow representing > > most exponential controls with linear values. If going this route, I > > don't see an application of V4L2_CTRL_FLAG_EXPONENTIAL. > > Yes, I think the flag can be dropped as I suggested. > > > Thus, I think that control over the rounding is tightly related to this > > patchset and needs to be discussed together. > > It addresses some of the same problem area but the implementation is > orthogonal to this. > > Providing that would probably make the base field less useful: the valid > control values could be enumerated by the user using TRY_EXT_CTRLS without > the need to tell the valid values are powers of e.g. two. > > I don't really have a strong opinion on that actually when it comes to the > API itself. The imx208 driver could proceed to use linear relation between > the control value and the digital gain. My worry is just the driver > implementation: this may not be entirely trivial. There's still no way to > address this problem in a generic way otherwise. -- Regards, Laurent Pinchart
Re: [PATCH 1/5] videodev2.h: Use 8 hexadecimals (32 bits) for control flags
Hi Sakari, Thank you for the patch. On Tuesday, 25 September 2018 13:14:30 EEST Sakari Ailus wrote: > The V4L2 control flags are a 32-bit bitmask. Use 32-bit hexadecimal > numbers to specify the flags (was 16). > > Signed-off-by: Sakari Ailus This is the easy one, Reviewed-by: Laurent Pinchart > --- > Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 22 ++-- > include/uapi/linux/videodev2.h| 22 ++-- > 2 files changed, 22 insertions(+), 22 deletions(-) > > diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst index > 5bd26e8c9a1a0..ff2d131223b84 100644 > --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > @@ -439,37 +439,37 @@ See also the examples in :ref:`control`. > > :widths: 3 1 4 > > * - ``V4L2_CTRL_FLAG_DISABLED`` > - - 0x0001 > + - 0x0001 >- This control is permanently disabled and should be ignored by the > application. Any attempt to change the control will result in an > ``EINVAL`` error code. > * - ``V4L2_CTRL_FLAG_GRABBED`` > - - 0x0002 > + - 0x0002 >- This control is temporarily unchangeable, for example because > another application took over control of the respective resource. > Such controls may be displayed specially in a user interface. > Attempts to change the control may result in an ``EBUSY`` error code. > * - ``V4L2_CTRL_FLAG_READ_ONLY`` > - - 0x0004 > + - 0x0004 >- This control is permanently readable only. Any attempt to change > the control will result in an ``EINVAL`` error code. > * - ``V4L2_CTRL_FLAG_UPDATE`` > - - 0x0008 > + - 0x0008 >- A hint that changing this control may affect the value of other > controls within the same control class. Applications should update > their user interface accordingly. > * - ``V4L2_CTRL_FLAG_INACTIVE`` > - - 0x0010 > + - 0x0010 >- This control is not applicable to the current configuration and > should be displayed accordingly in a user interface. For example > the flag may be set on a MPEG audio level 2 bitrate control when > MPEG audio encoding level 1 was selected with another control. > * - ``V4L2_CTRL_FLAG_SLIDER`` > - - 0x0020 > + - 0x0020 >- A hint that this control is best represented as a slider-like > element in a user interface. > * - ``V4L2_CTRL_FLAG_WRITE_ONLY`` > - - 0x0040 > + - 0x0040 >- This control is permanently writable only. Any attempt to read the > control will result in an ``EACCES`` error code error code. This flag > is typically present for relative controls or action controls > @@ -477,7 +477,7 @@ See also the examples in :ref:`control`. > action (e. g. motor control) but no meaningful value can be > returned. > * - ``V4L2_CTRL_FLAG_VOLATILE`` > - - 0x0080 > + - 0x0080 >- This control is volatile, which means that the value of the > control changes continuously. A typical example would be the > current gain value if the device is in auto-gain mode. In such a > @@ -493,7 +493,7 @@ See also the examples in :ref:`control`. > Setting a new value for a volatile control will *never* trigger a > > :ref:`V4L2_EVENT_CTRL_CH_VALUE ` event. > > * - ``V4L2_CTRL_FLAG_HAS_PAYLOAD`` > - - 0x0100 > + - 0x0100 >- This control has a pointer type, so its value has to be accessed > using one of the pointer fields of struct > > :c:type:`v4l2_ext_control`. This flag is set > > @@ -503,7 +503,7 @@ See also the examples in :ref:`control`. > * .. _FLAG_EXECUTE_ON_WRITE: > >- ``V4L2_CTRL_FLAG_EXECUTE_ON_WRITE`` > - - 0x0200 > + - 0x0200 >- The value provided to the control will be propagated to the driver > even if it remains constant. This is required when the control > represents an action on the hardware. For example: clearing an > @@ -512,7 +512,7 @@ See also the examples in :ref:`control`. > * .. _FLAG_MODIFY_LAYOUT: > >- ``V4L2_CTRL_FLAG_MODIFY_LAYOUT`` > - - 0x0400 > + - 0x0400 >- Changing this control value may modify the layout of the > buffer (for video devices) or the media bus format (for > sub-devices). > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index 184e4dbe8f9c0..ae083978988f1 100644 > --- a/include/uapi
Re: [PATCH] [media] v4l: xilinx: fix typo in formats table
Hi Andrea,
Thank you for the patch.
On Friday, 28 September 2018 10:32:13 EEST Andrea Merello wrote:
> In formats table the entry for CFA pattern "rggb" has GRBG fourcc.
> This patch fixes it.
>
> Cc: linux-media@vger.kernel.org
> Signed-off-by: Mirco Di Salvo
> Signed-off-by: Andrea Merello
Reviewed-by: Laurent Pinchart
Michal, should I take the patch in my tree ?
> ---
> drivers/media/platform/xilinx/xilinx-vip.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/xilinx/xilinx-vip.c
> b/drivers/media/platform/xilinx/xilinx-vip.c index
> 311259129504..e9567cdfb89b 100644
> --- a/drivers/media/platform/xilinx/xilinx-vip.c
> +++ b/drivers/media/platform/xilinx/xilinx-vip.c
> @@ -36,7 +36,7 @@ static const struct xvip_video_format xvip_video_formats[]
> = { { XVIP_VF_MONO_SENSOR, 8, "mono", MEDIA_BUS_FMT_Y8_1X8,
> 1, V4L2_PIX_FMT_GREY, "Greyscale 8-bit" },
> { XVIP_VF_MONO_SENSOR, 8, "rggb", MEDIA_BUS_FMT_SRGGB8_1X8,
> - 1, V4L2_PIX_FMT_SGRBG8, "Bayer 8-bit RGGB" },
> + 1, V4L2_PIX_FMT_SRGGB8, "Bayer 8-bit RGGB" },
> { XVIP_VF_MONO_SENSOR, 8, "grbg", MEDIA_BUS_FMT_SGRBG8_1X8,
> 1, V4L2_PIX_FMT_SGRBG8, "Bayer 8-bit GRBG" },
> { XVIP_VF_MONO_SENSOR, 8, "gbrg", MEDIA_BUS_FMT_SGBRG8_1X8,
--
Regards,
Laurent Pinchart
Re: [PATCH 03/13] v4l2-mc: switch it to use the new approach to setup pipelines
> index 8bfbe6b59fa9..ac8b93e46167 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -675,6 +675,25 @@ static inline void media_entity_cleanup(struct
> media_entity *entity) {} #define media_entity_cleanup(entity) do { } while
> (false)
> #endif
>
> +
Extra blank line.
> +/**
> + * media_get_pad_index() - retrieves a pad index from an entity
I think a better name would be media_entity_find_pad(), similarly to
media_entity_find_link(), as the function searches for a pad given a direction
and signal type. A *_get_*() function name hints of reference counting.
> + *
> + * @entity: entity where the pads belong
> + * @is_sink: true if the pad is a sink, false if it is a source
Could we use pad flags instead ? It's easier to read
pad = media_get_pad_index(entity, MEDIA_PAD_FL_SINK, ...);
than
pad = media_get_pad_index(entity, true, ...);
As an added bonus that would allow the caller to search for any pad with a
given signal type by specifying MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE.
> + * @sig_type:type of signal of the pad to be search
> + *
> + * This helper function finds the first pad index inside an entity that
> + * satisfies both @is_sink and @sig_type conditions.
> + *
> + * Return:
> + *
> + * On success, return the pad number. If the pad was not found or the media
> + * entity is a NULL pointer, return -EINVAL.
> + */
> +int media_get_pad_index(struct media_entity *entity, bool is_sink,
> + enum media_pad_signal_type sig_type);
> +
> /**
> * media_create_pad_link() - creates a link between two entities.
> *
--
Regards,
Laurent Pinchart
Re: [PATCH 02/13] media: v4l2: taint pads with the signal types for consumer devices
Hi Mauro,
Thank you for the patch.
Could you please CC me on patches touching the media controller core ? I can
send a MAINTAINERS patch to make sure that gets handled automatically.
On Wednesday, 1 August 2018 18:55:04 EEST Mauro Carvalho Chehab wrote:
> Consumer devices are provided with a wide diferent range of types
> supported by the same driver, allowing different configutations.
>
> In order to make easier to setup media controller links, "taint"
> pads with the signal type it carries.
>
> While here, get rid of DEMOD_PAD_VBI_OUT, as the signal it carries
> is actually the same as the normal video output.
>
> The difference happens at the video/VBI interface:
> - for VBI, only the hidden lines are streamed;
> - for video, the stream is usually cropped to hide the
> vbi lines.
>
> Signed-off-by: Mauro Carvalho Chehab
> ---
> drivers/media/dvb-frontends/au8522_decoder.c | 3 ++
> drivers/media/i2c/msp3400-driver.c | 2 ++
> drivers/media/i2c/saa7115.c | 2 ++
> drivers/media/i2c/tvp5150.c | 2 ++
> drivers/media/pci/saa7134/saa7134-core.c | 2 ++
> drivers/media/tuners/si2157.c| 3 ++
> drivers/media/usb/dvb-usb-v2/mxl111sf.c | 2 ++
> drivers/media/v4l2-core/tuner-core.c | 5 +++
> include/media/media-entity.h | 35
> 9 files changed, 56 insertions(+)
[snip]
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 3aa3d58d1d58..8bfbe6b59fa9 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -155,6 +155,40 @@ struct media_link {
> bool is_backlink;
> };
>
> +/**
> + * struct media_pad_signal_type - type of the signal inside a media pad
I'd say "carried by a media pad" instead of "inside a media pad".
> + *
> + * @PAD_SIGNAL_DEFAULT
Shouldn't we use a MEDIA_PAD_ prefix ?
> + * Default signal. Use this when all inputs or all outputs are
> + * uniquely identified by the pad number.
How about "Use this when the pad can carry a single signal type" ? I
understand your formulation as meaning that all pads of the entity have to be
of the default type, or none can be.
> + * @PAD_SIGNAL_ANALOG
This isn't a very good name given that PAD_SIGNAL_TV_CARRIERS is also analog.
> + * The pad contains an analog signa. It can be Radio Frequency,
s/signa/signal/
> + * Intermediate Frequency or baseband signal.
> + * Tuner inputs, composite and s-video signals should use it.
> + * On tuner sources, this is used for digital TV demodulators and for
> + * IF-PLL demodulator like tda9887.
> + * @PAD_SIGNAL_TV_CARRIERS
> + * The pad contains analog signals carrying either a digital or an analog
> + * modulated (or baseband) signal.
As above, maybe "The pad carries either ...".
> This is provided by tuner source
> + * pads and used by analog TV standard decoders and by digital TV demods.
> + * @PAD_SIGNAL_DV
> + * Contains a digital video signal, with can be a bitstream of samples
> + * taken from an analog TV video source. On such case, it usually
> + * contains the VBI data on it.
> + * @PAD_SIGNAL_AUDIO
> + * Contains an Intermediate Frequency analog signal from an audio
> + * sub-carrier or an audio bitstream. IF signals are provided by tuners
> + * and consumed by audio AM/FM decoders. Bitstream audio is provided by
s/ / /
> + * an audio decoder.
Generally speaking the types you propose here seem quite ad-hoc, without much
coherency. For instance you split analog and digital video, but group all
audio under a single type. It's also not very clear from the description how
to handle analog video, as it could match both PAD_SIGNAL_ANALOG and
PAD_SIGNAL_TV_CARRIERS. Both of those types also accept analog baseband
signals. I think this should be reworked, it doesn't sound very usable except
for the specific use case that this series tries to address.
> + */
> +enum media_pad_signal_type {
> + PAD_SIGNAL_DEFAULT = 0,
> + PAD_SIGNAL_ANALOG,
> + PAD_SIGNAL_TV_CARRIERS,
> + PAD_SIGNAL_DV,
> + PAD_SIGNAL_AUDIO,
> +};
> +
> /**
> * struct media_pad - A media pad graph object.
> *
> @@ -169,6 +203,7 @@ struct media_pad {
> struct media_gobj graph_obj;/* must be first field in struct */
> struct media_entity *entity;
> u16 index;
> + enum media_pad_signal_type sig_type;
Missing kerneldoc and Documentation/ update ? It's important to document the
use cases, and in particular whether the type should be static or can vary (as
in whether a pad can carry different types over time).
> unsigned long flags;
> };
--
Regards,
Laurent Pinchart
Re: [PATCH v3 1/1] v4l: event: Prevent freeing event subscriptions while accessed
Hi Sakari,
Thank you for the patch.
On Friday, 14 September 2018 14:03:01 EEST Sakari Ailus wrote:
> The event subscriptions are added to the subscribed event list while
> holding a spinlock, but that lock is subsequently released while still
> accessing the subscription object. This makes it possible to unsubscribe
> the event --- and freeing the subscription object's memory --- while
> the subscription object is simultaneously accessed.
>
> Prevent this by adding a mutex to serialise the event subscription and
> unsubscription. This also gives a guarantee to the callback ops that the
> add op has returned before the del op is called.
>
> This change also results in making the elems field less special:
> subscriptions are only added to the event list once they are fully
> initialised.
>
> Signed-off-by: Sakari Ailus
> Reviewed-by: Hans Verkuil
It wasn't immediately clear to me that the !sev->elems check can be removed
because the subscriptions are *now* only added to the event list once they are
fully initialized, I thought the sentence documented the current
implementation. After realizing that,
Reviewed-by: Laurent Pinchart
> ---
> since v1:
>
> - Call the mutex field subscribe_lock instead.
>
> - Move the field that is now subscribe_lock above the subscribed field the
> write access to which it serialises.
>
> - Improve documentation of the subscribe_lock field.
>
> since v2:
>
> - Acquire spinlock for the duration of list_add() in v4l2_event_subscribe().
>
> - Remove a redundant comment in the same place.
>
> drivers/media/v4l2-core/v4l2-event.c | 38 +
> drivers/media/v4l2-core/v4l2-fh.c| 2 ++
> include/media/v4l2-fh.h | 4
> 3 files changed, 26 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-event.c
> b/drivers/media/v4l2-core/v4l2-event.c index 127fe6eb91d9..a3ef1f50a4b3
> 100644
> --- a/drivers/media/v4l2-core/v4l2-event.c
> +++ b/drivers/media/v4l2-core/v4l2-event.c
> @@ -115,14 +115,6 @@ static void __v4l2_event_queue_fh(struct v4l2_fh *fh,
> const struct v4l2_event *e if (sev == NULL)
> return;
>
> - /*
> - * If the event has been added to the fh->subscribed list, but its
> - * add op has not completed yet elems will be 0, treat this as
> - * not being subscribed.
> - */
> - if (!sev->elems)
> - return;
> -
> /* Increase event sequence number on fh. */
> fh->sequence++;
>
> @@ -208,6 +200,7 @@ int v4l2_event_subscribe(struct v4l2_fh *fh,
> struct v4l2_subscribed_event *sev, *found_ev;
> unsigned long flags;
> unsigned i;
> + int ret = 0;
>
> if (sub->type == V4L2_EVENT_ALL)
> return -EINVAL;
> @@ -225,31 +218,36 @@ int v4l2_event_subscribe(struct v4l2_fh *fh,
> sev->flags = sub->flags;
> sev->fh = fh;
> sev->ops = ops;
> + sev->elems = elems;
> +
> + mutex_lock(>subscribe_lock);
>
> spin_lock_irqsave(>vdev->fh_lock, flags);
> found_ev = v4l2_event_subscribed(fh, sub->type, sub->id);
> - if (!found_ev)
> - list_add(>list, >subscribed);
> spin_unlock_irqrestore(>vdev->fh_lock, flags);
>
> if (found_ev) {
> + /* Already listening */
> kvfree(sev);
> - return 0; /* Already listening */
> + goto out_unlock;
> }
>
> if (sev->ops && sev->ops->add) {
> - int ret = sev->ops->add(sev, elems);
> + ret = sev->ops->add(sev, elems);
> if (ret) {
> - sev->ops = NULL;
> - v4l2_event_unsubscribe(fh, sub);
> - return ret;
> + kvfree(sev);
> + goto out_unlock;
> }
> }
>
> - /* Mark as ready for use */
> - sev->elems = elems;
> + spin_lock_irqsave(>vdev->fh_lock, flags);
> + list_add(>list, >subscribed);
> + spin_unlock_irqrestore(>vdev->fh_lock, flags);
>
> - return 0;
> +out_unlock:
> + mutex_unlock(>subscribe_lock);
> +
> + return ret;
> }
> EXPORT_SYMBOL_GPL(v4l2_event_subscribe);
>
> @@ -288,6 +286,8 @@ int v4l2_event_unsubscribe(struct v4l2_fh *fh,
> return 0;
> }
>
> + mutex_lock(>subscribe_lock);
> +
> spin_lock_irqsave(>vdev->fh_lock, flags);
>
> sev = v4l2_event_subscribed(fh, sub->type, sub->id);
> @@ -305,6 +305,8 @@ int v4l2_event_unsu
Re: [PATCH v5] media: imx208: Add imx208 camera sensor driver
Hi Sakari,
On Thursday, 20 September 2018 23:56:59 EEST Sakari Ailus wrote:
> On Thu, Sep 20, 2018 at 05:51:55PM +0900, Tomasz Figa wrote:
> > On Wed, Aug 8, 2018 at 4:08 PM Ping-chung Chen wrote:
> > [snip]
> >
> > > +
> > > +/* Digital gain control */
> > > +#define IMX208_REG_GR_DIGITAL_GAIN 0x020e
> > > +#define IMX208_REG_R_DIGITAL_GAIN 0x0210
> > > +#define IMX208_REG_B_DIGITAL_GAIN 0x0212
> > > +#define IMX208_REG_GB_DIGITAL_GAIN 0x0214
> > > +#define IMX208_DGTL_GAIN_MIN 0
> > > +#define IMX208_DGTL_GAIN_MAX 4096
> > > +#define IMX208_DGTL_GAIN_DEFAULT 0x100
> > > +#define IMX208_DGTL_GAIN_STEP 1
> > > +
> >
> > [snip]
> >
> > > +/* Initialize control handlers */
> > > +static int imx208_init_controls(struct imx208 *imx208)
> > > +{
> >
> > [snip]
> >
> > > + v4l2_ctrl_new_std(ctrl_hdlr, _ctrl_ops,
> > > V4L2_CID_DIGITAL_GAIN, + IMX208_DGTL_GAIN_MIN,
> > > IMX208_DGTL_GAIN_MAX, + IMX208_DGTL_GAIN_STEP,
> > > + IMX208_DGTL_GAIN_DEFAULT);
> >
> > We have a problem here. The sensor supports only a discrete range of
> > values here - {1, 2, 4, 8, 16} (multiplied by 256, since the value is
> > fixed point). This makes it possible for the userspace to set values
> > that are not allowed by the sensor specification and also leaves no
> > way to enumerate the supported values.
> >
> > I can see two solutions here:
> >
> > 1) Define the control range from 0 to 4 and treat it as an exponent of
> > 2, so that the value for the sensor becomes (1 << val) * 256.
> > (Suggested by Sakari offline.)
> >
> > This approach has the problem of losing the original unit (and scale)
> > of the value.
>
> I'd like to add that this is not a property of the proposed solution.
>
> Rather, the above needs to be accompanied by additional information
> provided through VIDIOC_QUERY_EXT_CTRL, i.e. the unit, prefix as well as
> other information such as whether the control is linear or exponential (as
> in this case).
>
> > 2) Use an integer menu control, which reports only the supported
> > discrete values - {1, 2, 4, 8, 16}.
> >
> > With this approach, userspace can enumerate the real gain values, but
> > we would either need to introduce a new control (e.g.
> > V4L2_CID_DIGITAL_GAIN_DISCRETE) or abuse the specification and
> > register V4L2_CID_DIGITAL_GAIN as an integer menu.
>
> New controls in V4L2 are, for the most part, created when there's something
> new to control. The documentation of some controls (similar to e.g. gain)
> documents a unit as well as a prefix but that's the case only because
> there's been no way to tell the unit or prefix otherwise in the API.
>
> An exception to this are EXPOSURE and EXPOSURE_ABSOLUTE. I'm not entirely
> sure how they came to be though. An accident is a possibility as far as I
> see.
If I remember correctly I introduced the absolute variant for the UVC driver
(even though git blame points to Brandon Philips). I don't really remember why
though.
> Controls that have a documented unit use that unit --- as long as that's
> the unit used by the hardware. If it's not, it tends to be that another
> unit is used but the user space has currently no way of knowing this. And
> the digital gain control is no exception to this.
>
> So if we want to improve the user space's ability to be informed how the
> control values reflect to device configuration, we do need to provide more
> information to the user space. One way to do that would be through
> VIDIOC_QUERY_EXT_CTRL. The IOCTL struct has plenty of reserved fields ---
> just for purposes such as this one.
I don't think we can come up with a good way to expose arbitrary mathematical
formulas through an ioctl. In my opinion the question is how far we want to
go, how precise we need to be.
> > Any opinions or better ideas?
--
Regards,
Laurent Pinchart
Re: [PATCH v5] media: imx208: Add imx208 camera sensor driver
Hello,
(CC'ing Helmut Grohne)
On Thursday, 20 September 2018 23:16:47 EEST Sylwester Nawrocki wrote:
> On 09/20/2018 06:49 PM, Grant Grundler wrote:
> > On Thu, Sep 20, 2018 at 1:52 AM Tomasz Figa wrote:
> >> On Wed, Aug 8, 2018 at 4:08 PM Ping-chung Chen wrote:
> >>> +/* Digital gain control */
> >>>
> >>> +#define IMX208_DGTL_GAIN_MIN 0
> >>> +#define IMX208_DGTL_GAIN_MAX 4096
> >>> +#define IMX208_DGTL_GAIN_DEFAULT 0x100
> >>> +#define IMX208_DGTL_GAIN_STEP 1
> >>>
> >>> +/* Initialize control handlers */
> >>> +static int imx208_init_controls(struct imx208 *imx208)
> >>> +{
> >>
> >> [snip]
> >>
> >>> + v4l2_ctrl_new_std(ctrl_hdlr, _ctrl_ops,
> >>> V4L2_CID_DIGITAL_GAIN, + IMX208_DGTL_GAIN_MIN,
> >>> IMX208_DGTL_GAIN_MAX, + IMX208_DGTL_GAIN_STEP,
> >>> + IMX208_DGTL_GAIN_DEFAULT);
> >>
> >> We have a problem here. The sensor supports only a discrete range of
> >> values here - {1, 2, 4, 8, 16} (multiplied by 256, since the value is
> >> fixed point). This makes it possible for the userspace to set values
> >> that are not allowed by the sensor specification and also leaves no
> >> way to enumerate the supported values.
>
> The driver could always adjust the value in set_ctrl callback so invalid
> settings are not allowed.
>
> I'm not sure if it's best approach but I once did something similar for
> the ov9650 sensor. The gain was fixed point 10-bits value with 4 bits
> for fractional part. The driver reports values multiplied by 16. See
> ov965x_set_gain() function in drivers/media/i2c/ov9650.c and "Table 4-1.
> Total Gain to Control Bit Correlation" in the OV9650 datasheet for details.
> The integer menu control just seemed not suitable for 2^10 values.
I've had a similar discussion on IRC recently with Helmut, who posted a nice
summary of the problem on the mailing list (see https://www.mail-archive.com/
linux-media@vger.kernel.org/msg134502.html). This is a known issue, and while
I proposed the same approach, I understand that in some cases userspace may
need to know exactly what values are acceptable. In such a case, however, I
would expect userspace to have knowledge of the particular sensor model, so
the information may not need to come from the kernel.
> Now the gain control has range 16...1984 out of which only 1024 values
> are valid. It might not be best approach for a GUI but at least the driver
> exposes mapping of all valid values, which could be enumerated with
> VIDIOC_TRY_EXT_CTRLS if required, without a need for a driver-specific
> user space code.
That would be ~2000 ioctl calls, I don't think that's very practical :-S
> >> I can see two solutions here:
> >>
> >> 1) Define the control range from 0 to 4 and treat it as an exponent of
> >> 2, so that the value for the sensor becomes (1 << val) * 256.
> >> (Suggested by Sakari offline.)
> >>
> >> This approach has the problem of losing the original unit (and scale)
> >> of the value.
> >
> > Exactly - will users be confused by this? If we have to explain it,
> > probably not the best choice.
> >
> >> 2) Use an integer menu control, which reports only the supported
> >> discrete values - {1, 2, 4, 8, 16}.
> >>
> >> With this approach, userspace can enumerate the real gain values, but
> >> we would either need to introduce a new control (e.g.
> >> V4L2_CID_DIGITAL_GAIN_DISCRETE) or abuse the specification and
> >> register V4L2_CID_DIGITAL_GAIN as an integer menu.
> >>
> >> Any opinions or better ideas?
> >
> > My $0.02: leave the user UI alone - let users specify/select anything
> > in the range the normal API or UI allows. But have sensor specific
> > code map all values in that range to values the sensor supports. Users
> > will notice how it works when they play with it. One can "adjust" the
> > mapping so it "feels right".
--
Regards,
Laurent Pinchart
Re: [PATCH v2 2/4] [media] ad5820: Add support for enable pin
Hi Ricardo,
On Thursday, 20 September 2018 22:12:44 EEST Ricardo Ribalda Delgado wrote:
> On Thu, Sep 20, 2018 at 9:08 PM Pavel Machek wrote:
> > On Thu 2018-09-20 21:06:16, Ricardo Ribalda Delgado wrote:
> >> On Thu, Sep 20, 2018 at 8:54 PM Pavel Machek wrote:
> >>> On Thu 2018-09-20 20:45:52, Ricardo Ribalda Delgado wrote:
> >>>> This patch adds support for a programmable enable pin. It can be
> >>>> used in situations where the ANA-vcc is not configurable (dummy-
> >>>> regulator), or just to have a more fine control of the power saving.
> >>>>
> >>>> The use of the enable pin is optional.
> >>>>
> >>>> Signed-off-by: Ricardo Ribalda Delgado
> >>>
> >>> Do we really want to do that?
> >>>
> >>> Would it make more sense to add gpio-regulator and connect ad5820 to
> >>> it in such case?
> >>
> >> My board (based on db820c) has both:
> >>
> >> ad5820: dac@0c {
> >>compatible = "adi,ad5820";
> >>reg = <0x0c>;
> >>
> >>VANA-supply = <_l23>;
> >>enable-gpios = < 26 GPIO_ACTIVE_HIGH>;
> >> };
> >
> > Well, I'm sure you could have gpio-based regulator powered from
> > pm8994_l23, and outputting to ad5820.
> >
> > Does ad5820 chip have a gpio input for enable?
>
> xshutdown pin:
> http://www.analog.com/media/en/technical-documentation/data-sheets/AD5821.pd
> f
>
> (AD5820,AD5821, and AD5823 are compatibles, or at least that is waht
> the module manufacturer says :)
Is that the pin that msmgpio 26 is connected to on your board ?
I'd argue that the GPIO should be called xshutdown in that case, as DT usually
uses the pin name, but there's precedent of using well-known names for pins
with the same functionality. In any case you should update the DT bindings to
document the new property, and clearly explain that it describes the GPIO
connected to the xshutdown pin. Please CC the devicet...@vger.kernel.org
mailing list on the bindings change (they usually like bindings changes to be
split to a separate patch).
--
Regards,
Laurent Pinchart
Re: [PATCH v2 2/4] [media] ad5820: Add support for enable pin
On Thursday, 20 September 2018 23:04:21 EEST Laurent Pinchart wrote:
> Hi Ricardo,
>
> On Thursday, 20 September 2018 22:12:44 EEST Ricardo Ribalda Delgado wrote:
> > On Thu, Sep 20, 2018 at 9:08 PM Pavel Machek wrote:
> > > On Thu 2018-09-20 21:06:16, Ricardo Ribalda Delgado wrote:
> > >> On Thu, Sep 20, 2018 at 8:54 PM Pavel Machek wrote:
> > >>> On Thu 2018-09-20 20:45:52, Ricardo Ribalda Delgado wrote:
> > >>>> This patch adds support for a programmable enable pin. It can be
> > >>>> used in situations where the ANA-vcc is not configurable (dummy-
> > >>>> regulator), or just to have a more fine control of the power saving.
> > >>>>
> > >>>> The use of the enable pin is optional.
> > >>>>
> > >>>> Signed-off-by: Ricardo Ribalda Delgado
> > >>>
> > >>> Do we really want to do that?
> > >>>
> > >>> Would it make more sense to add gpio-regulator and connect ad5820 to
> > >>> it in such case?
> > >>
> > >> My board (based on db820c) has both:
> > >>
> > >> ad5820: dac@0c {
> > >>
> > >>compatible = "adi,ad5820";
> > >>reg = <0x0c>;
> > >>
> > >>VANA-supply = <_l23>;
> > >>enable-gpios = < 26 GPIO_ACTIVE_HIGH>;
> > >>
> > >> };
> > >
> > > Well, I'm sure you could have gpio-based regulator powered from
> > > pm8994_l23, and outputting to ad5820.
> > >
> > > Does ad5820 chip have a gpio input for enable?
> >
> > xshutdown pin:
> > http://www.analog.com/media/en/technical-documentation/data-sheets/AD5821.
> > pd f
> >
> > (AD5820,AD5821, and AD5823 are compatibles, or at least that is waht
> > the module manufacturer says :)
>
> Is that the pin that msmgpio 26 is connected to on your board ?
>
> I'd argue that the GPIO should be called xshutdown in that case, as DT
> usually uses the pin name, but there's precedent of using well-known names
> for pins with the same functionality. In any case you should update the DT
> bindings to document the new property, and clearly explain that it
> describes the GPIO connected to the xshutdown pin. Please CC the
> devicet...@vger.kernel.org mailing list on the bindings change (they
> usually like bindings changes to be split to a separate patch).
And now I've noticed patch 3/4 :-/ Please scratch this.
--
Regards,
Laurent Pinchart
Re: [PATCH 5/6] media: isp: fix a warning about a wrong struct initializer
Hi Mauro,
On Thursday, 20 September 2018 17:44:53 EEST Mauro Carvalho Chehab wrote:
> Em Fri, 07 Sep 2018 14:46:34 +0300 Laurent Pinchart escreveu:
> > Hi Mauro,
> >
> > As maintainers should be held to the same level of obligations as
> > developers, and to avoid demotivating reviewers, could you handle
> > comments you receive before pushing your own patches to your tree ? There
> > should be no maintainer privilege here.
>
> Sorry, yeah, this was improperly handled. Not sure what happened.
>
> From whatever reason, I ended by mishandling this and lost the
> related emails. I had to do several e-mail changes at the beginning
> of August, as I'm not using s-opensource.com anymore, and had to
> reconfigure my inbox handling logic.
Thank you for the explanation.
> Do you want me to revert the patch?
I appreciate the proposal, but there's no need to :-) It's not a big issue, I
could easily submit a follow-up patch, but even that isn't really needed. My
main concern was the lack of reaction to the review, and now that this has be
cleared, the issue is closed for me.
> > On Wednesday, 8 August 2018 18:45:49 EEST Laurent Pinchart wrote:
> > > Hi Mauro,
> > >
> > > Thank you for the patch.
> > >
> > > The subject line should be "media: omap3isp: ...".
> > >
> > > On Wednesday, 8 August 2018 17:52:55 EEST Mauro Carvalho Chehab wrote:
> > > > As sparse complains:
> > > > drivers/media/platform/omap3isp/isp.c:303:39: warning: Using
> > > > plain
> > > > integer
> > > >
> > > > as NULL pointer
> > > >
> > > > when a struct is initialized with { 0 }, actually the first
> > > > element of the struct is initialized with zeros, initializing the
> > > > other elements recursively. That can even generate gcc warnings
> > > > on nested structs.
> > > >
> > > > So, instead, use the gcc-specific syntax for that (with is used
> > > > broadly inside the Kernel), initializing it with {};
> > > >
> > > > Signed-off-by: Mauro Carvalho Chehab
> > > > ---
> > > >
> > > > drivers/media/platform/omap3isp/isp.c | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/drivers/media/platform/omap3isp/isp.c
> > > > b/drivers/media/platform/omap3isp/isp.c index
> > > > 03354d513311..842e2235047d
> > > > 100644
> > > > --- a/drivers/media/platform/omap3isp/isp.c
> > > > +++ b/drivers/media/platform/omap3isp/isp.c
> > > > @@ -300,7 +300,7 @@ static struct clk *isp_xclk_src_get(struct
> > > > of_phandle_args *clkspec, void *data) static int isp_xclk_init(struct
> > > > isp_device *isp)
> > > >
> > > > {
> > > >
> > > > struct device_node *np = isp->dev->of_node;
> > > >
> > > > - struct clk_init_data init = { 0 };
> > > > + struct clk_init_data init = {};
> > >
> > > How about = { NULL }; to avoid a gcc-specific syntax ?
> > >
> > > > unsigned int i;
> > > >
> > > > for (i = 0; i < ARRAY_SIZE(isp->xclks); ++i)
>
> Thanks,
> Mauro
--
Regards,
Laurent Pinchart
Re: [PATCH v3 1/2] media: v4l2-subdev.h: allow V4L2_FRMIVAL_TYPE_CONTINUOUS & _STEPWISE
= fie.max_interval;
> + fival->stepwise.step = fie.step_interval;
> + }
> + }
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_fill_frmivalenum_from_subdev);
> diff --git a/include/media/v4l2-common.h b/include/media/v4l2-common.h
> index cdc87ec..3c62403 100644
> --- a/include/media/v4l2-common.h
> +++ b/include/media/v4l2-common.h
> @@ -384,4 +384,16 @@ int v4l2_g_parm_cap(struct video_device *vdev,
> int v4l2_s_parm_cap(struct video_device *vdev,
> struct v4l2_subdev *sd, struct v4l2_streamparm *a);
>
> +/**
> + * v4l2_fill_frmivalenum_from_subdev - helper for
> vidioc_enum_frameintervals + * calling the enum_frame_interval op of
> the given subdev.
> + *
> + * @sd: the sub-device pointer.
> + * @fival: the VIDIOC_ENUM_FRAMEINTERVALS argument.
> + * @code: the MEDIA_BUS_FMT_ code (not fival->pixel_format !)
> + */
> +int v4l2_fill_frmivalenum_from_subdev(struct v4l2_subdev *sd,
> + struct v4l2_frmivalenum *fival,
> + int code);
> +
> #endif /* V4L2_COMMON_H_ */
> diff --git a/include/uapi/linux/v4l2-subdev.h
> b/include/uapi/linux/v4l2-subdev.h index 03970ce..3faae35 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -100,6 +100,16 @@ struct v4l2_subdev_frame_size_enum {
> };
>
> /**
> + * enum v4l2_subdev_frmival_type - Frame interval type
> + */
> +enum v4l2_subdev_frmival_type {
> + V4L2_SUBDEV_FRMIVAL_TYPE_DISCRETE = 0,
> + V4L2_SUBDEV_FRMIVAL_TYPE_CONTINUOUS = 1,
> + V4L2_SUBDEV_FRMIVAL_TYPE_STEPWISE = 2,
> +};
> +
> +
> +/**
> * struct v4l2_subdev_frame_interval - Pad-level frame rate
> * @pad: pad number, as reported by the media API
> * @interval: frame interval in seconds
> @@ -117,8 +127,13 @@ struct v4l2_subdev_frame_interval {
> * @code: format code (MEDIA_BUS_FMT_ definitions)
> * @width: frame width in pixels
> * @height: frame height in pixels
> - * @interval: frame interval in seconds
> + * @interval: minimum frame interval in seconds
> * @which: format type (from enum v4l2_subdev_format_whence)
> + * @type: frame interval type (from enum v4l2_subdev_frmival_type)
> + * @max_interval: maximum frame interval in seconds,
> + *if type != V4L2_SUBDEV_FRMIVAL_TYPE_DISCRETE
> + * @step_interval: step between frame intervals, in seconds,
> + * if type == V4L2_SUBDEV_FRMIVAL_TYPE_STEPWISE
> */
> struct v4l2_subdev_frame_interval_enum {
> __u32 index;
> @@ -128,7 +143,10 @@ struct v4l2_subdev_frame_interval_enum {
> __u32 height;
> struct v4l2_fract interval;
> __u32 which;
> - __u32 reserved[8];
> + __u32 type;
> + struct v4l2_fract max_interval;
> + struct v4l2_fract step_interval;
> + __u32 reserved[3];
> };
>
> /**
--
Regards,
Laurent Pinchart
[GIT PULL FOR v4.20] R-Car VSP1 changes
Hi Mauro, The following changes since commit 78cf8c842c111df656c63b5d04997ea4e40ef26a: media: drxj: fix spelling mistake in fall-through annotations (2018-09-12 11:21:52 -0400) are available in the Git repository at: git://linuxtv.org/pinchartl/media.git tags/vsp1-next-20180914 for you to fetch changes up to 585e8c594967672aeca0ec90a6472f7c13dc30a2: media: vsp1: Document max_width restriction on UDS (2018-09-15 17:27:37 +0300) R-Car VSP1 changes for v4.20 Kieran Bingham (5): MAINTAINERS: VSP1: Add co-maintainer media: vsp1: Remove artificial minimum width/height limitation media: vsp1: use periods at the end of comment sentences media: vsp1: Document max_width restriction on SRU media: vsp1: Document max_width restriction on UDS Koji Matsuoka (1): media: vsp1: Fix YCbCr planar formats pitch calculation Laurent Pinchart (2): media: vsp1: Fix vsp1_regs.h license header media: vsp1: Update LIF buffer thresholds MAINTAINERS | 1 + drivers/media/platform/vsp1/vsp1_brx.c| 4 ++-- drivers/media/platform/vsp1/vsp1_drm.c| 11 ++- drivers/media/platform/vsp1/vsp1_drv.c| 6 +++--- drivers/media/platform/vsp1/vsp1_entity.c | 2 +- drivers/media/platform/vsp1/vsp1_lif.c| 29 + drivers/media/platform/vsp1/vsp1_regs.h | 2 +- drivers/media/platform/vsp1/vsp1_rpf.c| 4 ++-- drivers/media/platform/vsp1/vsp1_sru.c| 7 ++- drivers/media/platform/vsp1/vsp1_uds.c| 14 +++--- drivers/media/platform/vsp1/vsp1_video.c | 9 +++-- drivers/media/platform/vsp1/vsp1_wpf.c| 2 +- include/media/vsp1.h | 2 +- 13 files changed, 67 insertions(+), 26 deletions(-) -- Regards, Laurent Pinchart
Re: [PATCH 1/1] v4l: event: Prevent freeing event subscriptions while accessed
Hello,
On Wednesday, 12 September 2018 13:00:57 EEST Sakari Ailus wrote:
> On Wed, Sep 12, 2018 at 11:27:35AM +0200, Hans Verkuil wrote:
> > On 09/12/18 10:52, Sakari Ailus wrote:
> >> The event subscriptions are added to the subscribed event list while
> >> holding a spinlock, but that lock is subsequently released while still
> >> accessing the subscription object. This makes it possible to unsubscribe
> >> the event --- and freeing the subscription object's memory --- while
> >> the subscription object is simultaneously accessed.
> >
> > Hmm, the (un)subscribe ioctls are serialized through the ioctl lock,
> > so this could only be a scenario with drivers that do not use this
> > lock. Off-hand the only driver I know that does this is uvc.
> > Unfortunately,
> > that's a rather popular one.
>
> On video nodes, perhaps. But how about sub-device nodes? Generally drivers
> tend to do locking themselves, whether or not that is the best for most
> drivers.
I tend to agree with Sakari. Furthermore, having fine-grained locking is
better in my opinion than locking everything at the ioctl level, for drivers
that wish to do so. We should thus strive for self-contained locking in the
different helper libraries of V4L2.
> >> Prevent this by adding a mutex to serialise the event subscription and
> >> unsubscription. This also gives a guarantee to the callback ops that the
> >> add op has returned before the del op is called.
> >>
> >> This change also results in making the elems field less special:
> >> subscriptions are only added to the event list once they are fully
> >> initialised.
> >>
> >> Signed-off-by: Sakari Ailus
> >> ---
> >> Hi folks,
> >>
> >> I noticed this while working to add support for media events. This seems
> >> like material for the stable trees.
> >
> > I'd say 'no need for this' if it wasn't for uvc.
> >
> >> drivers/media/v4l2-core/v4l2-event.c | 35 -
> >> drivers/media/v4l2-core/v4l2-fh.c| 2 ++
> >> include/media/v4l2-fh.h | 4
> >> 3 files changed, 24 insertions(+), 17 deletions(-)
[snip]
> >> diff --git a/include/media/v4l2-fh.h b/include/media/v4l2-fh.h
> >> index ea73fef8bdc0..1be45a5f6383 100644
> >> --- a/include/media/v4l2-fh.h
> >> +++ b/include/media/v4l2-fh.h
> >> @@ -42,6 +42,9 @@ struct v4l2_ctrl_handler;
> >> * @available: list of events waiting to be dequeued
> >> * @navailable: number of available events at @available list
> >> * @sequence: event sequence number
> >> + * @mutex: hold event subscriptions during subscribing;
> >> + * guarantee that the add and del event callbacks are orderly
> >> called
Could you try to describe what this mutex protects in terms of data ?
> >> + *
Extra blank line ?
> >> * @m2m_ctx: pointer to v4l2_m2m_ctx
> >> */
> >>
> >> struct v4l2_fh {
> >> @@ -56,6 +59,7 @@ struct v4l2_fh {
> >>struct list_headavailable;
> >>unsigned intnavailable;
> >>u32 sequence;
> >> + struct mutexmutex;
> >
> > I don't like the name 'mutex'. Perhaps something more descriptive like:
> > 'subscribe_lock'?
> >
> >> #if IS_ENABLED(CONFIG_V4L2_MEM2MEM_DEV)
> >>
> >>struct v4l2_m2m_ctx *m2m_ctx;
> >
> > Overall I think this patch makes sense. The code is cleaner and easier to
> > follow. Just give 'mutex' a better name :-)
>
> How about "subscribe_mutex"? It's a mutex... "subscribe_lock" would use a
> similar convention elsewhere in V4L2 where mutexes are commonly called
> locks, so I'm certainly fine with that as well.
We indeed use lock for both mutexes and spinlocks. I have a slight preference
for the name lock myself, but I don't mind too much.
--
Regards,
Laurent Pinchart
Re: [PATCH 5/6] media: isp: fix a warning about a wrong struct initializer
Hi Mauro,
On Friday, 7 September 2018 14:46:34 EEST Laurent Pinchart wrote:
> Hi Mauro,
>
> As maintainers should be held to the same level of obligations as
> developers, and to avoid demotivating reviewers, could you handle comments
> you receive before pushing your own patches to your tree ? There should be
> no maintainer privilege here.
Ping ?
> On Wednesday, 8 August 2018 18:45:49 EEST Laurent Pinchart wrote:
> > Hi Mauro,
> >
> > Thank you for the patch.
> >
> > The subject line should be "media: omap3isp: ...".
> >
> > On Wednesday, 8 August 2018 17:52:55 EEST Mauro Carvalho Chehab wrote:
> >> As sparse complains:
> >>drivers/media/platform/omap3isp/isp.c:303:39: warning: Using plain
> >>integer as NULL pointer
> >>
> >> when a struct is initialized with { 0 }, actually the first
> >> element of the struct is initialized with zeros, initializing the
> >> other elements recursively. That can even generate gcc warnings
> >> on nested structs.
> >>
> > > So, instead, use the gcc-specific syntax for that (with is used
> >> broadly inside the Kernel), initializing it with {};
> >>
> >> Signed-off-by: Mauro Carvalho Chehab
> >> ---
> >>
> >> drivers/media/platform/omap3isp/isp.c | 2 +-
> >> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/media/platform/omap3isp/isp.c
> >> b/drivers/media/platform/omap3isp/isp.c index 03354d513311..842e2235047d
> >> 100644
> >> --- a/drivers/media/platform/omap3isp/isp.c
> >> +++ b/drivers/media/platform/omap3isp/isp.c
> >> @@ -300,7 +300,7 @@ static struct clk *isp_xclk_src_get(struct
> >> of_phandle_args *clkspec, void *data)
> >> static int isp_xclk_init(struct isp_device *isp)
> >> {
> >>struct device_node *np = isp->dev->of_node;
> >> - struct clk_init_data init = { 0 };
> >> + struct clk_init_data init = {};
> >
> > How about = { NULL }; to avoid a gcc-specific syntax ?
> >
> >>unsigned int i;
> >>
> >>for (i = 0; i < ARRAY_SIZE(isp->xclks); ++i)
--
Regards,
Laurent Pinchart
Re: [PATCH v2 5/5] media: ov5640: fix restore of last mode set
Hi Hugues,
On Monday, 10 September 2018 18:14:45 EEST Hugues FRUCHET wrote:
> On 09/07/2018 04:18 PM, Laurent Pinchart wrote:
> > On Thursday, 16 August 2018 18:07:54 EEST Hugues FRUCHET wrote:
> >> On 08/16/2018 12:10 PM, jacopo mondi wrote:
> >>> On Mon, Aug 13, 2018 at 12:19:46PM +0200, Hugues Fruchet wrote:
> >>>
> >>>> Mode setting depends on last mode set, in particular
> >>>> because of exposure calculation when downscale mode
> >>>> change between subsampling and scaling.
> >>>> At stream on the last mode was wrongly set to current mode,
> >>>> so no change was detected and exposure calculation
> >>>> was not made, fix this.
> >>>
> >>> I actually see a different issue here...
> >>
> >> Which problem do you have exactly, you got a VGA JPEG instead of a QVGA
> >> YUYV ?
> >>
> >>> The issue I see here depends on the format programmed through
> >>> set_fmt() never being applied when using the sensor with a media
> >>> controller equipped device (in this case an i.MX6 board) through
> >>> capture sessions, and the not properly calculated exposure you see may
> >>> be a consequence of this.
> >>>
> >>> I'll try to write down what I see, with the help of some debug output.
> >>>
> >>> - At probe time mode 640x460@30 is programmed:
> >>>
> >>> [1.651216] ov5640_probe: Initial mode with id: 2
> >>>
> >>> - I set the format on the sensor's pad and it gets not applied but
> >>> marked as pending as the sensor is powered off:
>>>
> >>> #media-ctl --set-v4l2 "'ov5640 2-003c':0[fmt:UYVY2X8/320x240
> >>> field:none]"
> >>> [ 65.611983] ov5640_set_fmt: NEW mode with id: 1 - PENDING
> >>
> >> So here sensor->current_mode is set to <1>;//QVGA
> >> and sensor->pending_mode_change is set to true;
> >>
> >>> - I start streaming with yavta, and the sensor receives a power on;
> >>> this causes the 'initial' format to be re-programmed and the
> >>> pending change to be ignored:
> >>>
> >>> #yavta -c10 -n4 -f YUYV -s $320x240 -F"../frame-#.yuv" /dev/video4
> >>>
> >>> [ 69.395018] ov5640_set_power:1805 - on
> >>> [ 69.431342] ov5640_restore_mode:1711
> >>> [ 69.996882] ov5640_set_mode: Apply mode with id: 0
> >>>
> >>> The 'ov5640_set_mode()' call from 'ov5640_restore_mode()' clears
> >>> the sensor->pending flag, discarding the newly requested format, for
> >>> this reason, at s_stream() time, the pending flag is not set
> >>> anymore.
> >>
> >> OK but before clearing sensor->pending_mode_change, set_mode() is
> >> loading registers corresponding to sensor->current_mode:
> >>
> >> static int ov5640_set_mode(struct ov5640_dev *sensor,
> >> const struct ov5640_mode_info *orig_mode)
> >> {
> >> ==>const struct ov5640_mode_info *mode = sensor->current_mode;
> >> ...
> >>ret = ov5640_set_mode_direct(sensor, mode, exposure);
> >>
> >> => so mode <1> is expected to be set now, so I don't understand your
> >> trace:
> >> "> [ 69.996882] ov5640_set_mode: Apply mode with id: 0"
> >> Which variable do you trace that shows "0" ?
> >>
> >>> Are you using a media-controller system? I suspect in non-mc cases,
> >>> the set_fmt is applied through a single power_on/power_off session, not
> >>> causing the 'restore_mode()' issue. Is this the case for you or your
> >>> issue is differnt?
> >>>
> >>> Edit:
> >>> Mita-san tried to address the issue of the output pixel format not
> >>> being restored when the image format was restored in
> >>> 19ad26f9e6e1 ("media: ov5640: add missing output pixel format setting")
> >>>
> >>> I understand the issue he tried to fix, but shouldn't the pending
> >>> format (if any) be applied instead of the initial one unconditionally?
> >>
> >> This is what does the ov5640_restore_mode(), set the current mode
> >> (sensor->current_mode), that is done through this line:
> >>
> >>/* now restore the last capture mode */
> >>ret = ov5640_set_mode(sensor,
Re: [PATCH v2 4/5] media: ov5640: fix auto controls values when switching to manual mode
Hi Hugues,
(Hans, there's a question for you below)
On Monday, 10 September 2018 17:43:27 EEST Hugues FRUCHET wrote:
> On 09/10/2018 12:46 PM, Laurent Pinchart wrote:
> > On Monday, 10 September 2018 13:23:41 EEST Hugues FRUCHET wrote:
> >> On 09/06/2018 03:31 PM, Laurent Pinchart wrote:
> >>> On Monday, 13 August 2018 13:19:45 EEST Hugues Fruchet wrote:
> >>>
> >>>> When switching from auto to manual mode, V4L2 core is calling
> >>>> g_volatile_ctrl() in manual mode in order to get the manual initial
> >>>> value. Remove the manual mode check/return to not break this
> >>>> behaviour.
> >>>>
> >>>> Signed-off-by: Hugues Fruchet
> >>>> ---
> >>>>drivers/media/i2c/ov5640.c | 4
> >>>>1 file changed, 4 deletions(-)
> >>>>
> >>>> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> >>>> index 9fb17b5..c110a6a 100644
> >>>> --- a/drivers/media/i2c/ov5640.c
> >>>> +++ b/drivers/media/i2c/ov5640.c
> >>>> @@ -2277,16 +2277,12 @@ static int ov5640_g_volatile_ctrl(struct
> >>>> v4l2_ctrl *ctrl)
> >>>>
> >>>> switch (ctrl->id) {
> >>>> case V4L2_CID_AUTOGAIN:
> >>>> -if (!ctrl->val)
> >>>> -return 0;
> >>>> val = ov5640_get_gain(sensor);
> >>>> if (val < 0)
> >>>> return val;
> >>>> sensor->ctrls.gain->val = val;
> >>>> break;
> >>>
> >>> What is this even supposed to do ? Only the V4L2_CID_GAIN and
> >>> V4L2_CID_EXPOSURE have the volatile flag set. Why can't this code be
> >>> replaced with
> >>
> >> This is because V4L2_CID_AUTOGAIN & V4L2_CID_GAIN are declared as
> >> auto-cluster:
> >>
> >>static int ov5640_init_controls(struct ov5640_dev *sensor)
> >>
> >>/* Auto/manual gain */
> >>ctrls->auto_gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTOGAIN,
> >> 0, 1, 1, 1);
> >>ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
> >>0, 1023, 1, 0);
> >>
> >> [...]
> >>
> >>v4l2_ctrl_auto_cluster(2, >auto_gain, 0, true);
> >>
> >> By checking many other drivers that are using clustered auto controls,
> >> they are all doing that way:
> >>
> >> ctrls->auto_x = v4l2_ctrl_new_std(CID_X_AUTO..
> >> ctrls->x = v4l2_ctrl_new_std(CID_X..
> >> v4l2_ctrl_auto_cluster(2, >auto, 0, true);
> >>
> >> g_volatile_ctrl(ctrl)
> >> switch (ctrl->id) {
> >> case CID_X_AUTO:
> >>ctrls->x->val = READ_REG()
> >
> > Seems like cargo-cult to me. Why is this better than the construct below
> > ?
>
> I have done the changes as per your suggestion, but behaviour is broken:
> when autogain control is on and I read gain value, gain is not refreshed
> with current gain value from sensor, but stick to last manual value set.
This looks like an issue in the control framework. We shouldn't have to force
all drivers to implement workarounds.
Hans, what's your opinion ?
> Moreover I've checked in vivid how it is done and still we have the
> structure of code I've already mentionned:
>
> static int vivid_user_vid_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
> {
> struct vivid_dev *dev = container_of(ctrl->handler, struct vivid_dev,
> ctrl_hdl_user_vid);
>
> switch (ctrl->id) {
> case V4L2_CID_AUTOGAIN:
> dev->gain->val = dev->jiffies_vid_cap & 0xff;
> break;
> }
> return 0;
> }
>
> >>> case V4L2_CID_GAIN:
> >>> val = ov5640_get_gain(sensor);
> >>> if (val < 0)
> >>> return val;
> >>> ctrl->val = val;
> >>> break;
> >>>
> >>>> case V4L2_CID_EXPOSURE_AUTO:
> >>>> -if (ctrl->val == V4L2_EXPOSURE_MANUAL)
> >>>> -return 0;
> >>>> val = ov5640_get_exposure(sensor);
> >>>> if (val < 0)
> >>>> return val;
> >>>
> >>> And same here.
--
Regards,
Laurent Pinchart
Re: [PATCH v2 4/5] media: ov5640: fix auto controls values when switching to manual mode
Hi Hugues,
On Monday, 10 September 2018 13:23:41 EEST Hugues FRUCHET wrote:
> On 09/06/2018 03:31 PM, Laurent Pinchart wrote:
> > On Monday, 13 August 2018 13:19:45 EEST Hugues Fruchet wrote:
> >
> >> When switching from auto to manual mode, V4L2 core is calling
> >> g_volatile_ctrl() in manual mode in order to get the manual initial
> >> value. Remove the manual mode check/return to not break this behaviour.
> >>
> >> Signed-off-by: Hugues Fruchet
> >> ---
> >> drivers/media/i2c/ov5640.c | 4
> >> 1 file changed, 4 deletions(-)
> >>
> >> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> >> index 9fb17b5..c110a6a 100644
> >> --- a/drivers/media/i2c/ov5640.c
> >> +++ b/drivers/media/i2c/ov5640.c
> >> @@ -2277,16 +2277,12 @@ static int ov5640_g_volatile_ctrl(struct
> >> v4l2_ctrl *ctrl)
> >>
> >>switch (ctrl->id) {
> >>case V4L2_CID_AUTOGAIN:
> >> - if (!ctrl->val)
> >> - return 0;
> >>
> >>val = ov5640_get_gain(sensor);
> >>if (val < 0)
> >>return val;
> >>
> >>sensor->ctrls.gain->val = val;
> >>break;
> >
> > What is this even supposed to do ? Only the V4L2_CID_GAIN and
> > V4L2_CID_EXPOSURE have the volatile flag set. Why can't this code be
> > replaced with
>
> This is because V4L2_CID_AUTOGAIN & V4L2_CID_GAIN are declared as
> auto-cluster:
> static int ov5640_init_controls(struct ov5640_dev *sensor)
> /* Auto/manual gain */
> ctrls->auto_gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTOGAIN,
>0, 1, 1, 1);
> ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
> 0, 1023, 1, 0);
> [...]
> v4l2_ctrl_auto_cluster(2, >auto_gain, 0, true);
>
> By checking many other drivers that are using clustered auto controls,
> they are all doing that way:
>
> ctrls->auto_x = v4l2_ctrl_new_std(CID_X_AUTO..
> ctrls->x = v4l2_ctrl_new_std(CID_X..
> v4l2_ctrl_auto_cluster(2, >auto, 0, true);
>
> g_volatile_ctrl(ctrl)
>switch (ctrl->id) {
> case CID_X_AUTO:
> ctrls->x->val = READ_REG()
Seems like cargo-cult to me. Why is this better than the construct below ?
> > case V4L2_CID_GAIN:
> > val = ov5640_get_gain(sensor);
> > if (val < 0)
> > return val;
> > ctrl->val = val;
> > break;
> >
> >>case V4L2_CID_EXPOSURE_AUTO:
> >> - if (ctrl->val == V4L2_EXPOSURE_MANUAL)
> >> - return 0;
> >>val = ov5640_get_exposure(sensor);
> >>if (val < 0)
> >>return val;
> >
> > And same here.
--
Regards,
Laurent Pinchart
Re: [PATCH v2 5/5] media: ov5640: fix restore of last mode set
Hello Hugues,
On Thursday, 16 August 2018 18:07:54 EEST Hugues FRUCHET wrote:
> On 08/16/2018 12:10 PM, jacopo mondi wrote:
> > On Mon, Aug 13, 2018 at 12:19:46PM +0200, Hugues Fruchet wrote:
> >
> >> Mode setting depends on last mode set, in particular
> >> because of exposure calculation when downscale mode
> >> change between subsampling and scaling.
> >> At stream on the last mode was wrongly set to current mode,
> >> so no change was detected and exposure calculation
> >> was not made, fix this.
> >
> > I actually see a different issue here...
>
> Which problem do you have exactly, you got a VGA JPEG instead of a QVGA
> YUYV ?
>
> > The issue I see here depends on the format programmed through
> > set_fmt() never being applied when using the sensor with a media
> > controller equipped device (in this case an i.MX6 board) through
> > capture sessions, and the not properly calculated exposure you see may
> > be a consequence of this.
> >
> > I'll try to write down what I see, with the help of some debug output.
> >
> > - At probe time mode 640x460@30 is programmed:
> >
> >[1.651216] ov5640_probe: Initial mode with id: 2
> >
> > - I set the format on the sensor's pad and it gets not applied but
> >marked as pending as the sensor is powered off:
> >
> >#media-ctl --set-v4l2 "'ov5640 2-003c':0[fmt:UYVY2X8/320x240
> >field:none]"
> > [ 65.611983] ov5640_set_fmt: NEW mode with id: 1 - PENDING
>
> So here sensor->current_mode is set to <1>;//QVGA
> and sensor->pending_mode_change is set to true;
>
> > - I start streaming with yavta, and the sensor receives a power on;
> >this causes the 'initial' format to be re-programmed and the pending
> >change to be ignored:
> >
> >#yavta -c10 -n4 -f YUYV -s $320x240 -F"../frame-#.yuv" /dev/video4
> >
> > [ 69.395018] ov5640_set_power:1805 - on
> > [ 69.431342] ov5640_restore_mode:1711
> > [ 69.996882] ov5640_set_mode: Apply mode with id: 0
> >
> >The 'ov5640_set_mode()' call from 'ov5640_restore_mode()' clears the
> >sensor->pending flag, discarding the newly requested format, for
> >this reason, at s_stream() time, the pending flag is not set
> >anymore.
>
> OK but before clearing sensor->pending_mode_change, set_mode() is
> loading registers corresponding to sensor->current_mode:
> static int ov5640_set_mode(struct ov5640_dev *sensor,
> const struct ov5640_mode_info *orig_mode)
> {
> ==> const struct ov5640_mode_info *mode = sensor->current_mode;
> ...
> ret = ov5640_set_mode_direct(sensor, mode, exposure);
>
> => so mode <1> is expected to be set now, so I don't understand your trace:
> "> [ 69.996882] ov5640_set_mode: Apply mode with id: 0"
> Which variable do you trace that shows "0" ?
>
> > Are you using a media-controller system? I suspect in non-mc cases,
> > the set_fmt is applied through a single power_on/power_off session, not
> > causing the 'restore_mode()' issue. Is this the case for you or your
> > issue is differnt?
> >
> > Edit:
> > Mita-san tried to address the issue of the output pixel format not
> > being restored when the image format was restored in
> > 19ad26f9e6e1 ("media: ov5640: add missing output pixel format setting")
> >
> > I understand the issue he tried to fix, but shouldn't the pending
> > format (if any) be applied instead of the initial one unconditionally?
>
> This is what does the ov5640_restore_mode(), set the current mode
> (sensor->current_mode), that is done through this line:
> /* now restore the last capture mode */
> ret = ov5640_set_mode(sensor, _mode_init_data);
> => note that the comment above is weird, in fact it is the "current"
> mode that is set.
> => note also that the 2nd parameter is not the mode to be set but the
> previously applied mode ! (ie loaded in ov5640 registers). This is used
> to decide if we have to go to the "set_mode_exposure_calc" or
> "set_mode_direct".
>
> the ov5640_restore_mode() also set the current pixel format
> (sensor->fmt), that is done through this line:
> return ov5640_set_framefmt(sensor, >fmt);
> ==> This is what have fixed Mita-san, this line was missing previously,
> leading to "mode registers" being loaded but not the "pixel format
> registers".
This seems overly complicated to me. Why do we have to set the
Re: [PATCH 5/6] media: isp: fix a warning about a wrong struct initializer
Hi Mauro,
As maintainers should be held to the same level of obligations as developers,
and to avoid demotivating reviewers, could you handle comments you receive
before pushing your own patches to your tree ? There should be no maintainer
privilege here.
On Wednesday, 8 August 2018 18:45:49 EEST Laurent Pinchart wrote:
> Hi Mauro,
>
> Thank you for the patch.
>
> The subject line should be "media: omap3isp: ...".
>
> On Wednesday, 8 August 2018 17:52:55 EEST Mauro Carvalho Chehab wrote:
> > As sparse complains:
> > drivers/media/platform/omap3isp/isp.c:303:39: warning: Using plain
> > integer
> >
> > as NULL pointer
> >
> > when a struct is initialized with { 0 }, actually the first
> > element of the struct is initialized with zeros, initializing the
> > other elements recursively. That can even generate gcc warnings
> > on nested structs.
> >
> > So, instead, use the gcc-specific syntax for that (with is used
> > broadly inside the Kernel), initializing it with {};
> >
> > Signed-off-by: Mauro Carvalho Chehab
> > ---
> >
> > drivers/media/platform/omap3isp/isp.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/platform/omap3isp/isp.c
> > b/drivers/media/platform/omap3isp/isp.c index 03354d513311..842e2235047d
> > 100644
> > --- a/drivers/media/platform/omap3isp/isp.c
> > +++ b/drivers/media/platform/omap3isp/isp.c
> > @@ -300,7 +300,7 @@ static struct clk *isp_xclk_src_get(struct
> > of_phandle_args *clkspec, void *data) static int isp_xclk_init(struct
> > isp_device *isp)
> >
> > {
> >
> > struct device_node *np = isp->dev->of_node;
> >
> > - struct clk_init_data init = { 0 };
> > + struct clk_init_data init = {};
>
> How about = { NULL }; to avoid a gcc-specific syntax ?
>
> > unsigned int i;
> >
> > for (i = 0; i < ARRAY_SIZE(isp->xclks); ++i)
--
Regards,
Laurent Pinchart
Re: [PATCH 2/2] media: ov5640: Fix auto-exposure disabling
/ov5640.c
> > >> index 12b3496..bc75cb7 100644
> > >> --- a/drivers/media/i2c/ov5640.c
> > >> +++ b/drivers/media/i2c/ov5640.c
> > >> @@ -1588,25 +1588,13 @@ static int ov5640_set_mode_exposure_calc(struct
> > >> ov5640_dev *sensor,> >>
> > >>* change mode directly
> > >>*/
> > >>
> > >> static int ov5640_set_mode_direct(struct ov5640_dev *sensor,
> > >>
> > >> - const struct ov5640_mode_info *mode,
> > >> - s32 exposure)
> > >> + const struct ov5640_mode_info *mode)
> > >>
> > >> {
> > >>
> > >> -int ret;
> > >> -
> > >>
> > >> if (!mode->reg_data)
> > >>
> > >> return -EINVAL;
> > >>
> > >> /* Write capture setting */
> > >>
> > >> -ret = ov5640_load_regs(sensor, mode);
> > >> -if (ret < 0)
> > >> -return ret;
> > >> -
> > >> -/* turn auto gain/exposure back on for direct mode */
> > >> -ret = __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_gain, 1);
> > >> -if (ret)
> > >> -return ret;
> > >> -
> > >> -return __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_exp, exposure);
> > >> +return ov5640_load_regs(sensor, mode);
> > >>
> > >> }
> > >>
> > >> static int ov5640_set_mode(struct ov5640_dev *sensor,
> > >>
> > >> @@ -1626,7 +1614,7 @@ static int ov5640_set_mode(struct ov5640_dev
> > >> *sensor,
> > >>
> > >> return ret;
> > >>
> > >> exposure = sensor->ctrls.auto_exp->val;
> > >>
> > >> -ret = ov5640_set_exposure(sensor, V4L2_EXPOSURE_MANUAL);
> > >> +ret = __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_exp,
> > >> V4L2_EXPOSURE_MANUAL);
> > >>
> > >> if (ret)
> > >>
> > >> return ret;
> > >>
> > >> @@ -1642,12 +1630,21 @@ static int ov5640_set_mode(struct ov5640_dev
> > >> *sensor,> >>
> > >> * change inside subsampling or scaling
> > >> * download firmware directly
> > >> */
> > >>
> > >> -ret = ov5640_set_mode_direct(sensor, mode, exposure);
> > >> +ret = ov5640_set_mode_direct(sensor, mode);
> > >>
> > >> }
> > >>
> > >> if (ret < 0)
> > >>
> > >> return ret;
> > >>
> > >> +/* Restore auto-gain and auto-exposure after mode has changed.
> > >> */
> > >> +ret = __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_gain, 1);
> > >> +if (ret)
> > >> +return ret;
> > >> +
> > >> +ret = __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_exp, exposure)
> > >> +if (ret)
> > >> +return ret;
> > >> +
> > >>
> > >> ret = ov5640_set_binning(sensor, dn_mode != SCALING);
> > >> if (ret < 0)
> > >>
> > >> return ret;
> > >>
> > >> --
> > >> 2.7.4
--
Regards,
Laurent Pinchart
Re: [PATCH v2 4/5] media: ov5640: fix auto controls values when switching to manual mode
Hi Hugues,
Thank you for the patch.
On Monday, 13 August 2018 13:19:45 EEST Hugues Fruchet wrote:
> When switching from auto to manual mode, V4L2 core is calling
> g_volatile_ctrl() in manual mode in order to get the manual initial value.
> Remove the manual mode check/return to not break this behaviour.
>
> Signed-off-by: Hugues Fruchet
> ---
> drivers/media/i2c/ov5640.c | 4
> 1 file changed, 4 deletions(-)
>
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 9fb17b5..c110a6a 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -2277,16 +2277,12 @@ static int ov5640_g_volatile_ctrl(struct v4l2_ctrl
> *ctrl)
>
> switch (ctrl->id) {
> case V4L2_CID_AUTOGAIN:
> - if (!ctrl->val)
> - return 0;
> val = ov5640_get_gain(sensor);
> if (val < 0)
> return val;
> sensor->ctrls.gain->val = val;
> break;
What is this even supposed to do ? Only the V4L2_CID_GAIN and
V4L2_CID_EXPOSURE have the volatile flag set. Why can't this code be replaced
with
case V4L2_CID_GAIN:
val = ov5640_get_gain(sensor);
if (val < 0)
return val;
ctrl->val = val;
break;
> case V4L2_CID_EXPOSURE_AUTO:
> - if (ctrl->val == V4L2_EXPOSURE_MANUAL)
> - return 0;
> val = ov5640_get_exposure(sensor);
> if (val < 0)
> return val;
And same here.
--
Regards,
Laurent Pinchart
Re: [PATCH v2 1/5] media: ov5640: fix exposure regression
Hi Hugues,
Thank you for the patch.
On Monday, 13 August 2018 13:19:42 EEST Hugues Fruchet wrote:
> fixes: bf4a4b518c20 ("media: ov5640: Don't force the auto exposure state at
> start time").
>
> Symptom was black image when capturing HD or 5Mp picture
> due to manual exposure set to 1 while it was intended to
> set autoexposure to "manual", fix this.
>
> Signed-off-by: Hugues Fruchet
Reviewed-by: Laurent Pinchart
> ---
> drivers/media/i2c/ov5640.c | 18 --
> 1 file changed, 12 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 1ecbb7a..4b9da8b 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -938,6 +938,12 @@ static int ov5640_load_regs(struct ov5640_dev *sensor,
> return ret;
> }
>
> +static int ov5640_set_autoexposure(struct ov5640_dev *sensor, bool on)
> +{
> + return ov5640_mod_reg(sensor, OV5640_REG_AEC_PK_MANUAL,
> + BIT(0), on ? 0 : BIT(0));
> +}
> +
> /* read exposure, in number of line periods */
> static int ov5640_get_exposure(struct ov5640_dev *sensor)
> {
> @@ -1593,7 +1599,7 @@ static int ov5640_set_mode_exposure_calc(struct
> ov5640_dev *sensor, */
> static int ov5640_set_mode_direct(struct ov5640_dev *sensor,
> const struct ov5640_mode_info *mode,
> - s32 exposure)
> + bool auto_exp)
> {
> int ret;
>
> @@ -1610,7 +1616,8 @@ static int ov5640_set_mode_direct(struct ov5640_dev
> *sensor, if (ret)
> return ret;
>
> - return __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_exp, exposure);
> + return __v4l2_ctrl_s_ctrl(sensor->ctrls.auto_exp, auto_exp ?
> + V4L2_EXPOSURE_AUTO : V4L2_EXPOSURE_MANUAL);
> }
>
> static int ov5640_set_mode(struct ov5640_dev *sensor,
> @@ -1618,7 +1625,7 @@ static int ov5640_set_mode(struct ov5640_dev *sensor,
> {
> const struct ov5640_mode_info *mode = sensor->current_mode;
> enum ov5640_downsize_mode dn_mode, orig_dn_mode;
> - s32 exposure;
> + bool auto_exp = sensor->ctrls.auto_exp->val == V4L2_EXPOSURE_AUTO;
> int ret;
>
> dn_mode = mode->dn_mode;
> @@ -1629,8 +1636,7 @@ static int ov5640_set_mode(struct ov5640_dev *sensor,
> if (ret)
> return ret;
>
> - exposure = sensor->ctrls.auto_exp->val;
> - ret = ov5640_set_exposure(sensor, V4L2_EXPOSURE_MANUAL);
> + ret = ov5640_set_autoexposure(sensor, false);
> if (ret)
> return ret;
>
> @@ -1646,7 +1652,7 @@ static int ov5640_set_mode(struct ov5640_dev *sensor,
>* change inside subsampling or scaling
> * download firmware directly
>*/
> - ret = ov5640_set_mode_direct(sensor, mode, exposure);
> + ret = ov5640_set_mode_direct(sensor, mode, auto_exp);
> }
>
> if (ret < 0)
--
Regards,
Laurent Pinchart
Re: [PATCH v2 3/5] media: ov5640: fix wrong binning value in exposure calculation
Hi Hugues,
Thank you for the patch.
On Monday, 13 August 2018 13:19:44 EEST Hugues Fruchet wrote:
> ov5640_set_mode_exposure_calc() is checking binning value but
> binning value read is buggy, fix this.
> Rename ov5640_binning_on() to ov5640_get_binning() as per other
> similar functions.
>
> Signed-off-by: Hugues Fruchet
Reiewed-by: Laurent Pinchart
> ---
> drivers/media/i2c/ov5640.c | 8
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
> index 7c569de..9fb17b5 100644
> --- a/drivers/media/i2c/ov5640.c
> +++ b/drivers/media/i2c/ov5640.c
> @@ -1349,7 +1349,7 @@ static int ov5640_set_ae_target(struct ov5640_dev
> *sensor, int target) return ov5640_write_reg(sensor, OV5640_REG_AEC_CTRL1F,
> fast_low); }
>
> -static int ov5640_binning_on(struct ov5640_dev *sensor)
> +static int ov5640_get_binning(struct ov5640_dev *sensor)
> {
> u8 temp;
> int ret;
> @@ -1357,8 +1357,8 @@ static int ov5640_binning_on(struct ov5640_dev
> *sensor) ret = ov5640_read_reg(sensor, OV5640_REG_TIMING_TC_REG21, );
> if (ret)
> return ret;
> - temp &= 0xfe;
> - return temp ? 1 : 0;
> +
> + return temp & BIT(0);
> }
>
> static int ov5640_set_binning(struct ov5640_dev *sensor, bool enable)
> @@ -1468,7 +1468,7 @@ static int ov5640_set_mode_exposure_calc(struct
> ov5640_dev *sensor, if (ret < 0)
> return ret;
> prev_shutter = ret;
> - ret = ov5640_binning_on(sensor);
> + ret = ov5640_get_binning(sensor);
> if (ret < 0)
> return ret;
> if (ret && mode->id != OV5640_MODE_720P_1280_720 &&
--
Regards,
Laurent Pinchart
Re: [PATCH 2/2] CNF4 pixel format for media subsystem
Hello Sergey,
On Thursday, 6 September 2018 12:37:36 EEST Hans Verkuil wrote:
> On 09/06/18 09:51, dorod...@gmail.com wrote:
> > From: Sergey Dorodnicov
> >
> > Registering new GUID used by Intel RealSense depth cameras with fourcc
> > CNF4, encoding sensor confidence information for every pixel.
s/confidence/depth confidence/
> >
> > Signed-off-by: Sergey Dorodnicov
> > Signed-off-by: Evgeni Raikhel
Could you please send me the output of lsusb -v (ideally run as root, or
through sudo) for a camera that uses this format ?
> > ---
> >
> > drivers/media/usb/uvc/uvc_driver.c | 5 +
> > drivers/media/usb/uvc/uvcvideo.h | 3 +++
> > 2 files changed, 8 insertions(+)
> >
> > diff --git a/drivers/media/usb/uvc/uvc_driver.c
> > b/drivers/media/usb/uvc/uvc_driver.c index d46dc43..c8d40a4 100644
> > --- a/drivers/media/usb/uvc/uvc_driver.c
> > +++ b/drivers/media/usb/uvc/uvc_driver.c
> > @@ -214,6 +214,11 @@ static struct uvc_format_desc uvc_fmts[] = {
> > .guid = UVC_GUID_FORMAT_INZI,
> > .fcc= V4L2_PIX_FMT_INZI,
> > },
> > + {
> > + .name = "Confidence 4-bit Packed (CNF4)",
>
> The name should correspond to what is set in v4l2-ioctl.c.
And should even be removed as it duplicates the names in v4l2-ioctl.c. I have
a half-baked patch to do so, I'll try to resurrect it. This isn't a blocking
issue, I'll rebase my patch on top of this one.
> > + .guid = UVC_GUID_FORMAT_CNF4,
> > + .fcc= V4L2_PIX_FMT_CNF4,
> > + },
> > };
> >
> > /* --
> > diff --git a/drivers/media/usb/uvc/uvcvideo.h
> > b/drivers/media/usb/uvc/uvcvideo.h index e5f5d84..779bab2 100644
> > --- a/drivers/media/usb/uvc/uvcvideo.h
> > +++ b/drivers/media/usb/uvc/uvcvideo.h
> > @@ -154,6 +154,9 @@
> > #define UVC_GUID_FORMAT_INVI \
> > { 'I', 'N', 'V', 'I', 0xdb, 0x57, 0x49, 0x5e, \
> > 0x8e, 0x3f, 0xf4, 0x79, 0x53, 0x2b, 0x94, 0x6f}
> > +#define UVC_GUID_FORMAT_CNF4 \
> > + { 'C', ' ', ' ', ' ', 0x00, 0x00, 0x10, 0x00, \
> > +0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
> >
> > #define UVC_GUID_FORMAT_D3DFMT_L8 \
> > {0x32, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, \
--
Regards,
Laurent Pinchart
[GIT PULL FOR v4.20] uvcvideo changes
Hi Mauro, The following changes since commit d842a7cf938b6e0f8a1aa9f1aec0476c9a599310: media: adv7842: enable reduced fps detection (2018-08-31 10:03:51 -0400) are available in the Git repository at: git://linuxtv.org/pinchartl/media.git tags/uvc-next-20180903 for you to fetch changes up to 8cefb491d85d35e9a2279a98f2545c10718524d7: media: uvcvideo: Add a D4M camera description (2018-09-03 19:00:47 +0300) UVC changes for v4.20 Colin Ian King (1): media: uvcvideo: Fix spelling mistake: "entites" -> "entities" Guennadi Liakhovetski (2): media: uvcvideo: Rename UVC_QUIRK_INFO to UVC_INFO_QUIRK media: uvcvideo: Add a D4M camera description Gustavo A. R. Silva (1): media: uvcvideo: Remove unnecessary NULL check before debugfs_remove_recursive Joe Perches (1): media: uvcvideo: Make some structs const Laurent Pinchart (2): media: uvcvideo: Make uvc_control_mapping menu_info field const media: uvcvideo: Store device information pointer in struct uvc_device Nadav Amit (1): media: uvcvideo: Fix uvc_alloc_entity() allocation alignment Documentation/media/uapi/v4l/meta-formats.rst | 1 + Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst | 210 + drivers/media/usb/uvc/uvc_ctrl.c | 14 +- drivers/media/usb/uvc/uvc_debugfs.c | 6 +- drivers/media/usb/uvc/uvc_driver.c| 53 +++--- drivers/media/usb/uvc/uvc_metadata.c | 7 +- drivers/media/usb/uvc/uvcvideo.h | 10 +- include/uapi/linux/videodev2.h| 1 + 8 files changed, 261 insertions(+), 41 deletions(-) create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst -- Regards, Laurent Pinchart
Re: [PATCH] uvcvideo: add a D4M camera description
Hi Guennadi, On Friday, 3 August 2018 14:07:12 EEST Guennadi Liakhovetski wrote: > Hi Laurent, > > Thanks for the review. A general note: I think you're requesting a rather > detailed information about many parameters. That isn't a problem by > itself, however, it is difficult to obtain some of that information. I'll > address whatever comments I can in an updated version, just answering some > questions here. I directed youor questions, that I couldn't answer myself > to respective people, but I have no idea if and when I get replies. So, > it's up to you whether to wait for that additional information or to take > at least what we have now. I've replied to v2, and apart from a few minor points, I think we can apply the current version. There are a few small questions I would still like to have answers to, but if it takes to long to obtain that, let's not miss v4.20. > On Sun, 29 Jul 2018, Laurent Pinchart wrote: > > On Saturday, 23 December 2017 13:11:00 EEST Guennadi Liakhovetski wrote: > >> From: Guennadi Liakhovetski > >> > >> D4M is a mobile model from the D4XX family of Intel RealSense cameras. > >> This patch adds a descriptor for it, which enables reading per-frame > >> metadata from it. > >> > >> Signed-off-by: Guennadi Liakhovetski > >> --- > >> > >> Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst | 202 > >> drivers/media/usb/uvc/uvc_driver.c| 11 ++ > >> include/uapi/linux/videodev2.h| 1 + > >> 3 files changed, 214 insertions(+) > >> create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst > >> > >> diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst > >> b/Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst new file mode 100644 > >> index 000..950780d > >> --- /dev/null > >> +++ b/Documentation/media/uapi/v4l/pixfmt-meta-d4xx.rst [snip] > >> +* - :cspan:`1` *Configuration* > >> +* - __u32 ID > >> + - 0x8002 > >> +* - __u32 Size > >> + - Size in bytes (currently 40) > >> +* - __u32 Version > >> + - Version of the struct > >> +* - __u32 Flags > >> + - A bitmask of flags: see [4_] below > >> +* - __u8 Hardware type > >> + - Camera hardware version [5_] > >> +* - __u8 SKU ID > >> + - Camera hardware configuration [6_] > >> +* - __u32 Cookie > >> + - Internal synchronisation > > > > Internal synchronisation with what ? :-) This is still something I'd like to understand (and I understand it may still take time to receive an answer from the right person). > >> +* - __u16 Format > >> + - Image format code [7_] > >> +* - __u16 Width > >> + - Width in pixels > >> +* - __u16 Height > >> + - Height in pixels > >> +* - __u16 Framerate > >> + - Requested framerate > > > > What's the unit of this value ? > > Is anything other than frames per second used in V4L? V4L2 expresses the frame rate as a fraction, hence my question, to know whether this field contained the number of frames per second as an integer, or used a different representation (such as a fixed point decimal value for instance). > >> +* - __u16 Trigger > >> + - Byte 0: bit 0: depth and RGB are synchronised, bit 1: external > >> trigger > >> + > >> +.. _1: > >> + > >> +[1] > >> https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-ext > >> ensions-1-5 > > > > Should we at some point replicate that documentation in the V4L2 spec ? > > Without copying it of course, as that would be a copyright violation. > > Well, we don't replicate the UVC itself or any other standards, do we? Of > course, that document doesn't have the same status as an official > vendor-neutral standard, but still, we don't replicate data sheets either. > Besides, I think there are cameras that use this, and windows supports > this, so, don't think it will disappear overnight... Probably not overnight, you're right. I'm a bit worried about the link becoming invalid though. In any case that's not a blocker, but I might at some point decide to replicate the documentation. [snip] -- Regards, Laurent Pinchart
Re: [PATCH v2 2/2] uvcvideo: add a D4M camera description
he left sensor, the fish eye camera, or both (or something else) ? > +* - __u32 ID > + - 0x8001 > +* - __u32 Size > + - Size in bytes (currently 40) > +* - __u32 Version > + - Version of the struct > +* - __u32 Flags > + - A bitmask of flags: see [3_] below > +* - __u32 Frame counter > + - Monotonically increasing counter I assume this increases by exactly one for every frame. I'll test it when I get back home, and will update the documentation accordingly. > +* - __u32 Optical time > + - Time in microseconds from the beginning of a frame till its middle I'm still puzzled by this value, as I expect it to be exactly half the exposure time, which is reported separately. If that's not the case I'd like to know what this represents. > +* - __u32 Readout time > + - Time, used to read out a frame in microseconds > +* - __u32 Exposure time > + - Frame exposure time in microseconds > +* - __u32 Frame interval > + - In microseconds = 100 / framerate > +* - __u32 Pipe latency > + - Time in microseconds from start of frame to data in USB buffer > +* - :cspan:`1` *Configuration* > +* - __u32 ID > + - 0x8002 > +* - __u32 Size > + - Size in bytes (currently 40) > +* - __u32 Version > + - Version of the struct > +* - __u32 Flags > + - A bitmask of flags: see [4_] below > +* - __u8 Hardware type > + - Camera hardware version [5_] > +* - __u8 SKU ID > + - Camera hardware configuration [6_] > +* - __u32 Cookie > + - Internal synchronisation > +* - __u16 Format > + - Image format code [7_] > +* - __u16 Width > + - Width in pixels > +* - __u16 Height > + - Height in pixels > +* - __u16 Framerate > + - Requested frame rate per second > +* - __u16 Trigger > + - Byte 0: bit 0: depth and RGB are synchronised, bit 1: external > trigger > + > +.. _1: > + > +[1] > https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-extens > ions-1-5 > + > +.. _2: > + > +[2] Depth Control flags specify which fields are valid: :: > + > + 0x0001 Gain > + 0x0002 Exposure > + 0x0004 Laser power > + 0x0008 AE mode > + 0x0010 Exposure priority > + 0x0020 AE ROI > + 0x0040 Preset > + > +.. _3: > + > +[3] Capture Timing flags specify which fields are valid: :: > + > + 0x0001 Frame counter > + 0x0002 Optical time > + 0x0004 Readout time > + 0x0008 Exposure time > + 0x0010 Frame interval > + 0x0020 Pipe latency > + > +.. _4: > + > +[4] Configuration flags specify which fields are valid: :: > + > + 0x0001 Hardware type > + 0x0002 SKU ID > + 0x0004 Cookie > + 0x0008 Format > + 0x0010 Width > + 0x0020 Height > + 0x0040 Framerate > + 0x0080 Trigger > + 0x0100 Cal count > + > +.. _5: > + > +[5] Camera model: :: > + > + 0 DS5 > + 1 IVCAM2 > + > +.. _6: > + > +[6] 8-bit camera hardware configuration bitfield: :: > + > + [1:0] depthCamera > + 00: no depth > + 01: standard depth > + 10: wide depth > + 11: reserved > + [2] depthIsActive - has a laser projector > + [3] RGB presence > + [4] IMU presence Do you mind if I write this "Inertial Measurement Unit (IMU) presense" ? Or is it just me who's not familiar enough with depth cameras ? :-) On a side note, does the camera expose IMU data to the host over USB ? > + [5] projectorType > + 0: HPTG > + 1: Princeton > + [6] 0: a projector, 1: an LED > + [7] reserved > + > +.. _7: > + > +[7] Image format codes per camera interface: I was initially puzzled by this until I read your explanation. How about wording it as "Image format codes per video streaming interface" to use the UVC vocabulary ? > + > +Depth: :: > + > + 1 Z16 > + 2 Z > + > +Left sensor: :: > + > + 1 Y8 > + 2 UYVY > + 3 R8L8 > + 4 Calibration > + 5 W10 > + > +Fish Eye sensor: :: > + > + 1 RAW8 If you agree with the comments above, I'll update the patch when applying it to my tree. I would however still like to get the following information - What are the version of the three structures documented in this patch ? I can check that when I get back home, but if you have the information it would be faster. - Do the fields in the Depth Control structure apply to the depth video stream only ? (I assume they do) - What do the fields in the Capture Control structure apply to ? (I assume the left sensor and/or fish eye video streams) - Does the optical time differ from half the exposure time ? If you think it will take time to get this information and we risk missing the next kernel version, I'm OK applying the patch already, but please then submit a follow-up patch (or just drop a mail in reply to this one with the information and I can turn that into a patch). [snip] -- Regards, Laurent Pinchart
Re: [PATCH] media: aptina-pll: allow approximating the requested pix_clock
Hi Helmut,
Thank you for the patch.
On Thursday, 23 August 2018 10:52:09 EEST Helmut Grohne wrote:
> Clock frequencies are not exact values, but rather imprecise, physical
> properties. The present pll computation however, treats them as exact.
> It tries to compute parameters that attain the requested pix_clock
> exactly. Failing that, it gives up.
>
> The new implementation approximates the requested pix_clock and reports
> the actual value resulting from the computed parameters. If the
> requested pix_clock can be attained, the original behaviour of
> maximizing p1 is retained.
>
> Experiments with the algorithm in userspace on an arm device show that
> the computational cost is negligible compared to executing a binary for
> all practical inputs.
Could you please share numbers, ideally when run in kernel space ?
> Signed-off-by: Helmut Grohne
> ---
> drivers/media/i2c/aptina-pll.c | 263 ++---
> drivers/media/i2c/mt9m032.c| 15 ++-
> 2 files changed, 165 insertions(+), 113 deletions(-)
>
> I tried using aptina_pll_calculate in a driver for a similar image sensor.
> Using it, I was unable to find a pix_clock value such that the computation
> was successful. Most of the practical parameters result in a non-integer
> pix_clock, something that struct pll cannot represent.
>
> The driver is not ready for submission at this point, but the changes to
> aptina_pll_calculate are reasonable on their own, which is why I submit them
> separately now.
This patch is very hard to review as you rewrite the whole, removing all the
documentation for the existing algorithm, without documenting the new one.
There's also no example of a failing case with the existing code that works
with yours.
Could you please document the algorithm in details (especially explaining the
background ideas), and share at least one use case, with with numbers for all
the input and output parameters ?
> diff --git a/drivers/media/i2c/aptina-pll.c b/drivers/media/i2c/aptina-pll.c
> index 224ae4e4cf8b..249018772b2b 100644
> --- a/drivers/media/i2c/aptina-pll.c
> +++ b/drivers/media/i2c/aptina-pll.c
> @@ -2,6 +2,7 @@
> * Aptina Sensor PLL Configuration
> *
> * Copyright (C) 2012 Laurent Pinchart
> + * Copyright (C) 2018 Intenta GmbH
> *
> * This program is free software; you can redistribute it and/or
> * modify it under the terms of the GNU General Public License
> @@ -14,23 +15,84 @@
> */
>
> #include
> -#include
> #include
> -#include
> +#include
> #include
>
> #include "aptina-pll.h"
>
> +/* A variant of mult_frac that works even when (x % denom) * numer produces
> a + * 32bit overflow.
> + */
> +#define mult_frac_exact(x, numer, denom) \
> + ((u32)div_u64(mul_u32_u32((x), (numer)), (denom)))
> +
> +static unsigned int absdiff(unsigned int x, unsigned int y)
> +{
> + return x > y ? x - y : y - x;
> +}
> +
> +/* Find n_min <= *np <= n_max and d_min <= *dp <= d_max such that *np / *dp
> + * approximates n_target / d_target.
> + */
> +static int approximate_fraction(unsigned int n_min, unsigned int n_max,
> + unsigned int d_min, unsigned int d_max,
> + unsigned int n_target, unsigned int d_target,
> + unsigned int *np, unsigned int *dp)
> +{
> + unsigned int d, best_err = UINT_MAX;
> +
> + d_min = max(d_min, mult_frac_exact(n_min, d_target, n_target));
> + d_max = min(d_max, mult_frac_exact(n_max, d_target, n_target) + 1);
> + if (d_min > d_max)
> + return -EINVAL;
> +
> + for (d = d_min; d < d_max; ++d) {
> + unsigned int n = mult_frac_exact(d, n_target, d_target);
> +
> + if (n >= n_min) {
> + unsigned int err = absdiff(
> + n_target, mult_frac_exact(n, d_target, d));
> +
> + if (err < best_err) {
> + best_err = err;
> + *np = n;
> + *dp = d;
> + }
> + if (err == 0)
> + return 0;
> + }
> + ++n;
> + if (n <= n_max) {
> + unsigned int err = absdiff(
> + n_target, mult_frac_exact(n, d_target, d));
> +
> + if (err < best_err) {
> + best_err = err;
> + *np = n;
> + *dp = d;
> + }
> + }
> + }
> + return best_err == UINT_MAX
Re: [PATCH v2 1/2] uvcvideo: rename UVC_QUIRK_INFO to UVC_INFO_QUIRK
Hi Guennadi, Thank you for the patch. On Monday, 20 August 2018 09:45:16 EEST Guennadi Liakhovetski wrote: > Hi Laurent, > > As far as I understand we've missed the 4.19 cycle. Can we move on with > this please? I've taken this patch in my tree and will try to handle the rest this week. -- Regards, Laurent Pinchart
Re: [PATCH v2 1/2] uvcvideo: rename UVC_QUIRK_INFO to UVC_INFO_QUIRK
Hi Guennadi,
On Monday, 20 August 2018 15:02:53 EEST Laurent Pinchart wrote:
> On Friday, 3 August 2018 14:36:56 EEST Guennadi Liakhovetski wrote:
> > This macro defines "information about quirks," not "quirks for
> > information."
To address Nicolas' objection, would you mind saying "device information
containing quirks" instead of "information about quirks" ? It you're fine with
that I'll fix it in my tree, there's no need to resubmit.
> > Signed-off-by: Guennadi Liakhovetski
>
> Reviewed-by: Laurent Pinchart
>
> and taken in my tree.
>
> > ---
> >
> > drivers/media/usb/uvc/uvc_driver.c | 18 +-
> > 1 file changed, 9 insertions(+), 9 deletions(-)
> >
> > diff --git a/drivers/media/usb/uvc/uvc_driver.c
> > b/drivers/media/usb/uvc/uvc_driver.c index d46dc43..699984b 100644
> > --- a/drivers/media/usb/uvc/uvc_driver.c
> > +++ b/drivers/media/usb/uvc/uvc_driver.c
> > @@ -2344,7 +2344,7 @@ static int uvc_clock_param_set(const char *val,
> > const
> > struct kernel_param *kp) .quirks = UVC_QUIRK_FORCE_Y8,
> >
> > };
> >
> > -#define UVC_QUIRK_INFO(q) (kernel_ulong_t)&(struct
> > uvc_device_info){.quirks = q} +#define UVC_INFO_QUIRK(q)
> > (kernel_ulong_t)&(struct
> > uvc_device_info){.quirks = q}
> >
> > /*
> >
> > * The Logitech cameras listed below have their interface class set to
> >
> > @@ -2453,7 +2453,7 @@ static int uvc_clock_param_set(const char *val,
> > const
> > struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> >
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info =
> > UVC_QUIRK_INFO(UVC_QUIRK_RESTORE_CTRLS_ON_INIT)
},
> > + .driver_info =
> > UVC_INFO_QUIRK(UVC_QUIRK_RESTORE_CTRLS_ON_INIT)
},
> >
> > /* Chicony CNF7129 (Asus EEE 100HE) */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > | USB_DEVICE_ID_MATCH_INT_INFO,
> >
> > @@ -2462,7 +2462,7 @@ static int uvc_clock_param_set(const char *val,
> > const
> > struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> >
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_RESTRICT_FRAME_RATE)
> > },
> > + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_RESTRICT_FRAME_RATE)
> > },
> >
> > /* Alcor Micro AU3820 (Future Boy PC USB Webcam) */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > | USB_DEVICE_ID_MATCH_INT_INFO,
> >
> > @@ -2525,7 +2525,7 @@ static int uvc_clock_param_set(const char *val,
> > const
> > struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> >
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
> >
> > | UVC_QUIRK_BUILTIN_ISIGHT) },
> >
> > /* Apple Built-In iSight via iBridge */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > @@ -2607,7 +2607,7 @@ static int uvc_clock_param_set(const char *val,
> > const
> > struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> >
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
> >
> > | UVC_QUIRK_PROBE_DEF) },
> >
> > /* IMC Networks (Medion Akoya) */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > @@ -2707,7 +2707,7 @@ static int uvc_clock_param_set(const char *val,
> > const
> > struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> >
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
> >
> > | UVC_QUIRK_PROBE_EXTRAFIELDS) },
> >
> > /* Aveo Technology USB 2.0 Camera (Tasco USB Microscope) */
> > { .match_flags = USB_DEVICE_ID_MATCH_D
Re: [PATCH v2 1/2] uvcvideo: rename UVC_QUIRK_INFO to UVC_INFO_QUIRK
Hi Guennadi,
Thank you for the patch.
On Friday, 3 August 2018 14:36:56 EEST Guennadi Liakhovetski wrote:
> This macro defines "information about quirks," not "quirks for
> information."
>
> Signed-off-by: Guennadi Liakhovetski
Reviewed-by: Laurent Pinchart
and taken in my tree.
> ---
> drivers/media/usb/uvc/uvc_driver.c | 18 +-
> 1 file changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index d46dc43..699984b 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -2344,7 +2344,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .quirks = UVC_QUIRK_FORCE_Y8,
> };
>
> -#define UVC_QUIRK_INFO(q) (kernel_ulong_t)&(struct uvc_device_info){.quirks
> = q} +#define UVC_INFO_QUIRK(q) (kernel_ulong_t)&(struct
> uvc_device_info){.quirks = q}
>
> /*
> * The Logitech cameras listed below have their interface class set to
> @@ -2453,7 +2453,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info =
> UVC_QUIRK_INFO(UVC_QUIRK_RESTORE_CTRLS_ON_INIT) },
> + .driver_info =
> UVC_INFO_QUIRK(UVC_QUIRK_RESTORE_CTRLS_ON_INIT) },
> /* Chicony CNF7129 (Asus EEE 100HE) */
> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
>
> | USB_DEVICE_ID_MATCH_INT_INFO,
>
> @@ -2462,7 +2462,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_RESTRICT_FRAME_RATE)
> },
> + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_RESTRICT_FRAME_RATE)
> },
> /* Alcor Micro AU3820 (Future Boy PC USB Webcam) */
> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
>
> | USB_DEVICE_ID_MATCH_INT_INFO,
>
> @@ -2525,7 +2525,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
>
> | UVC_QUIRK_BUILTIN_ISIGHT) },
>
> /* Apple Built-In iSight via iBridge */
> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> @@ -2607,7 +2607,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
>
> | UVC_QUIRK_PROBE_DEF) },
>
> /* IMC Networks (Medion Akoya) */
> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> @@ -2707,7 +2707,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
>
> | UVC_QUIRK_PROBE_EXTRAFIELDS) },
>
> /* Aveo Technology USB 2.0 Camera (Tasco USB Microscope) */
> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> @@ -2725,7 +2725,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_PROBE_EXTRAFIELDS) },
> + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_PROBE_EXTRAFIELDS) },
> /* Manta MM-353 Plako */
> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
>
> | USB_DEVICE_ID_MATCH_INT_INFO,
>
> @@ -2771,7 +2771,7 @@ static int uvc_clock_param_set(const char *val, const
> struct kernel_param *kp) .bInterfaceClass = USB_CLASS_VIDEO,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = UVC_QUIRK_INFO(UVC_QUIRK_STATUS_INTERVAL) },
> +
Re: [PATCH v2 1/2] uvcvideo: rename UVC_QUIRK_INFO to UVC_INFO_QUIRK
nt uvc_clock_param_set(const char *val,
> > const struct kernel_param *kp)
> >
> > .bInterfaceClass = USB_CLASS_VIDEO,
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info =
> > UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > + .driver_info =
> > UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
> >
> > | UVC_QUIRK_PROBE_EXTRAFIELDS)
> >
> > },
> >
> > /* Aveo Technology USB 2.0 Camera (Tasco USB Microscope) */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > @@ -2725,7 +2725,7 @@ static int uvc_clock_param_set(const char *val,
> > const struct kernel_param *kp)
> >
> > .bInterfaceClass = USB_CLASS_VIDEO,
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info =
> > UVC_QUIRK_INFO(UVC_QUIRK_PROBE_EXTRAFIELDS) },
> > + .driver_info =
> > UVC_INFO_QUIRK(UVC_QUIRK_PROBE_EXTRAFIELDS) },
> >
> > /* Manta MM-353 Plako */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > | USB_DEVICE_ID_MATCH_INT_INFO,
> >
> > @@ -2771,7 +2771,7 @@ static int uvc_clock_param_set(const char *val,
> > const struct kernel_param *kp)
> >
> > .bInterfaceClass = USB_CLASS_VIDEO,
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info =
> > UVC_QUIRK_INFO(UVC_QUIRK_STATUS_INTERVAL) },
> > + .driver_info =
> > UVC_INFO_QUIRK(UVC_QUIRK_STATUS_INTERVAL) },
> >
> > /* MSI StarCam 370i */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >
> > | USB_DEVICE_ID_MATCH_INT_INFO,
> >
> > @@ -2798,7 +2798,7 @@ static int uvc_clock_param_set(const char *val,
> > const struct kernel_param *kp)
> >
> > .bInterfaceClass = USB_CLASS_VIDEO,
> > .bInterfaceSubClass = 1,
> > .bInterfaceProtocol = 0,
> >
> > - .driver_info =
> > UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > + .driver_info =
> > UVC_INFO_QUIRK(UVC_QUIRK_PROBE_MINMAX
> >
> > UVC_QUIRK_IGNORE_SELECTOR_UNIT) },
> >
> > /* Oculus VR Positional Tracker DK2 */
> > { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
--
Regards,
Laurent Pinchart
Re: [PATCH] uvcvideo: add quirk to force Phytec CAM 004H to GBRG
Hi Philipp,
On Friday, 17 August 2018 20:46:33 EEST Philipp Zabel wrote:
> Am Donnerstag, den 16.08.2018, 19:39 +0300 schrieb Laurent Pinchart:
> > Hi Christoph,
> >
> > (Philipp, there's a question for you at the end)
>
> > On Thursday, 16 August 2018 15:48:15 EEST Christoph Fritz wrote:
> [...]
>
> >> format->fcc = dev->forced_color_format;
> >> format->bpp = 8;
> >> width_multiplier = 2;
> >
> > bpp and multiplier are more annoying. bpp is a property of the format,
> > which we could add to the uvc_fmts array.
> >
> > I believe the multiplier could be computed by device bpp / bpp from
> > uvc_fmts. That would work at least for the Oculus VR Positional Tracker
> > DK2, but I don't have the Oculus VR Rift Sensor descriptors to check
> > that. Philipp, if you still have access to the device, could you send
> > that to me ?
>
> Full lsusb -v output below, the UVC descriptors are not decoded because
> bFunctionClass is set to 255. The YUY2 uncompressed format descriptor
> looks like this:
>
>___guidFormat__
> 1b 24 04 01 04 59 55 59 32 00 00 10 00 80 00 00 aa 00 38 9b 71 10 01 00 00
> 00 00 ^^
> so, bBitsPerPixel == 16.
Thanks a lot, that's exactly the information I needed. We can thus compute the
multiplier with something like
if (dev->info->pixel_format) {
fmtdesc = uvc_format_by_fourcc(dev->info->pixel_format);
strlcpy(format->name, fmtdesc->name,
sizeof(format->name));
format->fcc = fmtdesc->fcc;
width_multiplier = format->bpp / fmtdesc->bpp;
format->bpp = fmtdesc->bpp;
}
(possibly with a better name for the pixel_format field)
--
Regards,
Laurent Pinchart
[PATCH] media: uvcvideo: Store device information pointer in struct uvc_device
The device information structure is currently copied field by field in
the uvc_device structure. As we only have two fields at the moment this
isn't much of an issue, but it prevents easy addition of new info
fields.
Fix this by storing the uvc_device_info pointer in the uvc_device
structure. As a result the uvc_device meta_format field can be removed.
The quirks field, however, needs to stay as it can be modified through a
module parameter.
As not all device have an information structure, we declare a global
"NULL" info instance that is used as a fallback when the driver_info is
empty.
Signed-off-by: Laurent Pinchart
---
drivers/media/usb/uvc/uvc_driver.c | 15 +--
drivers/media/usb/uvc/uvc_metadata.c | 7 ---
drivers/media/usb/uvc/uvcvideo.h | 8 +++-
3 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_driver.c
b/drivers/media/usb/uvc/uvc_driver.c
index 21b0270067c1..232da625faad 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -2040,10 +2040,7 @@ static int uvc_register_chains(struct uvc_device *dev)
* USB probe, disconnect, suspend and resume
*/
-struct uvc_device_info {
- u32 quirks;
- u32 meta_format;
-};
+static const struct uvc_device_info uvc_quirk_none = { 0 };
static int uvc_probe(struct usb_interface *intf,
const struct usb_device_id *id)
@@ -2052,7 +2049,6 @@ static int uvc_probe(struct usb_interface *intf,
struct uvc_device *dev;
const struct uvc_device_info *info =
(const struct uvc_device_info *)id->driver_info;
- u32 quirks = info ? info->quirks : 0;
int function;
int ret;
@@ -2079,10 +2075,9 @@ static int uvc_probe(struct usb_interface *intf,
dev->udev = usb_get_dev(udev);
dev->intf = usb_get_intf(intf);
dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
- dev->quirks = (uvc_quirks_param == -1)
- ? quirks : uvc_quirks_param;
- if (info)
- dev->meta_format = info->meta_format;
+ dev->info = info ? info : _quirk_none;
+ dev->quirks = uvc_quirks_param == -1
+ ? dev->info->quirks : uvc_quirks_param;
if (udev->product != NULL)
strlcpy(dev->name, udev->product, sizeof(dev->name));
@@ -2123,7 +2118,7 @@ static int uvc_probe(struct usb_interface *intf,
le16_to_cpu(udev->descriptor.idVendor),
le16_to_cpu(udev->descriptor.idProduct));
- if (dev->quirks != quirks) {
+ if (dev->quirks != dev->info->quirks) {
uvc_printk(KERN_INFO, "Forcing device quirks to 0x%x by module "
"parameter for testing purpose.\n", dev->quirks);
uvc_printk(KERN_INFO, "Please report required quirks to the "
diff --git a/drivers/media/usb/uvc/uvc_metadata.c
b/drivers/media/usb/uvc/uvc_metadata.c
index cd1aec19cc5b..ed0f0c0732cb 100644
--- a/drivers/media/usb/uvc/uvc_metadata.c
+++ b/drivers/media/usb/uvc/uvc_metadata.c
@@ -74,7 +74,8 @@ static int uvc_meta_v4l2_try_format(struct file *file, void
*fh,
memset(fmt, 0, sizeof(*fmt));
- fmt->dataformat = fmeta == dev->meta_format ? fmeta : V4L2_META_FMT_UVC;
+ fmt->dataformat = fmeta == dev->info->meta_format
+ ? fmeta : V4L2_META_FMT_UVC;
fmt->buffersize = UVC_METATADA_BUF_SIZE;
return 0;
@@ -118,14 +119,14 @@ static int uvc_meta_v4l2_enum_formats(struct file *file,
void *fh,
u32 index = fdesc->index;
if (fdesc->type != vfh->vdev->queue->type ||
- index > 1U || (index && !dev->meta_format))
+ index > 1U || (index && !dev->info->meta_format))
return -EINVAL;
memset(fdesc, 0, sizeof(*fdesc));
fdesc->type = vfh->vdev->queue->type;
fdesc->index = index;
- fdesc->pixelformat = index ? dev->meta_format : V4L2_META_FMT_UVC;
+ fdesc->pixelformat = index ? dev->info->meta_format : V4L2_META_FMT_UVC;
return 0;
}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 59d66e5b8887..591eae3d0b0d 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -572,15 +572,21 @@ struct uvc_streaming {
} clock;
};
+struct uvc_device_info {
+ u32 quirks;
+ u32 meta_format;
+};
+
struct uvc_device {
struct usb_device *udev;
struct usb_interface *intf;
unsigned long warnings;
u32 quirks;
- u32 meta_format;
int intfnum;
char name[32];
+ const struct uvc_device_info *info;
+
struct mutex lock; /* Protects users */
unsigned int users;
atomic_t nmappings;
--
Regards,
Laurent Pinchart
Re: [PATCH] uvcvideo: add quirk to force Phytec CAM 004H to GBRG
Hi Christoph,
On Friday, 17 August 2018 10:09:12 EEST Christoph Fritz wrote:
> On Thu, 2018-08-16 at 19:39 +0300, Laurent Pinchart wrote:
> > On Thursday, 16 August 2018 15:48:15 EEST Christoph Fritz wrote:
> >>> On Wednesday, 21 February 2018 23:24:36 EEST Laurent Pinchart wrote:
> >>>> On Wednesday, 21 February 2018 22:42:45 EET Christoph Fritz wrote:
[snip]
> >>>>>>> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> >>>>>>> b/drivers/media/usb/uvc/uvcvideo.h index 7e4d3ee..ad51002 100644
> >>>>>>> --- a/drivers/media/usb/uvc/uvcvideo.h
> >>>>>>> +++ b/drivers/media/usb/uvc/uvcvideo.h
> >>>>>>> @@ -164,6 +164,7 @@
> >>>>>>> #define UVC_QUIRK_RESTRICT_FRAME_RATE0x0200
> >>>>>>> #define UVC_QUIRK_RESTORE_CTRLS_ON_INIT 0x0400
> >>>>>>> #define UVC_QUIRK_FORCE_Y8 0x0800
> >>>>>>> +#define UVC_QUIRK_FORCE_GBRG 0x1000
> >>>>>>
> >>>>>> I don't think we should add a quirk flag for every format that
> >>>>>> needs to be forced. Instead, now that we have a new way to store
> >>>>>> per-device parameters since commit 3bc85817d798 ("media: uvcvideo:
> >>>>>> Add extensible device information"), how about making use of it and
> >>>>>> adding a field to the uvc_device_info structure to store the forced
> >>>>>> format ?
> >>
> >> you mean something like:
> >>
> >> struct uvc_device_info {
> >> u32 quirks;
> >> + u32 forced_color_format;
> >> u32 meta_format;
> >> };
> >>
> >> and
> >>
> >> +static const struct uvc_device_info uvc_forced_color_sgbrg8 = {
> >> + .forced_color_format = V4L2_PIX_FMT_SGBRG8,
> >> +};
> >>
> >> and
> >>
> >> @@ -2817,7 +2820,7 @@ static const struct usb_device_id uvc_ids[] = {
> >> .bInterfaceClass = USB_CLASS_VENDOR_SPEC,
> >> .bInterfaceSubClass = 1,
> >> .bInterfaceProtocol = 0,
> >> - .driver_info = (kernel_ulong_t)_quirk_force_y8 },
> >> + .driver_info = (kernel_ulong_t)_forced_color_y8
> >> },
> >>
> >> ?
> >
> > With an additional
> >
> > static const struct uvc_device_info uvc_forced_color_y8 = {
> > .forced_color_format = V4L2_PIX_FMT_GREY,
> > };
> >
> >> If yes:
> >> - there would be a need for forced_color_format in struct uvc_device
> >
> > Why so ?
>
> What is the alternative?
> Is there a better way to access "uvc_device_info->forced_color_format"
> from within function uvc_parse_format()?
I'd rather do that, yes. We can store a pointer to uvc_device_info in the
uvc_device structure to make that easier. I'll post a patch to do so in reply
to this e-mail.
> >> - module-parameter quirk would not test force color format any more
> >>
> >> - the actual force/quirk changes not only format->fcc:
> >> if (dev->forced_color_format == V4L2_PIX_FMT_SGBRG8) {
> >
> > The test should be if (dev->forced_color_format) to cover both the Y8 and
> > SGBRG8 cases.
> >
> >> strlcpy(format->name, "Greyscale 8-bit (Y8 )",
> >> sizeof(format->name));
> >
> > You can get the name from the uvc_fmts entry corresponding to dev->
> > forced_color_format.
> >>
> >> format->fcc = dev->forced_color_format;
> >> format->bpp = 8;
> >> width_multiplier = 2;
> >
> > bpp and multiplier are more annoying. bpp is a property of the format,
> > which we could add to the uvc_fmts array.
> >
> > I believe the multiplier could be computed by device bpp / bpp from
> > uvc_fmts. That would work at least for the Oculus VR Positional Tracker
> > DK2, but I don't have the Oculus VR Rift Sensor descriptors to check
> > that. Philipp, if you still have access to the device, could you send
> > that to me ?
> >
> >> }
> >>
> >> Is this the way you want me to go?
--
Regards,
Laurent Pinchart
Re: [PATCH] uvcvideo: add quirk to force Phytec CAM 004H to GBRG
Hi Christoph,
(Philipp, there's a question for you at the end)
On Thursday, 16 August 2018 15:48:15 EEST Christoph Fritz wrote:
> > On Wednesday, 21 February 2018 23:24:36 EEST Laurent Pinchart wrote:
> >> On Wednesday, 21 February 2018 22:42:45 EET Christoph Fritz wrote:
> >>>>> drivers/media/usb/uvc/uvc_driver.c | 16
> >>>>> drivers/media/usb/uvc/uvcvideo.h | 1 +
> >>>>> 2 files changed, 17 insertions(+)
> >>>>>
> >>>>> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> >>>>> b/drivers/media/usb/uvc/uvc_driver.c index cde43b6..8bfa40b 100644
> >>>>> --- a/drivers/media/usb/uvc/uvc_driver.c
> >>>>> +++ b/drivers/media/usb/uvc/uvc_driver.c
> >>>>> @@ -406,6 +406,13 @@ static int uvc_parse_format(struct uvc_device
> >>>>> *dev,
> >>>>> width_multiplier = 2;
> >>>>> }
> >>>>> }
> >>>>> + if (dev->quirks & UVC_QUIRK_FORCE_GBRG) {
> >>>>> + if (format->fcc == V4L2_PIX_FMT_SGRBG8) {
> >>>>> + strlcpy(format->name, "GBRG Bayer
> >>>>> (GBRG)",
> >>>>> + sizeof(format->name));
> >>>>> + format->fcc = V4L2_PIX_FMT_SGBRG8;
> >>>>> + }
> >>>>> + }
> >>>>>
> >>>>> if (buffer[2] == UVC_VS_FORMAT_UNCOMPRESSED) {
> >>>>> ftype = UVC_VS_FRAME_UNCOMPRESSED;
> >>>>> @@ -2631,6 +2638,15 @@ static struct usb_device_id uvc_ids[] = {
> >>>>> .bInterfaceClass = USB_CLASS_VENDOR_SPEC,
> >>>>> .bInterfaceSubClass = 1,
> >>>>> .bInterfaceProtocol = 0 },
> >>>>> + /* PHYTEC CAM 004H cameras */
> >>>>> + { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >>>>> + | USB_DEVICE_ID_MATCH_INT_INFO,
> >>>>> + .idVendor = 0x199e,
> >>>>> + .idProduct= 0x8302,
> >>>>> + .bInterfaceClass = USB_CLASS_VIDEO,
> >>>>> + .bInterfaceSubClass = 1,
> >>>>> + .bInterfaceProtocol = 0,
> >>>>> + .driver_info = UVC_QUIRK_FORCE_GBRG },
> >>>>> /* Bodelin ProScopeHR */
> >>>>> { .match_flags = USB_DEVICE_ID_MATCH_DEVICE
> >>>>> | USB_DEVICE_ID_MATCH_DEV_HI
> >>>>> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> >>>>> b/drivers/media/usb/uvc/uvcvideo.h index 7e4d3ee..ad51002 100644
> >>>>> --- a/drivers/media/usb/uvc/uvcvideo.h
> >>>>> +++ b/drivers/media/usb/uvc/uvcvideo.h
> >>>>> @@ -164,6 +164,7 @@
> >>>>> #define UVC_QUIRK_RESTRICT_FRAME_RATE 0x0200
> >>>>> #define UVC_QUIRK_RESTORE_CTRLS_ON_INIT0x0400
> >>>>> #define UVC_QUIRK_FORCE_Y8 0x0800
> >>>>> +#define UVC_QUIRK_FORCE_GBRG 0x1000
> >>>>
> >>>> I don't think we should add a quirk flag for every format that needs
> >>>> to be forced. Instead, now that we have a new way to store per-device
> >>>> parameters since commit 3bc85817d798 ("media: uvcvideo: Add
> >>>> extensible device information"), how about making use of it and adding
> >>>> a field to the uvc_device_info structure to store the forced format ?
>
> you mean something like:
>
> struct uvc_device_info {
> u32 quirks;
> + u32 forced_color_format;
> u32 meta_format;
> };
>
> and
>
> +static const struct uvc_device_info uvc_forced_color_sgbrg8 = {
> + .forced_color_format = V4L2_PIX_FMT_SGBRG8,
> +};
>
> and
>
> @@ -2817,7 +2820,7 @@ static const struct usb_device_id uvc_ids[] = {
> .bInterfaceClass = USB_CLASS_VENDOR_SPEC,
> .bInterfaceSubClass = 1,
> .bInterfaceProtocol = 0,
> - .driver_info = (kernel_ulong_t)_quirk_force_y8 },
> + .driver_info = (kernel_ulong_
Re: [PATCHv18 01/35] Documentation: v4l: document request API
buf.rst index > 9e448a4aa3aa..0e415f2551b2 100644 > --- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst > +++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst > @@ -65,7 +65,7 @@ To enqueue a :ref:`memory mapped ` buffer > applications set the with a pointer to this structure the driver sets the > ``V4L2_BUF_FLAG_MAPPED`` and ``V4L2_BUF_FLAG_QUEUED`` flags and clears > the ``V4L2_BUF_FLAG_DONE`` flag in the ``flags`` field, or it returns an > -EINVAL error code. > +``EINVAL`` error code. > > To enqueue a :ref:`user pointer ` buffer applications set the > ``memory`` field to ``V4L2_MEMORY_USERPTR``, the ``m.userptr`` field to > @@ -98,6 +98,25 @@ dequeued, until the :ref:`VIDIOC_STREAMOFF > ` or > :ref:`VIDIOC_REQBUFS` ioctl is called, or until the > > device is closed. > > +The ``request_fd`` field can be used with the ``VIDIOC_QBUF`` ioctl to > specify +the file descriptor of a :ref:`request `, if > requests are +in use. Setting it means that the buffer will not be passed > to the driver +until the request itself is queued. Also, the driver will > apply any +settings associated with the request for this buffer. This field > will +be ignored unless the ``V4L2_BUF_FLAG_REQUEST_FD`` flag is set. > +If the device does not support requests, then ``EPERM`` will be returned. > +If requests are supported but an invalid request FD is given, then > +``ENOENT`` will be returned. > + > +.. caution:: > + It is not allowed to mix queuing requests with queuing buffers directly. > + ``EPERM`` will be returned if the first buffer was queued directly and > + then the application tries to queue a request, or vice versa. How do drivers assert this constraint ? Is it per file handle ? Per device node ? If per device node, how is it reset ? i.e. if an application uses the device in request mode, closes and, and a legacy application tries to open and use it in normal mode, what happens ? > + For :ref:`memory-to-memory devices ` you can specify the > + ``request_fd`` only for output buffers, not for capture buffers. > Attempting + to specify this for a capture buffer will result in an > ``EPERM`` error. + I think this should be documented in more details in a new V4L2 request API section, as noted above. > Applications call the ``VIDIOC_DQBUF`` ioctl to dequeue a filled > (capturing) or displayed (output) buffer from the driver's outgoing > queue. They just set the ``type``, ``memory`` and ``reserved`` fields of > @@ -153,3 +172,14 @@ EPIPE > ``VIDIOC_DQBUF`` returns this on an empty capture queue for mem2mem > codecs if a buffer with the ``V4L2_BUF_FLAG_LAST`` was already > dequeued and no new buffers are expected to become available. > + > +EPERM > +The ``V4L2_BUF_FLAG_REQUEST_FD`` flag was set but the device does not > +support requests. Or the first buffer was queued via a request, but > +the application now tries to queue it directly, or vice versa (it is > +not permitted to mix the two APIs). Or an attempt is made to queue a > +CAPTURE buffer to a request for a :ref:`memory-to-memory device > `. + > +ENOENT > +The ``V4L2_BUF_FLAG_REQUEST_FD`` flag was set but the the given > +``request_fd`` was invalid. > diff --git a/Documentation/media/videodev2.h.rst.exceptions > b/Documentation/media/videodev2.h.rst.exceptions index > ca9f0edc579e..99256a2c003e 100644 > --- a/Documentation/media/videodev2.h.rst.exceptions > +++ b/Documentation/media/videodev2.h.rst.exceptions > @@ -514,6 +514,7 @@ ignore define V4L2_CTRL_DRIVER_PRIV > ignore define V4L2_CTRL_MAX_DIMS > ignore define V4L2_CTRL_WHICH_CUR_VAL > ignore define V4L2_CTRL_WHICH_DEF_VAL > +ignore define V4L2_CTRL_WHICH_REQUEST_VAL > ignore define V4L2_OUT_CAP_CUSTOM_TIMINGS > ignore define V4L2_CID_MAX_CTRLS -- Regards, Laurent Pinchart
Re: why does aptina_pll_calculate insist on exact division?
Hi Helmut,
(CC'ing Sakari Ailus who is our current PLL expert after spending so much time
on the SMIA PLL code)
On Tuesday, 14 August 2018 09:35:40 EEST Helmut Grohne wrote:
> Hi,
>
> I tried using the aptina_pll_calculate for a "new" imager and ran into
> problems. After filling out aptina_pll_limits from the data sheet, I was
> having a hard time finding a valid pix_clock. Most of the ones I tried
> are rejected by aptina_pll_calculate for various reasons. In particular,
> no pix_clock close to pix_clock_max is allowed.
How do you mean ? The only place where pix_clock_max is used is in the
following code:
if (pll->pix_clock == 0 || pll->pix_clock > limits->pix_clock_max) {
dev_err(dev, "pll: invalid pixel clock frequency.\n");
return -EINVAL;
}
aptina_pll_calculate() rejects a request pixel clock value higher than the
pixel clock frequency higher limit, which is also given by the caller. That
shouldn't happen, it would be a bug in the caller.
> Why does the calculation method insist on exact division and avoiding
> fractional numbers?
I'm not sure what you mean by avoiding fractional numbers. Could you please
elaborate ? Please keep in mind that I last touched the code 6 years, so my
memory isn't exactly fresh.
If you mean using floating point operations to calculate PLL parameters,
remember that the code runs in the Linux kernel, where floating point isn't
allowed. You would thus have to implement the algorithm using fixed-point
calculation.
> I'm using an ext_clock of 50 MHz. This clock is derived from a 33 MHz
> clock and the 50 MHz is not attained exactly. Rather it ends up being
> more like 49.76 Hz. This raises the question, what value I should
> put into ext_clock (or the corresponding device tree property). Should I
> use the requested frequency or the actual frequency?
There's no such thing as an exact frequency anyway, as it's a physical value.
I'd got for 50 MHz for simplicity.
> Worse, depending on the precise value of the ext_clock, aptina_pll_calculate
> may or may not be able to compute pll parameters.
>
> On the other hand, the manufacturer provided configuration tool happily
> computes pll parameters that result in close, but not exactly, the
> requested pix_clock. In particular, the pll parameters do not
> necessarily result in a whole number. It appears to merely approximate
> the requested frequency.
aptina_pll_calculate() also approximates the requested frequency, but as it
appears from your test, fails in some cases. That's certainly an issue in the
code and should be fixed. Feel free to convince the manufacturer to release
their PLL parameters computation code under the GPL ;-)
> Can you explain where the requirement to avoid fractional numbers comes
> from? Would it be reasonable to use a different algorithm that avoids
> this requirement?
Again, please elaborate on what you mean by avoiding fractional numbers. I
would certainly be open to a different algorithm (or possibly fixes to the
existing code), as long as it fulfills the requirements behind the current
implementation. In particular the code should compute the optimum PLL
parameters when multiple options are possible, and its complexity should be
lower than O(n^2) (ideally O(1), if not possible O(n)).
--
Regards,
Laurent Pinchart
