Re: [PATCH 1/3] media: stkwebcam: Support for ASUS A6VM notebook added.
Hi Andreas,
Thank you for the patch,
On 23/11/2018 16:14, Andreas Pape wrote:
> The ASUS A6VM notebook has a built in stk11xx webcam which is mounted
> in a way that the video is vertically and horizontally flipped.
> Therefore this notebook is added to the special handling in the driver
> to automatically flip the video into the correct orientation.
>
> Signed-off-by: Andreas Pape
> ---
> drivers/media/usb/stkwebcam/stk-webcam.c | 7 +++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/media/usb/stkwebcam/stk-webcam.c
> b/drivers/media/usb/stkwebcam/stk-webcam.c
> index e11d5d5b7c26..e61427e50525 100644
> --- a/drivers/media/usb/stkwebcam/stk-webcam.c
> +++ b/drivers/media/usb/stkwebcam/stk-webcam.c
> @@ -116,6 +116,13 @@ static const struct dmi_system_id
> stk_upside_down_dmi_table[] = {
> DMI_MATCH(DMI_PRODUCT_NAME, "T12Rg-H")
> }
> },
> + {
> + .ident = "ASUS A6VM",
> + .matches = {
> + DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK Computer Inc."),
> + DMI_MATCH(DMI_PRODUCT_NAME, "A6VM")
> + }
I guess these strings match the strings produced by dmi-decode on your
laptop?
Assuming so:
Reviewed-by: Kieran Bingham
> + },
> {}
> };
>
>
--
Regards
--
Kieran
Re: [PATCH 2/3] media: stkwebcam: Bugfix for not correctly initialized camera
Hi Andreas, Thank you for the patch, On 23/11/2018 16:14, Andreas Pape wrote: > stk_start_stream can only be called successfully if stk_initialise and > stk_setup_format are called before. When using e.g. cheese it was observed > that stk_initialise and stk_setup_format have not been called before which > leads to no picture in that software whereas other tools like guvcview > worked flawlessly. This patch solves the issue when using e.g. cheese. > This one worries me a little... (but hopefully not too much) > Signed-off-by: Andreas Pape > --- > drivers/media/usb/stkwebcam/stk-webcam.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/media/usb/stkwebcam/stk-webcam.c > b/drivers/media/usb/stkwebcam/stk-webcam.c > index e61427e50525..c64928e36a5a 100644 > --- a/drivers/media/usb/stkwebcam/stk-webcam.c > +++ b/drivers/media/usb/stkwebcam/stk-webcam.c > @@ -1155,6 +1155,8 @@ static int stk_vidioc_streamon(struct file *filp, > if (dev->sio_bufs == NULL) > return -EINVAL; > dev->sequence = 0; > + stk_initialise(dev); > + stk_setup_format(dev); Glancing through the code base - this seems to imply to me that s_fmt was not set/called (presumably by cheese) as stk_setup_format() is called only by stk_vidioc_s_fmt_vid_cap() and stk_camera_resume(). Is this an issue? I presume that this means the camera will just operate in a default configuration until cheese chooses something more specific. Actually - looking further this seems to be the case, as the mode is simply stored in dev->vsettings.mode, and so this last setup stage will just ensure the configuration of the hardware matches the driver. So it seems reasonable to me - but should it be set any earlier? Perhaps not. Are there any complaints when running v4l2-compliance on this device node? > return stk_start_stream(dev); > } > > -- Regards -- Kieran
Re: [PATCH 3/3] media: stkwebcam: Bugfix for wrong return values
Hi Andreas, Thank you for the patch, On 23/11/2018 16:14, Andreas Pape wrote: > usb_control_msg returns in case of a successfully sent message the number > of sent bytes as a positive number. Don't use this value as a return value > for stk_camera_read_reg, as a non-zero return value is used as an error > condition in some cases when stk_camera_read_reg is called. Yes, and I see the stk_camera_write_reg() also follows this pattern. > Signed-off-by: Andreas Pape > --- > drivers/media/usb/stkwebcam/stk-webcam.c | 6 +- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/drivers/media/usb/stkwebcam/stk-webcam.c > b/drivers/media/usb/stkwebcam/stk-webcam.c > index c64928e36a5a..66a3665fc826 100644 > --- a/drivers/media/usb/stkwebcam/stk-webcam.c > +++ b/drivers/media/usb/stkwebcam/stk-webcam.c > @@ -171,7 +171,11 @@ int stk_camera_read_reg(struct stk_camera *dev, u16 > index, u8 *value) > *value = *buf; > > kfree(buf); > - return ret; > + > + if (ret < 0) > + return ret; > + else > + return 0; I would have said the return 0; could be on it's own, and the else statement would then not be needed, but I see this follows the style used by stk_camera_write_reg() - so it looks good to me. Reviewed-by: Kieran Bingham > } > > static int stk_start_stream(struct stk_camera *dev) > -- Regards -- Kieran
Re: Bug in stkwebcam?
Hi Andreas, On 22/11/2018 19:23, Andreas Pape wrote: > Hello, > > I recently updated my old 2006 Asus A6VM notebook with the latest 32bit > Ubuntu 18.04 LTS (kernel 4.15.0) and found out that the driver for the > webcam (Syntek USB2.0, USB ID 174f:a311) was not working. I only got error > messages like "Sensor resetting failed" in dmesg when starting guvcview > for example. > > Far from being an expert for video devices, I tried to debug this and > figured out three patches to make the webcam work again on my old notebook > (at least I get a video again ;-). Excellent - that sounds like you've worked out the hard part :-) > I know the type of notebook and webcam is pretty old and the driver seems > not to be actively maintained anymore although still being part of actual > kernel versions. > > Is there still an interest in getting patches for such an old device? If > yes, I could try to rebase my patches to the actual version of media_tree.git > and post them to the mailing list. If it's a USB webcam, then it could be plugged into any newer PC/Laptop too, so I would say there is still merit in identifying the fault and fixing it if feasible. I don't think it would hurt to send the patches at least. Feel free to CC me when you do. Regards -- Kieran > Kind regards, > Andreas >
Re: [PATCH v6 08/10] media: uvcvideo: Split uvc_video_enable into two
Hi Laurent,
I see that you made changes to this patch before accepting it last time.
I'm afraid I haven't made those changes here, so please just cherry-pick
your previous incarnation. There are no changes here from me between v5,
and v6.
Regards
--
Kieran
On 09/11/2018 17:05, Kieran Bingham wrote:
> uvc_video_enable() is used both to start and stop the video stream
> object, however the single function entry point shares no code between
> the two operations.
>
> Split the function into two distinct calls, and rename to
> uvc_video_start_streaming() and uvc_video_stop_streaming() as
> appropriate.
>
> Signed-off-by: Kieran Bingham
> ---
> drivers/media/usb/uvc/uvc_queue.c | 4 +-
> drivers/media/usb/uvc/uvc_video.c | 56 +++-
> drivers/media/usb/uvc/uvcvideo.h | 3 +-
> 3 files changed, 31 insertions(+), 32 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c
> index 2752e386f1e8..d09b0c882938 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -176,7 +176,7 @@ static int uvc_start_streaming(struct vb2_queue *vq,
> unsigned int count)
>
> queue->buf_used = 0;
>
> - ret = uvc_video_enable(stream, 1);
> + ret = uvc_video_start_streaming(stream);
> if (ret == 0)
> return 0;
>
> @@ -194,7 +194,7 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
> lockdep_assert_irqs_enabled();
>
> if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
> - uvc_video_enable(uvc_queue_to_stream(queue), 0);
> + uvc_video_stop_streaming(uvc_queue_to_stream(queue));
>
> spin_lock_irq(>irqlock);
> uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c
> index e19bdf089cc4..cd67506dc696 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -2076,38 +2076,10 @@ int uvc_video_init(struct uvc_streaming *stream)
> return 0;
> }
>
> -/*
> - * Enable or disable the video stream.
> - */
> -int uvc_video_enable(struct uvc_streaming *stream, int enable)
> +int uvc_video_start_streaming(struct uvc_streaming *stream)
> {
> int ret;
>
> - if (!enable) {
> - uvc_uninit_video(stream, 1);
> - if (stream->intf->num_altsetting > 1) {
> - usb_set_interface(stream->dev->udev,
> - stream->intfnum, 0);
> - } else {
> - /* UVC doesn't specify how to inform a bulk-based device
> - * when the video stream is stopped. Windows sends a
> - * CLEAR_FEATURE(HALT) request to the video streaming
> - * bulk endpoint, mimic the same behaviour.
> - */
> - unsigned int epnum = stream->header.bEndpointAddress
> -& USB_ENDPOINT_NUMBER_MASK;
> - unsigned int dir = stream->header.bEndpointAddress
> - & USB_ENDPOINT_DIR_MASK;
> - unsigned int pipe;
> -
> - pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
> - usb_clear_halt(stream->dev->udev, pipe);
> - }
> -
> - uvc_video_clock_cleanup(stream);
> - return 0;
> - }
> -
> ret = uvc_video_clock_init(stream);
> if (ret < 0)
> return ret;
> @@ -2130,3 +2102,29 @@ int uvc_video_enable(struct uvc_streaming *stream, int
> enable)
>
> return ret;
> }
> +
> +int uvc_video_stop_streaming(struct uvc_streaming *stream)
> +{
> + uvc_uninit_video(stream, 1);
> + if (stream->intf->num_altsetting > 1) {
> + usb_set_interface(stream->dev->udev,
> + stream->intfnum, 0);
> + } else {
> + /* UVC doesn't specify how to inform a bulk-based device
> + * when the video stream is stopped. Windows sends a
> + * CLEAR_FEATURE(HALT) request to the video streaming
> + * bulk endpoint, mimic the same behaviour.
> + */
> + unsigned int epnum = stream->header.bEndpointAddress
> +& USB_ENDPOINT_NUMBER_MASK;
> + unsigned int dir = stream->header.bEndpointAddress
> + & USB_ENDPOINT_DIR_MASK;
> + unsigned int pipe;
> +
>
Re: [PATCH v6 09/10] media: uvcvideo: Rename uvc_{un,}init_video()
Hi Laurent,
I'm sorry - I didn't update the commit message on this one.
On 09/11/2018 17:05, Kieran Bingham wrote:
> We have both uvc_init_video() and uvc_video_init() calls which can be
> quite confusing to determine the process for each. Now that video
> uvc_video_enable() has been renamed to uvc_video_start_streaming(),
> adapt these calls to suit the new flow.
>
> Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
> uvc_video_stop().
>
Could you s/uvc_video_start/uvc_video_start_transfer/ and
s/uvc_video_stop/uvc_video_stop_transfer/ when applying please?
(Assuming you get to apply and don't find something else :D)
--
KB
> Signed-off-by: Kieran Bingham
>
> ---
>
> v6:
> - Append _transfer to {_stop,_start}
>
> drivers/media/usb/uvc/uvc_video.c | 20 +++-
> 1 file changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c
> index cd67506dc696..a81012c65280 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1641,7 +1641,8 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
> *stream,
> /*
> * Uninitialize isochronous/bulk URBs and free transfer buffers.
> */
> -static void uvc_uninit_video(struct uvc_streaming *stream, int free_buffers)
> +static void uvc_video_stop_transfer(struct uvc_streaming *stream,
> + int free_buffers)
> {
> struct uvc_urb *uvc_urb;
>
> @@ -1718,7 +1719,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
> *stream,
>
> urb = usb_alloc_urb(npackets, gfp_flags);
> if (urb == NULL) {
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop_transfer(stream, 1);
> return -ENOMEM;
> }
>
> @@ -1786,7 +1787,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
> *stream,
>
> urb = usb_alloc_urb(0, gfp_flags);
> if (urb == NULL) {
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop_transfer(stream, 1);
> return -ENOMEM;
> }
>
> @@ -1806,7 +1807,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
> *stream,
> /*
> * Initialize isochronous/bulk URBs and allocate transfer buffers.
> */
> -static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
> +static int uvc_video_start_transfer(struct uvc_streaming *stream,
> + gfp_t gfp_flags)
> {
> struct usb_interface *intf = stream->intf;
> struct usb_host_endpoint *ep;
> @@ -1894,7 +1896,7 @@ static int uvc_init_video(struct uvc_streaming *stream,
> gfp_t gfp_flags)
> if (ret < 0) {
> uvc_printk(KERN_ERR, "Failed to submit URB %u "
> "(%d).\n", i, ret);
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop_transfer(stream, 1);
> return ret;
> }
> }
> @@ -1925,7 +1927,7 @@ int uvc_video_suspend(struct uvc_streaming *stream)
> return 0;
>
> stream->frozen = 1;
> - uvc_uninit_video(stream, 0);
> + uvc_video_stop_transfer(stream, 0);
> usb_set_interface(stream->dev->udev, stream->intfnum, 0);
> return 0;
> }
> @@ -1961,7 +1963,7 @@ int uvc_video_resume(struct uvc_streaming *stream, int
> reset)
> if (ret < 0)
> return ret;
>
> - return uvc_init_video(stream, GFP_NOIO);
> + return uvc_video_start_transfer(stream, GFP_NOIO);
> }
>
> /*
> @@ -2089,7 +2091,7 @@ int uvc_video_start_streaming(struct uvc_streaming
> *stream)
> if (ret < 0)
> goto error_commit;
>
> - ret = uvc_init_video(stream, GFP_KERNEL);
> + ret = uvc_video_start_transfer(stream, GFP_KERNEL);
> if (ret < 0)
> goto error_video;
>
> @@ -2105,7 +2107,7 @@ int uvc_video_start_streaming(struct uvc_streaming
> *stream)
>
> int uvc_video_stop_streaming(struct uvc_streaming *stream)
> {
> - uvc_uninit_video(stream, 1);
> + uvc_video_stop_transfer(stream, 1);
> if (stream->intf->num_altsetting > 1) {
> usb_set_interface(stream->dev->udev,
> stream->intfnum, 0);
>
--
Regards
--
Kieran
[PATCH v6 09/10] media: uvcvideo: Rename uvc_{un,}init_video()
We have both uvc_init_video() and uvc_video_init() calls which can be
quite confusing to determine the process for each. Now that video
uvc_video_enable() has been renamed to uvc_video_start_streaming(),
adapt these calls to suit the new flow.
Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
uvc_video_stop().
Signed-off-by: Kieran Bingham
---
v6:
- Append _transfer to {_stop,_start}
drivers/media/usb/uvc/uvc_video.c | 20 +++-
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index cd67506dc696..a81012c65280 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1641,7 +1641,8 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
/*
* Uninitialize isochronous/bulk URBs and free transfer buffers.
*/
-static void uvc_uninit_video(struct uvc_streaming *stream, int free_buffers)
+static void uvc_video_stop_transfer(struct uvc_streaming *stream,
+ int free_buffers)
{
struct uvc_urb *uvc_urb;
@@ -1718,7 +1719,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
urb = usb_alloc_urb(npackets, gfp_flags);
if (urb == NULL) {
- uvc_uninit_video(stream, 1);
+ uvc_video_stop_transfer(stream, 1);
return -ENOMEM;
}
@@ -1786,7 +1787,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
urb = usb_alloc_urb(0, gfp_flags);
if (urb == NULL) {
- uvc_uninit_video(stream, 1);
+ uvc_video_stop_transfer(stream, 1);
return -ENOMEM;
}
@@ -1806,7 +1807,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
/*
* Initialize isochronous/bulk URBs and allocate transfer buffers.
*/
-static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
+static int uvc_video_start_transfer(struct uvc_streaming *stream,
+ gfp_t gfp_flags)
{
struct usb_interface *intf = stream->intf;
struct usb_host_endpoint *ep;
@@ -1894,7 +1896,7 @@ static int uvc_init_video(struct uvc_streaming *stream,
gfp_t gfp_flags)
if (ret < 0) {
uvc_printk(KERN_ERR, "Failed to submit URB %u "
"(%d).\n", i, ret);
- uvc_uninit_video(stream, 1);
+ uvc_video_stop_transfer(stream, 1);
return ret;
}
}
@@ -1925,7 +1927,7 @@ int uvc_video_suspend(struct uvc_streaming *stream)
return 0;
stream->frozen = 1;
- uvc_uninit_video(stream, 0);
+ uvc_video_stop_transfer(stream, 0);
usb_set_interface(stream->dev->udev, stream->intfnum, 0);
return 0;
}
@@ -1961,7 +1963,7 @@ int uvc_video_resume(struct uvc_streaming *stream, int
reset)
if (ret < 0)
return ret;
- return uvc_init_video(stream, GFP_NOIO);
+ return uvc_video_start_transfer(stream, GFP_NOIO);
}
/*
@@ -2089,7 +2091,7 @@ int uvc_video_start_streaming(struct uvc_streaming
*stream)
if (ret < 0)
goto error_commit;
- ret = uvc_init_video(stream, GFP_KERNEL);
+ ret = uvc_video_start_transfer(stream, GFP_KERNEL);
if (ret < 0)
goto error_video;
@@ -2105,7 +2107,7 @@ int uvc_video_start_streaming(struct uvc_streaming
*stream)
int uvc_video_stop_streaming(struct uvc_streaming *stream)
{
- uvc_uninit_video(stream, 1);
+ uvc_video_stop_transfer(stream, 1);
if (stream->intf->num_altsetting > 1) {
usb_set_interface(stream->dev->udev,
stream->intfnum, 0);
--
git-series 0.9.1
[PATCH v6 10/10] media: uvcvideo: Utilise for_each_uvc_urb iterator
A new iterator is available for processing UVC URB structures. This
simplifies the processing of the internal stream data.
Convert the manual loop iterators to the new helper, adding an index
helper to keep the existing debug print.
Signed-off-by: Kieran Bingham
---
v6:
- rename lone 'j' iterator to 'i'
- Remove conversion which doesn't make sense due to needing the
iterator value.
drivers/media/usb/uvc/uvc_video.c | 46 ++--
drivers/media/usb/uvc/uvcvideo.h | 3 ++-
2 files changed, 24 insertions(+), 25 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index a81012c65280..8d6d2fd8c74c 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1556,20 +1556,19 @@ static void uvc_video_complete(struct urb *urb)
*/
static void uvc_free_urb_buffers(struct uvc_streaming *stream)
{
- unsigned int i;
+ struct uvc_urb *uvc_urb;
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
+ for_each_uvc_urb(uvc_urb, stream) {
+ if (!uvc_urb->buffer)
+ continue;
- if (uvc_urb->buffer) {
#ifndef CONFIG_DMA_NONCOHERENT
- usb_free_coherent(stream->dev->udev, stream->urb_size,
- uvc_urb->buffer, uvc_urb->dma);
+ usb_free_coherent(stream->dev->udev, stream->urb_size,
+ uvc_urb->buffer, uvc_urb->dma);
#else
- kfree(uvc_urb->buffer);
+ kfree(uvc_urb->buffer);
#endif
- uvc_urb->buffer = NULL;
- }
+ uvc_urb->buffer = NULL;
}
stream->urb_size = 0;
@@ -1701,7 +1700,8 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
struct usb_host_endpoint *ep, gfp_t gfp_flags)
{
struct urb *urb;
- unsigned int npackets, i, j;
+ struct uvc_urb *uvc_urb;
+ unsigned int npackets, i;
u16 psize;
u32 size;
@@ -1714,9 +1714,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
size = npackets * psize;
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
-
+ for_each_uvc_urb(uvc_urb, stream) {
urb = usb_alloc_urb(npackets, gfp_flags);
if (urb == NULL) {
uvc_video_stop_transfer(stream, 1);
@@ -1739,9 +1737,9 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
urb->number_of_packets = npackets;
urb->transfer_buffer_length = size;
- for (j = 0; j < npackets; ++j) {
- urb->iso_frame_desc[j].offset = j * psize;
- urb->iso_frame_desc[j].length = psize;
+ for (i = 0; i < npackets; ++i) {
+ urb->iso_frame_desc[i].offset = i * psize;
+ urb->iso_frame_desc[i].length = psize;
}
uvc_urb->urb = urb;
@@ -1758,7 +1756,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
struct usb_host_endpoint *ep, gfp_t gfp_flags)
{
struct urb *urb;
- unsigned int npackets, pipe, i;
+ struct uvc_urb *uvc_urb;
+ unsigned int npackets, pipe;
u16 psize;
u32 size;
@@ -1782,9 +1781,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
if (stream->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
size = 0;
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
-
+ for_each_uvc_urb(uvc_urb, stream) {
urb = usb_alloc_urb(0, gfp_flags);
if (urb == NULL) {
uvc_video_stop_transfer(stream, 1);
@@ -1812,6 +1809,7 @@ static int uvc_video_start_transfer(struct uvc_streaming
*stream,
{
struct usb_interface *intf = stream->intf;
struct usb_host_endpoint *ep;
+ struct uvc_urb *uvc_urb;
unsigned int i;
int ret;
@@ -1889,13 +1887,11 @@ static int uvc_video_start_transfer(struct
uvc_streaming *stream,
return ret;
/* Submit the URBs. */
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
-
+ for_each_uvc_urb(uvc_urb, stream) {
ret = usb_submit_urb(uvc_urb->urb, gfp_flags);
if (ret < 0) {
- uvc_printk(KERN_ERR, "Failed to submit URB %u "
- "(%d).\n", i, ret);
+ uvc_printk(KERN_ERR, "Failed to submit URB %u (%d).\n",
+ uvc_urb_index(uvc_urb), ret);
uvc_vid
[PATCH v6 06/10] media: uvcvideo: Abstract streaming object lifetime
The streaming object is a key part of handling the UVC device. Although
not critical, we are currently missing a call to destroy the mutex on
clean up paths, and we are due to extend the objects complexity in the
near future.
Facilitate easy management of a stream object by creating a pair of
functions to handle creating and destroying the allocation. The new
uvc_stream_delete() function also performs the missing mutex_destroy()
operation.
Previously a failed streaming object allocation would cause
uvc_parse_streaming() to return -EINVAL, which is inappropriate. If the
constructor failes, we will instead return -ENOMEM.
While we're here, fix the trivial spelling error in the function banner
of uvc_delete().
Signed-off-by: Kieran Bingham
---
drivers/media/usb/uvc/uvc_driver.c | 54 +--
1 file changed, 38 insertions(+), 16 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_driver.c
b/drivers/media/usb/uvc/uvc_driver.c
index 67bd58c6f397..afb44d1c9d04 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -396,6 +396,39 @@ static struct uvc_streaming *uvc_stream_by_id(struct
uvc_device *dev, int id)
}
/*
+ * Streaming Object Management
+ */
+
+static void uvc_stream_delete(struct uvc_streaming *stream)
+{
+ mutex_destroy(>mutex);
+
+ usb_put_intf(stream->intf);
+
+ kfree(stream->format);
+ kfree(stream->header.bmaControls);
+ kfree(stream);
+}
+
+static struct uvc_streaming *uvc_stream_new(struct uvc_device *dev,
+ struct usb_interface *intf)
+{
+ struct uvc_streaming *stream;
+
+ stream = kzalloc(sizeof(*stream), GFP_KERNEL);
+ if (stream == NULL)
+ return NULL;
+
+ mutex_init(>mutex);
+
+ stream->dev = dev;
+ stream->intf = usb_get_intf(intf);
+ stream->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
+
+ return stream;
+}
+
+/*
* Descriptors parsing
*/
@@ -687,17 +720,12 @@ static int uvc_parse_streaming(struct uvc_device *dev,
return -EINVAL;
}
- streaming = kzalloc(sizeof(*streaming), GFP_KERNEL);
+ streaming = uvc_stream_new(dev, intf);
if (streaming == NULL) {
usb_driver_release_interface(_driver.driver, intf);
- return -EINVAL;
+ return -ENOMEM;
}
- mutex_init(>mutex);
- streaming->dev = dev;
- streaming->intf = usb_get_intf(intf);
- streaming->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
-
/* The Pico iMage webcam has its class-specific interface descriptors
* after the endpoint descriptors.
*/
@@ -904,10 +932,7 @@ static int uvc_parse_streaming(struct uvc_device *dev,
error:
usb_driver_release_interface(_driver.driver, intf);
- usb_put_intf(intf);
- kfree(streaming->format);
- kfree(streaming->header.bmaControls);
- kfree(streaming);
+ uvc_stream_delete(streaming);
return ret;
}
@@ -1815,7 +1840,7 @@ static int uvc_scan_device(struct uvc_device *dev)
* is released.
*
* As this function is called after or during disconnect(), all URBs have
- * already been canceled by the USB core. There is no need to kill the
+ * already been cancelled by the USB core. There is no need to kill the
* interrupt URB manually.
*/
static void uvc_delete(struct kref *kref)
@@ -1853,10 +1878,7 @@ static void uvc_delete(struct kref *kref)
streaming = list_entry(p, struct uvc_streaming, list);
usb_driver_release_interface(_driver.driver,
streaming->intf);
- usb_put_intf(streaming->intf);
- kfree(streaming->format);
- kfree(streaming->header.bmaControls);
- kfree(streaming);
+ uvc_stream_delete(streaming);
}
kfree(dev);
--
git-series 0.9.1
[PATCH v6 07/10] media: uvcvideo: Move decode processing to process context
Newer high definition cameras, and cameras with multiple lenses such as
the range of stereo-vision cameras now available have ever increasing
data rates.
The inclusion of a variable length packet header in URB packets mean
that we must memcpy the frame data out to our destination 'manually'.
This can result in data rates of up to 2 gigabits per second being
processed.
To improve efficiency, and maximise throughput, handle the URB decode
processing through a work queue to move it from interrupt context, and
allow multiple processors to work on URBs in parallel.
Signed-off-by: Kieran Bingham
---
v2:
- Lock full critical section of usb_submit_urb()
v3:
- Fix race on submitting uvc_video_decode_data_work() to work queue.
- Rename uvc_decode_op -> uvc_copy_op (Generic to encode/decode)
- Rename decodes -> copy_operations
- Don't queue work if there is no async task
- obtain copy op structure directly in uvc_video_decode_data()
- uvc_video_decode_data_work() -> uvc_video_copy_data_work()
v4:
- Provide for_each_uvc_urb()
- Simplify fix for shutdown race to flush queue before freeing URBs
- Rebase to v4.16-rc4 (linux-media/master) adjusting for metadata
conflicts.
v5:
- Rebase to media/v4.20-2
- Use GFP_KERNEL allocation in uvc_video_copy_data_work()
- Fix function documentation for uvc_video_copy_data_work()
- Add periods to the end of sentences
- Rename 'decode' variable to 'op' in uvc_video_decode_data()
- Move uvc_urb->async_operations initialisation to before use
- Move async workqueue to match uvc_streaming lifetime instead of
streamon/streamoff
v6:
- Utilise the new streaming object lifetime functions to perform
allocation and destruction of the async workqueue.
drivers/media/usb/uvc/uvc_driver.c | 11 +++-
drivers/media/usb/uvc/uvc_video.c | 104 +++---
drivers/media/usb/uvc/uvcvideo.h | 28 -
3 files changed, 119 insertions(+), 24 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_driver.c
b/drivers/media/usb/uvc/uvc_driver.c
index afb44d1c9d04..b62cbd800111 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -401,6 +401,9 @@ static struct uvc_streaming *uvc_stream_by_id(struct
uvc_device *dev, int id)
static void uvc_stream_delete(struct uvc_streaming *stream)
{
+ if (stream->async_wq)
+ destroy_workqueue(stream->async_wq);
+
mutex_destroy(>mutex);
usb_put_intf(stream->intf);
@@ -425,6 +428,14 @@ static struct uvc_streaming *uvc_stream_new(struct
uvc_device *dev,
stream->intf = usb_get_intf(intf);
stream->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
+ /* Allocate a stream specific work queue for asynchronous tasks. */
+ stream->async_wq = alloc_workqueue("uvcvideo", WQ_UNBOUND | WQ_HIGHPRI,
+ 0);
+ if (!stream->async_wq) {
+ uvc_stream_delete(stream);
+ return NULL;
+ }
+
return stream;
}
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 7a7779e1b466..e19bdf089cc4 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1094,21 +1094,54 @@ static int uvc_video_decode_start(struct uvc_streaming
*stream,
return data[0];
}
-static void uvc_video_decode_data(struct uvc_streaming *stream,
+/*
+ * uvc_video_decode_data_work: Asynchronous memcpy processing
+ *
+ * Copy URB data to video buffers in process context, releasing buffer
+ * references and requeuing the URB when done.
+ */
+static void uvc_video_copy_data_work(struct work_struct *work)
+{
+ struct uvc_urb *uvc_urb = container_of(work, struct uvc_urb, work);
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < uvc_urb->async_operations; i++) {
+ struct uvc_copy_op *op = _urb->copy_operations[i];
+
+ memcpy(op->dst, op->src, op->len);
+
+ /* Release reference taken on this buffer. */
+ uvc_queue_buffer_release(op->buf);
+ }
+
+ ret = usb_submit_urb(uvc_urb->urb, GFP_KERNEL);
+ if (ret < 0)
+ uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
+ ret);
+}
+
+static void uvc_video_decode_data(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, const u8 *data, int len)
{
- unsigned int maxlen, nbytes;
- void *mem;
+ unsigned int active_op = uvc_urb->async_operations;
+ struct uvc_copy_op *op = _urb->copy_operations[active_op];
+ unsigned int maxlen;
if (len <= 0)
return;
- /* Copy the video data to the buffer. */
maxlen = buf->length - buf->bytesused;
- mem = buf->mem + buf->bytesused;
- nbytes = min((unsigned int)len, maxlen);
- me
[PATCH v6 08/10] media: uvcvideo: Split uvc_video_enable into two
uvc_video_enable() is used both to start and stop the video stream
object, however the single function entry point shares no code between
the two operations.
Split the function into two distinct calls, and rename to
uvc_video_start_streaming() and uvc_video_stop_streaming() as
appropriate.
Signed-off-by: Kieran Bingham
---
drivers/media/usb/uvc/uvc_queue.c | 4 +-
drivers/media/usb/uvc/uvc_video.c | 56 +++-
drivers/media/usb/uvc/uvcvideo.h | 3 +-
3 files changed, 31 insertions(+), 32 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index 2752e386f1e8..d09b0c882938 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -176,7 +176,7 @@ static int uvc_start_streaming(struct vb2_queue *vq,
unsigned int count)
queue->buf_used = 0;
- ret = uvc_video_enable(stream, 1);
+ ret = uvc_video_start_streaming(stream);
if (ret == 0)
return 0;
@@ -194,7 +194,7 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
lockdep_assert_irqs_enabled();
if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
- uvc_video_enable(uvc_queue_to_stream(queue), 0);
+ uvc_video_stop_streaming(uvc_queue_to_stream(queue));
spin_lock_irq(>irqlock);
uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index e19bdf089cc4..cd67506dc696 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -2076,38 +2076,10 @@ int uvc_video_init(struct uvc_streaming *stream)
return 0;
}
-/*
- * Enable or disable the video stream.
- */
-int uvc_video_enable(struct uvc_streaming *stream, int enable)
+int uvc_video_start_streaming(struct uvc_streaming *stream)
{
int ret;
- if (!enable) {
- uvc_uninit_video(stream, 1);
- if (stream->intf->num_altsetting > 1) {
- usb_set_interface(stream->dev->udev,
- stream->intfnum, 0);
- } else {
- /* UVC doesn't specify how to inform a bulk-based device
-* when the video stream is stopped. Windows sends a
-* CLEAR_FEATURE(HALT) request to the video streaming
-* bulk endpoint, mimic the same behaviour.
-*/
- unsigned int epnum = stream->header.bEndpointAddress
- & USB_ENDPOINT_NUMBER_MASK;
- unsigned int dir = stream->header.bEndpointAddress
-& USB_ENDPOINT_DIR_MASK;
- unsigned int pipe;
-
- pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
- usb_clear_halt(stream->dev->udev, pipe);
- }
-
- uvc_video_clock_cleanup(stream);
- return 0;
- }
-
ret = uvc_video_clock_init(stream);
if (ret < 0)
return ret;
@@ -2130,3 +2102,29 @@ int uvc_video_enable(struct uvc_streaming *stream, int
enable)
return ret;
}
+
+int uvc_video_stop_streaming(struct uvc_streaming *stream)
+{
+ uvc_uninit_video(stream, 1);
+ if (stream->intf->num_altsetting > 1) {
+ usb_set_interface(stream->dev->udev,
+ stream->intfnum, 0);
+ } else {
+ /* UVC doesn't specify how to inform a bulk-based device
+* when the video stream is stopped. Windows sends a
+* CLEAR_FEATURE(HALT) request to the video streaming
+* bulk endpoint, mimic the same behaviour.
+*/
+ unsigned int epnum = stream->header.bEndpointAddress
+ & USB_ENDPOINT_NUMBER_MASK;
+ unsigned int dir = stream->header.bEndpointAddress
+& USB_ENDPOINT_DIR_MASK;
+ unsigned int pipe;
+
+ pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
+ usb_clear_halt(stream->dev->udev, pipe);
+ }
+
+ uvc_video_clock_cleanup(stream);
+ return 0;
+}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 94accfa3c009..b1b895c67122 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -786,7 +786,8 @@ void uvc_mc_cleanup_entity(struct uvc_entity *entity);
int uvc_video_init(struct uvc_streaming *stream);
int uvc_video_suspend(struct uvc_streaming *stream);
int uvc_video_resume(struct uvc_streaming *stream, int reset);
-int uvc_video_enable(struct uvc_streaming *stream, int enable);
+int uvc_video_start
[PATCH v6 03/10] media: uvcvideo: Protect queue internals with helper
The URB completion operation obtains the current buffer by reading
directly into the queue internal interface.
Protect this queue abstraction by providing a helper
uvc_queue_get_current_buffer() which can be used by both the decode
task, and the uvc_queue_next_buffer() functions.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v2:
- Fix coding style of conditional statements
v3:
- No change
v4:
- Rebase on top of linux-media/master (v4.16-rc4, metadata additions)
drivers/media/usb/uvc/uvc_queue.c | 33 +++-
drivers/media/usb/uvc/uvc_video.c | 6 +-
drivers/media/usb/uvc/uvcvideo.h | 1 +-
3 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index fecccb5e7628..adcc4928fae4 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -429,6 +429,33 @@ void uvc_queue_cancel(struct uvc_video_queue *queue, int
disconnect)
spin_unlock_irqrestore(>irqlock, flags);
}
+/*
+ * uvc_queue_get_current_buffer: Obtain the current working output buffer
+ *
+ * Buffers may span multiple packets, and even URBs, therefore the active
buffer
+ * remains on the queue until the EOF marker.
+ */
+static struct uvc_buffer *
+__uvc_queue_get_current_buffer(struct uvc_video_queue *queue)
+{
+ if (list_empty(>irqqueue))
+ return NULL;
+
+ return list_first_entry(>irqqueue, struct uvc_buffer, queue);
+}
+
+struct uvc_buffer *uvc_queue_get_current_buffer(struct uvc_video_queue *queue)
+{
+ struct uvc_buffer *nextbuf;
+ unsigned long flags;
+
+ spin_lock_irqsave(>irqlock, flags);
+ nextbuf = __uvc_queue_get_current_buffer(queue);
+ spin_unlock_irqrestore(>irqlock, flags);
+
+ return nextbuf;
+}
+
struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
struct uvc_buffer *buf)
{
@@ -445,11 +472,7 @@ struct uvc_buffer *uvc_queue_next_buffer(struct
uvc_video_queue *queue,
spin_lock_irqsave(>irqlock, flags);
list_del(>queue);
- if (!list_empty(>irqqueue))
- nextbuf = list_first_entry(>irqqueue, struct uvc_buffer,
- queue);
- else
- nextbuf = NULL;
+ nextbuf = __uvc_queue_get_current_buffer(queue);
spin_unlock_irqrestore(>irqlock, flags);
buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 6d4384695964..7a7779e1b466 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1484,11 +1484,7 @@ static void uvc_video_complete(struct urb *urb)
return;
}
- spin_lock_irqsave(>irqlock, flags);
- if (!list_empty(>irqqueue))
- buf = list_first_entry(>irqqueue, struct uvc_buffer,
- queue);
- spin_unlock_irqrestore(>irqlock, flags);
+ buf = uvc_queue_get_current_buffer(queue);
if (vb2_qmeta) {
spin_lock_irqsave(>irqlock, flags);
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 7c83185c5b87..16b8348f7ff0 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -727,6 +727,7 @@ int uvc_queue_streamoff(struct uvc_video_queue *queue, enum
v4l2_buf_type type);
void uvc_queue_cancel(struct uvc_video_queue *queue, int disconnect);
struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
struct uvc_buffer *buf);
+struct uvc_buffer *uvc_queue_get_current_buffer(struct uvc_video_queue *queue);
int uvc_queue_mmap(struct uvc_video_queue *queue,
struct vm_area_struct *vma);
__poll_t uvc_queue_poll(struct uvc_video_queue *queue, struct file *file,
--
git-series 0.9.1
[PATCH v6 02/10] media: uvcvideo: Convert decode functions to use new context structure
The URB completion handlers currently reference the stream context.
Now that each URB has its own context structure, convert the decode (and
one encode) functions to utilise this context for URB management.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v2:
- fix checkpatch warning (pre-existing in code)
v3: (none)
v4:
- Rebase on top of linux-media/master (v4.16-rc4, metadata additions)
drivers/media/usb/uvc/uvc_isight.c | 6 --
drivers/media/usb/uvc/uvc_video.c | 26 ++
drivers/media/usb/uvc/uvcvideo.h | 8 +---
3 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_isight.c
b/drivers/media/usb/uvc/uvc_isight.c
index 81e6f2187bfb..39a4e4482b23 100644
--- a/drivers/media/usb/uvc/uvc_isight.c
+++ b/drivers/media/usb/uvc/uvc_isight.c
@@ -99,9 +99,11 @@ static int isight_decode(struct uvc_video_queue *queue,
struct uvc_buffer *buf,
return 0;
}
-void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
- struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
+void uvc_video_decode_isight(struct uvc_urb *uvc_urb, struct uvc_buffer *buf,
+ struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
int ret, i;
for (i = 0; i < urb->number_of_packets; ++i) {
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 113881bed2a4..6d4384695964 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1291,9 +1291,11 @@ static void uvc_video_next_buffers(struct uvc_streaming
*stream,
*video_buf = uvc_queue_next_buffer(>queue, *video_buf);
}
-static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming
*stream,
+static void uvc_video_decode_isoc(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
u8 *mem;
int ret, i;
@@ -1334,9 +1336,11 @@ static void uvc_video_decode_isoc(struct urb *urb,
struct uvc_streaming *stream,
}
}
-static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming
*stream,
+static void uvc_video_decode_bulk(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
u8 *mem;
int len, ret;
@@ -1402,9 +1406,12 @@ static void uvc_video_decode_bulk(struct urb *urb,
struct uvc_streaming *stream,
}
}
-static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming
*stream,
+static void uvc_video_encode_bulk(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
+
u8 *mem = urb->transfer_buffer;
int len = stream->urb_size, ret;
@@ -1447,7 +1454,8 @@ static void uvc_video_encode_bulk(struct urb *urb, struct
uvc_streaming *stream,
static void uvc_video_complete(struct urb *urb)
{
- struct uvc_streaming *stream = urb->context;
+ struct uvc_urb *uvc_urb = urb->context;
+ struct uvc_streaming *stream = uvc_urb->stream;
struct uvc_video_queue *queue = >queue;
struct uvc_video_queue *qmeta = >meta.queue;
struct vb2_queue *vb2_qmeta = stream->meta.vdev.queue;
@@ -1490,7 +1498,7 @@ static void uvc_video_complete(struct urb *urb)
spin_unlock_irqrestore(>irqlock, flags);
}
- stream->decode(urb, stream, buf, buf_meta);
+ stream->decode(uvc_urb, buf, buf_meta);
if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) {
uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
@@ -1568,6 +1576,8 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
uvc_free_urb_buffers(stream);
break;
}
+
+ uvc_urb->stream = stream;
}
if (i == UVC_URBS) {
@@ -1666,7 +1676,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
}
urb->dev = stream->dev->udev;
- urb->context = stream;
+ urb->context = uvc_urb;
urb->pipe = usb_rcvisocpipe(stream->dev->udev,
ep->desc.bEndpointAddress);
#ifndef CONFIG_DMA_NONCOHERENT
@@ -1733,8 +1743,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
return -ENOMEM;
}
- usb_fill_bulk_urb(urb,
[PATCH v6 05/10] media: uvcvideo: queue: Support asynchronous buffer handling
The buffer queue interface currently operates sequentially, processing
buffers after they have fully completed.
In preparation for supporting parallel tasks operating on the buffers,
we will need to support buffers being processed on multiple CPUs.
Adapt the uvc_queue_next_buffer() such that a reference count tracks the
active use of the buffer, returning the buffer to the VB2 stack at
completion.
Signed-off-by: Kieran Bingham
v5:
- uvc_queue_requeue() -> uvc_queue_buffer_requeue()
- Fix comment
---
drivers/media/usb/uvc/uvc_queue.c | 61 ++--
drivers/media/usb/uvc/uvcvideo.h | 4 ++-
2 files changed, 54 insertions(+), 11 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index fa7059aab49a..2752e386f1e8 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -142,6 +142,7 @@ static void uvc_buffer_queue(struct vb2_buffer *vb)
spin_lock_irqsave(>irqlock, flags);
if (likely(!(queue->flags & UVC_QUEUE_DISCONNECTED))) {
+ kref_init(>ref);
list_add_tail(>queue, >irqqueue);
} else {
/* If the device is disconnected return the buffer to userspace
@@ -458,28 +459,66 @@ struct uvc_buffer *uvc_queue_get_current_buffer(struct
uvc_video_queue *queue)
return nextbuf;
}
-struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
+/*
+ * uvc_queue_buffer_requeue: Requeue a buffer on our internal irqqueue
+ *
+ * Reuse a buffer through our internal queue without the need to 'prepare'.
+ * The buffer will be returned to userspace through the uvc_buffer_queue call
if
+ * the device has been disconnected.
+ */
+static void uvc_queue_buffer_requeue(struct uvc_video_queue *queue,
struct uvc_buffer *buf)
{
- struct uvc_buffer *nextbuf;
- unsigned long flags;
+ buf->error = 0;
+ buf->state = UVC_BUF_STATE_QUEUED;
+ buf->bytesused = 0;
+ vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
+
+ uvc_buffer_queue(>buf.vb2_buf);
+}
+
+static void uvc_queue_buffer_complete(struct kref *ref)
+{
+ struct uvc_buffer *buf = container_of(ref, struct uvc_buffer, ref);
+ struct vb2_buffer *vb = >buf.vb2_buf;
+ struct uvc_video_queue *queue = vb2_get_drv_priv(vb->vb2_queue);
if ((queue->flags & UVC_QUEUE_DROP_CORRUPTED) && buf->error) {
- buf->error = 0;
- buf->state = UVC_BUF_STATE_QUEUED;
- buf->bytesused = 0;
- vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
- return buf;
+ uvc_queue_buffer_requeue(queue, buf);
+ return;
}
+ buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
+ vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
+ vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+/*
+ * Release a reference on the buffer. Complete the buffer when the last
+ * reference is released.
+ */
+void uvc_queue_buffer_release(struct uvc_buffer *buf)
+{
+ kref_put(>ref, uvc_queue_buffer_complete);
+}
+
+/*
+ * Remove this buffer from the queue. Lifetime will persist while async actions
+ * are still running (if any), and uvc_queue_buffer_release will give the
buffer
+ * back to VB2 when all users have completed.
+ */
+struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
+ struct uvc_buffer *buf)
+{
+ struct uvc_buffer *nextbuf;
+ unsigned long flags;
+
spin_lock_irqsave(>irqlock, flags);
list_del(>queue);
nextbuf = __uvc_queue_get_current_buffer(queue);
spin_unlock_irqrestore(>irqlock, flags);
- buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
- vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
- vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
+ uvc_queue_buffer_release(buf);
return nextbuf;
}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 16b8348f7ff0..7f884c60ae59 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -413,6 +413,9 @@ struct uvc_buffer {
unsigned int bytesused;
u32 pts;
+
+ /* Asynchronous buffer handling. */
+ struct kref ref;
};
#define UVC_QUEUE_DISCONNECTED (1 << 0)
@@ -728,6 +731,7 @@ void uvc_queue_cancel(struct uvc_video_queue *queue, int
disconnect);
struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
struct uvc_buffer *buf);
struct uvc_buffer *uvc_queue_get_current_buffer(struct uvc_video_queue *queue);
+void uvc_queue_buffer_release(struct uvc_buffer *buf);
int uvc_queue_mmap(struct uvc_video_queue *queue,
struct vm_area_struct *vma);
__poll_t uvc_queue_poll(struct uvc_video_queue *queue, struct file *file,
--
git-series 0.9.1
[PATCH v6 04/10] media: uvcvideo: queue: Simplify spin-lock usage
Both uvc_start_streaming(), and uvc_stop_streaming() are called from
userspace context, with interrupts enabled. As such, they do not need to
save the IRQ state, and can use spin_lock_irq() and spin_unlock_irq()
respectively.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v4:
- Rebase to v4.16 (linux-media/master)
v5:
- Provide lockdep validation
drivers/media/usb/uvc/uvc_queue.c | 14 --
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index adcc4928fae4..fa7059aab49a 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -169,18 +169,19 @@ static int uvc_start_streaming(struct vb2_queue *vq,
unsigned int count)
{
struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
struct uvc_streaming *stream = uvc_queue_to_stream(queue);
- unsigned long flags;
int ret;
+ lockdep_assert_irqs_enabled();
+
queue->buf_used = 0;
ret = uvc_video_enable(stream, 1);
if (ret == 0)
return 0;
- spin_lock_irqsave(>irqlock, flags);
+ spin_lock_irq(>irqlock);
uvc_queue_return_buffers(queue, UVC_BUF_STATE_QUEUED);
- spin_unlock_irqrestore(>irqlock, flags);
+ spin_unlock_irq(>irqlock);
return ret;
}
@@ -188,14 +189,15 @@ static int uvc_start_streaming(struct vb2_queue *vq,
unsigned int count)
static void uvc_stop_streaming(struct vb2_queue *vq)
{
struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
- unsigned long flags;
+
+ lockdep_assert_irqs_enabled();
if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
uvc_video_enable(uvc_queue_to_stream(queue), 0);
- spin_lock_irqsave(>irqlock, flags);
+ spin_lock_irq(>irqlock);
uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
- spin_unlock_irqrestore(>irqlock, flags);
+ spin_unlock_irq(>irqlock);
}
static const struct vb2_ops uvc_queue_qops = {
--
git-series 0.9.1
[PATCH v6 00/10] Asynchronous UVC
The Linux UVC driver has long provided adequate performance capabilities for
web-cams and low data rate video devices in Linux while resolutions were low.
Modern USB cameras are now capable of high data rates thanks to USB3 with
1080p, and even 4k capture resolutions supported.
Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO
(isochronous transfers) can generate more data than an embedded ARM core is
able to process on a single core, resulting in frame loss.
A large part of this performance impact is from the requirement to
‘memcpy’ frames out from URB packets to destination frames. This unfortunate
requirement is due to the UVC protocol allowing a variable length header, and
thus it is not possible to provide the target frame buffers directly.
Extra throughput is possible by moving the actual memcpy actions to a work
queue, and moving the memcpy out of interrupt context thus allowing work tasks
to be scheduled across multiple cores.
This series has been tested on both the ZED and BRIO cameras on arm64
platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix USB2
Camera, and a built in Toshiba Laptop camera, and with thanks to Philipp Zabel
for testing on a Lite-On internal Laptop Webcam, Logitech C910 (USB2 isoc),
Oculus Sensor (USB3 isoc), and Microsoft HoloLens Sensors (USB3 bulk).
As far as I am aware iSight devices, and devices which use UVC to encode data
(output device) have not yet been tested - but should find no ill effect (at
least not until they are tested of course :D )
Tested-by: Randy Dunlap
Tested-by: Philipp Zabel
This series is based upon Laurent Pinchart's uvc/next available at:
git://linuxtv.org/pinchartl/media.git uvc/next
and the series itself is available at my kernel.org repo:
git://git.kernel.org/pub/scm/linux/kernel/git/kbingham/linux.git uvc/async
v2:
- Fix race reported by Guennadi
v3:
- Fix similar race reported by Laurent
- Only queue work if required (encode/isight do not queue work)
- Refactor/Rename variables for clarity
v4:
- (Yet another) Rework of the uninitialise path.
This time to hopefully clean up the shutdown races for good.
use usb_poison_urb() to halt all URBs, then flush the work queue
before freeing.
- Rebase to latest linux-media/master
v5:
- Provide lockdep validation
- rename uvc_queue_requeue -> uvc_queue_buffer_requeue()
- Fix comments and periods throughout
- Rebase to media/v4.20-2
- Use GFP_KERNEL allocation in uvc_video_copy_data_work()
- Fix function documentation for uvc_video_copy_data_work()
- Add periods to the end of sentences
- Rename 'decode' variable to 'op' in uvc_video_decode_data()
- Move uvc_urb->async_operations initialisation to before use
- Move async workqueue to match uvc_streaming lifetime instead of
streamon/streamoff
- bracket the for_each_uvc_urb() macro
- New patches added to series:
media: uvcvideo: Split uvc_video_enable into two
media: uvcvideo: Rename uvc_{un,}init_video()
media: uvcvideo: Utilise for_each_uvc_urb iterator
v6:
- New patch added to series
media: uvcvideo: Abstract streaming object lifetime
- Utilise the new streaming object lifetime functions to perform
allocation and destruction of the async workqueue.
- Append _transfer to {_stop,_start} in uvc_video_{stop,start}_transfer
- rename lone 'j' iterator to 'i'
- Remove conversion which doesn't make sense due to needing the
iterator value.
Kieran Bingham (10):
media: uvcvideo: Refactor URB descriptors
media: uvcvideo: Convert decode functions to use new context structure
media: uvcvideo: Protect queue internals with helper
media: uvcvideo: queue: Simplify spin-lock usage
media: uvcvideo: queue: Support asynchronous buffer handling
media: uvcvideo: Abstract streaming object lifetime
media: uvcvideo: Move decode processing to process context
media: uvcvideo: Split uvc_video_enable into two
media: uvcvideo: Rename uvc_{un,}init_video()
media: uvcvideo: Utilise for_each_uvc_urb iterator
drivers/media/usb/uvc/uvc_driver.c | 65 +--
drivers/media/usb/uvc/uvc_isight.c | 6 +-
drivers/media/usb/uvc/uvc_queue.c | 110 +---
drivers/media/usb/uvc/uvc_video.c | 275 ++
drivers/media/usb/uvc/uvcvideo.h | 65 ++-
5 files changed, 370 insertions(+), 151 deletions(-)
base-commit: b82b45e8c9d5d015c6887216868e3335897662d3
--
git-series 0.9.1
[PATCH v6 01/10] media: uvcvideo: Refactor URB descriptors
We currently store three separate arrays for each URB reference we hold.
Objectify the data needed to track URBs into a single uvc_urb structure,
allowing better object management and tracking of the URB.
All accesses to the data pointers through stream, are converted to use a
uvc_urb pointer for consistency.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v2:
- Re-describe URB context structure
- Re-name uvc_urb->{urb_buffer,urb_dma}{buffer,dma}
v3:
- No change
v4:
- Rebase on top of linux-media/master (v4.16-rc4, metadata additions)
drivers/media/usb/uvc/uvc_video.c | 49 +++-
drivers/media/usb/uvc/uvcvideo.h | 18 ++--
2 files changed, 45 insertions(+), 22 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 86a99f461fd8..113881bed2a4 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1506,14 +1506,16 @@ static void uvc_free_urb_buffers(struct uvc_streaming
*stream)
unsigned int i;
for (i = 0; i < UVC_URBS; ++i) {
- if (stream->urb_buffer[i]) {
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
+ if (uvc_urb->buffer) {
#ifndef CONFIG_DMA_NONCOHERENT
usb_free_coherent(stream->dev->udev, stream->urb_size,
- stream->urb_buffer[i], stream->urb_dma[i]);
+ uvc_urb->buffer, uvc_urb->dma);
#else
- kfree(stream->urb_buffer[i]);
+ kfree(uvc_urb->buffer);
#endif
- stream->urb_buffer[i] = NULL;
+ uvc_urb->buffer = NULL;
}
}
@@ -1551,16 +1553,18 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
/* Retry allocations until one succeed. */
for (; npackets > 1; npackets /= 2) {
for (i = 0; i < UVC_URBS; ++i) {
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
stream->urb_size = psize * npackets;
#ifndef CONFIG_DMA_NONCOHERENT
- stream->urb_buffer[i] = usb_alloc_coherent(
+ uvc_urb->buffer = usb_alloc_coherent(
stream->dev->udev, stream->urb_size,
- gfp_flags | __GFP_NOWARN, >urb_dma[i]);
+ gfp_flags | __GFP_NOWARN, _urb->dma);
#else
- stream->urb_buffer[i] =
+ uvc_urb->buffer =
kmalloc(stream->urb_size, gfp_flags | __GFP_NOWARN);
#endif
- if (!stream->urb_buffer[i]) {
+ if (!uvc_urb->buffer) {
uvc_free_urb_buffers(stream);
break;
}
@@ -1590,13 +1594,15 @@ static void uvc_uninit_video(struct uvc_streaming
*stream, int free_buffers)
uvc_video_stats_stop(stream);
for (i = 0; i < UVC_URBS; ++i) {
- urb = stream->urb[i];
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
+ urb = uvc_urb->urb;
if (urb == NULL)
continue;
usb_kill_urb(urb);
usb_free_urb(urb);
- stream->urb[i] = NULL;
+ uvc_urb->urb = NULL;
}
if (free_buffers)
@@ -1651,6 +1657,8 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
size = npackets * psize;
for (i = 0; i < UVC_URBS; ++i) {
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
urb = usb_alloc_urb(npackets, gfp_flags);
if (urb == NULL) {
uvc_uninit_video(stream, 1);
@@ -1663,12 +1671,12 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
ep->desc.bEndpointAddress);
#ifndef CONFIG_DMA_NONCOHERENT
urb->transfer_flags = URB_ISO_ASAP | URB_NO_TRANSFER_DMA_MAP;
- urb->transfer_dma = stream->urb_dma[i];
+ urb->transfer_dma = uvc_urb->dma;
#else
urb->transfer_flags = URB_ISO_ASAP;
#endif
urb->interval = ep->desc.bInterval;
- urb->transfer_buffer = stream->urb_buffer[i];
+ urb->transfer_buffer = uvc_urb->buffer;
urb->complete = uvc_video_complete;
urb->number_of_packets = npackets;
urb->transfer_buffer_length = size;
@@ -1678,7 +1686,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
urb->iso_frame_desc[j].length = psize;
}
- stream->urb[i] = urb;
+ uvc_urb->
Re: [PATCH v5 8/9] media: uvcvideo: Rename uvc_{un,}init_video()
On 09/11/2018 15:41, Philipp Zabel wrote:
> On Wed, 2018-11-07 at 22:25 +0200, Laurent Pinchart wrote:
>> Hi Kieran,
>>
>> On Wednesday, 7 November 2018 16:30:46 EET Kieran Bingham wrote:
>>> On 06/11/2018 23:13, Laurent Pinchart wrote:
>>>> On Tuesday, 6 November 2018 23:27:19 EET Kieran Bingham wrote:
>>>>> From: Kieran Bingham
>>>>>
>>>>> We have both uvc_init_video() and uvc_video_init() calls which can be
>>>>> quite confusing to determine the process for each. Now that video
>>>>> uvc_video_enable() has been renamed to uvc_video_start_streaming(),
>>>>> adapt these calls to suit the new flow.
>>>>>
>>>>> Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
>>>>> uvc_video_stop().
>>>>
>>>> I agree that these functions are badly named and should be renamed. We are
>>>> however entering the nitpicking territory :-) The two functions do more
>>>> that starting and stopping, they also allocate and free URBs and the
>>>> associated buffers. It could also be argued that they don't actually
>>>> start and stop anything, as beyond URB management, they just queue the
>>>> URBs initially and kill them. I thus wonder if we could come up with
>>>> better names.
>>>
>>> Well the act of killing (poisoning now) the URBs will certainly stop the
>>> stream, but I guess submitting the URBs isn't necessarily the key act to
>>> starting the stream.
>>>
>>> I believe that needs the interface to be set correctly, and the buffers
>>> to be available?
>>>
>>> Although - I've just double-checked uvc_{video_start,init_video}() and
>>> that is indeed what it does?
>>>
>>> - start stats
>>> - Initialise endpoints
>>>- Perform allocations
>>> - Submit URBs
>>>
>>> Am I missing something? Is there another step that is pivotal to
>>> starting the USB packet/urb stream flow after this point ?
>>>
>>>
>>> Is it not true that the USB stack will start processing data at
>>> submitting URB completion callbacks after the end of uvc_video_start();
>>> and will no longer process data at the end of uvc_video_stop() (and thus
>>> no more completion callbacks)?
>>>
>>> (That's a real question to verify my interpretation)
>>>
>>> To me - these functions feel like the real 'start' and 'stop' components
>>> of the data stream - hence my choice in naming.
>>
>> The other part of the start operation is committing the streaming parameters
>> (see uvc_video_start_streaming()). For the stop operation it's issuing a
>> SET_INTERFACE or CLEAR_FEATURE(HALT) request (see
>> uvc_video_stop_streaming()).
>>
>>> Is your concern that you would like the functions to be more descriptive
>>> over their other actions such as? :
>>>
>>> uvc_video_initialise_start()
>>> uvc_video_allocate_init_start()
>>>
>>> Or something else? (I don't think those two are good names though)
>>
>> Probably something else :-) A possibly equally bad proposal would be
>> uvc_video_start_transfer() and uvc_video_stop_transfer().
>
> I think this is still better than what we have now.
>
> At least it contains "transfer" to make it clear it deals with the
> isoc/bulk transfer setup/teardown part of streaming, not actually
> starting or stopping the device streaming.
Ok - uvc_video_start_transfer() and uvc_video_stop_transfer() it shall
be then :)
v6 with fixed stream object / workqueue lifetime and updates to the last
two patches coming ... soon :)
--
Kieran
>
> regards
> Philipp
>
--
Regards
--
Kieran
Re: [PATCH v5 0/9] Asynchronous UVC
Hi Laurent,
On 08/11/2018 17:01, Laurent Pinchart wrote:
> Hi Kieran,
>
> On Wednesday, 7 November 2018 01:31:29 EET Laurent Pinchart wrote:
>> On Tuesday, 6 November 2018 23:27:11 EET Kieran Bingham wrote:
>>> From: Kieran Bingham
>>>
>>> The Linux UVC driver has long provided adequate performance capabilities
>>> for web-cams and low data rate video devices in Linux while resolutions
>>> were low.
>>>
>>> Modern USB cameras are now capable of high data rates thanks to USB3 with
>>> 1080p, and even 4k capture resolutions supported.
>>>
>>> Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO
>>> (isochronous transfers) can generate more data than an embedded ARM core
>>> is able to process on a single core, resulting in frame loss.
>>>
>>> A large part of this performance impact is from the requirement to
>>> ‘memcpy’ frames out from URB packets to destination frames. This
>>> unfortunate requirement is due to the UVC protocol allowing a variable
>>> length header, and thus it is not possible to provide the target frame
>>> buffers directly.
>>>
>>> Extra throughput is possible by moving the actual memcpy actions to a work
>>> queue, and moving the memcpy out of interrupt context thus allowing work
>>> tasks to be scheduled across multiple cores.
>>>
>>> This series has been tested on both the ZED and BRIO cameras on arm64
>>> platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix
>>> USB2 Camera, and a built in Toshiba Laptop camera, and with thanks to
>>> Philipp Zabel for testing on a Lite-On internal Laptop Webcam, Logitech
>>> C910 (USB2 isoc), Oculus Sensor (USB3 isoc), and Microsoft HoloLens
>>> Sensors (USB3 bulk).
>>>
>>> As far as I am aware iSight devices, and devices which use UVC to encode
>>> data (output device) have not yet been tested - but should find no ill
>>> effect (at least not until they are tested of course :D )
>>
>> :-D
>>
>> I'm not sure whether anyone is still using those devices with Linux. I
>> wouldn't be surprised if we realized down the road that they already don't
>> work.
>>
>>> Tested-by: Randy Dunlap
>>> Tested-by: Philipp Zabel
>>>
>>> v2:
>>> - Fix race reported by Guennadi
>>>
>>> v3:
>>> - Fix similar race reported by Laurent
>>> - Only queue work if required (encode/isight do not queue work)
>>> - Refactor/Rename variables for clarity
>>>
>>> v4:
>>> - (Yet another) Rework of the uninitialise path.
>>>
>>>This time to hopefully clean up the shutdown races for good.
>>>use usb_poison_urb() to halt all URBs, then flush the work queue
>>>before freeing.
>>>
>>> - Rebase to latest linux-media/master
>>>
>>> v5:
>>> - Provide lockdep validation
>>> - rename uvc_queue_requeue -> uvc_queue_buffer_requeue()
>>> - Fix comments and periods throughout
>>> - Rebase to media/v4.20-2
>>> - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
>>> - Fix function documentation for uvc_video_copy_data_work()
>>> - Add periods to the end of sentences
>>> - Rename 'decode' variable to 'op' in uvc_video_decode_data()
>>> - Move uvc_urb->async_operations initialisation to before use
>>> - Move async workqueue to match uvc_streaming lifetime instead of
>>>
>>>streamon/streamoff
>>>
>>> - bracket the for_each_uvc_urb() macro
>>>
>>> - New patches added to series:
>>> media: uvcvideo: Split uvc_video_enable into two
>>> media: uvcvideo: Rename uvc_{un,}init_video()
>>> media: uvcvideo: Utilise for_each_uvc_urb iterator
>>>
>>> Kieran Bingham (9):
>>> media: uvcvideo: Refactor URB descriptors
>>> media: uvcvideo: Convert decode functions to use new context structure
>>> media: uvcvideo: Protect queue internals with helper
>>> media: uvcvideo: queue: Simplify spin-lock usage
>>> media: uvcvideo: queue: Support asynchronous buffer handling
>>> media: uvcvideo: Move decode processing to process context
>>> media: uvcvideo: Split uvc_video_enable into two
>>
>> I've taken the above patches in my tree.
>
> I'm afraid I'll have to drop them :-( If I disconnect the camera while in use,
> I get the following oops:
Argh - that's what we get for that 'last minute' change o
Re: [PATCH v5 8/9] media: uvcvideo: Rename uvc_{un,}init_video()
Hi Laurent,
On 06/11/2018 23:13, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Tuesday, 6 November 2018 23:27:19 EET Kieran Bingham wrote:
>> From: Kieran Bingham
>>
>> We have both uvc_init_video() and uvc_video_init() calls which can be
>> quite confusing to determine the process for each. Now that video
>> uvc_video_enable() has been renamed to uvc_video_start_streaming(),
>> adapt these calls to suit the new flow.
>>
>> Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
>> uvc_video_stop().
>
> I agree that these functions are badly named and should be renamed. We are
> however entering the nitpicking territory :-) The two functions do more that
> starting and stopping, they also allocate and free URBs and the associated
> buffers. It could also be argued that they don't actually start and stop
> anything, as beyond URB management, they just queue the URBs initially and
> kill them. I thus wonder if we could come up with better names.
Well the act of killing (poisoning now) the URBs will certainly stop the
stream, but I guess submitting the URBs isn't necessarily the key act to
starting the stream.
I believe that needs the interface to be set correctly, and the buffers
to be available?
Although - I've just double-checked uvc_{video_start,init_video}() and
that is indeed what it does?
- start stats
- Initialise endpoints
- Perform allocations
- Submit URBs
Am I missing something? Is there another step that is pivotal to
starting the USB packet/urb stream flow after this point ?
Is it not true that the USB stack will start processing data at
submitting URB completion callbacks after the end of uvc_video_start();
and will no longer process data at the end of uvc_video_stop() (and thus
no more completion callbacks)?
(That's a real question to verify my interpretation)
To me - these functions feel like the real 'start' and 'stop' components
of the data stream - hence my choice in naming.
Is your concern that you would like the functions to be more descriptive
over their other actions such as? :
uvc_video_initialise_start()
uvc_video_allocate_init_start()
Or something else? (I don't think those two are good names though)
--
Kieran
>> Signed-off-by: Kieran Bingham
>> ---
>> drivers/media/usb/uvc/uvc_video.c | 18 +-
>> 1 file changed, 9 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/media/usb/uvc/uvc_video.c
>> b/drivers/media/usb/uvc/uvc_video.c index 0d35e933856a..020022e6ade4 100644
>> --- a/drivers/media/usb/uvc/uvc_video.c
>> +++ b/drivers/media/usb/uvc/uvc_video.c
>> @@ -1641,7 +1641,7 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
>> *stream, /*
>> * Uninitialize isochronous/bulk URBs and free transfer buffers.
>> */
>> -static void uvc_uninit_video(struct uvc_streaming *stream, int
>> free_buffers) +static void uvc_video_stop(struct uvc_streaming *stream, int
>> free_buffers) {
>> struct uvc_urb *uvc_urb;
>>
>> @@ -1718,7 +1718,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
>> *stream,
>>
>> urb = usb_alloc_urb(npackets, gfp_flags);
>> if (urb == NULL) {
>> -uvc_uninit_video(stream, 1);
>> +uvc_video_stop(stream, 1);
>> return -ENOMEM;
>> }
>>
>> @@ -1786,7 +1786,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
>> *stream,
>>
>> urb = usb_alloc_urb(0, gfp_flags);
>> if (urb == NULL) {
>> -uvc_uninit_video(stream, 1);
>> +uvc_video_stop(stream, 1);
>> return -ENOMEM;
>> }
>>
>> @@ -1806,7 +1806,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
>> *stream, /*
>> * Initialize isochronous/bulk URBs and allocate transfer buffers.
>> */
>> -static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
>> +static int uvc_video_start(struct uvc_streaming *stream, gfp_t gfp_flags)
>> {
>> struct usb_interface *intf = stream->intf;
>> struct usb_host_endpoint *ep;
>> @@ -1894,7 +1894,7 @@ static int uvc_init_video(struct uvc_streaming
>> *stream, gfp_t gfp_flags) if (ret < 0) {
>> uvc_printk(KERN_ERR, "Failed to submit URB %u "
>> "(%d).\n", i, ret);
>> -uvc_uninit_video(stream, 1);
>> +uvc_video_stop(stream, 1);
>> return ret;
>> }
>> }
>> @@ -1
Re: [PATCH v5 9/9] media: uvcvideo: Utilise for_each_uvc_urb iterator
Hi Laurent,
On 06/11/2018 23:21, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Tuesday, 6 November 2018 23:27:20 EET Kieran Bingham wrote:
>> From: Kieran Bingham
>>
>> A new iterator is available for processing UVC URB structures. This
>> simplifies the processing of the internal stream data.
>>
>> Convert the manual loop iterators to the new helper, adding an index
>> helper to keep the existing debug print.
>>
>> Signed-off-by: Kieran Bingham
>>
>> ---
>> This patch converts to using the iterator which for most hunks makes
>> sense. The only one with uncertainty is in uvc_alloc_urb_buffers() where
>> the loop index is used to determine if all the buffers were successfully
>> allocated.
>>
>> As the loop index is not incremented by the loops, we can obtain the
>> buffer index - but then we are offset and out-by-one.
>>
>> Adjusting this in the code is fine - but at that point it feels like
>> it's not adding much value. I've left that hunk in for this patch - but
>> that part could be reverted if desired - unless anyone has a better
>> rework of the buffer check?
>>
>> drivers/media/usb/uvc/uvc_video.c | 51
>> drivers/media/usb/uvc/uvcvideo.h | 3 ++-
>> 2 files changed, 29 insertions(+), 25 deletions(-)
>>
>> diff --git a/drivers/media/usb/uvc/uvc_video.c
>> b/drivers/media/usb/uvc/uvc_video.c index 020022e6ade4..f6e5db7ea50e 100644
>> --- a/drivers/media/usb/uvc/uvc_video.c
>> +++ b/drivers/media/usb/uvc/uvc_video.c
>> @@ -1556,20 +1556,19 @@ static void uvc_video_complete(struct urb *urb)
>> */
>> static void uvc_free_urb_buffers(struct uvc_streaming *stream)
>> {
>> -unsigned int i;
>> +struct uvc_urb *uvc_urb;
>>
>> -for (i = 0; i < UVC_URBS; ++i) {
>> -struct uvc_urb *uvc_urb = >uvc_urb[i];
>> +for_each_uvc_urb(uvc_urb, stream) {
>> +if (!uvc_urb->buffer)
>> +continue;
>>
>> -if (uvc_urb->buffer) {
>> #ifndef CONFIG_DMA_NONCOHERENT
>> -usb_free_coherent(stream->dev->udev, stream->urb_size,
>> -uvc_urb->buffer, uvc_urb->dma);
>> +usb_free_coherent(stream->dev->udev, stream->urb_size,
>> + uvc_urb->buffer, uvc_urb->dma);
>> #else
>> -kfree(uvc_urb->buffer);
>> +kfree(uvc_urb->buffer);
>> #endif
>> -uvc_urb->buffer = NULL;
>> -}
>> +uvc_urb->buffer = NULL;
>> }
>>
>> stream->urb_size = 0;
>> @@ -1589,8 +1588,9 @@ static void uvc_free_urb_buffers(struct uvc_streaming
>> *stream) static int uvc_alloc_urb_buffers(struct uvc_streaming *stream,
>> unsigned int size, unsigned int psize, gfp_t gfp_flags)
>> {
>> +struct uvc_urb *uvc_urb;
>> unsigned int npackets;
>> -unsigned int i;
>> +unsigned int i = 0;
>>
>> /* Buffers are already allocated, bail out. */
>> if (stream->urb_size)
>> @@ -1605,8 +1605,12 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
>> *stream,
>>
>> /* Retry allocations until one succeed. */
>> for (; npackets > 1; npackets /= 2) {
>> -for (i = 0; i < UVC_URBS; ++i) {
>> -struct uvc_urb *uvc_urb = >uvc_urb[i];
>> +for_each_uvc_urb(uvc_urb, stream) {
>> +/*
>> + * Track how many URBs we allocate, adding one to the
>> + * index to account for our zero index.
>> + */
>> +i = uvc_urb_index(uvc_urb) + 1;
>
> That's a bit ugly indeed, I think we could keep the existing loop;
I do agree, as stated I left it in for 'completeness' of the patch. But
I don't think it adds value here.
Will remove.
>> stream->urb_size = psize * npackets;
>> #ifndef CONFIG_DMA_NONCOHERENT
>> @@ -1700,7 +1704,8 @@ static int uvc_init_video_isoc(struct uvc_streaming
>> *stream, struct usb_host_endpoint *ep, gfp_t gfp_flags)
>> {
>> struct urb *urb;
>> -unsigned int npackets, i, j;
>> +struct uvc_urb *uvc_urb;
>> +unsigned int npackets, j;
>
> j without i seems weird, could you rename it ?
Sure. Done
>
>> u16 psize;
>> u32 size;
>>
>> @@ -1713,9 +1718,7 @@ static in
Re: [PATCH v5 6/9] media: uvcvideo: Move decode processing to process context
On 06/11/2018 22:58, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Tuesday, 6 November 2018 23:27:17 EET Kieran Bingham wrote:
>> From: Kieran Bingham
>>
>> Newer high definition cameras, and cameras with multiple lenses such as
>> the range of stereo-vision cameras now available have ever increasing
>> data rates.
>>
>> The inclusion of a variable length packet header in URB packets mean
>> that we must memcpy the frame data out to our destination 'manually'.
>> This can result in data rates of up to 2 gigabits per second being
>> processed.
>>
>> To improve efficiency, and maximise throughput, handle the URB decode
>> processing through a work queue to move it from interrupt context, and
>> allow multiple processors to work on URBs in parallel.
>>
>> Signed-off-by: Kieran Bingham
>
> Reviewed-by: Laurent Pinchart
>
> I wonder if we shouldn't, as a future improvement, only queue async work when
> the quantity of data to be copied is above a certain threshold.
Possibly - lets keep it in mind for when we get back to looking at
Keiichi's patch and any cache management for further performance
improvements
--
Kieran
>
>> ---
>> v2:
>> - Lock full critical section of usb_submit_urb()
>>
>> v3:
>> - Fix race on submitting uvc_video_decode_data_work() to work queue.
>> - Rename uvc_decode_op -> uvc_copy_op (Generic to encode/decode)
>> - Rename decodes -> copy_operations
>> - Don't queue work if there is no async task
>> - obtain copy op structure directly in uvc_video_decode_data()
>> - uvc_video_decode_data_work() -> uvc_video_copy_data_work()
>>
>> v4:
>> - Provide for_each_uvc_urb()
>> - Simplify fix for shutdown race to flush queue before freeing URBs
>> - Rebase to v4.16-rc4 (linux-media/master) adjusting for metadata
>>conflicts.
>>
>> v5:
>> - Rebase to media/v4.20-2
>> - Use GFP_KERNEL allocation in uvc_video_copy_data_work()
>> - Fix function documentation for uvc_video_copy_data_work()
>> - Add periods to the end of sentences
>> - Rename 'decode' variable to 'op' in uvc_video_decode_data()
>> - Move uvc_urb->async_operations initialisation to before use
>> - Move async workqueue to match uvc_streaming lifetime instead of
>>streamon/streamoff
>>
>> drivers/media/usb/uvc/uvc_driver.c | 2 +-
>> drivers/media/usb/uvc/uvc_video.c | 110 +++---
>> drivers/media/usb/uvc/uvcvideo.h | 28 -
>> 3 files changed, 116 insertions(+), 24 deletions(-)
>>
>> diff --git a/drivers/media/usb/uvc/uvc_driver.c
>> b/drivers/media/usb/uvc/uvc_driver.c index bc369a0934a3..e61a6d26e812
>> 100644
>> --- a/drivers/media/usb/uvc/uvc_driver.c
>> +++ b/drivers/media/usb/uvc/uvc_driver.c
>> @@ -1883,6 +1883,8 @@ static void uvc_unregister_video(struct uvc_device
>> *dev) video_unregister_device(>vdev);
>> video_unregister_device(>meta.vdev);
>>
>> +destroy_workqueue(stream->async_wq);
>> +
>> uvc_debugfs_cleanup_stream(stream);
>> }
>> }
>> diff --git a/drivers/media/usb/uvc/uvc_video.c
>> b/drivers/media/usb/uvc/uvc_video.c index 7a7779e1b466..ce9e40444507 100644
>> --- a/drivers/media/usb/uvc/uvc_video.c
>> +++ b/drivers/media/usb/uvc/uvc_video.c
>> @@ -1094,21 +1094,54 @@ static int uvc_video_decode_start(struct
>> uvc_streaming *stream, return data[0];
>> }
>>
>> -static void uvc_video_decode_data(struct uvc_streaming *stream,
>> +/*
>> + * uvc_video_decode_data_work: Asynchronous memcpy processing
>> + *
>> + * Copy URB data to video buffers in process context, releasing buffer
>> + * references and requeuing the URB when done.
>> + */
>> +static void uvc_video_copy_data_work(struct work_struct *work)
>> +{
>> +struct uvc_urb *uvc_urb = container_of(work, struct uvc_urb, work);
>> +unsigned int i;
>> +int ret;
>> +
>> +for (i = 0; i < uvc_urb->async_operations; i++) {
>> +struct uvc_copy_op *op = _urb->copy_operations[i];
>> +
>> +memcpy(op->dst, op->src, op->len);
>> +
>> +/* Release reference taken on this buffer. */
>> +uvc_queue_buffer_release(op->buf);
>> +}
>> +
>> +ret = usb_submit_urb(uvc_urb->urb, GFP_KERNEL);
>> +if (ret < 0)
>> +uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
>> +
Re: [PATCH v5 7/9] media: uvcvideo: Split uvc_video_enable into two
Hi Laurent,
On 06/11/2018 23:08, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Tuesday, 6 November 2018 23:27:18 EET Kieran Bingham wrote:
>> From: Kieran Bingham
>>
>> uvc_video_enable() is used both to start and stop the video stream
>> object, however the single function entry point shares no code between
>> the two operations.
>>
>> Split the function into two distinct calls, and rename to
>> uvc_video_start_streaming() and uvc_video_stop_streaming() as
>> appropriate.
>>
>> Signed-off-by: Kieran Bingham
>> ---
>> drivers/media/usb/uvc/uvc_queue.c | 4 +-
>> drivers/media/usb/uvc/uvc_video.c | 56 +++-
>> drivers/media/usb/uvc/uvcvideo.h | 3 +-
>> 3 files changed, 31 insertions(+), 32 deletions(-)
>>
>> diff --git a/drivers/media/usb/uvc/uvc_queue.c
>> b/drivers/media/usb/uvc/uvc_queue.c index cd8c03341de0..682698ec1118 100644
>> --- a/drivers/media/usb/uvc/uvc_queue.c
>> +++ b/drivers/media/usb/uvc/uvc_queue.c
>> @@ -176,7 +176,7 @@ static int uvc_start_streaming(struct vb2_queue *vq,
>> unsigned int count)
>>
>> queue->buf_used = 0;
>>
>> -ret = uvc_video_enable(stream, 1);
>> +ret = uvc_video_start_streaming(stream);
>> if (ret == 0)
>> return 0;
>>
>> @@ -194,7 +194,7 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
>> lockdep_assert_irqs_enabled();
>>
>> if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
>> -uvc_video_enable(uvc_queue_to_stream(queue), 0);
>> +uvc_video_stop_streaming(uvc_queue_to_stream(queue));
>>
>> spin_lock_irq(>irqlock);
>> uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
>> diff --git a/drivers/media/usb/uvc/uvc_video.c
>> b/drivers/media/usb/uvc/uvc_video.c index ce9e40444507..0d35e933856a 100644
>> --- a/drivers/media/usb/uvc/uvc_video.c
>> +++ b/drivers/media/usb/uvc/uvc_video.c
>> @@ -2082,38 +2082,10 @@ int uvc_video_init(struct uvc_streaming *stream)
>> return 0;
>> }
>>
>> -/*
>> - * Enable or disable the video stream.
>> - */
>> -int uvc_video_enable(struct uvc_streaming *stream, int enable)
>> +int uvc_video_start_streaming(struct uvc_streaming *stream)
>> {
>> int ret;
>>
>> -if (!enable) {
>> -uvc_uninit_video(stream, 1);
>> -if (stream->intf->num_altsetting > 1) {
>> -usb_set_interface(stream->dev->udev,
>> - stream->intfnum, 0);
>> -} else {
>> -/* UVC doesn't specify how to inform a bulk-based device
>> - * when the video stream is stopped. Windows sends a
>> - * CLEAR_FEATURE(HALT) request to the video streaming
>> - * bulk endpoint, mimic the same behaviour.
>> - */
>> -unsigned int epnum = stream->header.bEndpointAddress
>> - & USB_ENDPOINT_NUMBER_MASK;
>> -unsigned int dir = stream->header.bEndpointAddress
>> - & USB_ENDPOINT_DIR_MASK;
>> -unsigned int pipe;
>> -
>> -pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
>> -usb_clear_halt(stream->dev->udev, pipe);
>> -}
>> -
>> -uvc_video_clock_cleanup(stream);
>> -return 0;
>> -}
>> -
>> ret = uvc_video_clock_init(stream);
>> if (ret < 0)
>> return ret;
>> @@ -2136,3 +2108,29 @@ int uvc_video_enable(struct uvc_streaming *stream,
>> int enable)
>>
>> return ret;
>> }
>> +
>> +int uvc_video_stop_streaming(struct uvc_streaming *stream)
>> +{
>> +uvc_uninit_video(stream, 1);
>> +if (stream->intf->num_altsetting > 1) {
>> +usb_set_interface(stream->dev->udev,
>> + stream->intfnum, 0);
>
> This now holds on a single line.
Ah yes.
>
>> +} else {
>> +/* UVC doesn't specify how to inform a bulk-based device
>
> Let's fix the checkpatch.pl warning here.
Oh ? I didn't get any checkpatch warnings. Do I need to add some
parameters to my checkpatch now?
>
>> + * when the video stream is stopped. Windows sends a
>> + * CLEAR_FEATURE(
[PATCH v5 7/9] media: uvcvideo: Split uvc_video_enable into two
From: Kieran Bingham
uvc_video_enable() is used both to start and stop the video stream
object, however the single function entry point shares no code between
the two operations.
Split the function into two distinct calls, and rename to
uvc_video_start_streaming() and uvc_video_stop_streaming() as
appropriate.
Signed-off-by: Kieran Bingham
---
drivers/media/usb/uvc/uvc_queue.c | 4 +-
drivers/media/usb/uvc/uvc_video.c | 56 +++-
drivers/media/usb/uvc/uvcvideo.h | 3 +-
3 files changed, 31 insertions(+), 32 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index cd8c03341de0..682698ec1118 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -176,7 +176,7 @@ static int uvc_start_streaming(struct vb2_queue *vq,
unsigned int count)
queue->buf_used = 0;
- ret = uvc_video_enable(stream, 1);
+ ret = uvc_video_start_streaming(stream);
if (ret == 0)
return 0;
@@ -194,7 +194,7 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
lockdep_assert_irqs_enabled();
if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
- uvc_video_enable(uvc_queue_to_stream(queue), 0);
+ uvc_video_stop_streaming(uvc_queue_to_stream(queue));
spin_lock_irq(>irqlock);
uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index ce9e40444507..0d35e933856a 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -2082,38 +2082,10 @@ int uvc_video_init(struct uvc_streaming *stream)
return 0;
}
-/*
- * Enable or disable the video stream.
- */
-int uvc_video_enable(struct uvc_streaming *stream, int enable)
+int uvc_video_start_streaming(struct uvc_streaming *stream)
{
int ret;
- if (!enable) {
- uvc_uninit_video(stream, 1);
- if (stream->intf->num_altsetting > 1) {
- usb_set_interface(stream->dev->udev,
- stream->intfnum, 0);
- } else {
- /* UVC doesn't specify how to inform a bulk-based device
-* when the video stream is stopped. Windows sends a
-* CLEAR_FEATURE(HALT) request to the video streaming
-* bulk endpoint, mimic the same behaviour.
-*/
- unsigned int epnum = stream->header.bEndpointAddress
- & USB_ENDPOINT_NUMBER_MASK;
- unsigned int dir = stream->header.bEndpointAddress
-& USB_ENDPOINT_DIR_MASK;
- unsigned int pipe;
-
- pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
- usb_clear_halt(stream->dev->udev, pipe);
- }
-
- uvc_video_clock_cleanup(stream);
- return 0;
- }
-
ret = uvc_video_clock_init(stream);
if (ret < 0)
return ret;
@@ -2136,3 +2108,29 @@ int uvc_video_enable(struct uvc_streaming *stream, int
enable)
return ret;
}
+
+int uvc_video_stop_streaming(struct uvc_streaming *stream)
+{
+ uvc_uninit_video(stream, 1);
+ if (stream->intf->num_altsetting > 1) {
+ usb_set_interface(stream->dev->udev,
+ stream->intfnum, 0);
+ } else {
+ /* UVC doesn't specify how to inform a bulk-based device
+* when the video stream is stopped. Windows sends a
+* CLEAR_FEATURE(HALT) request to the video streaming
+* bulk endpoint, mimic the same behaviour.
+*/
+ unsigned int epnum = stream->header.bEndpointAddress
+ & USB_ENDPOINT_NUMBER_MASK;
+ unsigned int dir = stream->header.bEndpointAddress
+& USB_ENDPOINT_DIR_MASK;
+ unsigned int pipe;
+
+ pipe = usb_sndbulkpipe(stream->dev->udev, epnum) | dir;
+ usb_clear_halt(stream->dev->udev, pipe);
+ }
+
+ uvc_video_clock_cleanup(stream);
+ return 0;
+}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 0953e2e59a79..c0a120496a1f 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -784,7 +784,8 @@ void uvc_mc_cleanup_entity(struct uvc_entity *entity);
int uvc_video_init(struct uvc_streaming *stream);
int uvc_video_suspend(struct uvc_streaming *stream);
int uvc_video_resume(struct uvc_streaming *stream, int reset);
-int uvc_video_enable(struct uvc_streaming *stream, int enable)
[PATCH v5 9/9] media: uvcvideo: Utilise for_each_uvc_urb iterator
From: Kieran Bingham
A new iterator is available for processing UVC URB structures. This
simplifies the processing of the internal stream data.
Convert the manual loop iterators to the new helper, adding an index
helper to keep the existing debug print.
Signed-off-by: Kieran Bingham
---
This patch converts to using the iterator which for most hunks makes
sense. The only one with uncertainty is in uvc_alloc_urb_buffers() where
the loop index is used to determine if all the buffers were successfully
allocated.
As the loop index is not incremented by the loops, we can obtain the
buffer index - but then we are offset and out-by-one.
Adjusting this in the code is fine - but at that point it feels like
it's not adding much value. I've left that hunk in for this patch - but
that part could be reverted if desired - unless anyone has a better
rework of the buffer check?
drivers/media/usb/uvc/uvc_video.c | 51
drivers/media/usb/uvc/uvcvideo.h | 3 ++-
2 files changed, 29 insertions(+), 25 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 020022e6ade4..f6e5db7ea50e 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1556,20 +1556,19 @@ static void uvc_video_complete(struct urb *urb)
*/
static void uvc_free_urb_buffers(struct uvc_streaming *stream)
{
- unsigned int i;
+ struct uvc_urb *uvc_urb;
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
+ for_each_uvc_urb(uvc_urb, stream) {
+ if (!uvc_urb->buffer)
+ continue;
- if (uvc_urb->buffer) {
#ifndef CONFIG_DMA_NONCOHERENT
- usb_free_coherent(stream->dev->udev, stream->urb_size,
- uvc_urb->buffer, uvc_urb->dma);
+ usb_free_coherent(stream->dev->udev, stream->urb_size,
+ uvc_urb->buffer, uvc_urb->dma);
#else
- kfree(uvc_urb->buffer);
+ kfree(uvc_urb->buffer);
#endif
- uvc_urb->buffer = NULL;
- }
+ uvc_urb->buffer = NULL;
}
stream->urb_size = 0;
@@ -1589,8 +1588,9 @@ static void uvc_free_urb_buffers(struct uvc_streaming
*stream)
static int uvc_alloc_urb_buffers(struct uvc_streaming *stream,
unsigned int size, unsigned int psize, gfp_t gfp_flags)
{
+ struct uvc_urb *uvc_urb;
unsigned int npackets;
- unsigned int i;
+ unsigned int i = 0;
/* Buffers are already allocated, bail out. */
if (stream->urb_size)
@@ -1605,8 +1605,12 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
/* Retry allocations until one succeed. */
for (; npackets > 1; npackets /= 2) {
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
+ for_each_uvc_urb(uvc_urb, stream) {
+ /*
+* Track how many URBs we allocate, adding one to the
+* index to account for our zero index.
+*/
+ i = uvc_urb_index(uvc_urb) + 1;
stream->urb_size = psize * npackets;
#ifndef CONFIG_DMA_NONCOHERENT
@@ -1700,7 +1704,8 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
struct usb_host_endpoint *ep, gfp_t gfp_flags)
{
struct urb *urb;
- unsigned int npackets, i, j;
+ struct uvc_urb *uvc_urb;
+ unsigned int npackets, j;
u16 psize;
u32 size;
@@ -1713,9 +1718,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
size = npackets * psize;
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
-
+ for_each_uvc_urb(uvc_urb, stream) {
urb = usb_alloc_urb(npackets, gfp_flags);
if (urb == NULL) {
uvc_video_stop(stream, 1);
@@ -1757,7 +1760,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
struct usb_host_endpoint *ep, gfp_t gfp_flags)
{
struct urb *urb;
- unsigned int npackets, pipe, i;
+ struct uvc_urb *uvc_urb;
+ unsigned int npackets, pipe;
u16 psize;
u32 size;
@@ -1781,9 +1785,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
if (stream->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
size = 0;
- for (i = 0; i < UVC_URBS; ++i) {
- struct uvc_urb *uvc_urb = >uvc_urb[i];
-
+ for_each_uvc_urb(uvc_urb, stream) {
urb = usb_alloc_urb(0, gfp_flags);
if (urb == NULL) {
uvc_video_stop(stream, 1);
@@ -1810,6 +1812,7 @@ stati
[PATCH v5 8/9] media: uvcvideo: Rename uvc_{un,}init_video()
From: Kieran Bingham
We have both uvc_init_video() and uvc_video_init() calls which can be
quite confusing to determine the process for each. Now that video
uvc_video_enable() has been renamed to uvc_video_start_streaming(),
adapt these calls to suit the new flow.
Rename uvc_init_video() to uvc_video_start() and uvc_uninit_video() to
uvc_video_stop().
Signed-off-by: Kieran Bingham
---
drivers/media/usb/uvc/uvc_video.c | 18 +-
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 0d35e933856a..020022e6ade4 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1641,7 +1641,7 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
/*
* Uninitialize isochronous/bulk URBs and free transfer buffers.
*/
-static void uvc_uninit_video(struct uvc_streaming *stream, int free_buffers)
+static void uvc_video_stop(struct uvc_streaming *stream, int free_buffers)
{
struct uvc_urb *uvc_urb;
@@ -1718,7 +1718,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
urb = usb_alloc_urb(npackets, gfp_flags);
if (urb == NULL) {
- uvc_uninit_video(stream, 1);
+ uvc_video_stop(stream, 1);
return -ENOMEM;
}
@@ -1786,7 +1786,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
urb = usb_alloc_urb(0, gfp_flags);
if (urb == NULL) {
- uvc_uninit_video(stream, 1);
+ uvc_video_stop(stream, 1);
return -ENOMEM;
}
@@ -1806,7 +1806,7 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
/*
* Initialize isochronous/bulk URBs and allocate transfer buffers.
*/
-static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
+static int uvc_video_start(struct uvc_streaming *stream, gfp_t gfp_flags)
{
struct usb_interface *intf = stream->intf;
struct usb_host_endpoint *ep;
@@ -1894,7 +1894,7 @@ static int uvc_init_video(struct uvc_streaming *stream,
gfp_t gfp_flags)
if (ret < 0) {
uvc_printk(KERN_ERR, "Failed to submit URB %u "
"(%d).\n", i, ret);
- uvc_uninit_video(stream, 1);
+ uvc_video_stop(stream, 1);
return ret;
}
}
@@ -1925,7 +1925,7 @@ int uvc_video_suspend(struct uvc_streaming *stream)
return 0;
stream->frozen = 1;
- uvc_uninit_video(stream, 0);
+ uvc_video_stop(stream, 0);
usb_set_interface(stream->dev->udev, stream->intfnum, 0);
return 0;
}
@@ -1961,7 +1961,7 @@ int uvc_video_resume(struct uvc_streaming *stream, int
reset)
if (ret < 0)
return ret;
- return uvc_init_video(stream, GFP_NOIO);
+ return uvc_video_start(stream, GFP_NOIO);
}
/*
@@ -2095,7 +2095,7 @@ int uvc_video_start_streaming(struct uvc_streaming
*stream)
if (ret < 0)
goto error_commit;
- ret = uvc_init_video(stream, GFP_KERNEL);
+ ret = uvc_video_start(stream, GFP_KERNEL);
if (ret < 0)
goto error_video;
@@ -2111,7 +2111,7 @@ int uvc_video_start_streaming(struct uvc_streaming
*stream)
int uvc_video_stop_streaming(struct uvc_streaming *stream)
{
- uvc_uninit_video(stream, 1);
+ uvc_video_stop(stream, 1);
if (stream->intf->num_altsetting > 1) {
usb_set_interface(stream->dev->udev,
stream->intfnum, 0);
--
git-series 0.9.1
[PATCH v5 0/9] Asynchronous UVC
From: Kieran Bingham
The Linux UVC driver has long provided adequate performance capabilities for
web-cams and low data rate video devices in Linux while resolutions were low.
Modern USB cameras are now capable of high data rates thanks to USB3 with
1080p, and even 4k capture resolutions supported.
Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO
(isochronous transfers) can generate more data than an embedded ARM core is
able to process on a single core, resulting in frame loss.
A large part of this performance impact is from the requirement to
‘memcpy’ frames out from URB packets to destination frames. This unfortunate
requirement is due to the UVC protocol allowing a variable length header, and
thus it is not possible to provide the target frame buffers directly.
Extra throughput is possible by moving the actual memcpy actions to a work
queue, and moving the memcpy out of interrupt context thus allowing work tasks
to be scheduled across multiple cores.
This series has been tested on both the ZED and BRIO cameras on arm64
platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix USB2
Camera, and a built in Toshiba Laptop camera, and with thanks to Philipp Zabel
for testing on a Lite-On internal Laptop Webcam, Logitech C910 (USB2 isoc),
Oculus Sensor (USB3 isoc), and Microsoft HoloLens Sensors (USB3 bulk).
As far as I am aware iSight devices, and devices which use UVC to encode data
(output device) have not yet been tested - but should find no ill effect (at
least not until they are tested of course :D )
Tested-by: Randy Dunlap
Tested-by: Philipp Zabel
v2:
- Fix race reported by Guennadi
v3:
- Fix similar race reported by Laurent
- Only queue work if required (encode/isight do not queue work)
- Refactor/Rename variables for clarity
v4:
- (Yet another) Rework of the uninitialise path.
This time to hopefully clean up the shutdown races for good.
use usb_poison_urb() to halt all URBs, then flush the work queue
before freeing.
- Rebase to latest linux-media/master
v5:
- Provide lockdep validation
- rename uvc_queue_requeue -> uvc_queue_buffer_requeue()
- Fix comments and periods throughout
- Rebase to media/v4.20-2
- Use GFP_KERNEL allocation in uvc_video_copy_data_work()
- Fix function documentation for uvc_video_copy_data_work()
- Add periods to the end of sentences
- Rename 'decode' variable to 'op' in uvc_video_decode_data()
- Move uvc_urb->async_operations initialisation to before use
- Move async workqueue to match uvc_streaming lifetime instead of
streamon/streamoff
- bracket the for_each_uvc_urb() macro
- New patches added to series:
media: uvcvideo: Split uvc_video_enable into two
media: uvcvideo: Rename uvc_{un,}init_video()
media: uvcvideo: Utilise for_each_uvc_urb iterator
Kieran Bingham (9):
media: uvcvideo: Refactor URB descriptors
media: uvcvideo: Convert decode functions to use new context structure
media: uvcvideo: Protect queue internals with helper
media: uvcvideo: queue: Simplify spin-lock usage
media: uvcvideo: queue: Support asynchronous buffer handling
media: uvcvideo: Move decode processing to process context
media: uvcvideo: Split uvc_video_enable into two
media: uvcvideo: Rename uvc_{un,}init_video()
media: uvcvideo: Utilise for_each_uvc_urb iterator
drivers/media/usb/uvc/uvc_driver.c | 2 +-
drivers/media/usb/uvc/uvc_isight.c | 6 +-
drivers/media/usb/uvc/uvc_queue.c | 110 +---
drivers/media/usb/uvc/uvc_video.c | 282 +++---
drivers/media/usb/uvc/uvcvideo.h | 65 ++-
5 files changed, 331 insertions(+), 134 deletions(-)
base-commit: dafb7f9aef2fd44991ff1691721ff765a23be27b
--
git-series 0.9.1
[PATCH v5 2/9] media: uvcvideo: Convert decode functions to use new context structure
From: Kieran Bingham
The URB completion handlers currently reference the stream context.
Now that each URB has its own context structure, convert the decode (and
one encode) functions to utilise this context for URB management.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v2:
- fix checkpatch warning (pre-existing in code)
v3: (none)
v4:
- Rebase on top of linux-media/master (v4.16-rc4, metadata additions)
drivers/media/usb/uvc/uvc_isight.c | 6 --
drivers/media/usb/uvc/uvc_video.c | 26 ++
drivers/media/usb/uvc/uvcvideo.h | 8 +---
3 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_isight.c
b/drivers/media/usb/uvc/uvc_isight.c
index 81e6f2187bfb..39a4e4482b23 100644
--- a/drivers/media/usb/uvc/uvc_isight.c
+++ b/drivers/media/usb/uvc/uvc_isight.c
@@ -99,9 +99,11 @@ static int isight_decode(struct uvc_video_queue *queue,
struct uvc_buffer *buf,
return 0;
}
-void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
- struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
+void uvc_video_decode_isight(struct uvc_urb *uvc_urb, struct uvc_buffer *buf,
+ struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
int ret, i;
for (i = 0; i < urb->number_of_packets; ++i) {
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 113881bed2a4..6d4384695964 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1291,9 +1291,11 @@ static void uvc_video_next_buffers(struct uvc_streaming
*stream,
*video_buf = uvc_queue_next_buffer(>queue, *video_buf);
}
-static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming
*stream,
+static void uvc_video_decode_isoc(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
u8 *mem;
int ret, i;
@@ -1334,9 +1336,11 @@ static void uvc_video_decode_isoc(struct urb *urb,
struct uvc_streaming *stream,
}
}
-static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming
*stream,
+static void uvc_video_decode_bulk(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
u8 *mem;
int len, ret;
@@ -1402,9 +1406,12 @@ static void uvc_video_decode_bulk(struct urb *urb,
struct uvc_streaming *stream,
}
}
-static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming
*stream,
+static void uvc_video_encode_bulk(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
{
+ struct urb *urb = uvc_urb->urb;
+ struct uvc_streaming *stream = uvc_urb->stream;
+
u8 *mem = urb->transfer_buffer;
int len = stream->urb_size, ret;
@@ -1447,7 +1454,8 @@ static void uvc_video_encode_bulk(struct urb *urb, struct
uvc_streaming *stream,
static void uvc_video_complete(struct urb *urb)
{
- struct uvc_streaming *stream = urb->context;
+ struct uvc_urb *uvc_urb = urb->context;
+ struct uvc_streaming *stream = uvc_urb->stream;
struct uvc_video_queue *queue = >queue;
struct uvc_video_queue *qmeta = >meta.queue;
struct vb2_queue *vb2_qmeta = stream->meta.vdev.queue;
@@ -1490,7 +1498,7 @@ static void uvc_video_complete(struct urb *urb)
spin_unlock_irqrestore(>irqlock, flags);
}
- stream->decode(urb, stream, buf, buf_meta);
+ stream->decode(uvc_urb, buf, buf_meta);
if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) {
uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
@@ -1568,6 +1576,8 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
uvc_free_urb_buffers(stream);
break;
}
+
+ uvc_urb->stream = stream;
}
if (i == UVC_URBS) {
@@ -1666,7 +1676,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
}
urb->dev = stream->dev->udev;
- urb->context = stream;
+ urb->context = uvc_urb;
urb->pipe = usb_rcvisocpipe(stream->dev->udev,
ep->desc.bEndpointAddress);
#ifndef CONFIG_DMA_NONCOHERENT
@@ -1733,8 +1743,8 @@ static int uvc_init_video_bulk(struct uvc_streaming
*stream,
return -ENOMEM;
}
-
[PATCH v5 1/9] media: uvcvideo: Refactor URB descriptors
From: Kieran Bingham
We currently store three separate arrays for each URB reference we hold.
Objectify the data needed to track URBs into a single uvc_urb structure,
allowing better object management and tracking of the URB.
All accesses to the data pointers through stream, are converted to use a
uvc_urb pointer for consistency.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v2:
- Re-describe URB context structure
- Re-name uvc_urb->{urb_buffer,urb_dma}{buffer,dma}
v3:
- No change
v4:
- Rebase on top of linux-media/master (v4.16-rc4, metadata additions)
drivers/media/usb/uvc/uvc_video.c | 49 +++-
drivers/media/usb/uvc/uvcvideo.h | 18 ++--
2 files changed, 45 insertions(+), 22 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 86a99f461fd8..113881bed2a4 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1506,14 +1506,16 @@ static void uvc_free_urb_buffers(struct uvc_streaming
*stream)
unsigned int i;
for (i = 0; i < UVC_URBS; ++i) {
- if (stream->urb_buffer[i]) {
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
+ if (uvc_urb->buffer) {
#ifndef CONFIG_DMA_NONCOHERENT
usb_free_coherent(stream->dev->udev, stream->urb_size,
- stream->urb_buffer[i], stream->urb_dma[i]);
+ uvc_urb->buffer, uvc_urb->dma);
#else
- kfree(stream->urb_buffer[i]);
+ kfree(uvc_urb->buffer);
#endif
- stream->urb_buffer[i] = NULL;
+ uvc_urb->buffer = NULL;
}
}
@@ -1551,16 +1553,18 @@ static int uvc_alloc_urb_buffers(struct uvc_streaming
*stream,
/* Retry allocations until one succeed. */
for (; npackets > 1; npackets /= 2) {
for (i = 0; i < UVC_URBS; ++i) {
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
stream->urb_size = psize * npackets;
#ifndef CONFIG_DMA_NONCOHERENT
- stream->urb_buffer[i] = usb_alloc_coherent(
+ uvc_urb->buffer = usb_alloc_coherent(
stream->dev->udev, stream->urb_size,
- gfp_flags | __GFP_NOWARN, >urb_dma[i]);
+ gfp_flags | __GFP_NOWARN, _urb->dma);
#else
- stream->urb_buffer[i] =
+ uvc_urb->buffer =
kmalloc(stream->urb_size, gfp_flags | __GFP_NOWARN);
#endif
- if (!stream->urb_buffer[i]) {
+ if (!uvc_urb->buffer) {
uvc_free_urb_buffers(stream);
break;
}
@@ -1590,13 +1594,15 @@ static void uvc_uninit_video(struct uvc_streaming
*stream, int free_buffers)
uvc_video_stats_stop(stream);
for (i = 0; i < UVC_URBS; ++i) {
- urb = stream->urb[i];
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
+ urb = uvc_urb->urb;
if (urb == NULL)
continue;
usb_kill_urb(urb);
usb_free_urb(urb);
- stream->urb[i] = NULL;
+ uvc_urb->urb = NULL;
}
if (free_buffers)
@@ -1651,6 +1657,8 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
size = npackets * psize;
for (i = 0; i < UVC_URBS; ++i) {
+ struct uvc_urb *uvc_urb = >uvc_urb[i];
+
urb = usb_alloc_urb(npackets, gfp_flags);
if (urb == NULL) {
uvc_uninit_video(stream, 1);
@@ -1663,12 +1671,12 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
ep->desc.bEndpointAddress);
#ifndef CONFIG_DMA_NONCOHERENT
urb->transfer_flags = URB_ISO_ASAP | URB_NO_TRANSFER_DMA_MAP;
- urb->transfer_dma = stream->urb_dma[i];
+ urb->transfer_dma = uvc_urb->dma;
#else
urb->transfer_flags = URB_ISO_ASAP;
#endif
urb->interval = ep->desc.bInterval;
- urb->transfer_buffer = stream->urb_buffer[i];
+ urb->transfer_buffer = uvc_urb->buffer;
urb->complete = uvc_video_complete;
urb->number_of_packets = npackets;
urb->transfer_buffer_length = size;
@@ -1678,7 +1686,7 @@ static int uvc_init_video_isoc(struct uvc_streaming
*stream,
urb->iso_frame_desc[j].length = psize;
}
- stream->urb[i] = urb;
+
[PATCH v5 4/9] media: uvcvideo: queue: Simplify spin-lock usage
From: Kieran Bingham
Both uvc_start_streaming(), and uvc_stop_streaming() are called from
userspace context, with interrupts enabled. As such, they do not need to
save the IRQ state, and can use spin_lock_irq() and spin_unlock_irq()
respectively.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v4:
- Rebase to v4.16 (linux-media/master)
v5:
- Provide lockdep validation
drivers/media/usb/uvc/uvc_queue.c | 14 --
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index 74f9483911d0..bebf2415d9de 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -169,18 +169,19 @@ static int uvc_start_streaming(struct vb2_queue *vq,
unsigned int count)
{
struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
struct uvc_streaming *stream = uvc_queue_to_stream(queue);
- unsigned long flags;
int ret;
+ lockdep_assert_irqs_enabled();
+
queue->buf_used = 0;
ret = uvc_video_enable(stream, 1);
if (ret == 0)
return 0;
- spin_lock_irqsave(>irqlock, flags);
+ spin_lock_irq(>irqlock);
uvc_queue_return_buffers(queue, UVC_BUF_STATE_QUEUED);
- spin_unlock_irqrestore(>irqlock, flags);
+ spin_unlock_irq(>irqlock);
return ret;
}
@@ -188,14 +189,15 @@ static int uvc_start_streaming(struct vb2_queue *vq,
unsigned int count)
static void uvc_stop_streaming(struct vb2_queue *vq)
{
struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
- unsigned long flags;
+
+ lockdep_assert_irqs_enabled();
if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
uvc_video_enable(uvc_queue_to_stream(queue), 0);
- spin_lock_irqsave(>irqlock, flags);
+ spin_lock_irq(>irqlock);
uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
- spin_unlock_irqrestore(>irqlock, flags);
+ spin_unlock_irq(>irqlock);
}
static const struct vb2_ops uvc_queue_qops = {
--
git-series 0.9.1
[PATCH v5 3/9] media: uvcvideo: Protect queue internals with helper
From: Kieran Bingham
The URB completion operation obtains the current buffer by reading
directly into the queue internal interface.
Protect this queue abstraction by providing a helper
uvc_queue_get_current_buffer() which can be used by both the decode
task, and the uvc_queue_next_buffer() functions.
Signed-off-by: Kieran Bingham
Reviewed-by: Laurent Pinchart
---
v2:
- Fix coding style of conditional statements
v3:
- No change
v4:
- Rebase on top of linux-media/master (v4.16-rc4, metadata additions)
drivers/media/usb/uvc/uvc_queue.c | 33 +++-
drivers/media/usb/uvc/uvc_video.c | 6 +-
drivers/media/usb/uvc/uvcvideo.h | 1 +-
3 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index 8964e16f2b22..74f9483911d0 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -430,6 +430,33 @@ void uvc_queue_cancel(struct uvc_video_queue *queue, int
disconnect)
spin_unlock_irqrestore(>irqlock, flags);
}
+/*
+ * uvc_queue_get_current_buffer: Obtain the current working output buffer
+ *
+ * Buffers may span multiple packets, and even URBs, therefore the active
buffer
+ * remains on the queue until the EOF marker.
+ */
+static struct uvc_buffer *
+__uvc_queue_get_current_buffer(struct uvc_video_queue *queue)
+{
+ if (list_empty(>irqqueue))
+ return NULL;
+
+ return list_first_entry(>irqqueue, struct uvc_buffer, queue);
+}
+
+struct uvc_buffer *uvc_queue_get_current_buffer(struct uvc_video_queue *queue)
+{
+ struct uvc_buffer *nextbuf;
+ unsigned long flags;
+
+ spin_lock_irqsave(>irqlock, flags);
+ nextbuf = __uvc_queue_get_current_buffer(queue);
+ spin_unlock_irqrestore(>irqlock, flags);
+
+ return nextbuf;
+}
+
struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
struct uvc_buffer *buf)
{
@@ -446,11 +473,7 @@ struct uvc_buffer *uvc_queue_next_buffer(struct
uvc_video_queue *queue,
spin_lock_irqsave(>irqlock, flags);
list_del(>queue);
- if (!list_empty(>irqqueue))
- nextbuf = list_first_entry(>irqqueue, struct uvc_buffer,
- queue);
- else
- nextbuf = NULL;
+ nextbuf = __uvc_queue_get_current_buffer(queue);
spin_unlock_irqrestore(>irqlock, flags);
buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 6d4384695964..7a7779e1b466 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1484,11 +1484,7 @@ static void uvc_video_complete(struct urb *urb)
return;
}
- spin_lock_irqsave(>irqlock, flags);
- if (!list_empty(>irqqueue))
- buf = list_first_entry(>irqqueue, struct uvc_buffer,
- queue);
- spin_unlock_irqrestore(>irqlock, flags);
+ buf = uvc_queue_get_current_buffer(queue);
if (vb2_qmeta) {
spin_lock_irqsave(>irqlock, flags);
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index f7f8db6fc91a..bdb6d8daedab 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -725,6 +725,7 @@ int uvc_queue_streamoff(struct uvc_video_queue *queue, enum
v4l2_buf_type type);
void uvc_queue_cancel(struct uvc_video_queue *queue, int disconnect);
struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
struct uvc_buffer *buf);
+struct uvc_buffer *uvc_queue_get_current_buffer(struct uvc_video_queue *queue);
int uvc_queue_mmap(struct uvc_video_queue *queue,
struct vm_area_struct *vma);
__poll_t uvc_queue_poll(struct uvc_video_queue *queue, struct file *file,
--
git-series 0.9.1
[PATCH v5 6/9] media: uvcvideo: Move decode processing to process context
From: Kieran Bingham
Newer high definition cameras, and cameras with multiple lenses such as
the range of stereo-vision cameras now available have ever increasing
data rates.
The inclusion of a variable length packet header in URB packets mean
that we must memcpy the frame data out to our destination 'manually'.
This can result in data rates of up to 2 gigabits per second being
processed.
To improve efficiency, and maximise throughput, handle the URB decode
processing through a work queue to move it from interrupt context, and
allow multiple processors to work on URBs in parallel.
Signed-off-by: Kieran Bingham
---
v2:
- Lock full critical section of usb_submit_urb()
v3:
- Fix race on submitting uvc_video_decode_data_work() to work queue.
- Rename uvc_decode_op -> uvc_copy_op (Generic to encode/decode)
- Rename decodes -> copy_operations
- Don't queue work if there is no async task
- obtain copy op structure directly in uvc_video_decode_data()
- uvc_video_decode_data_work() -> uvc_video_copy_data_work()
v4:
- Provide for_each_uvc_urb()
- Simplify fix for shutdown race to flush queue before freeing URBs
- Rebase to v4.16-rc4 (linux-media/master) adjusting for metadata
conflicts.
v5:
- Rebase to media/v4.20-2
- Use GFP_KERNEL allocation in uvc_video_copy_data_work()
- Fix function documentation for uvc_video_copy_data_work()
- Add periods to the end of sentences
- Rename 'decode' variable to 'op' in uvc_video_decode_data()
- Move uvc_urb->async_operations initialisation to before use
- Move async workqueue to match uvc_streaming lifetime instead of
streamon/streamoff
drivers/media/usb/uvc/uvc_driver.c | 2 +-
drivers/media/usb/uvc/uvc_video.c | 110 +++---
drivers/media/usb/uvc/uvcvideo.h | 28 -
3 files changed, 116 insertions(+), 24 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_driver.c
b/drivers/media/usb/uvc/uvc_driver.c
index bc369a0934a3..e61a6d26e812 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -1883,6 +1883,8 @@ static void uvc_unregister_video(struct uvc_device *dev)
video_unregister_device(>vdev);
video_unregister_device(>meta.vdev);
+ destroy_workqueue(stream->async_wq);
+
uvc_debugfs_cleanup_stream(stream);
}
}
diff --git a/drivers/media/usb/uvc/uvc_video.c
b/drivers/media/usb/uvc/uvc_video.c
index 7a7779e1b466..ce9e40444507 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1094,21 +1094,54 @@ static int uvc_video_decode_start(struct uvc_streaming
*stream,
return data[0];
}
-static void uvc_video_decode_data(struct uvc_streaming *stream,
+/*
+ * uvc_video_decode_data_work: Asynchronous memcpy processing
+ *
+ * Copy URB data to video buffers in process context, releasing buffer
+ * references and requeuing the URB when done.
+ */
+static void uvc_video_copy_data_work(struct work_struct *work)
+{
+ struct uvc_urb *uvc_urb = container_of(work, struct uvc_urb, work);
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < uvc_urb->async_operations; i++) {
+ struct uvc_copy_op *op = _urb->copy_operations[i];
+
+ memcpy(op->dst, op->src, op->len);
+
+ /* Release reference taken on this buffer. */
+ uvc_queue_buffer_release(op->buf);
+ }
+
+ ret = usb_submit_urb(uvc_urb->urb, GFP_KERNEL);
+ if (ret < 0)
+ uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
+ ret);
+}
+
+static void uvc_video_decode_data(struct uvc_urb *uvc_urb,
struct uvc_buffer *buf, const u8 *data, int len)
{
- unsigned int maxlen, nbytes;
- void *mem;
+ unsigned int active_op = uvc_urb->async_operations;
+ struct uvc_copy_op *op = _urb->copy_operations[active_op];
+ unsigned int maxlen;
if (len <= 0)
return;
- /* Copy the video data to the buffer. */
maxlen = buf->length - buf->bytesused;
- mem = buf->mem + buf->bytesused;
- nbytes = min((unsigned int)len, maxlen);
- memcpy(mem, data, nbytes);
- buf->bytesused += nbytes;
+
+ /* Take a buffer reference for async work. */
+ kref_get(>ref);
+
+ op->buf = buf;
+ op->src = data;
+ op->dst = buf->mem + buf->bytesused;
+ op->len = min_t(unsigned int, len, maxlen);
+
+ buf->bytesused += op->len;
/* Complete the current frame if the buffer size was exceeded. */
if (len > maxlen) {
@@ -1116,6 +1149,8 @@ static void uvc_video_decode_data(struct uvc_streaming
*stream,
buf->error = 1;
buf->state = UVC_BUF_STATE_READY;
}
+
+ uvc_urb->async_operations++;
}
static void
[PATCH v5 5/9] media: uvcvideo: queue: Support asynchronous buffer handling
From: Kieran Bingham
The buffer queue interface currently operates sequentially, processing
buffers after they have fully completed.
In preparation for supporting parallel tasks operating on the buffers,
we will need to support buffers being processed on multiple CPUs.
Adapt the uvc_queue_next_buffer() such that a reference count tracks the
active use of the buffer, returning the buffer to the VB2 stack at
completion.
Signed-off-by: Kieran Bingham
v5:
- uvc_queue_requeue() -> uvc_queue_buffer_requeue()
- Fix comment
---
drivers/media/usb/uvc/uvc_queue.c | 61 ++--
drivers/media/usb/uvc/uvcvideo.h | 4 ++-
2 files changed, 54 insertions(+), 11 deletions(-)
diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
index bebf2415d9de..cd8c03341de0 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -142,6 +142,7 @@ static void uvc_buffer_queue(struct vb2_buffer *vb)
spin_lock_irqsave(>irqlock, flags);
if (likely(!(queue->flags & UVC_QUEUE_DISCONNECTED))) {
+ kref_init(>ref);
list_add_tail(>queue, >irqqueue);
} else {
/* If the device is disconnected return the buffer to userspace
@@ -459,28 +460,66 @@ struct uvc_buffer *uvc_queue_get_current_buffer(struct
uvc_video_queue *queue)
return nextbuf;
}
-struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
+/*
+ * uvc_queue_buffer_requeue: Requeue a buffer on our internal irqqueue
+ *
+ * Reuse a buffer through our internal queue without the need to 'prepare'.
+ * The buffer will be returned to userspace through the uvc_buffer_queue call
if
+ * the device has been disconnected.
+ */
+static void uvc_queue_buffer_requeue(struct uvc_video_queue *queue,
struct uvc_buffer *buf)
{
- struct uvc_buffer *nextbuf;
- unsigned long flags;
+ buf->error = 0;
+ buf->state = UVC_BUF_STATE_QUEUED;
+ buf->bytesused = 0;
+ vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
+
+ uvc_buffer_queue(>buf.vb2_buf);
+}
+
+static void uvc_queue_buffer_complete(struct kref *ref)
+{
+ struct uvc_buffer *buf = container_of(ref, struct uvc_buffer, ref);
+ struct vb2_buffer *vb = >buf.vb2_buf;
+ struct uvc_video_queue *queue = vb2_get_drv_priv(vb->vb2_queue);
if ((queue->flags & UVC_QUEUE_DROP_CORRUPTED) && buf->error) {
- buf->error = 0;
- buf->state = UVC_BUF_STATE_QUEUED;
- buf->bytesused = 0;
- vb2_set_plane_payload(>buf.vb2_buf, 0, 0);
- return buf;
+ uvc_queue_buffer_requeue(queue, buf);
+ return;
}
+ buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
+ vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
+ vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+/*
+ * Release a reference on the buffer. Complete the buffer when the last
+ * reference is released.
+ */
+void uvc_queue_buffer_release(struct uvc_buffer *buf)
+{
+ kref_put(>ref, uvc_queue_buffer_complete);
+}
+
+/*
+ * Remove this buffer from the queue. Lifetime will persist while async actions
+ * are still running (if any), and uvc_queue_buffer_release will give the
buffer
+ * back to VB2 when all users have completed.
+ */
+struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
+ struct uvc_buffer *buf)
+{
+ struct uvc_buffer *nextbuf;
+ unsigned long flags;
+
spin_lock_irqsave(>irqlock, flags);
list_del(>queue);
nextbuf = __uvc_queue_get_current_buffer(queue);
spin_unlock_irqrestore(>irqlock, flags);
- buf->state = buf->error ? UVC_BUF_STATE_ERROR : UVC_BUF_STATE_DONE;
- vb2_set_plane_payload(>buf.vb2_buf, 0, buf->bytesused);
- vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_DONE);
+ uvc_queue_buffer_release(buf);
return nextbuf;
}
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index bdb6d8daedab..1bc17da7f3d4 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -410,6 +410,9 @@ struct uvc_buffer {
unsigned int bytesused;
u32 pts;
+
+ /* Asynchronous buffer handling. */
+ struct kref ref;
};
#define UVC_QUEUE_DISCONNECTED (1 << 0)
@@ -726,6 +729,7 @@ void uvc_queue_cancel(struct uvc_video_queue *queue, int
disconnect);
struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
struct uvc_buffer *buf);
struct uvc_buffer *uvc_queue_get_current_buffer(struct uvc_video_queue *queue);
+void uvc_queue_buffer_release(struct uvc_buffer *buf);
int uvc_queue_mmap(struct uvc_video_queue *queue,
Re: uvcvideo: IR camera lights only every second frame
Hi Jiri, On 30/10/2018 14:36, Jiri Slaby wrote: > Hi, > > I have a Dell Lattitude 7280 with two webcams. The standard one works > fine (/dev/video0). The other one is an IR camera (/dev/video1). The > camera proper works fine and produces 340x374 frames. But there is an IR > led supposed to light the object. The video is 30fps, but the LED seems > to emit light only on half of the frames, i.e. on every second frame (15 > fps). This makes the video blink a lot. The two consecutive frames look > like: > https://www.fi.muni.cz/~xslaby/sklad/mpv-shot0002.jpg > https://www.fi.muni.cz/~xslaby/sklad/mpv-shot0003.jpg > > Do you have any ideas what to check/test? I have an HP Spectre with IR camera, and it also 'flashes' alternate frames. I assumed this was something to do with controlling the lighting for face recognition some how. I'm fairly sure we don't control the 'IR flash' from the UVC. I wonder if there is a control parameter for the IR led in the extension-units? -- Regards Kieran > $ v4l2-ctl --all -d /dev/video2 > Driver Info (not using libv4l2): > Driver name : uvcvideo > Card type : Integrated_Webcam_HD: Integrate > Bus info : usb-:00:14.0-5 > Driver version: 4.18.15 > Capabilities : 0x84A1 > Video Capture > Metadata Capture > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x0421 > Video Capture > Streaming > Extended Pix Format > Priority: 2 > Video input : 0 (Camera 11: ok) > Format Video Capture: > Width/Height : 340/374 > Pixel Format : 'YUYV' > Field : None > Bytes per Line: 680 > Size Image: 254320 > Colorspace: sRGB > Transfer Function : Default (maps to sRGB) > YCbCr/HSV Encoding: Default (maps to ITU-R 601) > Quantization : Default (maps to Limited Range) > Flags : > Crop Capability Video Capture: > Bounds : Left 0, Top 0, Width 340, Height 374 > Default : Left 0, Top 0, Width 340, Height 374 > Pixel Aspect: 1/1 > Selection: crop_default, Left 0, Top 0, Width 340, Height 374 > Selection: crop_bounds, Left 0, Top 0, Width 340, Height 374 > Streaming Parameters Video Capture: > Capabilities : timeperframe > Frames per second: 30.000 (30/1) > Read buffers : 0 > > > > > > $ lsusb -vs 1:3 > Bus 001 Device 003: ID 0bda:5691 Realtek Semiconductor Corp. > Device Descriptor: > bLength18 > bDescriptorType 1 > bcdUSB 2.00 > bDeviceClass 239 Miscellaneous Device > bDeviceSubClass 2 > bDeviceProtocol 1 Interface Association > bMaxPacketSize064 > idVendor 0x0bda Realtek Semiconductor Corp. > idProduct 0x5691 > bcdDevice 60.12 > iManufacturer 3 CNFGE16N5214300025C2 > iProduct1 Integrated_Webcam_HD > iSerial 2 0001 > bNumConfigurations 1 > Configuration Descriptor: > bLength 9 > bDescriptorType 2 > wTotalLength 1041 > bNumInterfaces 4 > bConfigurationValue 1 > iConfiguration 4 USB Camera > bmAttributes 0x80 > (Bus Powered) > MaxPower 500mA > ** UNRECOGNIZED: 28 ff 42 49 53 54 00 01 06 06 10 00 00 00 00 00 01 > 07 f4 01 02 08 f4 01 03 09 f4 01 04 0a f4 01 05 0b f4 01 06 0c e8 03 > Interface Association: > bLength 8 > bDescriptorType11 > bFirstInterface 0 > bInterfaceCount 2 > bFunctionClass 14 Video > bFunctionSubClass 3 Video Interface Collection > bFunctionProtocol 0 > iFunction 5 Integrated Webcam > Interface Descriptor: > bLength 9 > bDescriptorType 4 > bInterfaceNumber0 > bAlternateSetting 0 > bNumEndpoints 1 > bInterfaceClass14 Video > bInterfaceSubClass 1 Video Control > bInterfaceProtocol 0 > iInterface 5 Integrated Webcam > VideoControl Interface Descriptor: > bLength13 > bDescriptorType36 > bDescriptorSubtype 1 (HEADER) > bcdUVC 1.00 > wTotalLength 107 > dwClockFrequency 15.00MHz > bInCollection 1 > baInterfaceNr( 0) 1 > VideoControl Interface Descriptor: > bLength18 > bDescriptorType36 > bDescriptorSubtype 2 (INPUT_TERMINAL) > bTerminalID 1 > wTerminalType 0x0201 Camera Sensor
Re: [PATCH] media: intel-ipu3: cio2: Remove redundant definitions
Hi Rajmohan
Thank you for the patch,
On 10/10/18 00:42, Rajmohan Mani wrote:
> Removed redundant CIO2_IMAGE_MAX_* definitions
>
> Fixes: c2a6a07afe4a ("media: intel-ipu3: cio2: add new MIPI-CSI2 driver")
>
> Signed-off-by: Rajmohan Mani
Reviewed-by: Kieran Bingham
Looks like this {sh,c}ould be bundled in with
"[PATCH 0/2] Trivial CIO2 patches" from Sakari at integration.
--
Regards
Kieran Bingham
> ---
> drivers/media/pci/intel/ipu3/ipu3-cio2.h | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.h
> b/drivers/media/pci/intel/ipu3/ipu3-cio2.h
> index 240635be7a31..7caab9b8c2b9 100644
> --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.h
> +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.h
> @@ -10,8 +10,6 @@
> #define CIO2_PCI_ID 0x9d32
> #define CIO2_PCI_BAR 0
> #define CIO2_DMA_MASKDMA_BIT_MASK(39)
> -#define CIO2_IMAGE_MAX_WIDTH 4224
> -#define CIO2_IMAGE_MAX_LENGTH3136
>
> #define CIO2_IMAGE_MAX_WIDTH 4224
> #define CIO2_IMAGE_MAX_LENGTH3136
>
Re: [PATCH] vicodec: lower minimum height to 360
Hi Hans, Thank you for the patch, On 10/10/18 08:03, Hans Verkuil wrote: > Lower the minimum height to 360 to be consistent with the webcam input of > vivid. > > The 480 was rather arbitrary but it made it harder to use vivid as a source > for > encoding since the default resolution when you load vivid is 640x360. As this is a virtual codec, is the minimum width and height really so 'large' ? What about 320x240 or such? (or even 32x32...) Or is the aim to provide minimum frame sizes and a means to verify userspace correctly handles the minimum frame sizes too ? I could certainly acknowledge it's worth providing a means for a userspace app to test that it handles minimum sizes correctly. > Signed-off-by: Hans Verkuil If the minimum is desired: Reviewed-by: Kieran Bingham > --- > diff --git a/drivers/media/platform/vicodec/vicodec-core.c > b/drivers/media/platform/vicodec/vicodec-core.c > index 1eb9132bfc85..b292cff26c86 100644 > --- a/drivers/media/platform/vicodec/vicodec-core.c > +++ b/drivers/media/platform/vicodec/vicodec-core.c > @@ -42,7 +42,7 @@ MODULE_PARM_DESC(debug, " activates debug info"); > #define MAX_WIDTH4096U > #define MIN_WIDTH640U > #define MAX_HEIGHT 2160U > -#define MIN_HEIGHT 480U > +#define MIN_HEIGHT 360U > > #define dprintk(dev, fmt, arg...) \ > v4l2_dbg(1, debug, >v4l2_dev, "%s: " fmt, __func__, ## arg) >
Re: [RFC] Informal meeting during ELCE to discuss userspace support for stateless codecs
Hi Hans On 08/10/18 12:53, Hans Verkuil wrote: > Hi all, > > I would like to meet up somewhere during the ELCE to discuss userspace support > for stateless (and perhaps stateful as well?) codecs. > > It is also planned as a topic during the summit, but I would prefer to prepare > for that in advance, esp. since I myself do not have any experience writing > userspace SW for such devices. > > Nicolas, it would be really great if you can participate in this meeting > since you probably have the most experience with this by far. > > Looking through the ELCE program I found two timeslots that are likely to work > for most of us (because the topics in the program appear to be boring for us > media types!): > > Tuesday from 10:50-15:50 > > or: > > Monday from 15:45 onward > > My guess is that we need 2-3 hours or so. Hard to predict. > > The basic question that I would like to have answered is what the userspace > component should look like? libv4l-like plugin or a library that userspace can > link with? Do we want more general support for stateful codecs as well that > deals > with resolution changes and the more complex parts of the codec API? > > I've mailed this directly to those that I expect are most interested in this, > but if someone want to join in let me know. Depending on time and availability I might like to join in on this. Consider me non-essential however and if I'm not around don't wait for me. I have a desire to work on the codecs for Renesas (although it seems unlikely to be possible due to licensing issues) > I want to keep the group small though, so you need to bring relevant > experience > to the table. I can offer outdated experience managing the codec firmwares, and decode/encode stacks for the ST SDK2 player implementation. However that was 4 or 5 years ago - so I now deny all knowledge of knowing anything from back then :-) -- Regards Kieran
Re: [ANN] Draft Agenda for the media summit on Thursday Oct 25th in Edinburgh
Hi Hans, Mauro, I believe I selected attendance when registering for the conference, but please add my name to the list here as well. I have a small topic which might start off in the hallway track before hand that I'd like to kick off too. "Fault tolerant V4L2" ... In other words, how should we handle complex devices which do not 'fully probe'. In our instance, with our GMSL driver implementation - we have an 8 camera system, described as such in DT. If one of the cameras is disconnected, or does not fully probe - then all of the remaining cameras are unavailable, as they are all subdevices from the MAX9286 => CSI2 => VIN chain. Originally we had async subdevice registration at probe time so that they appear as they are ready - but I believe earlier review comments determined that this goes against the V4L2 spec, and if the system state is not correct - then it's just a flat out failure. I'd like to discuss (or start to discuss) how we should handle cases such as a missing *non-essential* subdevice in a safety critical system. For instance, imagine here that two cameras are digital wing mirrors, and one gets knocked off by a lorry ... I'd like to know that all the remaining 'digital eyes' can still function :) I had hoped to send a more clear proposal on this topic with patch examples based on our driver - but I'm afraid my talk didn't get accepted for ELCE and so I haven't had chance (/been forced) to spend time on this topic yet :-) Further discussion on this could be how to get devices to re-probe at runtime too - or retry their probe sequences at least, which might perhaps even be a higher level than linux-media. Regards Kieran On 24/09/18 15:42, Hans Verkuil wrote: > Hi all, > > We are organizing a media mini-summit on Thursday October 25th in > Edinburgh, Edinburgh International Conference Centre. > > If you plan to attend, please let Mauro know. It is open for all, but > we have a limited number of seats. > > Name of the room for the summit: TBD > > Currently known attendees (please add/remove names as needed): > > Sakari Ailus > Mauro Carvalho Chehab > Ezequiel Garcia > Michael Ira Krufky > Laurent Pinchart > Ricardo Ribalda Delgado > Hans Verkuil > Sean Young > > Agenda (First draft!) > = > > General remarks: the given start/end times for the various topics are > approximate since it is always hard to predict how long a discussion will > take. > If people are attending other summits and those conflict with specific media > topics they want to be part of, then let me know and we can rearrange the > schedule to (hopefully) accommodate that. > > 9:00-9:15: Introduction (Hans Verkuil) > > 9:15-9:30: Status of the HDMI CEC kernel support (Hans Verkuil) > Give a quick overview of the status: what has been merged, what is > still pending, what is under development. > > 9:30-9:45: Save/restore controls from MTD (Ricardo Ribalda Delgado) > Industrial/Scientific sensors usually come with very extensive > calibration information such as: per column gain, list of dead > pixels, temperature sensor offset... etc > > We are saving that information on an flash device that is located > by the sensor. > > Show how we are integrating that calibration flash with v4l2-ctrl. > And if this feature is useful for someone else and upstream it. > > 9:45-11:00: Complex Cameras (Mauro Carvalho Chehab) > I expect that we could have something to discuss there about complex > cameras. So, I'd reserve a 50 mins slot for it. > > The idea is to discuss about the undergoing work with complex camera > development is happening. > > As we're working to merge request API, another topic for discussion > is how to add support for requests on it (or on a separate but related > library). > > 11:00-11:15: Break > > 11:15-12:00: Automated Testing (Ezequiel Garcia) > There is a lot of discussion going on around testing, > so it's a good opportunity for us to talk about our > current testing infrastructure. > > We are already doing a good job with v4l2-compliance. > Can we do more? > > Lunch > > 13:30-14:30: Stateless Codec userspace (Hans Verkuil) > Support for stateless codecs and Request API should be merged for > 4.20, and the next step is to discuss how to organize the userspace > support. > > Hopefully by the time the media summit starts we'll have some better > ideas of what we want in this area. > > 14:30-15:15: Which ioctls should be replaced with better versions? (Hans > Verkuil) > Some parts of the V4L2 API are awkward to use and I think it would be > a good idea to look at possible candidates for that. > > Examples are the ioctls that use struct v4l2_buffer: the multiplanar > support is > really horrible, and writing code to support both single and > multiplanar is hard. > We are
Re: [PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
Hi Steve, On 08/06/18 18:43, Steve Longerbeam wrote: > The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. Just spotted this :D s/HMDI/HDMI/ Regards Kieran > > Signed-off-by: Steve Longerbeam > --- > drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c > b/drivers/media/i2c/adv748x/adv748x-csi2.c > index 820b44e..469be87 100644 > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c > @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct > adv748x_csi2 *tx) > adv748x_csi2_set_virtual_channel(tx, 0); > > adv748x_subdev_init(>sd, state, _csi2_ops, > - MEDIA_ENT_F_UNKNOWN, > + MEDIA_ENT_F_VID_IF_BRIDGE, > is_txa(tx) ? "txa" : "txb"); > > /* Ensure that matching is based upon the endpoint fwnodes */ > signature.asc Description: OpenPGP digital signature
Re: [PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
Hi Steve, On 08/06/18 22:34, Steve Longerbeam wrote: > Hi Kieran, > > > On 06/08/2018 02:29 PM, Kieran Bingham wrote: >> Hi Steve, >> >> Thankyou for the patch. >> >> On 08/06/18 18:43, Steve Longerbeam wrote: >>> The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. >>> >> Reading the documentation for MEDIA_ENT_F_VID_IF_BRIDGE, this seems >> reasonable. >> >> Out of interest, have you stumbled across this as part of your other work on >> CSI2 drivers - or have you been looking to test the ADV748x with your CSI2 >> receiver? I'd love to know if the driver works with other (non-renesas) >> platforms! > > This isn't really related to my other work on the i.MX CSI2 receiver driver > in imx-media. I've only tested this on Renesas (Salvator-X). No problem. I was just curious :D And this will get rid of that annoying warning message that I've been ignoring! Regards Kieran > > Steve > >>> Signed-off-by: Steve Longerbeam >> Acked-by: Kieran Bingham >> >> >>> --- >>> drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c >>> b/drivers/media/i2c/adv748x/adv748x-csi2.c >>> index 820b44e..469be87 100644 >>> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c >>> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c >>> @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, >>> struct >>> adv748x_csi2 *tx) >>> adv748x_csi2_set_virtual_channel(tx, 0); >>> adv748x_subdev_init(>sd, state, _csi2_ops, >>> - MEDIA_ENT_F_UNKNOWN, >>> + MEDIA_ENT_F_VID_IF_BRIDGE, >>> is_txa(tx) ? "txa" : "txb"); >>> /* Ensure that matching is based upon the endpoint fwnodes */ >>> > signature.asc Description: OpenPGP digital signature
Re: [PATCH] media: i2c: adv748x: csi2: set entity function to video interface bridge
Hi Steve, Thankyou for the patch. On 08/06/18 18:43, Steve Longerbeam wrote: > The ADV748x CSI-2 subdevices are HMDI/AFE to MIPI CSI-2 bridges. > Reading the documentation for MEDIA_ENT_F_VID_IF_BRIDGE, this seems reasonable. Out of interest, have you stumbled across this as part of your other work on CSI2 drivers - or have you been looking to test the ADV748x with your CSI2 receiver? I'd love to know if the driver works with other (non-renesas) platforms! > Signed-off-by: Steve Longerbeam Acked-by: Kieran Bingham > --- > drivers/media/i2c/adv748x/adv748x-csi2.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c > b/drivers/media/i2c/adv748x/adv748x-csi2.c > index 820b44e..469be87 100644 > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c > @@ -284,7 +284,7 @@ int adv748x_csi2_init(struct adv748x_state *state, struct > adv748x_csi2 *tx) > adv748x_csi2_set_virtual_channel(tx, 0); > > adv748x_subdev_init(>sd, state, _csi2_ops, > - MEDIA_ENT_F_UNKNOWN, > + MEDIA_ENT_F_VID_IF_BRIDGE, > is_txa(tx) ? "txa" : "txb"); > > /* Ensure that matching is based upon the endpoint fwnodes */ > signature.asc Description: OpenPGP digital signature
Re: [ANN] Meeting to discuss improvements to support MC-based cameras on generic apps
Hi Mauro On 31/05/18 14:22, Mauro Carvalho Chehab wrote: > Em Mon, 28 May 2018 10:43:51 -0300 > More details about the meeting: > > Date: June, 19 > Site: Google > Address: 〒106-6126 Tokyo, Minato, Roppongi, 6 Chome−10−1 Roppongi Hills Mori > Tower 44F > > Please confirm who will be attending the meeting. I'll be in Japan, so I will attend the meeting if that's OK. Regards Kieran signature.asc Description: OpenPGP digital signature
[PATCH] media: vsp1: Document vsp1_dl_body refcnt
In commit 2d9445db0ee9 ("media: vsp1: Use reference counting for
bodies"), a new field was introduced to the vsp1_dl_body structure to
account for usage tracking of the body.
Document the newly added field in the kerneldoc.
Signed-off-by: Kieran Bingham <kieran.bing...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index d9b9cdd8fbe2..10a24bde2299 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -43,6 +43,7 @@ struct vsp1_dl_entry {
* struct vsp1_dl_body - Display list body
* @list: entry in the display list list of bodies
* @free: entry in the pool free body list
+ * @refcnt: reference tracking for the body
* @pool: pool to which this body belongs
* @vsp1: the VSP1 device
* @entries: array of entries
--
2.17.0
Re: [GIT PULL FOR v4.18] R-Car VSP1 TLB optimisation
Hi Mauro, Laurent, On 26/05/18 00:39, Laurent Pinchart wrote: > Hi Mauro, > > (CC'ing Kieran) > > On Saturday, 26 May 2018 02:10:27 EEST Mauro Carvalho Chehab wrote: >> Em Sun, 20 May 2018 15:10:50 +0300 Laurent Pinchart escreveu: >>> Hi Mauro, >>> >>> The following changes since commit >>> 8ed8bba70b4355b1ba029b151ade84475dd12991: >>> media: imx274: remove non-indexed pointers from mode_table (2018-05-17 >>> 06:22:08 -0400) >>> >>> are available in the Git repository at: >>> git://linuxtv.org/pinchartl/media.git v4l2/vsp1/next >>> >>> for you to fetch changes up to 429f256501652c90a4ed82f2416618f82a77d37c: >>> media: vsp1: Move video configuration to a cached dlb (2018-05-20 >>> 09:46:51 +0300) >>> >>> The branch passes the VSP and DU test suites, both on its own and when >>> merged with the drm-next branch. >> >> This series added a new warning: >> >> drivers/media/platform/vsp1/vsp1_dl.c:69: warning: Function parameter or >> member 'refcnt' not described in 'vsp1_dl_body' > > We'll fix that. Kieran, as you authored the code, would you like to give it a > go ? Sent! Thanks for the catch. -- Kieran >> To the already existing one: >> >> drivers/media/platform/vsp1/vsp1_drm.c:336 vsp1_du_pipeline_setup_brx() >> error: we previously assumed 'pipe->brx' could be null (see line 244) > > That's still on my todo list. I tried to give it a go but received plenty of > SQL errors. How do you run smatch ? > >> (there's also a Spectre warning too, but I'll looking into those >> in separate). >> >> For now, I'll apply it, but I reserve the right of not pulling any >> new patchsets that would add more warnings. >> >>> >>> >>> Geert Uytterhoeven (1): >>> media: vsp1: Drop OF dependency of VIDEO_RENESAS_VSP1 >>> >>> Kieran Bingham (10): >>> media: vsp1: Release buffers for each video node >>> media: vsp1: Move video suspend resume handling to video object >>> media: vsp1: Reword uses of 'fragment' as 'body' >>> media: vsp1: Protect bodies against overflow >>> media: vsp1: Provide a body pool >>> media: vsp1: Convert display lists to use new body pool >>> media: vsp1: Use reference counting for bodies >>> media: vsp1: Refactor display list configure operations >>> media: vsp1: Adapt entities to configure into a body >>> media: vsp1: Move video configuration to a cached dlb >>> >>> drivers/media/platform/Kconfig| 2 +- >>> drivers/media/platform/vsp1/vsp1_brx.c| 32 ++-- >>> drivers/media/platform/vsp1/vsp1_clu.c| 113 ++- >>> drivers/media/platform/vsp1/vsp1_clu.h| 1 + >>> drivers/media/platform/vsp1/vsp1_dl.c | 388 ++--- >>> drivers/media/platform/vsp1/vsp1_dl.h | 21 ++- >>> drivers/media/platform/vsp1/vsp1_drm.c| 18 +- >>> drivers/media/platform/vsp1/vsp1_drv.c| 4 +- >>> drivers/media/platform/vsp1/vsp1_entity.c | 34 +++- >>> drivers/media/platform/vsp1/vsp1_entity.h | 45 +++-- >>> drivers/media/platform/vsp1/vsp1_hgo.c| 26 ++- >>> drivers/media/platform/vsp1/vsp1_hgt.c| 28 ++- >>> drivers/media/platform/vsp1/vsp1_hsit.c | 20 +- >>> drivers/media/platform/vsp1/vsp1_lif.c| 25 ++- >>> drivers/media/platform/vsp1/vsp1_lut.c| 80 +--- >>> drivers/media/platform/vsp1/vsp1_lut.h| 1 + >>> drivers/media/platform/vsp1/vsp1_pipe.c | 74 +--- >>> drivers/media/platform/vsp1/vsp1_pipe.h | 12 +- >>> drivers/media/platform/vsp1/vsp1_rpf.c| 189 ++- >>> drivers/media/platform/vsp1/vsp1_sru.c| 24 +-- >>> drivers/media/platform/vsp1/vsp1_uds.c| 73 +++ >>> drivers/media/platform/vsp1/vsp1_uds.h| 2 +- >>> drivers/media/platform/vsp1/vsp1_uif.c| 35 ++-- >>> drivers/media/platform/vsp1/vsp1_video.c | 177 - >>> drivers/media/platform/vsp1/vsp1_video.h | 3 + >>> drivers/media/platform/vsp1/vsp1_wpf.c| 326 ++--- >>> 26 files changed, 967 insertions(+), 786 deletions(-) >
Re: [PATCH v11 01/10] media: v4l: vsp1: Release buffers for each video node
On 18/05/18 21:53, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Friday, 18 May 2018 23:41:54 EEST Kieran Bingham wrote:
>> From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
>>
>> Commit 372b2b0399fc ("media: v4l: vsp1: Release buffers in
>> start_streaming error path") introduced a helper to clean up buffers on
>> error paths, but inadvertently changed the code such that only the
>> output WPF buffers were cleaned, rather than the video node being
>> operated on.
>>
>> Since then vsp1_video_cleanup_pipeline() has grown to perform both video
>> node cleanup, as well as pipeline cleanup. Split the implementation into
>> two distinct functions that perform the required work, so that each
>> video node can release it's buffers correctly on streamoff. The pipe
>
> s/it's/its/
>
>> cleanup that was performed in the vsp1_video_stop_streaming() (releasing
>> the pipe->dl) is moved to the function for clarity.
>>
>> Fixes: 372b2b0399fc ("media: v4l: vsp1: Release buffers in start_streaming
>> error path")
>> Cc: sta...@vger.kernel.org # v4.13+
>
> Commit 372b2b0399fc was introduced in v4.14, should this be v4.14+ ?
Yes, thank you - that's me mis-interpreting my own scripts to get the version
for fixes.
>>
>> Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
>
> Reviewed-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com>
>
> No need to resubmit for this, I'll fix the commit message when applying.
Great.
--
Kieran
>
>> ---
>> drivers/media/platform/vsp1/vsp1_video.c | 21 +
>> 1 file changed, 13 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/media/platform/vsp1/vsp1_video.c
>> b/drivers/media/platform/vsp1/vsp1_video.c index c8c12223a267..ba89dd176a13
>> 100644
>> --- a/drivers/media/platform/vsp1/vsp1_video.c
>> +++ b/drivers/media/platform/vsp1/vsp1_video.c
>> @@ -842,9 +842,8 @@ static int vsp1_video_setup_pipeline(struct
>> vsp1_pipeline *pipe) return 0;
>> }
>>
>> -static void vsp1_video_cleanup_pipeline(struct vsp1_pipeline *pipe)
>> +static void vsp1_video_release_buffers(struct vsp1_video *video)
>> {
>> -struct vsp1_video *video = pipe->output->video;
>> struct vsp1_vb2_buffer *buffer;
>> unsigned long flags;
>>
>> @@ -854,12 +853,18 @@ static void vsp1_video_cleanup_pipeline(struct
>> vsp1_pipeline *pipe) vb2_buffer_done(>buf.vb2_buf,
>> VB2_BUF_STATE_ERROR);
>> INIT_LIST_HEAD(>irqqueue);
>> spin_unlock_irqrestore(>irqlock, flags);
>> +}
>> +
>> +static void vsp1_video_cleanup_pipeline(struct vsp1_pipeline *pipe)
>> +{
>> +lockdep_assert_held(>lock);
>>
>> /* Release our partition table allocation */
>> -mutex_lock(>lock);
>> kfree(pipe->part_table);
>> pipe->part_table = NULL;
>> -mutex_unlock(>lock);
>> +
>> +vsp1_dl_list_put(pipe->dl);
>> +pipe->dl = NULL;
>> }
>>
>> static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int
>> count) @@ -874,8 +879,9 @@ static int vsp1_video_start_streaming(struct
>> vb2_queue *vq, unsigned int count) if (pipe->stream_count ==
>> pipe->num_inputs) {
>> ret = vsp1_video_setup_pipeline(pipe);
>> if (ret < 0) {
>> -mutex_unlock(>lock);
>> +vsp1_video_release_buffers(video);
>> vsp1_video_cleanup_pipeline(pipe);
>> +mutex_unlock(>lock);
>> return ret;
>> }
>>
>> @@ -925,13 +931,12 @@ static void vsp1_video_stop_streaming(struct vb2_queue
>> *vq) if (ret == -ETIMEDOUT)
>> dev_err(video->vsp1->dev, "pipeline stop timeout\n");
>>
>> -vsp1_dl_list_put(pipe->dl);
>> -pipe->dl = NULL;
>> +vsp1_video_cleanup_pipeline(pipe);
>> }
>> mutex_unlock(>lock);
>>
>> media_pipeline_stop(>video.entity);
>> -vsp1_video_cleanup_pipeline(pipe);
>> +vsp1_video_release_buffers(video);
>> vsp1_video_pipeline_put(pipe);
>> }
>
[PATCH v11 08/10] media: vsp1: Refactor display list configure operations
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
The entities provide a single .configure operation which configures the
object into the target display list, based on the vsp1_entity_params
selection.
Split the configure function into three parts, '.configure_stream()',
'.configure_frame()', and '.configure_partition()' to facilitate
splitting the configuration of each parameter class into separate
display list bodies.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
[Blank line reformatting, remote unneeded local variable initialization]
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_brx.c| 12 +-
drivers/media/platform/vsp1/vsp1_clu.c| 78 ++
drivers/media/platform/vsp1/vsp1_drm.c| 12 +-
drivers/media/platform/vsp1/vsp1_entity.c | 24 ++-
drivers/media/platform/vsp1/vsp1_entity.h | 39 +--
drivers/media/platform/vsp1/vsp1_hgo.c| 12 +-
drivers/media/platform/vsp1/vsp1_hgt.c| 12 +-
drivers/media/platform/vsp1/vsp1_hsit.c | 12 +-
drivers/media/platform/vsp1/vsp1_lif.c| 12 +-
drivers/media/platform/vsp1/vsp1_lut.c| 47 +---
drivers/media/platform/vsp1/vsp1_rpf.c| 168 ++---
drivers/media/platform/vsp1/vsp1_sru.c| 12 +-
drivers/media/platform/vsp1/vsp1_uds.c| 56 ++--
drivers/media/platform/vsp1/vsp1_uif.c| 16 +-
drivers/media/platform/vsp1/vsp1_video.c | 28 +--
drivers/media/platform/vsp1/vsp1_wpf.c| 302 ---
16 files changed, 422 insertions(+), 420 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_brx.c
b/drivers/media/platform/vsp1/vsp1_brx.c
index 3beec18fd863..011edac5ebc1 100644
--- a/drivers/media/platform/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/vsp1/vsp1_brx.c
@@ -281,19 +281,15 @@ static const struct v4l2_subdev_ops brx_ops = {
* VSP1 Entity Operations
*/
-static void brx_configure(struct vsp1_entity *entity,
- struct vsp1_pipeline *pipe,
- struct vsp1_dl_list *dl,
- enum vsp1_entity_params params)
+static void brx_configure_stream(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl)
{
struct vsp1_brx *brx = to_brx(>subdev);
struct v4l2_mbus_framefmt *format;
unsigned int flags;
unsigned int i;
- if (params != VSP1_ENTITY_PARAMS_INIT)
- return;
-
format = vsp1_entity_get_pad_format(>entity, brx->entity.config,
brx->entity.source_pad);
@@ -400,7 +396,7 @@ static void brx_configure(struct vsp1_entity *entity,
}
static const struct vsp1_entity_operations brx_entity_ops = {
- .configure = brx_configure,
+ .configure_stream = brx_configure_stream,
};
/*
-
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index ea83f1b7d125..34f17a82ac1f 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -169,57 +169,50 @@ static const struct v4l2_subdev_ops clu_ops = {
* VSP1 Entity Operations
*/
-static void clu_configure(struct vsp1_entity *entity,
- struct vsp1_pipeline *pipe,
- struct vsp1_dl_list *dl,
- enum vsp1_entity_params params)
+static void clu_configure_stream(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl)
+{
+ struct vsp1_clu *clu = to_clu(>subdev);
+ struct v4l2_mbus_framefmt *format;
+
+ /*
+* The yuv_mode can't be changed during streaming. Cache it internally
+* for future runtime configuration calls.
+*/
+ format = vsp1_entity_get_pad_format(>entity,
+ clu->entity.config,
+ CLU_PAD_SINK);
+ clu->yuv_mode = format->code == MEDIA_BUS_FMT_AYUV8_1X32;
+}
+
+static void clu_configure_frame(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
{
struct vsp1_clu *clu = to_clu(>subdev);
struct vsp1_dl_body *dlb;
unsigned long flags;
u32 ctrl = VI6_CLU_CTRL_AAI | VI6_CLU_CTRL_MVS | VI6_CLU_CTRL_EN;
- switch (params) {
- case VSP1_ENTITY_PARAMS_INIT: {
- /*
-* The format can't be changed during streaming, only verify it
-* at setup time and store the information internally for
[PATCH v11 02/10] media: vsp1: Move video suspend resume handling to video object
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
The suspend and resume handlers are only utilised by video pipelines,
yet the functions currently reside in the vsp1_pipe object.
This causes an issue with resume, as the functions incorrectly call
vsp1_pipeline_run() directly instead of processing the video object
through vsp1_video_pipeline_run().
Move the functions to the video object, renaming accordingly and update
the resume handler to call vsp1_video_pipeline_run() as appropriate.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_drv.c | 4 +-
drivers/media/platform/vsp1/vsp1_pipe.c | 70 +---
drivers/media/platform/vsp1/vsp1_pipe.h | 3 +-
drivers/media/platform/vsp1/vsp1_video.c | 75 +-
drivers/media/platform/vsp1/vsp1_video.h | 3 +-
5 files changed, 80 insertions(+), 75 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c
b/drivers/media/platform/vsp1/vsp1_drv.c
index d29f9c4baebe..5d82f6ee56ea 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -589,7 +589,7 @@ static int __maybe_unused vsp1_pm_suspend(struct device
*dev)
* restarted explicitly by the DU.
*/
if (!vsp1->drm)
- vsp1_pipelines_suspend(vsp1);
+ vsp1_video_suspend(vsp1);
pm_runtime_force_suspend(vsp1->dev);
@@ -607,7 +607,7 @@ static int __maybe_unused vsp1_pm_resume(struct device *dev)
* restarted explicitly by the DU.
*/
if (!vsp1->drm)
- vsp1_pipelines_resume(vsp1);
+ vsp1_video_resume(vsp1);
return 0;
}
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c
b/drivers/media/platform/vsp1/vsp1_pipe.c
index 6fde4c0b9844..da21f1a7cd75 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -386,73 +386,3 @@ void vsp1_pipeline_propagate_partition(struct
vsp1_pipeline *pipe,
}
}
-void vsp1_pipelines_suspend(struct vsp1_device *vsp1)
-{
- unsigned long flags;
- unsigned int i;
- int ret;
-
- /*
-* To avoid increasing the system suspend time needlessly, loop over the
-* pipelines twice, first to set them all to the stopping state, and
-* then to wait for the stop to complete.
-*/
- for (i = 0; i < vsp1->info->wpf_count; ++i) {
- struct vsp1_rwpf *wpf = vsp1->wpf[i];
- struct vsp1_pipeline *pipe;
-
- if (wpf == NULL)
- continue;
-
- pipe = wpf->entity.pipe;
- if (pipe == NULL)
- continue;
-
- spin_lock_irqsave(>irqlock, flags);
- if (pipe->state == VSP1_PIPELINE_RUNNING)
- pipe->state = VSP1_PIPELINE_STOPPING;
- spin_unlock_irqrestore(>irqlock, flags);
- }
-
- for (i = 0; i < vsp1->info->wpf_count; ++i) {
- struct vsp1_rwpf *wpf = vsp1->wpf[i];
- struct vsp1_pipeline *pipe;
-
- if (wpf == NULL)
- continue;
-
- pipe = wpf->entity.pipe;
- if (pipe == NULL)
- continue;
-
- ret = wait_event_timeout(pipe->wq, vsp1_pipeline_stopped(pipe),
-msecs_to_jiffies(500));
- if (ret == 0)
- dev_warn(vsp1->dev, "pipeline %u stop timeout\n",
-wpf->entity.index);
- }
-}
-
-void vsp1_pipelines_resume(struct vsp1_device *vsp1)
-{
- unsigned long flags;
- unsigned int i;
-
- /* Resume all running pipelines. */
- for (i = 0; i < vsp1->info->wpf_count; ++i) {
- struct vsp1_rwpf *wpf = vsp1->wpf[i];
- struct vsp1_pipeline *pipe;
-
- if (wpf == NULL)
- continue;
-
- pipe = wpf->entity.pipe;
- if (pipe == NULL)
- continue;
-
- spin_lock_irqsave(>irqlock, flags);
- if (vsp1_pipeline_ready(pipe))
- vsp1_pipeline_run(pipe);
- spin_unlock_irqrestore(>irqlock, flags);
- }
-}
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h
b/drivers/media/platform/vsp1/vsp1_pipe.h
index 663d7fed7929..69858ba6cb31 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -164,9 +164,6 @@ void vsp1_pipeline_propagate_partition(struct vsp1_pipeline
*pipe,
unsigned int index,
struct vsp1_partition_window *window);
-void vsp1_pipelines_suspend(struct vsp1_device *vsp1);
-void vsp1_pipelines_resume(str
[PATCH v11 09/10] media: vsp1: Adapt entities to configure into a body
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Currently the entities store their configurations into a display list.
Adapt this such that the code can be configured into a body directly,
allowing greater flexibility and control of the content.
All users of vsp1_dl_list_write() are removed in this process, thus it
too is removed.
A helper, vsp1_dl_list_get_body0() is provided to access the internal body0
from the display list.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
[Don't remove blank line unnecessarily]
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_brx.c| 22 ++--
drivers/media/platform/vsp1/vsp1_clu.c| 23 ++---
drivers/media/platform/vsp1/vsp1_dl.c | 12 ++-
drivers/media/platform/vsp1/vsp1_dl.h | 2 +-
drivers/media/platform/vsp1/vsp1_drm.c| 12 ---
drivers/media/platform/vsp1/vsp1_entity.c | 22 ++--
drivers/media/platform/vsp1/vsp1_entity.h | 18 ++
drivers/media/platform/vsp1/vsp1_hgo.c| 16 -
drivers/media/platform/vsp1/vsp1_hgt.c| 18 +-
drivers/media/platform/vsp1/vsp1_hsit.c | 10 +++---
drivers/media/platform/vsp1/vsp1_lif.c| 15
drivers/media/platform/vsp1/vsp1_lut.c| 23 ++---
drivers/media/platform/vsp1/vsp1_pipe.c | 4 +-
drivers/media/platform/vsp1/vsp1_pipe.h | 3 +-
drivers/media/platform/vsp1/vsp1_rpf.c| 43
drivers/media/platform/vsp1/vsp1_sru.c| 14
drivers/media/platform/vsp1/vsp1_uds.c| 25 +++---
drivers/media/platform/vsp1/vsp1_uds.h| 2 +-
drivers/media/platform/vsp1/vsp1_uif.c| 21 ++--
drivers/media/platform/vsp1/vsp1_video.c | 16 ++---
drivers/media/platform/vsp1/vsp1_wpf.c| 42 ---
21 files changed, 194 insertions(+), 169 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_brx.c
b/drivers/media/platform/vsp1/vsp1_brx.c
index 011edac5ebc1..359917b5d842 100644
--- a/drivers/media/platform/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/vsp1/vsp1_brx.c
@@ -26,10 +26,10 @@
* Device Access
*/
-static inline void vsp1_brx_write(struct vsp1_brx *brx, struct vsp1_dl_list
*dl,
- u32 reg, u32 data)
+static inline void vsp1_brx_write(struct vsp1_brx *brx,
+ struct vsp1_dl_body *dlb, u32 reg, u32 data)
{
- vsp1_dl_list_write(dl, brx->base + reg, data);
+ vsp1_dl_body_write(dlb, brx->base + reg, data);
}
/*
-
@@ -283,7 +283,7 @@ static const struct v4l2_subdev_ops brx_ops = {
static void brx_configure_stream(struct vsp1_entity *entity,
struct vsp1_pipeline *pipe,
-struct vsp1_dl_list *dl)
+struct vsp1_dl_body *dlb)
{
struct vsp1_brx *brx = to_brx(>subdev);
struct v4l2_mbus_framefmt *format;
@@ -305,7 +305,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
* format at the pipeline output is premultiplied.
*/
flags = pipe->output ? pipe->output->format.flags : 0;
- vsp1_brx_write(brx, dl, VI6_BRU_INCTRL,
+ vsp1_brx_write(brx, dlb, VI6_BRU_INCTRL,
flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
0 : VI6_BRU_INCTRL_NRM);
@@ -313,12 +313,12 @@ static void brx_configure_stream(struct vsp1_entity
*entity,
* Set the background position to cover the whole output image and
* configure its color.
*/
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_SIZE,
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_SIZE,
(format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
(format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_LOC, 0);
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_LOC, 0);
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_COL, brx->bgcolor |
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_COL, brx->bgcolor |
(0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
/*
@@ -328,7 +328,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
* unit.
*/
if (entity->type == VSP1_ENTITY_BRU)
- vsp1_brx_write(brx, dl, VI6_BRU_ROP,
+ vsp1_brx_write(brx, dlb, VI6_BRU_ROP,
VI6_BRU_ROP_DSTSEL_BRUIN(1) |
VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
VI6_BRU_ROP_AROP(VI6_ROP_NOP));
@@ -370,7 +370,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
[PATCH v11 06/10] media: vsp1: Convert display lists to use new body pool
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Adapt the dl->body0 object to use an object from the body pool. This
greatly reduces the pressure on the TLB for IPMMU use cases, as all of
the lists use a single allocation for the main body.
The CLU and LUT objects pre-allocate a pool containing three bodies,
allowing a userspace update before the hardware has committed a previous
set of tables.
Bodies are no longer 'freed' in interrupt context, but instead released
back to their respective pools. This allows us to remove the garbage
collector in the DLM.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_clu.c | 27 ++-
drivers/media/platform/vsp1/vsp1_clu.h | 1 +-
drivers/media/platform/vsp1/vsp1_dl.c | 221 ++
drivers/media/platform/vsp1/vsp1_dl.h | 3 +-
drivers/media/platform/vsp1/vsp1_lut.c | 27 ++-
drivers/media/platform/vsp1/vsp1_lut.h | 1 +-
6 files changed, 100 insertions(+), 180 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index ebfbb915dcdc..8efa12f5e53f 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -19,6 +19,8 @@
#define CLU_MIN_SIZE 4U
#define CLU_MAX_SIZE 8190U
+#define CLU_SIZE (17 * 17 * 17)
+
/*
-
* Device Access
*/
@@ -43,19 +45,19 @@ static int clu_set_table(struct vsp1_clu *clu, struct
v4l2_ctrl *ctrl)
struct vsp1_dl_body *dlb;
unsigned int i;
- dlb = vsp1_dl_body_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
+ dlb = vsp1_dl_body_get(clu->pool);
if (!dlb)
return -ENOMEM;
vsp1_dl_body_write(dlb, VI6_CLU_ADDR, 0);
- for (i = 0; i < 17 * 17 * 17; ++i)
+ for (i = 0; i < CLU_SIZE; ++i)
vsp1_dl_body_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
spin_lock_irq(>lock);
swap(clu->clu, dlb);
spin_unlock_irq(>lock);
- vsp1_dl_body_free(dlb);
+ vsp1_dl_body_put(dlb);
return 0;
}
@@ -216,8 +218,16 @@ static void clu_configure(struct vsp1_entity *entity,
}
}
+static void clu_destroy(struct vsp1_entity *entity)
+{
+ struct vsp1_clu *clu = to_clu(>subdev);
+
+ vsp1_dl_body_pool_destroy(clu->pool);
+}
+
static const struct vsp1_entity_operations clu_entity_ops = {
.configure = clu_configure,
+ .destroy = clu_destroy,
};
/*
-
@@ -243,6 +253,17 @@ struct vsp1_clu *vsp1_clu_create(struct vsp1_device *vsp1)
if (ret < 0)
return ERR_PTR(ret);
+ /*
+* Pre-allocate a body pool, with 3 bodies allowing a userspace update
+* before the hardware has committed a previous set of tables, handling
+* both the queued and pending dl entries. One extra entry is added to
+* the CLU_SIZE to allow for the VI6_CLU_ADDR header.
+*/
+ clu->pool = vsp1_dl_body_pool_create(clu->entity.vsp1, 3, CLU_SIZE + 1,
+0);
+ if (!clu->pool)
+ return ERR_PTR(-ENOMEM);
+
/* Initialize the control handler. */
v4l2_ctrl_handler_init(>ctrls, 2);
v4l2_ctrl_new_custom(>ctrls, _table_control, NULL);
diff --git a/drivers/media/platform/vsp1/vsp1_clu.h
b/drivers/media/platform/vsp1/vsp1_clu.h
index c45e6e707592..cef2f44481ba 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.h
+++ b/drivers/media/platform/vsp1/vsp1_clu.h
@@ -32,6 +32,7 @@ struct vsp1_clu {
spinlock_t lock;
unsigned int mode;
struct vsp1_dl_body *clu;
+ struct vsp1_dl_body_pool *pool;
};
static inline struct vsp1_clu *to_clu(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 41ace89a585b..617c46a03dec 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -108,7 +108,7 @@ struct vsp1_dl_list {
struct vsp1_dl_header *header;
dma_addr_t dma;
- struct vsp1_dl_body body0;
+ struct vsp1_dl_body *body0;
struct list_head bodies;
bool has_chain;
@@ -128,14 +128,12 @@ enum vsp1_dl_mode {
* @mode: display list operation mode (header or headerless)
* @singleshot: execute the display list in single-shot mode
* @vsp1: the VSP1 device
- * @lock: protects the free, active, queued, pending and gc_bodies lists
+ * @lock: protects the free, active, queued, and
[PATCH v11 07/10] media: vsp1: Use reference counting for bodies
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Extend the display list body with a reference count, allowing bodies to
be kept as long as a reference is maintained. This provides the ability
to keep a cached copy of bodies which will not change, so that they can
be re-applied to multiple display lists.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_clu.c | 7 ++-
drivers/media/platform/vsp1/vsp1_dl.c | 16 ++--
drivers/media/platform/vsp1/vsp1_lut.c | 7 ++-
3 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index 8efa12f5e53f..ea83f1b7d125 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -212,8 +212,13 @@ static void clu_configure(struct vsp1_entity *entity,
clu->clu = NULL;
spin_unlock_irqrestore(>lock, flags);
- if (dlb)
+ if (dlb) {
vsp1_dl_list_add_body(dl, dlb);
+
+ /* Release our local reference. */
+ vsp1_dl_body_put(dlb);
+ }
+
break;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 617c46a03dec..1407c90c6880 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -10,6 +10,7 @@
#include
#include
#include
+#include
#include
#include
@@ -54,6 +55,8 @@ struct vsp1_dl_body {
struct list_head list;
struct list_head free;
+ refcount_t refcnt;
+
struct vsp1_dl_body_pool *pool;
struct vsp1_device *vsp1;
@@ -258,6 +261,7 @@ struct vsp1_dl_body *vsp1_dl_body_get(struct
vsp1_dl_body_pool *pool)
if (!list_empty(>free)) {
dlb = list_first_entry(>free, struct vsp1_dl_body, free);
list_del(>free);
+ refcount_set(>refcnt, 1);
}
spin_unlock_irqrestore(>lock, flags);
@@ -278,6 +282,9 @@ void vsp1_dl_body_put(struct vsp1_dl_body *dlb)
if (!dlb)
return;
+ if (!refcount_dec_and_test(>refcnt))
+ return;
+
dlb->num_entries = 0;
spin_lock_irqsave(>pool->lock, flags);
@@ -463,8 +470,11 @@ void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg,
u32 data)
* which bodies are added.
*
* Adding a body to a display list passes ownership of the body to the list.
The
- * caller must not touch the body after this call, and must not release it
- * explicitly with vsp1_dl_body_put().
+ * caller retains its reference to the fragment when adding it to the display
+ * list, but is not allowed to add new entries to the body.
+ *
+ * The reference must be explicitly released by a call to vsp1_dl_body_put()
+ * when the body isn't needed anymore.
*
* Additional bodies are only usable for display lists in header mode.
* Attempting to add a body to a header-less display list will return an error.
@@ -475,6 +485,8 @@ int vsp1_dl_list_add_body(struct vsp1_dl_list *dl, struct
vsp1_dl_body *dlb)
if (dl->dlm->mode != VSP1_DL_MODE_HEADER)
return -EINVAL;
+ refcount_inc(>refcnt);
+
list_add_tail(>list, >bodies);
return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c
b/drivers/media/platform/vsp1/vsp1_lut.c
index 6b358617ce15..b3ea90172439 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -168,8 +168,13 @@ static void lut_configure(struct vsp1_entity *entity,
lut->lut = NULL;
spin_unlock_irqrestore(>lock, flags);
- if (dlb)
+ if (dlb) {
vsp1_dl_list_add_body(dl, dlb);
+
+ /* Release our local reference. */
+ vsp1_dl_body_put(dlb);
+ }
+
break;
}
}
--
git-series 0.9.1
[PATCH v11 04/10] media: vsp1: Protect bodies against overflow
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
The body write function relies on the code never asking it to write more
than the entries available in the list.
Currently with each list body containing 256 entries, this is fine, but
we can reduce this number greatly saving memory. In preparation of this
add a level of protection to catch any buffer overflows.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 7 +++
1 file changed, 7 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 083da4f05c20..51965c30dec2 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -46,6 +46,7 @@ struct vsp1_dl_entry {
* @dma: DMA address of the entries
* @size: size of the DMA memory in bytes
* @num_entries: number of stored entries
+ * @max_entries: number of entries available
*/
struct vsp1_dl_body {
struct list_head list;
@@ -56,6 +57,7 @@ struct vsp1_dl_body {
size_t size;
unsigned int num_entries;
+ unsigned int max_entries;
};
/**
@@ -138,6 +140,7 @@ static int vsp1_dl_body_init(struct vsp1_device *vsp1,
dlb->vsp1 = vsp1;
dlb->size = size;
+ dlb->max_entries = num_entries;
dlb->entries = dma_alloc_wc(vsp1->bus_master, dlb->size, >dma,
GFP_KERNEL);
@@ -219,6 +222,10 @@ void vsp1_dl_body_free(struct vsp1_dl_body *dlb)
*/
void vsp1_dl_body_write(struct vsp1_dl_body *dlb, u32 reg, u32 data)
{
+ if (WARN_ONCE(dlb->num_entries >= dlb->max_entries,
+ "DLB size exceeded (max %u)", dlb->max_entries))
+ return;
+
dlb->entries[dlb->num_entries].addr = reg;
dlb->entries[dlb->num_entries].data = data;
dlb->num_entries++;
--
git-series 0.9.1
[PATCH v11 03/10] media: vsp1: Reword uses of 'fragment' as 'body'
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Throughout the codebase, the term 'fragment' is used to represent a
display list body. This term duplicates the 'body' which is already in
use.
The datasheet references these objects as a body, therefore replace all
mentions of a fragment with a body, along with the corresponding
pluralised terms.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_clu.c | 10 +-
drivers/media/platform/vsp1/vsp1_dl.c | 111 --
drivers/media/platform/vsp1/vsp1_dl.h | 13 +--
drivers/media/platform/vsp1/vsp1_lut.c | 8 +-
4 files changed, 70 insertions(+), 72 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index 96a448e1504c..ebfbb915dcdc 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -43,19 +43,19 @@ static int clu_set_table(struct vsp1_clu *clu, struct
v4l2_ctrl *ctrl)
struct vsp1_dl_body *dlb;
unsigned int i;
- dlb = vsp1_dl_fragment_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
+ dlb = vsp1_dl_body_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
if (!dlb)
return -ENOMEM;
- vsp1_dl_fragment_write(dlb, VI6_CLU_ADDR, 0);
+ vsp1_dl_body_write(dlb, VI6_CLU_ADDR, 0);
for (i = 0; i < 17 * 17 * 17; ++i)
- vsp1_dl_fragment_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
+ vsp1_dl_body_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
spin_lock_irq(>lock);
swap(clu->clu, dlb);
spin_unlock_irq(>lock);
- vsp1_dl_fragment_free(dlb);
+ vsp1_dl_body_free(dlb);
return 0;
}
@@ -211,7 +211,7 @@ static void clu_configure(struct vsp1_entity *entity,
spin_unlock_irqrestore(>lock, flags);
if (dlb)
- vsp1_dl_list_add_fragment(dl, dlb);
+ vsp1_dl_list_add_body(dl, dlb);
break;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 801dea475740..083da4f05c20 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -65,7 +65,7 @@ struct vsp1_dl_body {
* @header: display list header, NULL for headerless lists
* @dma: DMA address for the header
* @body0: first display list body
- * @fragments: list of extra display list bodies
+ * @bodies: list of extra display list bodies
* @has_chain: if true, indicates that there's a partition chain
* @chain: entry in the display list partition chain
* @internal: whether the display list is used for internal purpose
@@ -78,7 +78,7 @@ struct vsp1_dl_list {
dma_addr_t dma;
struct vsp1_dl_body body0;
- struct list_head fragments;
+ struct list_head bodies;
bool has_chain;
struct list_head chain;
@@ -97,13 +97,13 @@ enum vsp1_dl_mode {
* @mode: display list operation mode (header or headerless)
* @singleshot: execute the display list in single-shot mode
* @vsp1: the VSP1 device
- * @lock: protects the free, active, queued, pending and gc_fragments lists
+ * @lock: protects the free, active, queued, pending and gc_bodies lists
* @free: array of all free display lists
* @active: list currently being processed (loaded) by hardware
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
- * @gc_work: fragments garbage collector work struct
- * @gc_fragments: array of display list fragments waiting to be freed
+ * @gc_work: bodies garbage collector work struct
+ * @gc_bodies: array of display list bodies waiting to be freed
*/
struct vsp1_dl_manager {
unsigned int index;
@@ -118,7 +118,7 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *pending;
struct work_struct gc_work;
- struct list_head gc_fragments;
+ struct list_head gc_bodies;
};
/*
-
@@ -156,18 +156,17 @@ static void vsp1_dl_body_cleanup(struct vsp1_dl_body *dlb)
}
/**
- * vsp1_dl_fragment_alloc - Allocate a display list fragment
+ * vsp1_dl_body_alloc - Allocate a display list body
* @vsp1: The VSP1 device
- * @num_entries: The maximum number of entries that the fragment can contain
+ * @num_entries: The maximum number of entries that the body can contain
*
- * Allocate a display list fragment with enough memory to contain the requested
+ * Allocate a display list body with enough memory to contain the requested
* number of entries.
*
- * Return a pointer to a fragment on success or NULL if memory can
[PATCH v11 05/10] media: vsp1: Provide a body pool
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Each display list allocates a body to store register values in a dma
accessible buffer from a dma_alloc_wc() allocation. Each of these
results in an entry in the IOMMU TLB, and a large number of display list
allocations adds pressure to this resource.
Reduce TLB pressure on the IPMMUs by allocating multiple display list
bodies in a single allocation, and providing these to the display list
through a 'body pool'. A pool can be allocated by the display list
manager or entities which require their own body allocations.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 163 +++-
drivers/media/platform/vsp1/vsp1_dl.h | 8 +-
2 files changed, 171 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 51965c30dec2..41ace89a585b 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -41,6 +41,8 @@ struct vsp1_dl_entry {
/**
* struct vsp1_dl_body - Display list body
* @list: entry in the display list list of bodies
+ * @free: entry in the pool free body list
+ * @pool: pool to which this body belongs
* @vsp1: the VSP1 device
* @entries: array of entries
* @dma: DMA address of the entries
@@ -50,6 +52,9 @@ struct vsp1_dl_entry {
*/
struct vsp1_dl_body {
struct list_head list;
+ struct list_head free;
+
+ struct vsp1_dl_body_pool *pool;
struct vsp1_device *vsp1;
struct vsp1_dl_entry *entries;
@@ -61,6 +66,30 @@ struct vsp1_dl_body {
};
/**
+ * struct vsp1_dl_body_pool - display list body pool
+ * @dma: DMA address of the entries
+ * @size: size of the full DMA memory pool in bytes
+ * @mem: CPU memory pointer for the pool
+ * @bodies: Array of DLB structures for the pool
+ * @free: List of free DLB entries
+ * @lock: Protects the free list
+ * @vsp1: the VSP1 device
+ */
+struct vsp1_dl_body_pool {
+ /* DMA allocation */
+ dma_addr_t dma;
+ size_t size;
+ void *mem;
+
+ /* Body management */
+ struct vsp1_dl_body *bodies;
+ struct list_head free;
+ spinlock_t lock;
+
+ struct vsp1_device *vsp1;
+};
+
+/**
* struct vsp1_dl_list - Display list
* @list: entry in the display list manager lists
* @dlm: the display list manager
@@ -104,6 +133,7 @@ enum vsp1_dl_mode {
* @active: list currently being processed (loaded) by hardware
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
+ * @pool: body pool for the display list bodies
* @gc_work: bodies garbage collector work struct
* @gc_bodies: array of display list bodies waiting to be freed
*/
@@ -119,6 +149,8 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *queued;
struct vsp1_dl_list *pending;
+ struct vsp1_dl_body_pool *pool;
+
struct work_struct gc_work;
struct list_head gc_bodies;
};
@@ -127,6 +159,137 @@ struct vsp1_dl_manager {
* Display List Body Management
*/
+/**
+ * vsp1_dl_body_pool_create - Create a pool of bodies from a single allocation
+ * @vsp1: The VSP1 device
+ * @num_bodies: The number of bodies to allocate
+ * @num_entries: The maximum number of entries that a body can contain
+ * @extra_size: Extra allocation provided for the bodies
+ *
+ * Allocate a pool of display list bodies each with enough memory to contain
the
+ * requested number of entries plus the @extra_size.
+ *
+ * Return a pointer to a pool on success or NULL if memory can't be allocated.
+ */
+struct vsp1_dl_body_pool *
+vsp1_dl_body_pool_create(struct vsp1_device *vsp1, unsigned int num_bodies,
+unsigned int num_entries, size_t extra_size)
+{
+ struct vsp1_dl_body_pool *pool;
+ size_t dlb_size;
+ unsigned int i;
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool)
+ return NULL;
+
+ pool->vsp1 = vsp1;
+
+ /*
+* TODO: 'extra_size' is only used by vsp1_dlm_create(), to allocate
+* extra memory for the display list header. We need only one header per
+* display list, not per display list body, thus this allocation is
+* extraneous and should be reworked in the future.
+*/
+ dlb_size = num_entries * sizeof(struct vsp1_dl_entry) + extra_size;
+ pool->size = dlb_size * num_bodies;
+
+ pool->bodies = kcalloc(num_bodies, sizeof(*pool->bodies), GFP_KERNEL);
+ if (!pool->bodies) {
+ kfree(pool);
+ return NULL;
+ }
+
+ pool->mem = dma_alloc_wc(vsp1->bus_master, pool->size, >dma,
+
[PATCH v11 10/10] media: vsp1: Move video configuration to a cached dlb
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
We are now able to configure a pipeline directly into a local display
list body. Take advantage of this fact, and create a cacheable body to
store the configuration of the pipeline in the video object.
vsp1_video_pipeline_run() is now the last user of the pipe->dl object.
Convert this function to use the cached video->config body and obtain a
local display list reference.
Attach the video->stream_config body to the display list when needed
before committing to hardware.
Use a flag 'configured' to know when we should attach our stream_config
to the next outgoing display list to reconfigure the hardware in the
event of our first frame, or the first frame following a suspend/resume
cycle.
Our video DL usage now looks like the below output:
dl->body0 contains our disposable runtime configuration. Max 41.
dl_child->body0 is our partition specific configuration. Max 12.
dl->bodies shows our constant configuration and LUTs.
These two are LUT/CLU:
* dl->bodies[x]->num_entries 256 / max 256
* dl->bodies[x]->num_entries 4914 / max 4914
Which shows that our 'constant' configuration cache is currently
utilised to a maximum of 64 entries.
trace-cmd report | \
grep max | sed 's/.*vsp1_dl_list_commit://g' | sort | uniq;
dl->body0->num_entries 13 / max 128
dl->body0->num_entries 14 / max 128
dl->body0->num_entries 16 / max 128
dl->body0->num_entries 20 / max 128
dl->body0->num_entries 27 / max 128
dl->body0->num_entries 34 / max 128
dl->body0->num_entries 41 / max 128
dl_child->body0->num_entries 10 / max 128
dl_child->body0->num_entries 12 / max 128
dl->bodies[x]->num_entries 15 / max 128
dl->bodies[x]->num_entries 16 / max 128
dl->bodies[x]->num_entries 17 / max 128
dl->bodies[x]->num_entries 18 / max 128
dl->bodies[x]->num_entries 20 / max 128
dl->bodies[x]->num_entries 21 / max 128
dl->bodies[x]->num_entries 256 / max 256
dl->bodies[x]->num_entries 31 / max 128
dl->bodies[x]->num_entries 32 / max 128
dl->bodies[x]->num_entries 39 / max 128
dl->bodies[x]->num_entries 40 / max 128
dl->bodies[x]->num_entries 47 / max 128
dl->bodies[x]->num_entries 48 / max 128
dl->bodies[x]->num_entries 4914 / max 4914
dl->bodies[x]->num_entries 55 / max 128
dl->bodies[x]->num_entries 56 / max 128
dl->bodies[x]->num_entries 63 / max 128
dl->bodies[x]->num_entries 64 / max 128
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v11:
- Remove dlbs pool from video object.
Utilise the DLM pool for video->stream_config
- Improve comments
- clear the video->stream_config after it is released
object.
- stream_config and configured flag return to the pipe object.
v10:
- Removed pipe->configured flag, and use
pipe->state == VSP1_PIPELINE_STOPPED instead.
v8:
- Fix comments
- Rename video->pipe_config -> video->stream_config
v4:
- Adjust pipe configured flag to be reset on resume rather than suspend
- rename dl_child, dl_next
v3:
- 's/fragment/body/', 's/fragments/bodies/'
- video dlb cache allocation increased from 2 to 3 dlbs
drivers/media/platform/vsp1/vsp1_dl.c| 10 +++-
drivers/media/platform/vsp1/vsp1_dl.h| 1 +-
drivers/media/platform/vsp1/vsp1_pipe.h | 6 +-
drivers/media/platform/vsp1/vsp1_video.c | 69 +++--
4 files changed, 56 insertions(+), 30 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index c7fa1cb088cd..0f97305de965 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -813,6 +813,11 @@ void vsp1_dlm_reset(struct vsp1_dl_manager *dlm)
dlm->pending = NULL;
}
+struct vsp1_dl_body *vsp1_dlm_dl_body_get(struct vsp1_dl_manager *dlm)
+{
+ return vsp1_dl_body_get(dlm->pool);
+}
+
struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
unsigned int index,
unsigned int prealloc)
@@ -838,13 +843,14 @@ struct vsp1_dl_manager *vsp1_dlm_create(struct
vsp1_device *vsp1,
* Initialize the display list body and allocate DMA memory for the body
* and the optional header. Both are allocated together to avoid memory
* fragmentation, with the header located right after the body in
-* memory.
+* memory. An extra body is allocated on top of the prealloc to account
+* for the cached body used by the vsp1_video object.
*/
header_size = dlm->mode == VSP1_DL_MODE_HEADER
? ALIGN(sizeof(struct vsp1_dl_header), 8)
: 0;
- dlm->pool = vsp1_dl_body_pool_create(vsp1, prealloc,
+
[PATCH v11 01/10] media: v4l: vsp1: Release buffers for each video node
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Commit 372b2b0399fc ("media: v4l: vsp1: Release buffers in
start_streaming error path") introduced a helper to clean up buffers on
error paths, but inadvertently changed the code such that only the
output WPF buffers were cleaned, rather than the video node being
operated on.
Since then vsp1_video_cleanup_pipeline() has grown to perform both video
node cleanup, as well as pipeline cleanup. Split the implementation into
two distinct functions that perform the required work, so that each
video node can release it's buffers correctly on streamoff. The pipe
cleanup that was performed in the vsp1_video_stop_streaming() (releasing
the pipe->dl) is moved to the function for clarity.
Fixes: 372b2b0399fc ("media: v4l: vsp1: Release buffers in start_streaming
error path")
Cc: sta...@vger.kernel.org # v4.13+
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_video.c | 21 +
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_video.c
b/drivers/media/platform/vsp1/vsp1_video.c
index c8c12223a267..ba89dd176a13 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -842,9 +842,8 @@ static int vsp1_video_setup_pipeline(struct vsp1_pipeline
*pipe)
return 0;
}
-static void vsp1_video_cleanup_pipeline(struct vsp1_pipeline *pipe)
+static void vsp1_video_release_buffers(struct vsp1_video *video)
{
- struct vsp1_video *video = pipe->output->video;
struct vsp1_vb2_buffer *buffer;
unsigned long flags;
@@ -854,12 +853,18 @@ static void vsp1_video_cleanup_pipeline(struct
vsp1_pipeline *pipe)
vb2_buffer_done(>buf.vb2_buf, VB2_BUF_STATE_ERROR);
INIT_LIST_HEAD(>irqqueue);
spin_unlock_irqrestore(>irqlock, flags);
+}
+
+static void vsp1_video_cleanup_pipeline(struct vsp1_pipeline *pipe)
+{
+ lockdep_assert_held(>lock);
/* Release our partition table allocation */
- mutex_lock(>lock);
kfree(pipe->part_table);
pipe->part_table = NULL;
- mutex_unlock(>lock);
+
+ vsp1_dl_list_put(pipe->dl);
+ pipe->dl = NULL;
}
static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
@@ -874,8 +879,9 @@ static int vsp1_video_start_streaming(struct vb2_queue *vq,
unsigned int count)
if (pipe->stream_count == pipe->num_inputs) {
ret = vsp1_video_setup_pipeline(pipe);
if (ret < 0) {
- mutex_unlock(>lock);
+ vsp1_video_release_buffers(video);
vsp1_video_cleanup_pipeline(pipe);
+ mutex_unlock(>lock);
return ret;
}
@@ -925,13 +931,12 @@ static void vsp1_video_stop_streaming(struct vb2_queue
*vq)
if (ret == -ETIMEDOUT)
dev_err(video->vsp1->dev, "pipeline stop timeout\n");
- vsp1_dl_list_put(pipe->dl);
- pipe->dl = NULL;
+ vsp1_video_cleanup_pipeline(pipe);
}
mutex_unlock(>lock);
media_pipeline_stop(>video.entity);
- vsp1_video_cleanup_pipeline(pipe);
+ vsp1_video_release_buffers(video);
vsp1_video_pipeline_put(pipe);
}
--
git-series 0.9.1
[PATCH v11 00/10] vsp1: TLB optimisation and DL caching
From: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com> Each display list currently allocates an area of DMA memory to store register settings for the VSP1 to process. Each of these allocations adds pressure to the IPMMU TLB entries. We can reduce the pressure by pre-allocating larger areas and dividing the area across multiple bodies represented as a pool. With this reconfiguration of bodies, we can adapt the configuration code to separate out constant hardware configuration and cache it for re-use. The patches provided in this series can be found at: git://git.kernel.org/pub/scm/linux/kernel/git/kbingham/rcar.git tags/vsp1/tlb-optimise/v11 Changelog: -- v11: - Introduce two new patches as fixes to the VSP1. * media: v4l: vsp1: Release buffers for each video node * media: vsp1: Move video suspend resume handling to video object - media: vsp1: Move video configuration to a cached dlb - Now uses dlb's from the DLM pool - clears the pipe->stream_config after it is released - moves stream_config and configured flags to pipe objects. v10: - Rebase to latest linux-media/master - Remove pipe->configured flag, as pipe->state is suitable for the same purpose, as: (!pipe->configured) == (pipe->state == VSP1_PIPELINE_STOPPED) v9: - Pass the DL through configure_partition() calls - Remove redundant reference to gc_bodies v8: - Fix formatting and white space - Reword vsp1_dl_list_add_body() documentation - Update commit message on "Provide a body pool" - No longer pass unnecessary dlm->pool through vsp1_dl_list_alloc() - Add support for the new UIF entity - Fix comment location for clu_configure_stream() - Implement configure_partition separation - Rename video->pipe_config to video->stream_config v7: - Rebased on to linux-media/master (v4.16-rc4) - Clean up the formatting of the vsp1_dl_list_add_body() - Fix formatting and white space - s/prepare/configure_stream/ - s/configure/configure_frame/ v6: - Rebased on to linux-media/master (v4.16-rc1) - Removed DRM/UIF (DISCOM/ColorKey) updates v5: - Rebased on to renesas-drivers-2018-01-09-v4.15-rc7 to fix conflicts with DRM and UIF updates on VSP1 driver v4: - Rebased to v4.14 * v4l: vsp1: Use reference counting for bodies - Fix up reference handling comments * v4l: vsp1: Provide a body pool - Provide comment explaining extra allocation on body pool highlighting area for optimisation later. * v4l: vsp1: Refactor display list configure operations - Fix up comment to describe yuv_mode caching rather than format * vsp1: Adapt entities to configure into a body - Rename vsp1_dl_list_get_body() to vsp1_dl_list_get_body0() * v4l: vsp1: Move video configuration to a cached dlb - Adjust pipe configured flag to be reset on resume rather than suspend - rename dl_child, dl_next Testing: The VSP unit tests have been run on this patch set with the following results: root@Ubuntu-ARM64:~/vsp-tests# ./vsp-tests.sh --- Test loop 1 --- - vsp-unit-test-.sh Test Conditions: Platform Renesas Salvator-X 2nd version board based on r8a7795 ES2.0+ Kernel release4.17.0-rc4-arm64-renesas-00397-g3d2f6f2901b0 convert /usr/bin/convert compare /usr/bin/compare killall /usr/bin/killall raw2rgbpnm/usr/bin/raw2rgbpnm stress/usr/bin/stress yavta /usr/bin/yavta - vsp-unit-test-0001.sh Testing WPF packing in RGB332: pass Testing WPF packing in ARGB555: pass Testing WPF packing in XRGB555: pass Testing WPF packing in RGB565: pass Testing WPF packing in BGR24: pass Testing WPF packing in RGB24: pass Testing WPF packing in ABGR32: pass Testing WPF packing in ARGB32: pass Testing WPF packing in XBGR32: pass Testing WPF packing in XRGB32: pass - vsp-unit-test-0002.sh Testing WPF packing in NV12M: pass Testing WPF packing in NV16M: pass Testing WPF packing in NV21M: pass Testing WPF packing in NV61M: pass Testing WPF packing in UYVY: pass Testing WPF packing in VYUY: skip Testing WPF packing in YUV420M: pass Testing WPF packing in YUV422M: pass Testing WPF packing in YUV444M: pass Testing WPF packing in YVU420M: pass Testing WPF packing in YVU422M: pass Testing WPF packing in YVU444M: pass Testing WPF packing in YUYV: pass Testing WPF packing in YVYU: pass - vsp-unit-test-0003.sh Testing scaling from 640x640 to 640x480 in RGB24: pass Testing scaling from 1024x768 to 640x480 in RGB24: pass Testing scaling from 640x480 to 1024x768 in RGB24: pass Testing scaling from 640x640 to 640x480 in YUV444M: pass Testing scaling from 1024x768 to 640x480 in YUV444M: pass Testing scaling from 640x480 to 1024x768 in YUV444M: pass - vsp-unit-test-0004.sh Testing histogram in RGB24: pass Testing histogram in YUV444M: pass - vsp-unit-test-0005.sh Testing RPF.0: pass Testing RPF.1: pass Testing RPF.2: pass Testing RPF.3: pass Testing RPF.4: pass - vsp-unit-test-0006.sh Testing in
Re: [PATCH v10 8/8] media: vsp1: Move video configuration to a cached dlb
Hi Laurent,
On 17/05/18 20:57, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Thursday, 17 May 2018 20:24:01 EEST Kieran Bingham wrote:
>> We are now able to configure a pipeline directly into a local display
>> list body. Take advantage of this fact, and create a cacheable body to
>> store the configuration of the pipeline in the video object.
>>
>> vsp1_video_pipeline_run() is now the last user of the pipe->dl object.
>> Convert this function to use the cached video->config body and obtain a
>> local display list reference.
>>
>> Attach the video->config body to the display list when needed before
>> committing to hardware.
>>
>> The pipe object is marked as un-configured when resuming from a suspend.
>
> Is this comment still valid ?
Nope :D
Updated in latest local patch.
>
>> This ensures that when the hardware is reset - our cached configuration
>> will be re-attached to the next committed DL.
>>
>> Our video DL usage now looks like the below output:
>>
>> dl->body0 contains our disposable runtime configuration. Max 41.
>> dl_child->body0 is our partition specific configuration. Max 12.
>> dl->bodies shows our constant configuration and LUTs.
>>
>> These two are LUT/CLU:
>> * dl->bodies[x]->num_entries 256 / max 256
>> * dl->bodies[x]->num_entries 4914 / max 4914
>>
>> Which shows that our 'constant' configuration cache is currently
>> utilised to a maximum of 64 entries.
>>
>> trace-cmd report | \
>> grep max | sed 's/.*vsp1_dl_list_commit://g' | sort | uniq;
>>
>> dl->body0->num_entries 13 / max 128
>> dl->body0->num_entries 14 / max 128
>> dl->body0->num_entries 16 / max 128
>> dl->body0->num_entries 20 / max 128
>> dl->body0->num_entries 27 / max 128
>> dl->body0->num_entries 34 / max 128
>> dl->body0->num_entries 41 / max 128
>> dl_child->body0->num_entries 10 / max 128
>> dl_child->body0->num_entries 12 / max 128
>> dl->bodies[x]->num_entries 15 / max 128
>> dl->bodies[x]->num_entries 16 / max 128
>> dl->bodies[x]->num_entries 17 / max 128
>> dl->bodies[x]->num_entries 18 / max 128
>> dl->bodies[x]->num_entries 20 / max 128
>> dl->bodies[x]->num_entries 21 / max 128
>> dl->bodies[x]->num_entries 256 / max 256
>> dl->bodies[x]->num_entries 31 / max 128
>> dl->bodies[x]->num_entries 32 / max 128
>> dl->bodies[x]->num_entries 39 / max 128
>> dl->bodies[x]->num_entries 40 / max 128
>> dl->bodies[x]->num_entries 47 / max 128
>> dl->bodies[x]->num_entries 48 / max 128
>> dl->bodies[x]->num_entries 4914 / max 4914
>> dl->bodies[x]->num_entries 55 / max 128
>> dl->bodies[x]->num_entries 56 / max 128
>> dl->bodies[x]->num_entries 63 / max 128
>> dl->bodies[x]->num_entries 64 / max 128
>>
>> Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
>> ---
>> v10:
>> - Removed pipe->configured flag, and use
>>pipe->state == VSP1_PIPELINE_STOPPED instead.
>>
>> v8:
>> - Fix comments
>> - Rename video->pipe_config -> video->stream_config
>>
>> v4:
>> - Adjust pipe configured flag to be reset on resume rather than suspend
>> - rename dl_child, dl_next
>>
>> v3:
>> - 's/fragment/body/', 's/fragments/bodies/'
>> - video dlb cache allocation increased from 2 to 3 dlbs
>>
>> drivers/media/platform/vsp1/vsp1_pipe.h | 3 +-
>> drivers/media/platform/vsp1/vsp1_video.c | 67 +++--
>> drivers/media/platform/vsp1/vsp1_video.h | 2 +-
>> 3 files changed, 43 insertions(+), 29 deletions(-)
>>
>> diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h
>> b/drivers/media/platform/vsp1/vsp1_pipe.h index e00010693eef..be6ecab3cbed
>> 100644
>> --- a/drivers/media/platform/vsp1/vsp1_pipe.h
>> +++ b/drivers/media/platform/vsp1/vsp1_pipe.h
>> @@ -102,7 +102,6 @@ struct vsp1_partition {
>> * @uds: UDS entity, if present
>> * @uds_input: entity at the input of the UDS, if the UDS is present
>> * @entities: list of entities in the pipeline
>> - * @dl: display list associated with the pipeline
>> * @partitions: The number of partitions used to process one frame
>> * @partition: The current partition for configuration to process
>> * @part_table: The pre-calculated partitions use
[PATCH v10 6/8] media: vsp1: Refactor display list configure operations
The entities provide a single .configure operation which configures the
object into the target display list, based on the vsp1_entity_params
selection.
Split the configure function into three parts, '.configure_stream()',
'.configure_frame()', and '.configure_partition()' to facilitate
splitting the configuration of each parameter class into separate
display list bodies.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
[Blank line reformatting, remote unneeded local variable initialization]
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_brx.c| 12 +-
drivers/media/platform/vsp1/vsp1_clu.c| 78 ++
drivers/media/platform/vsp1/vsp1_drm.c| 12 +-
drivers/media/platform/vsp1/vsp1_entity.c | 24 ++-
drivers/media/platform/vsp1/vsp1_entity.h | 39 +--
drivers/media/platform/vsp1/vsp1_hgo.c| 12 +-
drivers/media/platform/vsp1/vsp1_hgt.c| 12 +-
drivers/media/platform/vsp1/vsp1_hsit.c | 12 +-
drivers/media/platform/vsp1/vsp1_lif.c| 12 +-
drivers/media/platform/vsp1/vsp1_lut.c| 47 +---
drivers/media/platform/vsp1/vsp1_rpf.c| 168 ++---
drivers/media/platform/vsp1/vsp1_sru.c| 12 +-
drivers/media/platform/vsp1/vsp1_uds.c| 56 ++--
drivers/media/platform/vsp1/vsp1_uif.c| 16 +-
drivers/media/platform/vsp1/vsp1_video.c | 28 +--
drivers/media/platform/vsp1/vsp1_wpf.c| 302 ---
16 files changed, 422 insertions(+), 420 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_brx.c
b/drivers/media/platform/vsp1/vsp1_brx.c
index 3beec18fd863..011edac5ebc1 100644
--- a/drivers/media/platform/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/vsp1/vsp1_brx.c
@@ -281,19 +281,15 @@ static const struct v4l2_subdev_ops brx_ops = {
* VSP1 Entity Operations
*/
-static void brx_configure(struct vsp1_entity *entity,
- struct vsp1_pipeline *pipe,
- struct vsp1_dl_list *dl,
- enum vsp1_entity_params params)
+static void brx_configure_stream(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl)
{
struct vsp1_brx *brx = to_brx(>subdev);
struct v4l2_mbus_framefmt *format;
unsigned int flags;
unsigned int i;
- if (params != VSP1_ENTITY_PARAMS_INIT)
- return;
-
format = vsp1_entity_get_pad_format(>entity, brx->entity.config,
brx->entity.source_pad);
@@ -400,7 +396,7 @@ static void brx_configure(struct vsp1_entity *entity,
}
static const struct vsp1_entity_operations brx_entity_ops = {
- .configure = brx_configure,
+ .configure_stream = brx_configure_stream,
};
/*
-
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index ea83f1b7d125..34f17a82ac1f 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -169,57 +169,50 @@ static const struct v4l2_subdev_ops clu_ops = {
* VSP1 Entity Operations
*/
-static void clu_configure(struct vsp1_entity *entity,
- struct vsp1_pipeline *pipe,
- struct vsp1_dl_list *dl,
- enum vsp1_entity_params params)
+static void clu_configure_stream(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl)
+{
+ struct vsp1_clu *clu = to_clu(>subdev);
+ struct v4l2_mbus_framefmt *format;
+
+ /*
+* The yuv_mode can't be changed during streaming. Cache it internally
+* for future runtime configuration calls.
+*/
+ format = vsp1_entity_get_pad_format(>entity,
+ clu->entity.config,
+ CLU_PAD_SINK);
+ clu->yuv_mode = format->code == MEDIA_BUS_FMT_AYUV8_1X32;
+}
+
+static void clu_configure_frame(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
{
struct vsp1_clu *clu = to_clu(>subdev);
struct vsp1_dl_body *dlb;
unsigned long flags;
u32 ctrl = VI6_CLU_CTRL_AAI | VI6_CLU_CTRL_MVS | VI6_CLU_CTRL_EN;
- switch (params) {
- case VSP1_ENTITY_PARAMS_INIT: {
- /*
-* The format can't be changed during streaming, only verify it
-* at setup time and store the information internally for future
-* runtime configuration calls.
-*/
- struct v4l2
[PATCH v10 2/8] media: vsp1: Protect bodies against overflow
The body write function relies on the code never asking it to write more
than the entries available in the list.
Currently with each list body containing 256 entries, this is fine, but
we can reduce this number greatly saving memory. In preparation of this
add a level of protection to catch any buffer overflows.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 7 +++
1 file changed, 7 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 083da4f05c20..51965c30dec2 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -46,6 +46,7 @@ struct vsp1_dl_entry {
* @dma: DMA address of the entries
* @size: size of the DMA memory in bytes
* @num_entries: number of stored entries
+ * @max_entries: number of entries available
*/
struct vsp1_dl_body {
struct list_head list;
@@ -56,6 +57,7 @@ struct vsp1_dl_body {
size_t size;
unsigned int num_entries;
+ unsigned int max_entries;
};
/**
@@ -138,6 +140,7 @@ static int vsp1_dl_body_init(struct vsp1_device *vsp1,
dlb->vsp1 = vsp1;
dlb->size = size;
+ dlb->max_entries = num_entries;
dlb->entries = dma_alloc_wc(vsp1->bus_master, dlb->size, >dma,
GFP_KERNEL);
@@ -219,6 +222,10 @@ void vsp1_dl_body_free(struct vsp1_dl_body *dlb)
*/
void vsp1_dl_body_write(struct vsp1_dl_body *dlb, u32 reg, u32 data)
{
+ if (WARN_ONCE(dlb->num_entries >= dlb->max_entries,
+ "DLB size exceeded (max %u)", dlb->max_entries))
+ return;
+
dlb->entries[dlb->num_entries].addr = reg;
dlb->entries[dlb->num_entries].data = data;
dlb->num_entries++;
--
git-series 0.9.1
[PATCH v10 3/8] media: vsp1: Provide a body pool
Each display list allocates a body to store register values in a dma
accessible buffer from a dma_alloc_wc() allocation. Each of these
results in an entry in the IOMMU TLB, and a large number of display list
allocations adds pressure to this resource.
Reduce TLB pressure on the IPMMUs by allocating multiple display list
bodies in a single allocation, and providing these to the display list
through a 'body pool'. A pool can be allocated by the display list
manager or entities which require their own body allocations.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 163 +++-
drivers/media/platform/vsp1/vsp1_dl.h | 8 +-
2 files changed, 171 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 51965c30dec2..41ace89a585b 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -41,6 +41,8 @@ struct vsp1_dl_entry {
/**
* struct vsp1_dl_body - Display list body
* @list: entry in the display list list of bodies
+ * @free: entry in the pool free body list
+ * @pool: pool to which this body belongs
* @vsp1: the VSP1 device
* @entries: array of entries
* @dma: DMA address of the entries
@@ -50,6 +52,9 @@ struct vsp1_dl_entry {
*/
struct vsp1_dl_body {
struct list_head list;
+ struct list_head free;
+
+ struct vsp1_dl_body_pool *pool;
struct vsp1_device *vsp1;
struct vsp1_dl_entry *entries;
@@ -61,6 +66,30 @@ struct vsp1_dl_body {
};
/**
+ * struct vsp1_dl_body_pool - display list body pool
+ * @dma: DMA address of the entries
+ * @size: size of the full DMA memory pool in bytes
+ * @mem: CPU memory pointer for the pool
+ * @bodies: Array of DLB structures for the pool
+ * @free: List of free DLB entries
+ * @lock: Protects the free list
+ * @vsp1: the VSP1 device
+ */
+struct vsp1_dl_body_pool {
+ /* DMA allocation */
+ dma_addr_t dma;
+ size_t size;
+ void *mem;
+
+ /* Body management */
+ struct vsp1_dl_body *bodies;
+ struct list_head free;
+ spinlock_t lock;
+
+ struct vsp1_device *vsp1;
+};
+
+/**
* struct vsp1_dl_list - Display list
* @list: entry in the display list manager lists
* @dlm: the display list manager
@@ -104,6 +133,7 @@ enum vsp1_dl_mode {
* @active: list currently being processed (loaded) by hardware
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
+ * @pool: body pool for the display list bodies
* @gc_work: bodies garbage collector work struct
* @gc_bodies: array of display list bodies waiting to be freed
*/
@@ -119,6 +149,8 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *queued;
struct vsp1_dl_list *pending;
+ struct vsp1_dl_body_pool *pool;
+
struct work_struct gc_work;
struct list_head gc_bodies;
};
@@ -127,6 +159,137 @@ struct vsp1_dl_manager {
* Display List Body Management
*/
+/**
+ * vsp1_dl_body_pool_create - Create a pool of bodies from a single allocation
+ * @vsp1: The VSP1 device
+ * @num_bodies: The number of bodies to allocate
+ * @num_entries: The maximum number of entries that a body can contain
+ * @extra_size: Extra allocation provided for the bodies
+ *
+ * Allocate a pool of display list bodies each with enough memory to contain
the
+ * requested number of entries plus the @extra_size.
+ *
+ * Return a pointer to a pool on success or NULL if memory can't be allocated.
+ */
+struct vsp1_dl_body_pool *
+vsp1_dl_body_pool_create(struct vsp1_device *vsp1, unsigned int num_bodies,
+unsigned int num_entries, size_t extra_size)
+{
+ struct vsp1_dl_body_pool *pool;
+ size_t dlb_size;
+ unsigned int i;
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool)
+ return NULL;
+
+ pool->vsp1 = vsp1;
+
+ /*
+* TODO: 'extra_size' is only used by vsp1_dlm_create(), to allocate
+* extra memory for the display list header. We need only one header per
+* display list, not per display list body, thus this allocation is
+* extraneous and should be reworked in the future.
+*/
+ dlb_size = num_entries * sizeof(struct vsp1_dl_entry) + extra_size;
+ pool->size = dlb_size * num_bodies;
+
+ pool->bodies = kcalloc(num_bodies, sizeof(*pool->bodies), GFP_KERNEL);
+ if (!pool->bodies) {
+ kfree(pool);
+ return NULL;
+ }
+
+ pool->mem = dma_alloc_wc(vsp1->bus_master, pool->size, >dma,
+GFP_KERNEL);
+ if (!pool->mem) {
+
[PATCH v10 0/8] vsp1: TLB optimisation and DL caching
esting active pipeline suspend/resume in suspend:core: pass - vsp-unit-test-0021.sh Testing WPF packing in RGB332 during stress testing: pass Testing WPF packing in ARGB555 during stress testing: pass Testing WPF packing in XRGB555 during stress testing: pass Testing WPF packing in RGB565 during stress testing: pass Testing WPF packing in BGR24 during stress testing: pass Testing WPF packing in RGB24 during stress testing: pass Testing WPF packing in ABGR32 during stress testing: pass Testing WPF packing in ARGB32 during stress testing: pass Testing WPF packing in XBGR32 during stress testing: pass Testing WPF packing in XRGB32 during stress testing: pass ./vsp-unit-test-0021.sh: line 34: 25075 Killed stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M - vsp-unit-test-0022.sh Testing long duration pipelines under stress: pass ./vsp-unit-test-0022.sh: line 38: 27041 Killed stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M - vsp-unit-test-0023.sh Testing histogram HGT with hue areas 0,255,255,255,255,255,255,255,255,255,255,255: pass Testing histogram HGT with hue areas 0,40,40,80,80,120,120,160,160,200,200,255: pass Testing histogram HGT with hue areas 220,40,40,80,80,120,120,160,160,200,200,220: pass Testing histogram HGT with hue areas 0,10,50,60,100,110,150,160,200,210,250,255: pass Testing histogram HGT with hue areas 10,20,50,60,100,110,150,160,200,210,230,240: pass Testing histogram HGT with hue areas 240,20,60,80,100,120,140,160,180,200,210,220: pass - vsp-unit-test-0024.sh Test requires unavailable feature set `rpf.0 rpf.1 brs wpf.0': skipped 168 tests: 147 passed, 0 failed, 3 skipped Kieran Bingham (8): media: vsp1: Reword uses of 'fragment' as 'body' media: vsp1: Protect bodies against overflow media: vsp1: Provide a body pool media: vsp1: Convert display lists to use new body pool media: vsp1: Use reference counting for bodies media: vsp1: Refactor display list configure operations media: vsp1: Adapt entities to configure into a body media: vsp1: Move video configuration to a cached dlb drivers/media/platform/vsp1/vsp1_brx.c| 32 +-- drivers/media/platform/vsp1/vsp1_clu.c| 113 --- drivers/media/platform/vsp1/vsp1_clu.h| 1 +- drivers/media/platform/vsp1/vsp1_dl.c | 388 +-- drivers/media/platform/vsp1/vsp1_dl.h | 20 +- drivers/media/platform/vsp1/vsp1_drm.c| 18 +- drivers/media/platform/vsp1/vsp1_entity.c | 34 +- drivers/media/platform/vsp1/vsp1_entity.h | 45 +-- drivers/media/platform/vsp1/vsp1_hgo.c| 26 +-- drivers/media/platform/vsp1/vsp1_hgt.c| 28 +-- drivers/media/platform/vsp1/vsp1_hsit.c | 20 +- drivers/media/platform/vsp1/vsp1_lif.c| 25 +- drivers/media/platform/vsp1/vsp1_lut.c| 80 +++-- drivers/media/platform/vsp1/vsp1_lut.h| 1 +- drivers/media/platform/vsp1/vsp1_pipe.c | 4 +- drivers/media/platform/vsp1/vsp1_pipe.h | 6 +- drivers/media/platform/vsp1/vsp1_rpf.c| 189 +-- drivers/media/platform/vsp1/vsp1_sru.c| 24 +- drivers/media/platform/vsp1/vsp1_uds.c| 73 ++-- drivers/media/platform/vsp1/vsp1_uds.h| 2 +- drivers/media/platform/vsp1/vsp1_uif.c| 35 +-- drivers/media/platform/vsp1/vsp1_video.c | 89 ++--- drivers/media/platform/vsp1/vsp1_video.h | 2 +- drivers/media/platform/vsp1/vsp1_wpf.c| 326 ++- 24 files changed, 870 insertions(+), 711 deletions(-) base-commit: 7e6b6b945272c20f6b78d319e07f27897a8373c9 -- git-series 0.9.1
[PATCH v10 7/8] media: vsp1: Adapt entities to configure into a body
Currently the entities store their configurations into a display list.
Adapt this such that the code can be configured into a body directly,
allowing greater flexibility and control of the content.
All users of vsp1_dl_list_write() are removed in this process, thus it
too is removed.
A helper, vsp1_dl_list_get_body0() is provided to access the internal body0
from the display list.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
[Don't remove blank line unnecessarily]
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_brx.c| 22 ++--
drivers/media/platform/vsp1/vsp1_clu.c| 23 ++---
drivers/media/platform/vsp1/vsp1_dl.c | 12 ++-
drivers/media/platform/vsp1/vsp1_dl.h | 2 +-
drivers/media/platform/vsp1/vsp1_drm.c| 12 ---
drivers/media/platform/vsp1/vsp1_entity.c | 22 ++--
drivers/media/platform/vsp1/vsp1_entity.h | 18 ++
drivers/media/platform/vsp1/vsp1_hgo.c| 16 -
drivers/media/platform/vsp1/vsp1_hgt.c| 18 +-
drivers/media/platform/vsp1/vsp1_hsit.c | 10 +++---
drivers/media/platform/vsp1/vsp1_lif.c| 15
drivers/media/platform/vsp1/vsp1_lut.c| 23 ++---
drivers/media/platform/vsp1/vsp1_pipe.c | 4 +-
drivers/media/platform/vsp1/vsp1_pipe.h | 3 +-
drivers/media/platform/vsp1/vsp1_rpf.c| 43
drivers/media/platform/vsp1/vsp1_sru.c| 14
drivers/media/platform/vsp1/vsp1_uds.c| 25 +++---
drivers/media/platform/vsp1/vsp1_uds.h| 2 +-
drivers/media/platform/vsp1/vsp1_uif.c| 21 ++--
drivers/media/platform/vsp1/vsp1_video.c | 16 ++---
drivers/media/platform/vsp1/vsp1_wpf.c| 42 ---
21 files changed, 194 insertions(+), 169 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_brx.c
b/drivers/media/platform/vsp1/vsp1_brx.c
index 011edac5ebc1..359917b5d842 100644
--- a/drivers/media/platform/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/vsp1/vsp1_brx.c
@@ -26,10 +26,10 @@
* Device Access
*/
-static inline void vsp1_brx_write(struct vsp1_brx *brx, struct vsp1_dl_list
*dl,
- u32 reg, u32 data)
+static inline void vsp1_brx_write(struct vsp1_brx *brx,
+ struct vsp1_dl_body *dlb, u32 reg, u32 data)
{
- vsp1_dl_list_write(dl, brx->base + reg, data);
+ vsp1_dl_body_write(dlb, brx->base + reg, data);
}
/*
-
@@ -283,7 +283,7 @@ static const struct v4l2_subdev_ops brx_ops = {
static void brx_configure_stream(struct vsp1_entity *entity,
struct vsp1_pipeline *pipe,
-struct vsp1_dl_list *dl)
+struct vsp1_dl_body *dlb)
{
struct vsp1_brx *brx = to_brx(>subdev);
struct v4l2_mbus_framefmt *format;
@@ -305,7 +305,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
* format at the pipeline output is premultiplied.
*/
flags = pipe->output ? pipe->output->format.flags : 0;
- vsp1_brx_write(brx, dl, VI6_BRU_INCTRL,
+ vsp1_brx_write(brx, dlb, VI6_BRU_INCTRL,
flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
0 : VI6_BRU_INCTRL_NRM);
@@ -313,12 +313,12 @@ static void brx_configure_stream(struct vsp1_entity
*entity,
* Set the background position to cover the whole output image and
* configure its color.
*/
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_SIZE,
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_SIZE,
(format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
(format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_LOC, 0);
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_LOC, 0);
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_COL, brx->bgcolor |
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_COL, brx->bgcolor |
(0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
/*
@@ -328,7 +328,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
* unit.
*/
if (entity->type == VSP1_ENTITY_BRU)
- vsp1_brx_write(brx, dl, VI6_BRU_ROP,
+ vsp1_brx_write(brx, dlb, VI6_BRU_ROP,
VI6_BRU_ROP_DSTSEL_BRUIN(1) |
VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
VI6_BRU_ROP_AROP(VI6_ROP_NOP));
@@ -370,7 +370,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
if (!(entity->type == VSP1_ENTITY_BRU && i == 1))
[PATCH v10 5/8] media: vsp1: Use reference counting for bodies
Extend the display list body with a reference count, allowing bodies to
be kept as long as a reference is maintained. This provides the ability
to keep a cached copy of bodies which will not change, so that they can
be re-applied to multiple display lists.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_clu.c | 7 ++-
drivers/media/platform/vsp1/vsp1_dl.c | 16 ++--
drivers/media/platform/vsp1/vsp1_lut.c | 7 ++-
3 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index 8efa12f5e53f..ea83f1b7d125 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -212,8 +212,13 @@ static void clu_configure(struct vsp1_entity *entity,
clu->clu = NULL;
spin_unlock_irqrestore(>lock, flags);
- if (dlb)
+ if (dlb) {
vsp1_dl_list_add_body(dl, dlb);
+
+ /* Release our local reference. */
+ vsp1_dl_body_put(dlb);
+ }
+
break;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 617c46a03dec..1407c90c6880 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -10,6 +10,7 @@
#include
#include
#include
+#include
#include
#include
@@ -54,6 +55,8 @@ struct vsp1_dl_body {
struct list_head list;
struct list_head free;
+ refcount_t refcnt;
+
struct vsp1_dl_body_pool *pool;
struct vsp1_device *vsp1;
@@ -258,6 +261,7 @@ struct vsp1_dl_body *vsp1_dl_body_get(struct
vsp1_dl_body_pool *pool)
if (!list_empty(>free)) {
dlb = list_first_entry(>free, struct vsp1_dl_body, free);
list_del(>free);
+ refcount_set(>refcnt, 1);
}
spin_unlock_irqrestore(>lock, flags);
@@ -278,6 +282,9 @@ void vsp1_dl_body_put(struct vsp1_dl_body *dlb)
if (!dlb)
return;
+ if (!refcount_dec_and_test(>refcnt))
+ return;
+
dlb->num_entries = 0;
spin_lock_irqsave(>pool->lock, flags);
@@ -463,8 +470,11 @@ void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg,
u32 data)
* which bodies are added.
*
* Adding a body to a display list passes ownership of the body to the list.
The
- * caller must not touch the body after this call, and must not release it
- * explicitly with vsp1_dl_body_put().
+ * caller retains its reference to the fragment when adding it to the display
+ * list, but is not allowed to add new entries to the body.
+ *
+ * The reference must be explicitly released by a call to vsp1_dl_body_put()
+ * when the body isn't needed anymore.
*
* Additional bodies are only usable for display lists in header mode.
* Attempting to add a body to a header-less display list will return an error.
@@ -475,6 +485,8 @@ int vsp1_dl_list_add_body(struct vsp1_dl_list *dl, struct
vsp1_dl_body *dlb)
if (dl->dlm->mode != VSP1_DL_MODE_HEADER)
return -EINVAL;
+ refcount_inc(>refcnt);
+
list_add_tail(>list, >bodies);
return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c
b/drivers/media/platform/vsp1/vsp1_lut.c
index 6b358617ce15..b3ea90172439 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -168,8 +168,13 @@ static void lut_configure(struct vsp1_entity *entity,
lut->lut = NULL;
spin_unlock_irqrestore(>lock, flags);
- if (dlb)
+ if (dlb) {
vsp1_dl_list_add_body(dl, dlb);
+
+ /* Release our local reference. */
+ vsp1_dl_body_put(dlb);
+ }
+
break;
}
}
--
git-series 0.9.1
[PATCH v10 8/8] media: vsp1: Move video configuration to a cached dlb
We are now able to configure a pipeline directly into a local display
list body. Take advantage of this fact, and create a cacheable body to
store the configuration of the pipeline in the video object.
vsp1_video_pipeline_run() is now the last user of the pipe->dl object.
Convert this function to use the cached video->config body and obtain a
local display list reference.
Attach the video->config body to the display list when needed before
committing to hardware.
The pipe object is marked as un-configured when resuming from a suspend.
This ensures that when the hardware is reset - our cached configuration
will be re-attached to the next committed DL.
Our video DL usage now looks like the below output:
dl->body0 contains our disposable runtime configuration. Max 41.
dl_child->body0 is our partition specific configuration. Max 12.
dl->bodies shows our constant configuration and LUTs.
These two are LUT/CLU:
* dl->bodies[x]->num_entries 256 / max 256
* dl->bodies[x]->num_entries 4914 / max 4914
Which shows that our 'constant' configuration cache is currently
utilised to a maximum of 64 entries.
trace-cmd report | \
grep max | sed 's/.*vsp1_dl_list_commit://g' | sort | uniq;
dl->body0->num_entries 13 / max 128
dl->body0->num_entries 14 / max 128
dl->body0->num_entries 16 / max 128
dl->body0->num_entries 20 / max 128
dl->body0->num_entries 27 / max 128
dl->body0->num_entries 34 / max 128
dl->body0->num_entries 41 / max 128
dl_child->body0->num_entries 10 / max 128
dl_child->body0->num_entries 12 / max 128
dl->bodies[x]->num_entries 15 / max 128
dl->bodies[x]->num_entries 16 / max 128
dl->bodies[x]->num_entries 17 / max 128
dl->bodies[x]->num_entries 18 / max 128
dl->bodies[x]->num_entries 20 / max 128
dl->bodies[x]->num_entries 21 / max 128
dl->bodies[x]->num_entries 256 / max 256
dl->bodies[x]->num_entries 31 / max 128
dl->bodies[x]->num_entries 32 / max 128
dl->bodies[x]->num_entries 39 / max 128
dl->bodies[x]->num_entries 40 / max 128
dl->bodies[x]->num_entries 47 / max 128
dl->bodies[x]->num_entries 48 / max 128
dl->bodies[x]->num_entries 4914 / max 4914
dl->bodies[x]->num_entries 55 / max 128
dl->bodies[x]->num_entries 56 / max 128
dl->bodies[x]->num_entries 63 / max 128
dl->bodies[x]->num_entries 64 / max 128
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v10:
- Removed pipe->configured flag, and use
pipe->state == VSP1_PIPELINE_STOPPED instead.
v8:
- Fix comments
- Rename video->pipe_config -> video->stream_config
v4:
- Adjust pipe configured flag to be reset on resume rather than suspend
- rename dl_child, dl_next
v3:
- 's/fragment/body/', 's/fragments/bodies/'
- video dlb cache allocation increased from 2 to 3 dlbs
drivers/media/platform/vsp1/vsp1_pipe.h | 3 +-
drivers/media/platform/vsp1/vsp1_video.c | 67 +++--
drivers/media/platform/vsp1/vsp1_video.h | 2 +-
3 files changed, 43 insertions(+), 29 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h
b/drivers/media/platform/vsp1/vsp1_pipe.h
index e00010693eef..be6ecab3cbed 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -102,7 +102,6 @@ struct vsp1_partition {
* @uds: UDS entity, if present
* @uds_input: entity at the input of the UDS, if the UDS is present
* @entities: list of entities in the pipeline
- * @dl: display list associated with the pipeline
* @partitions: The number of partitions used to process one frame
* @partition: The current partition for configuration to process
* @part_table: The pre-calculated partitions used by the pipeline
@@ -139,8 +138,6 @@ struct vsp1_pipeline {
*/
struct list_head entities;
- struct vsp1_dl_list *dl;
-
unsigned int partitions;
struct vsp1_partition *partition;
struct vsp1_partition *part_table;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c
b/drivers/media/platform/vsp1/vsp1_video.c
index 72f29773eb1c..f2bc26d28396 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -390,44 +390,48 @@ static void vsp1_video_pipeline_run_partition(struct
vsp1_pipeline *pipe,
static void vsp1_video_pipeline_run(struct vsp1_pipeline *pipe)
{
struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+ struct vsp1_video *video = pipe->output->video;
struct vsp1_entity *entity;
struct vsp1_dl_body *dlb;
+ struct vsp1_dl_list *dl;
unsigned int partition;
- if (!pipe->dl)
- pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
+ dl = vsp1_dl_list_get(pipe->output->dlm);
+
+ /*
[PATCH v10 4/8] media: vsp1: Convert display lists to use new body pool
Adapt the dl->body0 object to use an object from the body pool. This
greatly reduces the pressure on the TLB for IPMMU use cases, as all of
the lists use a single allocation for the main body.
The CLU and LUT objects pre-allocate a pool containing three bodies,
allowing a userspace update before the hardware has committed a previous
set of tables.
Bodies are no longer 'freed' in interrupt context, but instead released
back to their respective pools. This allows us to remove the garbage
collector in the DLM.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_clu.c | 27 ++-
drivers/media/platform/vsp1/vsp1_clu.h | 1 +-
drivers/media/platform/vsp1/vsp1_dl.c | 221 ++
drivers/media/platform/vsp1/vsp1_dl.h | 3 +-
drivers/media/platform/vsp1/vsp1_lut.c | 27 ++-
drivers/media/platform/vsp1/vsp1_lut.h | 1 +-
6 files changed, 100 insertions(+), 180 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index ebfbb915dcdc..8efa12f5e53f 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -19,6 +19,8 @@
#define CLU_MIN_SIZE 4U
#define CLU_MAX_SIZE 8190U
+#define CLU_SIZE (17 * 17 * 17)
+
/*
-
* Device Access
*/
@@ -43,19 +45,19 @@ static int clu_set_table(struct vsp1_clu *clu, struct
v4l2_ctrl *ctrl)
struct vsp1_dl_body *dlb;
unsigned int i;
- dlb = vsp1_dl_body_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
+ dlb = vsp1_dl_body_get(clu->pool);
if (!dlb)
return -ENOMEM;
vsp1_dl_body_write(dlb, VI6_CLU_ADDR, 0);
- for (i = 0; i < 17 * 17 * 17; ++i)
+ for (i = 0; i < CLU_SIZE; ++i)
vsp1_dl_body_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
spin_lock_irq(>lock);
swap(clu->clu, dlb);
spin_unlock_irq(>lock);
- vsp1_dl_body_free(dlb);
+ vsp1_dl_body_put(dlb);
return 0;
}
@@ -216,8 +218,16 @@ static void clu_configure(struct vsp1_entity *entity,
}
}
+static void clu_destroy(struct vsp1_entity *entity)
+{
+ struct vsp1_clu *clu = to_clu(>subdev);
+
+ vsp1_dl_body_pool_destroy(clu->pool);
+}
+
static const struct vsp1_entity_operations clu_entity_ops = {
.configure = clu_configure,
+ .destroy = clu_destroy,
};
/*
-
@@ -243,6 +253,17 @@ struct vsp1_clu *vsp1_clu_create(struct vsp1_device *vsp1)
if (ret < 0)
return ERR_PTR(ret);
+ /*
+* Pre-allocate a body pool, with 3 bodies allowing a userspace update
+* before the hardware has committed a previous set of tables, handling
+* both the queued and pending dl entries. One extra entry is added to
+* the CLU_SIZE to allow for the VI6_CLU_ADDR header.
+*/
+ clu->pool = vsp1_dl_body_pool_create(clu->entity.vsp1, 3, CLU_SIZE + 1,
+0);
+ if (!clu->pool)
+ return ERR_PTR(-ENOMEM);
+
/* Initialize the control handler. */
v4l2_ctrl_handler_init(>ctrls, 2);
v4l2_ctrl_new_custom(>ctrls, _table_control, NULL);
diff --git a/drivers/media/platform/vsp1/vsp1_clu.h
b/drivers/media/platform/vsp1/vsp1_clu.h
index c45e6e707592..cef2f44481ba 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.h
+++ b/drivers/media/platform/vsp1/vsp1_clu.h
@@ -32,6 +32,7 @@ struct vsp1_clu {
spinlock_t lock;
unsigned int mode;
struct vsp1_dl_body *clu;
+ struct vsp1_dl_body_pool *pool;
};
static inline struct vsp1_clu *to_clu(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 41ace89a585b..617c46a03dec 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -108,7 +108,7 @@ struct vsp1_dl_list {
struct vsp1_dl_header *header;
dma_addr_t dma;
- struct vsp1_dl_body body0;
+ struct vsp1_dl_body *body0;
struct list_head bodies;
bool has_chain;
@@ -128,14 +128,12 @@ enum vsp1_dl_mode {
* @mode: display list operation mode (header or headerless)
* @singleshot: execute the display list in single-shot mode
* @vsp1: the VSP1 device
- * @lock: protects the free, active, queued, pending and gc_bodies lists
+ * @lock: protects the free, active, queued, and pending lists
* @free: array of all free display lists
* @active: list cu
[PATCH v10 1/8] media: vsp1: Reword uses of 'fragment' as 'body'
Throughout the codebase, the term 'fragment' is used to represent a
display list body. This term duplicates the 'body' which is already in
use.
The datasheet references these objects as a body, therefore replace all
mentions of a fragment with a body, along with the corresponding
pluralised terms.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_clu.c | 10 +-
drivers/media/platform/vsp1/vsp1_dl.c | 111 --
drivers/media/platform/vsp1/vsp1_dl.h | 13 +--
drivers/media/platform/vsp1/vsp1_lut.c | 8 +-
4 files changed, 70 insertions(+), 72 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index 96a448e1504c..ebfbb915dcdc 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -43,19 +43,19 @@ static int clu_set_table(struct vsp1_clu *clu, struct
v4l2_ctrl *ctrl)
struct vsp1_dl_body *dlb;
unsigned int i;
- dlb = vsp1_dl_fragment_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
+ dlb = vsp1_dl_body_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
if (!dlb)
return -ENOMEM;
- vsp1_dl_fragment_write(dlb, VI6_CLU_ADDR, 0);
+ vsp1_dl_body_write(dlb, VI6_CLU_ADDR, 0);
for (i = 0; i < 17 * 17 * 17; ++i)
- vsp1_dl_fragment_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
+ vsp1_dl_body_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
spin_lock_irq(>lock);
swap(clu->clu, dlb);
spin_unlock_irq(>lock);
- vsp1_dl_fragment_free(dlb);
+ vsp1_dl_body_free(dlb);
return 0;
}
@@ -211,7 +211,7 @@ static void clu_configure(struct vsp1_entity *entity,
spin_unlock_irqrestore(>lock, flags);
if (dlb)
- vsp1_dl_list_add_fragment(dl, dlb);
+ vsp1_dl_list_add_body(dl, dlb);
break;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 801dea475740..083da4f05c20 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -65,7 +65,7 @@ struct vsp1_dl_body {
* @header: display list header, NULL for headerless lists
* @dma: DMA address for the header
* @body0: first display list body
- * @fragments: list of extra display list bodies
+ * @bodies: list of extra display list bodies
* @has_chain: if true, indicates that there's a partition chain
* @chain: entry in the display list partition chain
* @internal: whether the display list is used for internal purpose
@@ -78,7 +78,7 @@ struct vsp1_dl_list {
dma_addr_t dma;
struct vsp1_dl_body body0;
- struct list_head fragments;
+ struct list_head bodies;
bool has_chain;
struct list_head chain;
@@ -97,13 +97,13 @@ enum vsp1_dl_mode {
* @mode: display list operation mode (header or headerless)
* @singleshot: execute the display list in single-shot mode
* @vsp1: the VSP1 device
- * @lock: protects the free, active, queued, pending and gc_fragments lists
+ * @lock: protects the free, active, queued, pending and gc_bodies lists
* @free: array of all free display lists
* @active: list currently being processed (loaded) by hardware
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
- * @gc_work: fragments garbage collector work struct
- * @gc_fragments: array of display list fragments waiting to be freed
+ * @gc_work: bodies garbage collector work struct
+ * @gc_bodies: array of display list bodies waiting to be freed
*/
struct vsp1_dl_manager {
unsigned int index;
@@ -118,7 +118,7 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *pending;
struct work_struct gc_work;
- struct list_head gc_fragments;
+ struct list_head gc_bodies;
};
/*
-
@@ -156,18 +156,17 @@ static void vsp1_dl_body_cleanup(struct vsp1_dl_body *dlb)
}
/**
- * vsp1_dl_fragment_alloc - Allocate a display list fragment
+ * vsp1_dl_body_alloc - Allocate a display list body
* @vsp1: The VSP1 device
- * @num_entries: The maximum number of entries that the fragment can contain
+ * @num_entries: The maximum number of entries that the body can contain
*
- * Allocate a display list fragment with enough memory to contain the requested
+ * Allocate a display list body with enough memory to contain the requested
* number of entries.
*
- * Return a pointer to a fragment on success or NULL if memory can't be
- * allocated.
+ * Return a pointer to a body on success or NU
Re: [PATCH v7 8/8] media: vsp1: Move video configuration to a cached dlb
Hi Laurent, On 17/05/18 15:35, Laurent Pinchart wrote: > Hi Kieran, > > On Monday, 30 April 2018 20:48:03 EEST Kieran Bingham wrote: >> On 07/04/18 01:23, Laurent Pinchart wrote: >>> On Thursday, 8 March 2018 02:05:31 EEST Kieran Bingham wrote: >>>> We are now able to configure a pipeline directly into a local display >>>> list body. Take advantage of this fact, and create a cacheable body to >>>> store the configuration of the pipeline in the video object. >>>> >>>> vsp1_video_pipeline_run() is now the last user of the pipe->dl object. >>>> Convert this function to use the cached video->config body and obtain a >>>> local display list reference. >>>> >>>> Attach the video->config body to the display list when needed before >>>> committing to hardware. >>>> >>>> The pipe object is marked as un-configured when resuming from a suspend. >>>> This ensures that when the hardware is reset - our cached configuration >>>> will be re-attached to the next committed DL. >>>> >>>> Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com> >>>> --- >>>> >>>> v3: >>>> - 's/fragment/body/', 's/fragments/bodies/' >>>> - video dlb cache allocation increased from 2 to 3 dlbs >>>> >>>> Our video DL usage now looks like the below output: >>>> >>>> dl->body0 contains our disposable runtime configuration. Max 41. >>>> dl_child->body0 is our partition specific configuration. Max 12. >>>> dl->bodies shows our constant configuration and LUTs. >>>> >>>> These two are LUT/CLU: >>>> * dl->bodies[x]->num_entries 256 / max 256 >>>> * dl->bodies[x]->num_entries 4914 / max 4914 >>>> >>>> Which shows that our 'constant' configuration cache is currently >>>> utilised to a maximum of 64 entries. >>>> >>>> trace-cmd report | \ >>>> >>>> grep max | sed 's/.*vsp1_dl_list_commit://g' | sort | uniq; >>>> >>>> dl->body0->num_entries 13 / max 128 >>>> dl->body0->num_entries 14 / max 128 >>>> dl->body0->num_entries 16 / max 128 >>>> dl->body0->num_entries 20 / max 128 >>>> dl->body0->num_entries 27 / max 128 >>>> dl->body0->num_entries 34 / max 128 >>>> dl->body0->num_entries 41 / max 128 >>>> dl_child->body0->num_entries 10 / max 128 >>>> dl_child->body0->num_entries 12 / max 128 >>>> dl->bodies[x]->num_entries 15 / max 128 >>>> dl->bodies[x]->num_entries 16 / max 128 >>>> dl->bodies[x]->num_entries 17 / max 128 >>>> dl->bodies[x]->num_entries 18 / max 128 >>>> dl->bodies[x]->num_entries 20 / max 128 >>>> dl->bodies[x]->num_entries 21 / max 128 >>>> dl->bodies[x]->num_entries 256 / max 256 >>>> dl->bodies[x]->num_entries 31 / max 128 >>>> dl->bodies[x]->num_entries 32 / max 128 >>>> dl->bodies[x]->num_entries 39 / max 128 >>>> dl->bodies[x]->num_entries 40 / max 128 >>>> dl->bodies[x]->num_entries 47 / max 128 >>>> dl->bodies[x]->num_entries 48 / max 128 >>>> dl->bodies[x]->num_entries 4914 / max 4914 >>>> dl->bodies[x]->num_entries 55 / max 128 >>>> dl->bodies[x]->num_entries 56 / max 128 >>>> dl->bodies[x]->num_entries 63 / max 128 >>>> dl->bodies[x]->num_entries 64 / max 128 >>> >>> This might be useful to capture in the main part of the commit message. >>> >>>> v4: >>>> - Adjust pipe configured flag to be reset on resume rather than suspend >>>> - rename dl_child, dl_next >>>> >>>> drivers/media/platform/vsp1/vsp1_pipe.c | 7 +++- >>>> drivers/media/platform/vsp1/vsp1_pipe.h | 4 +- >>>> drivers/media/platform/vsp1/vsp1_video.c | 67 - >>>> drivers/media/platform/vsp1/vsp1_video.h | 2 +- >>>> 4 files changed, 54 insertions(+), 26 deletions(-) >>>> >>>> diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c >>>> b/drivers/media/platform/vsp1/vsp1_pipe.c index >>>> 5012643583b6..fa445b1a2e38 >>>> 100644 >>>> --
Re: [PATCH v9 6/8] media: vsp1: Refactor display list configure operations
Hi Laurent,
Thanks for the review,
On 17/05/18 10:41, Laurent Pinchart wrote:
> Hi Kieran,
>
> Thank you for the patch.
>
> On Thursday, 3 May 2018 16:35:45 EEST Kieran Bingham wrote:
>> The entities provide a single .configure operation which configures the
>> object into the target display list, based on the vsp1_entity_params
>> selection.
>>
>> Split the configure function into three parts, '.configure_stream()',
>> '.configure_frame()', and '.configure_partition()' to facilitate
>> splitting the configuration of each parameter class into separate
>> display list bodies.
>>
>> Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
>>
>> ---
>> The checkpatch warning:
>>
>> WARNING: function definition argument 'struct vsp1_dl_list *' should
>> also have an identifier name
>>
>> has been ignored to match the existing code style.
>>
>> v8:
>> - Add support for the UIF
>> - Remove unrelated whitespace change
>> - Fix comment location for clu_configure_stream()
>> - Update configure documentations
>> - Implement configure_partition separation.
>>
>> v7
>> - Fix formatting and white space
>> - s/prepare/configure_stream/
>> - s/configure/configure_frame/
>> ---
>> drivers/media/platform/vsp1/vsp1_brx.c| 12 +-
>> drivers/media/platform/vsp1/vsp1_clu.c| 77 ++
>> drivers/media/platform/vsp1/vsp1_drm.c| 12 +-
>> drivers/media/platform/vsp1/vsp1_entity.c | 24 ++-
>> drivers/media/platform/vsp1/vsp1_entity.h | 39 +--
>> drivers/media/platform/vsp1/vsp1_hgo.c| 12 +-
>> drivers/media/platform/vsp1/vsp1_hgt.c| 12 +-
>> drivers/media/platform/vsp1/vsp1_hsit.c | 12 +-
>> drivers/media/platform/vsp1/vsp1_lif.c| 12 +-
>> drivers/media/platform/vsp1/vsp1_lut.c| 47 +---
>> drivers/media/platform/vsp1/vsp1_rpf.c| 168 ++---
>> drivers/media/platform/vsp1/vsp1_sru.c| 12 +-
>> drivers/media/platform/vsp1/vsp1_uds.c| 56 ++--
>> drivers/media/platform/vsp1/vsp1_uif.c| 16 +-
>> drivers/media/platform/vsp1/vsp1_video.c | 28 +--
>> drivers/media/platform/vsp1/vsp1_wpf.c| 303 ---
>> 16 files changed, 422 insertions(+), 420 deletions(-)
>
> [snip]
>
>> diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
>> b/drivers/media/platform/vsp1/vsp1_clu.c index ea83f1b7d125..0a978980d447
>> 100644
>> --- a/drivers/media/platform/vsp1/vsp1_clu.c
>> +++ b/drivers/media/platform/vsp1/vsp1_clu.c
>> @@ -168,58 +168,50 @@ static const struct v4l2_subdev_ops clu_ops = {
>> /*
>> * VSP1 Entity Operations
>> */
>> +static void clu_configure_stream(struct vsp1_entity *entity,
>> + struct vsp1_pipeline *pipe,
>> + struct vsp1_dl_list *dl)
>> +{
>> +struct vsp1_clu *clu = to_clu(>subdev);
>> +struct v4l2_mbus_framefmt *format;
>>
>
> I would have kept this blank line before the function.
I would have thought I would too :)
>
>> -static void clu_configure(struct vsp1_entity *entity,
>> - struct vsp1_pipeline *pipe,
>> - struct vsp1_dl_list *dl,
>> - enum vsp1_entity_params params)
>> +/*
>> + * The yuv_mode can't be changed during streaming. Cache it internally
>> + * for future runtime configuration calls.
>> + */
>> +format = vsp1_entity_get_pad_format(>entity,
>> +clu->entity.config,
>> +CLU_PAD_SINK);
>> +clu->yuv_mode = format->code == MEDIA_BUS_FMT_AYUV8_1X32;
>> +}
>
> [snip]
>
>> diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c
>> b/drivers/media/platform/vsp1/vsp1_wpf.c index 65ed2f849551..da287c27b324
>> 100644
>> --- a/drivers/media/platform/vsp1/vsp1_wpf.c
>> +++ b/drivers/media/platform/vsp1/vsp1_wpf.c
>
> [snip]
>
>> +static void wpf_configure_frame(struct vsp1_entity *entity,
>> +struct vsp1_pipeline *pipe,
>> +struct vsp1_dl_list *dl)
>> +{
>> +struct vsp1_rwpf *wpf = to_rwpf(>subdev);
>> +unsigned long flags;
>> +u32 outfmt = 0;
>
> No need to initialize outfmt to 0.
Ah yes, indeed!.
>> +
>
> This blank line isn't needed.
>
>> +const unsigned int mask = BIT(WPF_CTRL_VFLIP)
>&
Re: [PATCH v4 0/6] Asynchronous UVC
Hi Mauro, On 15/05/18 20:22, Mauro Carvalho Chehab wrote: > Em Tue, 27 Mar 2018 17:45:57 +0100 > Kieran Bingham <kieran.bing...@ideasonboard.com> escreveu: > >> The Linux UVC driver has long provided adequate performance capabilities for >> web-cams and low data rate video devices in Linux while resolutions were low. >> >> Modern USB cameras are now capable of high data rates thanks to USB3 with >> 1080p, and even 4k capture resolutions supported. >> >> Cameras such as the Stereolabs ZED (bulk transfers) or the Logitech BRIO >> (isochronous transfers) can generate more data than an embedded ARM core is >> able to process on a single core, resulting in frame loss. >> >> A large part of this performance impact is from the requirement to >> ‘memcpy’ frames out from URB packets to destination frames. This unfortunate >> requirement is due to the UVC protocol allowing a variable length header, and >> thus it is not possible to provide the target frame buffers directly. >> >> Extra throughput is possible by moving the actual memcpy actions to a work >> queue, and moving the memcpy out of interrupt context thus allowing work >> tasks >> to be scheduled across multiple cores. >> >> This series has been tested on both the ZED and BRIO cameras on arm64 >> platforms, and with thanks to Randy Dunlap, a Dynex 1.3MP Webcam, a Sonix >> USB2 >> Camera, and a built in Toshiba Laptop camera, and with thanks to Philipp >> Zabel >> for testing on a Lite-On internal Laptop Webcam, Logitech C910 (USB2 isoc), >> Oculus Sensor (USB3 isoc), and Microsoft HoloLens Sensors (USB3 bulk). >> >> As far as I am aware iSight devices, and devices which use UVC to encode data >> (output device) have not yet been tested - but should find no ill effect (at >> least not until they are tested of course :D ) >> >> Tested-by: Randy Dunlap <rdun...@infradead.org> >> Tested-by: Philipp Zabel <philipp.za...@gmail.com> >> >> v2: >> - Fix race reported by Guennadi >> >> v3: >> - Fix similar race reported by Laurent >> - Only queue work if required (encode/isight do not queue work) >> - Refactor/Rename variables for clarity >> >> v4: >> - (Yet another) Rework of the uninitialise path. >>This time to hopefully clean up the shutdown races for good. >>use usb_poison_urb() to halt all URBs, then flush the work queue >>before freeing. >> - Rebase to latest linux-media/master > > Kieran/Laurent, > > What's the status of this patchset? I believe v4 was my final version (until someone tells me otherwise), and fixed all known races. Unless there has been bitrot since I posted (was last rebased at v4.16-rc4 ?) ... It would be good to at least get it in -next for a while if not mainline... Laurent ? The latest version is available in a branch at : git://git.kernel.org/pub/scm/linux/kernel/git/kbingham/rcar.git uvc/async/v4 Let me know if I need to post a rebased version. -- Kieran > > Regards, > Mauro > >> >> Kieran Bingham (6): >> media: uvcvideo: Refactor URB descriptors >> media: uvcvideo: Convert decode functions to use new context structure >> media: uvcvideo: Protect queue internals with helper >> media: uvcvideo: queue: Simplify spin-lock usage >> media: uvcvideo: queue: Support asynchronous buffer handling >> media: uvcvideo: Move decode processing to process context >> >> drivers/media/usb/uvc/uvc_isight.c | 6 +- >> drivers/media/usb/uvc/uvc_queue.c | 102 + >> drivers/media/usb/uvc/uvc_video.c | 180 +- >> drivers/media/usb/uvc/uvcvideo.h | 59 -- >> 4 files changed, 266 insertions(+), 81 deletions(-) >> >> base-commit: a77cfdf6bd06eef0dadea2b541a7c01502b1b4f6 > > > > Thanks, > Mauro > signature.asc Description: OpenPGP digital signature
Re: [PATCH v2 1/2] Revert "media: rcar-vin: enable field toggle after a set number of lines for Gen3"
Hi Niklas,
On 11/05/18 15:41, Niklas Söderlund wrote:
> The offending commit was an attempt to fix the issue of writing outside
> the capture buffer for VIN Gen3. Unfortunately it only fixed the symptom
> of the problem to such a degree I could no longer reproduce it. Revert
> the offending commit before a proper fix can be added in a follow-up
> patch.
>
> This reverts commit 015060cb7795eac34454696cc9c9f8b76926a401.
>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+rene...@ragnatech.se>
Acked-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
> ---
> drivers/media/platform/rcar-vin/rcar-dma.c | 20 +---
> 1 file changed, 5 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/media/platform/rcar-vin/rcar-dma.c
> b/drivers/media/platform/rcar-vin/rcar-dma.c
> index b41ba9a4a2b3ac90..ac07f99e3516a620 100644
> --- a/drivers/media/platform/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/rcar-vin/rcar-dma.c
> @@ -124,9 +124,7 @@
> #define VNDMR2_VPS (1 << 30)
> #define VNDMR2_HPS (1 << 29)
> #define VNDMR2_FTEV (1 << 17)
> -#define VNDMR2_FTEH (1 << 16)
> #define VNDMR2_VLV(n)((n & 0xf) << 12)
> -#define VNDMR2_HLV(n)((n) & 0xfff)
>
> /* Video n CSI2 Interface Mode Register (Gen3) */
> #define VNCSI_IFMD_DES1 (1 << 26)
> @@ -614,9 +612,8 @@ void rvin_crop_scale_comp(struct rvin_dev *vin)
>
> static int rvin_setup(struct rvin_dev *vin)
> {
> - u32 vnmc, dmr, dmr2, interrupts, lines;
> + u32 vnmc, dmr, dmr2, interrupts;
> bool progressive = false, output_is_yuv = false, input_is_yuv = false;
> - bool halfsize = false;
>
> switch (vin->format.field) {
> case V4L2_FIELD_TOP:
> @@ -631,15 +628,12 @@ static int rvin_setup(struct rvin_dev *vin)
> /* Use BT if video standard can be read and is 60 Hz format */
> if (!vin->info->use_mc && vin->std & V4L2_STD_525_60)
> vnmc = VNMC_IM_FULL | VNMC_FOC;
> - halfsize = true;
> break;
> case V4L2_FIELD_INTERLACED_TB:
> vnmc = VNMC_IM_FULL;
> - halfsize = true;
> break;
> case V4L2_FIELD_INTERLACED_BT:
> vnmc = VNMC_IM_FULL | VNMC_FOC;
> - halfsize = true;
> break;
> case V4L2_FIELD_NONE:
> vnmc = VNMC_IM_ODD_EVEN;
> @@ -682,15 +676,11 @@ static int rvin_setup(struct rvin_dev *vin)
> break;
> }
>
> - if (vin->info->model == RCAR_GEN3) {
> - /* Enable HSYNC Field Toggle mode after height HSYNC inputs. */
> - lines = vin->format.height / (halfsize ? 2 : 1);
> - dmr2 = VNDMR2_FTEH | VNDMR2_HLV(lines);
> - vin_dbg(vin, "Field Toogle after %u lines\n", lines);
> - } else {
> - /* Enable VSYNC Field Toogle mode after one VSYNC input. */
> + /* Enable VSYNC Field Toogle mode after one VSYNC input */
> + if (vin->info->model == RCAR_GEN3)
> + dmr2 = VNDMR2_FTEV;
> + else
> dmr2 = VNDMR2_FTEV | VNDMR2_VLV(1);
> - }
>
> /* Hsync Signal Polarity Select */
> if (!(vin->mbus_cfg.flags & V4L2_MBUS_HSYNC_ACTIVE_LOW))
>
Re: [PATCH v2 2/2] rcar-vin: fix crop and compose handling for Gen3
Hi Niklas,
This looks like quite the improvement :D
On 11/05/18 15:41, Niklas Söderlund wrote:
> When refactoring the Gen3 enablement series crop and compose handling
> where broken. This went unnoticed but can result in writing out side the
As well as Sergei's 'where/were', 'out side' is one word in this context.
'outside of the capture buffer'
> capture buffer. Fix this by restoring the crop and compose to reflect
> the format dimensions as we have not yet enabled the scaler for Gen3.
>
> Fixes: 5e7c623632fcf8f5 ("media: rcar-vin: use different v4l2 operations in
> media controller mode")
> Reported-by: Jacopo Mondi <jacopo+rene...@jmondi.org>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+rene...@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
> ---
> drivers/media/platform/rcar-vin/rcar-v4l2.c | 6 ++
> 1 file changed, 6 insertions(+)
>
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index 2fb8587116f25a4f..e78fba84d59028ef 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -702,6 +702,12 @@ static int rvin_mc_s_fmt_vid_cap(struct file *file, void
> *priv,
>
> vin->format = f->fmt.pix;
>
> + vin->crop.top = 0;
> + vin->crop.left = 0;
> + vin->crop.width = vin->format.width;
> + vin->crop.height = vin->format.height;
> + vin->compose = vin->crop;
> +
> return 0;
> }
>
>
Re: [PATCH v2] media: i2c: adv748x: Fix pixel rate values
Hi Niklas, Laurent,
Thanks for the updated patch and detailed investigation to get this right.
On 11/05/18 15:04, Niklas Söderlund wrote:
> From: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
>
> The pixel rate, as reported by the V4L2_CID_PIXEL_RATE control, must
> include both horizontal and vertical blanking. Both the AFE and HDMI
> receiver program it incorrectly:
>
> - The HDMI receiver goes to the trouble of removing blanking to compute
> the rate of active pixels. This is easy to fix by removing the
> computation and returning the incoming pixel clock rate directly.
>
> - The AFE performs similar calculation, while it should simply return
> the fixed pixel rate for analog sources, mandated by the ADV748x to be
> 14.3180180 MHz.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
> [Niklas: Update AFE fixed pixel rate]
> Signed-off-by: Niklas Söderlund <niklas.soderlund+rene...@ragnatech.se>
Reviewed-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Does this still require changes to the CSI2 driver to be synchronised?
(or does it not matter as the CSI2 isn't upstream yet)...
--
Kieran
> ---
>
> * Changes since v1
> - Update AFE fixed pixel rate.
> ---
> drivers/media/i2c/adv748x/adv748x-afe.c | 12 ++--
> drivers/media/i2c/adv748x/adv748x-hdmi.c | 8 +---
> 2 files changed, 7 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c
> b/drivers/media/i2c/adv748x/adv748x-afe.c
> index 61514bae7e5ceb42..edd25e895e5dec3c 100644
> --- a/drivers/media/i2c/adv748x/adv748x-afe.c
> +++ b/drivers/media/i2c/adv748x/adv748x-afe.c
> @@ -321,17 +321,17 @@ static const struct v4l2_subdev_video_ops
> adv748x_afe_video_ops = {
> static int adv748x_afe_propagate_pixelrate(struct adv748x_afe *afe)
> {
> struct v4l2_subdev *tx;
> - unsigned int width, height, fps;
>
> tx = adv748x_get_remote_sd(>pads[ADV748X_AFE_SOURCE]);
> if (!tx)
> return -ENOLINK;
>
> - width = 720;
> - height = afe->curr_norm & V4L2_STD_525_60 ? 480 : 576;
> - fps = afe->curr_norm & V4L2_STD_525_60 ? 30 : 25;
> -
> - return adv748x_csi2_set_pixelrate(tx, width * height * fps);
> + /*
> + * The ADV748x ADC sampling frequency is twice the externally supplied
> + * clock whose frequency is required to be 28.63636 MHz. It oversamples
> + * with a factor of 4 resulting in a pixel rate of 14.3180180 MHz.
> + */
> + return adv748x_csi2_set_pixelrate(tx, 14318180);
> }
>
> static int adv748x_afe_enum_mbus_code(struct v4l2_subdev *sd,
> diff --git a/drivers/media/i2c/adv748x/adv748x-hdmi.c
> b/drivers/media/i2c/adv748x/adv748x-hdmi.c
> index 10d229a4f08868f7..aecc2a84dfecbec8 100644
> --- a/drivers/media/i2c/adv748x/adv748x-hdmi.c
> +++ b/drivers/media/i2c/adv748x/adv748x-hdmi.c
> @@ -402,8 +402,6 @@ static int adv748x_hdmi_propagate_pixelrate(struct
> adv748x_hdmi *hdmi)
> {
> struct v4l2_subdev *tx;
> struct v4l2_dv_timings timings;
> - struct v4l2_bt_timings *bt =
> - unsigned int fps;
>
> tx = adv748x_get_remote_sd(>pads[ADV748X_HDMI_SOURCE]);
> if (!tx)
> @@ -411,11 +409,7 @@ static int adv748x_hdmi_propagate_pixelrate(struct
> adv748x_hdmi *hdmi)
>
> adv748x_hdmi_query_dv_timings(>sd, );
>
> - fps = DIV_ROUND_CLOSEST_ULL(bt->pixelclock,
> - V4L2_DV_BT_FRAME_WIDTH(bt) *
> - V4L2_DV_BT_FRAME_HEIGHT(bt));
> -
> - return adv748x_csi2_set_pixelrate(tx, bt->width * bt->height * fps);
> + return adv748x_csi2_set_pixelrate(tx, timings.bt.pixelclock);
> }
>
> static int adv748x_hdmi_enum_mbus_code(struct v4l2_subdev *sd,
>
signature.asc
Description: OpenPGP digital signature
Re: 4.17-rc3 regression in UVC driver
On 05/05/18 11:32, Mauro Carvalho Chehab wrote: > Hi Kieran, >> I manually pull from git://linuxtv.org/pinchartl/media.git uvc/fixes >> and picked the patch. > >> It should be at media_tree.git at the fixes branch. Thank you Mauro - that's great! -- Kieran
Re: 4.17-rc3 regression in UVC driver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hello again, On 05/05/18 08:34, Kieran Bingham wrote: > Hi Sebastian, > > On 04/05/18 19:45, Sebastian Reichel wrote: >> Hi, >> >> I just got the following error message every ms with 4.17-rc3 after >> upgrading to for first ~192 seconds after system start (Debian >> 4.17~rc3-1~exp1 kernel) on my Thinkpad X250: >> >>> uvcvideo: Failed to query (GET_MIN) UVC control 2 on unit 1: -32 (exp. >>> 1). > > I have submitted a patch to fix this ... (and I thought it would have got > in by now ... so I'll chase this up) Mauro - I just saw Laurent sent a pull-request for this last week, so I guess maybe it's on it's way: [GIT FIXES FOR v4.17] UVC fixes (25/04/2018) But perhaps it got missed ? - I don't see this patch in media/v4.17-4: git log media/v4.17-4 --grep="media: uvcvideo: Prevent setting unavailable flags " Anyway, I'll leave this in your hands. - -- Regards Kieran > Please see : https://patchwork.linuxtv.org/patch/48043/ and apply the > patch to bring your system logs back to a reasonable state :D > > Laurent, Mauro, > > This is the second bug report I've had on this topic. Can we aim to get > this patch merged please? > >> I see /dev/video0 and /dev/video1. The first one seems to be functional. >> The second one does not work and does not make sense to me (the system >> has only one webcam). I did not try to bisect anything. Here is some >> more information, that might be useful: > > There are two device nodes now, as one is provided to output meta-data or > such. > > >>> sre@earth ~ % mpv /dev/video1 Playing: /dev/video1 [ffmpeg/demuxer] >>> video4linux2,v4l2: ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for >>> device [lavf] avformat_open_input() failed Failed to recognize file >>> format. sre@earth ~ % udevadm info /dev/video0 P: >>> /devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video0 >>> N: video0 E: DEVNAME=/dev/video0 E: >>> DEVPATH=/devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video0 >>> >>> >>> E: MAJOR=81 >>> E: MINOR=0 E: SUBSYSTEM=video4linux sre@earth ~ % udevadm info >>> /dev/video1 P: >>> /devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video1 >>> N: video1 E: DEVNAME=/dev/video1 E: >>> DEVPATH=/devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video1 >>> >>> >>> E: MAJOR=81 >>> E: MINOR=1 E: SUBSYSTEM=video4linux sre@earth ~ % lsusb -d 04ca:703c >>> Bus 001 Device 004: ID 04ca:703c Lite-On Technology Corp. >> >> -- Sebastian > > > Regards > > Kieran > -BEGIN PGP SIGNATURE- iQIzBAEBCgAdFiEEkC3XmD+9KP3jctR6oR5GchCkYf0FAlrtYWoACgkQoR5GchCk Yf3tQg/7BgBFIj9more5vTaCFEU2L+tapdOBItO+mgzHI9fFXVHAmaVUlSjVAiMX PlyRx1NsSnCRSZ4b6R/ydjEuNG5dLr2B8LfS5QP/6ABmUCONdCBIbx/2mDk+DtoX XZeI65MMr3nZG7f6ZNq2EoVPIcFF4WLY4CsfixBOye3ps4e91IkTDFA/EVG0qkD8 XLRHACYSe/7lMS2TcAyOlmzWecYLYqnFBxfjlBD80NfdWITsDZdooZ3KUdHTCqN4 ooWjyifkeh79D8M9PGhnNGoB4gbHjTMhbZEH4RaABdv/jO/L4kPExX+2Wjc4LnRj QlWmLm9svIisXJaPO6sPg+rIUr6xLTs5Dv5yYU+UpR9AVfVusHy7FfFoYkcHpThZ h0rc1yi6eU4/Nnz1BvQbLCdjab2yEF5nD05PQs22r4ZJy04ymiz+y3SZvlr8vYfu 2ZRJzjkvFHtWV3yd/LfjxmuzmYXH4Yn+yN8jMGu8sY4JtJEsSF8qPbjDpicOhXvI pv4V9xvDGscsZdKFaNv0IES+NHNRsgPvx0sR0DOb3MC8T9p+aEJPfQEBly1XBPG0 4GkvuA//DKpBASFdlSpKt4YW6OX56X+g8BPOcaBJV8BlBRzumDk4cAZ/T2t4DLm5 w3GnEwWLu0BiWl1LKqgh4xVgOg45IqcJJmDgupM3WzjCrYwVCbo= =B80a -END PGP SIGNATURE-
Re: 4.17-rc3 regression in UVC driver
Hi Sebastian, On 04/05/18 19:45, Sebastian Reichel wrote: > Hi, > > I just got the following error message every ms with 4.17-rc3 after > upgrading to for first ~192 seconds after system start (Debian > 4.17~rc3-1~exp1 kernel) on my Thinkpad X250: > >> uvcvideo: Failed to query (GET_MIN) UVC control 2 on unit 1: -32 (exp. 1). I have submitted a patch to fix this ... (and I thought it would have got in by now ... so I'll chase this up) Please see : https://patchwork.linuxtv.org/patch/48043/ and apply the patch to bring your system logs back to a reasonable state :D Laurent, Mauro, This is the second bug report I've had on this topic. Can we aim to get this patch merged please? > I see /dev/video0 and /dev/video1. The first one seems to be > functional. The second one does not work and does not make > sense to me (the system has only one webcam). I did not try to > bisect anything. Here is some more information, that might > be useful: There are two device nodes now, as one is provided to output meta-data or such. >> sre@earth ~ % mpv /dev/video1 >> Playing: /dev/video1 >> [ffmpeg/demuxer] video4linux2,v4l2: ioctl(VIDIOC_G_INPUT): Inappropriate >> ioctl for device >> [lavf] avformat_open_input() failed >> Failed to recognize file format. >> sre@earth ~ % udevadm info /dev/video0 >> P: /devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video0 >> N: video0 >> E: DEVNAME=/dev/video0 >> E: >> DEVPATH=/devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video0 >> E: MAJOR=81 >> E: MINOR=0 >> E: SUBSYSTEM=video4linux >> sre@earth ~ % udevadm info /dev/video1 >> P: /devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video1 >> N: video1 >> E: DEVNAME=/dev/video1 >> E: >> DEVPATH=/devices/pci:00/:00:14.0/usb1/1-8/1-8:1.0/video4linux/video1 >> E: MAJOR=81 >> E: MINOR=1 >> E: SUBSYSTEM=video4linux >> sre@earth ~ % lsusb -d 04ca:703c >> Bus 001 Device 004: ID 04ca:703c Lite-On Technology Corp. > > -- Sebastian Regards Kieran signature.asc Description: OpenPGP digital signature
Re: [PATCH v3 10/11] media: vsp1: Support Interlaced display pipelines
Hi Laurent,
On 03/05/18 12:13, Laurent Pinchart wrote:
> Hi Kieran,
>>> + } else {
>>> + vsp1_rpf_write(rpf, dlb, VI6_RPF_SRCM_ADDR_Y, mem.addr[0]);
>>> + vsp1_rpf_write(rpf, dlb, VI6_RPF_SRCM_ADDR_C0, mem.addr[1]);
>>> + vsp1_rpf_write(rpf, dlb, VI6_RPF_SRCM_ADDR_C1, mem.addr[2]);
>>> + }
>>> }
>>> +
>>> static void rpf_partition(struct vsp1_entity *entity,
>>> struct vsp1_pipeline *pipe,
>>> struct vsp1_partition *partition,
>>> diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h
>>> b/drivers/media/platform/vsp1/vsp1_rwpf.h index
>>> 70742ecf766f..8d6e42f27908 100644
>>> --- a/drivers/media/platform/vsp1/vsp1_rwpf.h
>>> +++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
>>> @@ -42,6 +42,7 @@ struct vsp1_rwpf {
>>> struct v4l2_pix_format_mplane format;
>>> const struct vsp1_format_info *fmtinfo;
>>> unsigned int brx_input;
>>> + bool interlaced;
>
> kerneldoc might be nice :-)
There's no existing kerneldoc on struct vsp1_rwpf ?
>>>
>>> unsigned int alpha;
>>>
>
> [snip]
>
>>> diff --git a/include/media/vsp1.h b/include/media/vsp1.h
>>> index 678c24de1ac6..c10883f30980 100644
>>> --- a/include/media/vsp1.h
>>> +++ b/include/media/vsp1.h
>>> @@ -50,6 +50,7 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int
>>> pipe_index,>
>>> * @dst: destination rectangle on the display (integer coordinates)
>>> * @alpha: alpha value (0: fully transparent, 255: fully opaque)
>>> * @zpos: Z position of the plane (from 0 to number of planes minus 1)
>>> + * @interlaced: true for interlaced pipelines
>
> Maybe "true if the pipeline outputs an interlaced stream" ?
That's fine - but I've neglected to incorporate this into my v4 repost :-(
If by any magic - v4 is suitable for integration already, and you're happy to
take it into your tree - please feel free to update this comment.
Otherwise it will be in any next update.
--
KB
[PATCH v4 06/11] media: vsp1: Provide VSP1 feature helper macro
The VSP1 devices define their specific capabilities through features
marked in their device info structure. Various parts of the code read
this info structure to infer if the features are available.
Wrap this into a more readable vsp1_feature(vsp1, f) macro to ensure
that usage is consistent throughout the driver.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1.h | 2 ++
drivers/media/platform/vsp1/vsp1_drv.c | 16
drivers/media/platform/vsp1/vsp1_wpf.c | 6 +++---
3 files changed, 13 insertions(+), 11 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1.h
b/drivers/media/platform/vsp1/vsp1.h
index 33f632331474..f0d21cc8e9ab 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -68,6 +68,8 @@ struct vsp1_device_info {
bool uapi;
};
+#define vsp1_feature(vsp1, f) ((vsp1)->info->features & (f))
+
struct vsp1_device {
struct device *dev;
const struct vsp1_device_info *info;
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c
b/drivers/media/platform/vsp1/vsp1_drv.c
index d29f9c4baebe..0fc388bf5a33 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -265,7 +265,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
}
/* Instantiate all the entities. */
- if (vsp1->info->features & VSP1_HAS_BRS) {
+ if (vsp1_feature(vsp1, VSP1_HAS_BRS)) {
vsp1->brs = vsp1_brx_create(vsp1, VSP1_ENTITY_BRS);
if (IS_ERR(vsp1->brs)) {
ret = PTR_ERR(vsp1->brs);
@@ -275,7 +275,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
list_add_tail(>brs->entity.list_dev, >entities);
}
- if (vsp1->info->features & VSP1_HAS_BRU) {
+ if (vsp1_feature(vsp1, VSP1_HAS_BRU)) {
vsp1->bru = vsp1_brx_create(vsp1, VSP1_ENTITY_BRU);
if (IS_ERR(vsp1->bru)) {
ret = PTR_ERR(vsp1->bru);
@@ -285,7 +285,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
list_add_tail(>bru->entity.list_dev, >entities);
}
- if (vsp1->info->features & VSP1_HAS_CLU) {
+ if (vsp1_feature(vsp1, VSP1_HAS_CLU)) {
vsp1->clu = vsp1_clu_create(vsp1);
if (IS_ERR(vsp1->clu)) {
ret = PTR_ERR(vsp1->clu);
@@ -311,7 +311,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
list_add_tail(>hst->entity.list_dev, >entities);
- if (vsp1->info->features & VSP1_HAS_HGO && vsp1->info->uapi) {
+ if (vsp1_feature(vsp1, VSP1_HAS_HGO) && vsp1->info->uapi) {
vsp1->hgo = vsp1_hgo_create(vsp1);
if (IS_ERR(vsp1->hgo)) {
ret = PTR_ERR(vsp1->hgo);
@@ -322,7 +322,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
>entities);
}
- if (vsp1->info->features & VSP1_HAS_HGT && vsp1->info->uapi) {
+ if (vsp1_feature(vsp1, VSP1_HAS_HGT) && vsp1->info->uapi) {
vsp1->hgt = vsp1_hgt_create(vsp1);
if (IS_ERR(vsp1->hgt)) {
ret = PTR_ERR(vsp1->hgt);
@@ -353,7 +353,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
}
}
- if (vsp1->info->features & VSP1_HAS_LUT) {
+ if (vsp1_feature(vsp1, VSP1_HAS_LUT)) {
vsp1->lut = vsp1_lut_create(vsp1);
if (IS_ERR(vsp1->lut)) {
ret = PTR_ERR(vsp1->lut);
@@ -387,7 +387,7 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
}
}
- if (vsp1->info->features & VSP1_HAS_SRU) {
+ if (vsp1_feature(vsp1, VSP1_HAS_SRU)) {
vsp1->sru = vsp1_sru_create(vsp1);
if (IS_ERR(vsp1->sru)) {
ret = PTR_ERR(vsp1->sru);
@@ -537,7 +537,7 @@ static int vsp1_device_init(struct vsp1_device *vsp1)
vsp1_write(vsp1, VI6_DPR_HSI_ROUTE, VI6_DPR_NODE_UNUSED);
vsp1_write(vsp1, VI6_DPR_BRU_ROUTE, VI6_DPR_NODE_UNUSED);
- if (vsp1->info->features & VSP1_HAS_BRS)
+ if (vsp1_feature(vsp1, VSP1_HAS_BRS))
vsp1_write(vsp1, VI6_DPR_ILV_BRS_ROUTE, VI6_DPR_NODE_UNUSED);
vsp1_write(vsp1, VI6_DPR_HGO_SMPPT, (7 << VI6_DPR_SMPPT_TGW_SHIFT) |
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c
b/drivers/media/platform/vsp1/vsp1_wpf.c
index 2edea361eee4..ea1d226371b2 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -141,13 +141,13 @@ static int wpf_init_contro
[PATCH v4 04/11] media: vsp1: Remove unused display list structure field
The vsp1 reference in the vsp1_dl_body structure is not used.
Remove it.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index ec6fc21fabe0..b23e88cda49f 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -44,7 +44,6 @@ struct vsp1_dl_entry {
* @list: entry in the display list list of bodies
* @free: entry in the pool free body list
* @pool: pool to which this body belongs
- * @vsp1: the VSP1 device
* @entries: array of entries
* @dma: DMA address of the entries
* @size: size of the DMA memory in bytes
@@ -58,7 +57,6 @@ struct vsp1_dl_body {
refcount_t refcnt;
struct vsp1_dl_body_pool *pool;
- struct vsp1_device *vsp1;
struct vsp1_dl_entry *entries;
dma_addr_t dma;
--
git-series 0.9.1
[PATCH v4 03/11] media: vsp1: Rename dl_child to dl_next
Both vsp1_dl_list_commit() and __vsp1_dl_list_put() walk the display
list chain referencing the nodes as children, when in reality they are
siblings.
Update the terminology to 'dl_next' to be consistent with the
vsp1_video_pipeline_run() usage.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index f4cede9b9b43..ec6fc21fabe0 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -398,7 +398,7 @@ struct vsp1_dl_list *vsp1_dl_list_get(struct
vsp1_dl_manager *dlm)
/* This function must be called with the display list manager lock held.*/
static void __vsp1_dl_list_put(struct vsp1_dl_list *dl)
{
- struct vsp1_dl_list *dl_child;
+ struct vsp1_dl_list *dl_next;
if (!dl)
return;
@@ -408,8 +408,8 @@ static void __vsp1_dl_list_put(struct vsp1_dl_list *dl)
* hardware operation.
*/
if (dl->has_chain) {
- list_for_each_entry(dl_child, >chain, chain)
- __vsp1_dl_list_put(dl_child);
+ list_for_each_entry(dl_next, >chain, chain)
+ __vsp1_dl_list_put(dl_next);
}
dl->has_chain = false;
@@ -673,17 +673,17 @@ static void vsp1_dl_list_commit_singleshot(struct
vsp1_dl_list *dl)
void vsp1_dl_list_commit(struct vsp1_dl_list *dl, bool internal)
{
struct vsp1_dl_manager *dlm = dl->dlm;
- struct vsp1_dl_list *dl_child;
+ struct vsp1_dl_list *dl_next;
unsigned long flags;
if (dlm->mode == VSP1_DL_MODE_HEADER) {
/* Fill the header for the head and chained display lists. */
vsp1_dl_list_fill_header(dl, list_empty(>chain));
- list_for_each_entry(dl_child, >chain, chain) {
- bool last = list_is_last(_child->chain, >chain);
+ list_for_each_entry(dl_next, >chain, chain) {
+ bool last = list_is_last(_next->chain, >chain);
- vsp1_dl_list_fill_header(dl_child, last);
+ vsp1_dl_list_fill_header(dl_next, last);
}
}
--
git-series 0.9.1
[PATCH v4 02/11] media: vsp1: Remove packed attributes from aligned structures
The use of the packed attribute can cause a performance penalty for
all accesses to the struct members, as the compiler will assume that the
structure has the potential to have an unaligned base.
These structures are all correctly aligned and contain no holes, thus
the attribute is redundant and negatively impacts performance, so we
remove the attributes entirely.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v2
- Remove attributes entirely
---
drivers/media/platform/vsp1/vsp1_dl.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index c7fa1cb088cd..f4cede9b9b43 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -25,19 +25,19 @@
struct vsp1_dl_header_list {
u32 num_bytes;
u32 addr;
-} __attribute__((__packed__));
+};
struct vsp1_dl_header {
u32 num_lists;
struct vsp1_dl_header_list lists[8];
u32 next_header;
u32 flags;
-} __attribute__((__packed__));
+};
struct vsp1_dl_entry {
u32 addr;
u32 data;
-} __attribute__((__packed__));
+};
/**
* struct vsp1_dl_body - Display list body
--
git-series 0.9.1
[PATCH v4 10/11] media: vsp1: Support Interlaced display pipelines
Calculate the top and bottom fields for the interlaced frames and
utilise the extended display list command feature to implement the
auto-field operations. This allows the DU to update the VSP2 registers
dynamically based upon the currently processing field.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v3:
- Pass DL through partition calls to allow autocmd's to be retrieved
- Document interlaced field in struct vsp1_du_atomic_config
v2:
- fix erroneous BIT value which enabled interlaced
- fix field handling at frame_end interrupt
---
drivers/media/platform/vsp1/vsp1_dl.c | 10 -
drivers/media/platform/vsp1/vsp1_drm.c | 11 -
drivers/media/platform/vsp1/vsp1_regs.h | 1 +-
drivers/media/platform/vsp1/vsp1_rpf.c | 71 --
drivers/media/platform/vsp1/vsp1_rwpf.h | 1 +-
include/media/vsp1.h| 2 +-
6 files changed, 93 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index d33ae5f125bd..bbe9f3006f71 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -906,6 +906,8 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl, bool
internal)
*/
unsigned int vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
{
+ struct vsp1_device *vsp1 = dlm->vsp1;
+ u32 status = vsp1_read(vsp1, VI6_STATUS);
unsigned int flags = 0;
spin_lock(>lock);
@@ -931,6 +933,14 @@ unsigned int vsp1_dlm_irq_frame_end(struct vsp1_dl_manager
*dlm)
goto done;
/*
+* Progressive streams report only TOP fields. If we have a BOTTOM
+* field, we are interlaced, and expect the frame to complete on the
+* next frame end interrupt.
+*/
+ if (status & VI6_STATUS_FLD_STD(dlm->index))
+ goto done;
+
+ /*
* The device starts processing the queued display list right after the
* frame end interrupt. The display list thus becomes active.
*/
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c
b/drivers/media/platform/vsp1/vsp1_drm.c
index 2c3db8b8adce..cc29c9d96bb7 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -811,6 +811,17 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int
pipe_index,
return -EINVAL;
}
+ if (!(vsp1_feature(vsp1, VSP1_HAS_EXT_DL)) && cfg->interlaced) {
+ /*
+* Interlaced support requires extended display lists to
+* provide the auto-fld feature with the DU.
+*/
+ dev_dbg(vsp1->dev, "Interlaced unsupported on this output\n");
+ return -EINVAL;
+ }
+
+ rpf->interlaced = cfg->interlaced;
+
rpf->fmtinfo = fmtinfo;
rpf->format.num_planes = fmtinfo->planes;
rpf->format.plane_fmt[0].bytesperline = cfg->pitch;
diff --git a/drivers/media/platform/vsp1/vsp1_regs.h
b/drivers/media/platform/vsp1/vsp1_regs.h
index d054767570c1..a2ac65cc5155 100644
--- a/drivers/media/platform/vsp1/vsp1_regs.h
+++ b/drivers/media/platform/vsp1/vsp1_regs.h
@@ -28,6 +28,7 @@
#define VI6_SRESET_SRTS(n) (1 << (n))
#define VI6_STATUS 0x0038
+#define VI6_STATUS_FLD_STD(n) (1 << ((n) + 28))
#define VI6_STATUS_SYS_ACT(n) (1 << ((n) + 8))
#define VI6_WPF_IRQ_ENB(n) (0x0048 + (n) * 12)
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c
b/drivers/media/platform/vsp1/vsp1_rpf.c
index 8fae7c485642..511b2e127820 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -20,6 +20,20 @@
#define RPF_MAX_WIDTH 8190
#define RPF_MAX_HEIGHT 8190
+/* Pre extended display list command data structure */
+struct vsp1_extcmd_auto_fld_body {
+ u32 top_y0;
+ u32 bottom_y0;
+ u32 top_c0;
+ u32 bottom_c0;
+ u32 top_c1;
+ u32 bottom_c1;
+ u32 reserved0;
+ u32 reserved1;
+} __packed;
+
+#define VSP1_DL_EXT_AUTOFLD_INTBIT(0)
+
/*
-
* Device Access
*/
@@ -64,6 +78,14 @@ static void rpf_configure_stream(struct vsp1_entity *entity,
pstride |= format->plane_fmt[1].bytesperline
<< VI6_RPF_SRCM_PSTRIDE_C_SHIFT;
+ /*
+* pstride has both STRIDE_Y and STRIDE_C, but multiplying the whole
+* of pstride by 2 is conveniently OK here as we are multiplying both
+* values.
+*/
+ if (rpf->interlaced)
+ pstride *= 2;
+
vsp1_rpf_write(rpf, dlb, VI6_RPF_SRCM_PSTRIDE, pstride);
/* Format */
@@ -100,6 +122,9 @@ s
[PATCH v4 00/11] R-Car DU Interlaced support through VSP1
The Gen3 R-Car DU devices make use of the VSP to handle frame processing. In this series we implement support for handling interlaced pipelines by using the auto-fld feature of the VSP hardware. The implementation is preceded by some cleanup work and refactoring, through patches 1 to 6. These are trivial and could be collected earlier and independently if this series requires further revisions. Patch 7 makes a key distinctive change to remove all existing support for headerless display lists throughout the VSP1 driver, and ensures that all pipelines use the same code path. This simplifies the code and reduces opportunity for untested code paths to exist. Patches 8, 9 and 10 implement the relevant support in the VSP1 driver, before patch 11 finally enables the feature through the drm R-Car DU driver. This series is based upon my previous TLB optimise and body rework (v9), and is available from the following URL: git://git.kernel.org/pub/scm/linux/kernel/git/kbingham/rcar.git tags/vsp1/du/interlaced/v4 ChangeLog: v4: - Move configure_partition() call changes into tlb-optimise-v9 v3: - Rebased on top of TLB Optimise rework v8 - Added the DL parameter back into the configure_partition() calls. - This change could be moved into the TLB-Optimise series. - Document interlaced field in struct vsp1_du_atomic_config v2: - media: vsp1: use kernel __packed for structures became: media: vsp1: Remove packed attributes from aligned structures - media: vsp1: Add support for extended display list headers - No longer declares structs __packed - media: vsp1: Provide support for extended command pools - Fix spelling typo in commit message - constify, and staticify the instantiation of vsp1_extended_commands - s/autfld_cmds/autofld_cmds/ - staticify cmd pool functions (Thanks kbuild-bot) - media: vsp1: Support Interlaced display pipelines - fix erroneous BIT value which enabled interlaced - fix field handling at frame_end interrupt Kieran Bingham (11): media: vsp1: drm: Fix minor grammar error media: vsp1: Remove packed attributes from aligned structures media: vsp1: Rename dl_child to dl_next media: vsp1: Remove unused display list structure field media: vsp1: Clean up DLM objects on error media: vsp1: Provide VSP1 feature helper macro media: vsp1: Use header display lists for all WPF outputs linked to the DU media: vsp1: Add support for extended display list headers media: vsp1: Provide support for extended command pools media: vsp1: Support Interlaced display pipelines drm: rcar-du: Support interlaced video output through vsp1 drivers/gpu/drm/rcar-du/rcar_du_crtc.c | 1 +- drivers/gpu/drm/rcar-du/rcar_du_vsp.c | 3 +- drivers/media/platform/vsp1/vsp1.h | 3 +- drivers/media/platform/vsp1/vsp1_dl.c | 407 +++-- drivers/media/platform/vsp1/vsp1_dl.h | 32 +- drivers/media/platform/vsp1/vsp1_drm.c | 13 +- drivers/media/platform/vsp1/vsp1_drv.c | 23 +- drivers/media/platform/vsp1/vsp1_regs.h | 6 +- drivers/media/platform/vsp1/vsp1_rpf.c | 71 +++- drivers/media/platform/vsp1/vsp1_rwpf.h | 1 +- drivers/media/platform/vsp1/vsp1_wpf.c | 6 +- include/media/vsp1.h| 2 +- 12 files changed, 456 insertions(+), 112 deletions(-) base-commit: de53826416ea9c22ee985d1f0291aab7d7ea94ce -- git-series 0.9.1
[PATCH v4 07/11] media: vsp1: Use header display lists for all WPF outputs linked to the DU
Header mode display lists are now supported on all WPF outputs. To
support extended headers and auto-fld capabilities for interlaced mode
handling only header mode display lists can be used.
Disable the headerless display list configuration, and remove the dead
code.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 107 ++-
1 file changed, 27 insertions(+), 80 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index fbffbd407b29..56514cd51c51 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -94,7 +94,7 @@ struct vsp1_dl_body_pool {
* struct vsp1_dl_list - Display list
* @list: entry in the display list manager lists
* @dlm: the display list manager
- * @header: display list header, NULL for headerless lists
+ * @header: display list header
* @dma: DMA address for the header
* @body0: first display list body
* @bodies: list of extra display list bodies
@@ -118,15 +118,9 @@ struct vsp1_dl_list {
bool internal;
};
-enum vsp1_dl_mode {
- VSP1_DL_MODE_HEADER,
- VSP1_DL_MODE_HEADERLESS,
-};
-
/**
* struct vsp1_dl_manager - Display List manager
* @index: index of the related WPF
- * @mode: display list operation mode (header or headerless)
* @singleshot: execute the display list in single-shot mode
* @vsp1: the VSP1 device
* @lock: protects the free, active, queued, and pending lists
@@ -138,7 +132,6 @@ enum vsp1_dl_mode {
*/
struct vsp1_dl_manager {
unsigned int index;
- enum vsp1_dl_mode mode;
bool singleshot;
struct vsp1_device *vsp1;
@@ -318,6 +311,7 @@ void vsp1_dl_body_write(struct vsp1_dl_body *dlb, u32 reg,
u32 data)
static struct vsp1_dl_list *vsp1_dl_list_alloc(struct vsp1_dl_manager *dlm)
{
struct vsp1_dl_list *dl;
+ size_t header_offset;
dl = kzalloc(sizeof(*dl), GFP_KERNEL);
if (!dl)
@@ -330,16 +324,15 @@ static struct vsp1_dl_list *vsp1_dl_list_alloc(struct
vsp1_dl_manager *dlm)
dl->body0 = vsp1_dl_body_get(dlm->pool);
if (!dl->body0)
return NULL;
- if (dlm->mode == VSP1_DL_MODE_HEADER) {
- size_t header_offset = dl->body0->max_entries
-* sizeof(*dl->body0->entries);
- dl->header = ((void *)dl->body0->entries) + header_offset;
- dl->dma = dl->body0->dma + header_offset;
+ header_offset = dl->body0->max_entries
+* sizeof(*dl->body0->entries);
- memset(dl->header, 0, sizeof(*dl->header));
- dl->header->lists[0].addr = dl->body0->dma;
- }
+ dl->header = ((void *)dl->body0->entries) + header_offset;
+ dl->dma = dl->body0->dma + header_offset;
+
+ memset(dl->header, 0, sizeof(*dl->header));
+ dl->header->lists[0].addr = dl->body0->dma;
return dl;
}
@@ -471,16 +464,9 @@ struct vsp1_dl_body *vsp1_dl_list_get_body0(struct
vsp1_dl_list *dl)
*
* The reference must be explicitly released by a call to vsp1_dl_body_put()
* when the body isn't needed anymore.
- *
- * Additional bodies are only usable for display lists in header mode.
- * Attempting to add a body to a header-less display list will return an error.
*/
int vsp1_dl_list_add_body(struct vsp1_dl_list *dl, struct vsp1_dl_body *dlb)
{
- /* Multi-body lists are only available in header mode. */
- if (dl->dlm->mode != VSP1_DL_MODE_HEADER)
- return -EINVAL;
-
refcount_inc(>refcnt);
list_add_tail(>list, >bodies);
@@ -501,17 +487,10 @@ int vsp1_dl_list_add_body(struct vsp1_dl_list *dl, struct
vsp1_dl_body *dlb)
* Adding a display list to a chain passes ownership of the display list to
* the head display list item. The chain is released when the head dl item is
* put back with __vsp1_dl_list_put().
- *
- * Chained display lists are only usable in header mode. Attempts to add a
- * display list to a chain in header-less mode will return an error.
*/
int vsp1_dl_list_add_chain(struct vsp1_dl_list *head,
struct vsp1_dl_list *dl)
{
- /* Chained lists are only available in header mode. */
- if (head->dlm->mode != VSP1_DL_MODE_HEADER)
- return -EINVAL;
-
head->has_chain = true;
list_add_tail(>chain, >chain);
return 0;
@@ -579,17 +558,10 @@ static bool vsp1_dl_list_hw_update_pending(struct
vsp1_dl_manager *dlm)
return false;
/*
-* Check whether the VSP1 has taken the update. In headerless mode the
-* hardware indicates this by clearing the UPD bit in the DL_BODY_SIZE
-* register, and in header mode by clear
[PATCH v4 08/11] media: vsp1: Add support for extended display list headers
Extended display list headers allow pre and post command lists to be
executed by the VSP pipeline. This provides the base support for
features such as AUTO_FLD (for interlaced support) and AUTO_DISP (for
supporting continuous camera preview pipelines.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v2:
- remove __packed attributes
---
drivers/media/platform/vsp1/vsp1.h | 1 +-
drivers/media/platform/vsp1/vsp1_dl.c | 83 +-
drivers/media/platform/vsp1/vsp1_dl.h | 29 -
drivers/media/platform/vsp1/vsp1_drv.c | 7 +-
drivers/media/platform/vsp1/vsp1_regs.h | 5 +-
5 files changed, 116 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1.h
b/drivers/media/platform/vsp1/vsp1.h
index f0d21cc8e9ab..56c62122a81a 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -53,6 +53,7 @@ struct vsp1_uif;
#define VSP1_HAS_HGO (1 << 7)
#define VSP1_HAS_HGT (1 << 8)
#define VSP1_HAS_BRS (1 << 9)
+#define VSP1_HAS_EXT_DL(1 << 10)
struct vsp1_device_info {
u32 version;
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 56514cd51c51..b64d32535edc 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -22,6 +22,9 @@
#define VSP1_DLH_INT_ENABLE(1 << 1)
#define VSP1_DLH_AUTO_START(1 << 0)
+#define VSP1_DLH_EXT_PRE_CMD_EXEC (1 << 9)
+#define VSP1_DLH_EXT_POST_CMD_EXEC (1 << 8)
+
struct vsp1_dl_header_list {
u32 num_bytes;
u32 addr;
@@ -34,11 +37,34 @@ struct vsp1_dl_header {
u32 flags;
};
+struct vsp1_dl_ext_header {
+ u32 reserved0; /* alignment padding */
+
+ u16 pre_ext_cmd_qty;
+ u16 flags;
+ u32 pre_ext_cmd_plist;
+
+ u32 post_ext_cmd_qty;
+ u32 post_ext_cmd_plist;
+};
+
+struct vsp1_dl_header_extended {
+ struct vsp1_dl_header header;
+ struct vsp1_dl_ext_header ext;
+};
+
struct vsp1_dl_entry {
u32 addr;
u32 data;
};
+struct vsp1_dl_ext_cmd_header {
+ u32 cmd;
+ u32 flags;
+ u32 data;
+ u32 reserved;
+};
+
/**
* struct vsp1_dl_body - Display list body
* @list: entry in the display list list of bodies
@@ -95,9 +121,12 @@ struct vsp1_dl_body_pool {
* @list: entry in the display list manager lists
* @dlm: the display list manager
* @header: display list header
+ * @extended: extended display list header. NULL for normal lists
* @dma: DMA address for the header
* @body0: first display list body
* @bodies: list of extra display list bodies
+ * @pre_cmd: pre cmd to be issued through extended dl header
+ * @post_cmd: post cmd to be issued through extended dl header
* @has_chain: if true, indicates that there's a partition chain
* @chain: entry in the display list partition chain
* @internal: whether the display list is used for internal purpose
@@ -107,11 +136,15 @@ struct vsp1_dl_list {
struct vsp1_dl_manager *dlm;
struct vsp1_dl_header *header;
+ struct vsp1_dl_ext_header *extended;
dma_addr_t dma;
struct vsp1_dl_body *body0;
struct list_head bodies;
+ struct vsp1_dl_ext_cmd *pre_cmd;
+ struct vsp1_dl_ext_cmd *post_cmd;
+
bool has_chain;
struct list_head chain;
@@ -496,6 +529,14 @@ int vsp1_dl_list_add_chain(struct vsp1_dl_list *head,
return 0;
}
+static void vsp1_dl_ext_cmd_fill_header(struct vsp1_dl_ext_cmd *cmd)
+{
+ cmd->cmds[0].cmd = cmd->cmd_opcode;
+ cmd->cmds[0].flags = cmd->flags;
+ cmd->cmds[0].data = cmd->data_dma;
+ cmd->cmds[0].reserved = 0;
+}
+
static void vsp1_dl_list_fill_header(struct vsp1_dl_list *dl, bool is_last)
{
struct vsp1_dl_manager *dlm = dl->dlm;
@@ -548,6 +589,27 @@ static void vsp1_dl_list_fill_header(struct vsp1_dl_list
*dl, bool is_last)
*/
dl->header->flags = VSP1_DLH_INT_ENABLE;
}
+
+ if (!dl->extended)
+ return;
+
+ dl->extended->flags = 0;
+
+ if (dl->pre_cmd) {
+ dl->extended->pre_ext_cmd_plist = dl->pre_cmd->cmd_dma;
+ dl->extended->pre_ext_cmd_qty = dl->pre_cmd->num_cmds;
+ dl->extended->flags |= VSP1_DLH_EXT_PRE_CMD_EXEC;
+
+ vsp1_dl_ext_cmd_fill_header(dl->pre_cmd);
+ }
+
+ if (dl->post_cmd) {
+ dl->extended->pre_ext_cmd_plist = dl->post_cmd->cmd_dma;
+ dl->extended->pre_ext_cmd_qty = dl->post_cmd->num_cmds;
+ dl->extended->flags |= VSP1_DLH_EXT_POST_CMD_EXEC;
+
+ vsp1_dl_ext_cmd_fill_header(dl->pre_cmd);
+ }
}
static bool
[PATCH v4 09/11] media: vsp1: Provide support for extended command pools
VSPD and VSP-DL devices can provide extended display lists supporting
extended command display list objects.
These extended commands require their own dma memory areas for a header
and body specific to the command type.
Implement a command pool to allocate all necessary memory in a single
DMA allocation to reduce pressure on the TLB, and provide convenient
re-usable command objects for the entities to utilise.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v2:
- Fix spelling typo in commit message
- constify, and staticify the instantiation of vsp1_extended_commands
- s/autfld_cmds/autofld_cmds/
- staticify cmd pool functions (Thanks kbuild-bot)
---
drivers/media/platform/vsp1/vsp1_dl.c | 191 +++-
drivers/media/platform/vsp1/vsp1_dl.h | 3 +-
2 files changed, 194 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index b64d32535edc..d33ae5f125bd 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -117,6 +117,30 @@ struct vsp1_dl_body_pool {
};
/**
+ * struct vsp1_cmd_pool - display list body pool
+ * @dma: DMA address of the entries
+ * @size: size of the full DMA memory pool in bytes
+ * @mem: CPU memory pointer for the pool
+ * @bodies: Array of DLB structures for the pool
+ * @free: List of free DLB entries
+ * @lock: Protects the pool and free list
+ * @vsp1: the VSP1 device
+ */
+struct vsp1_dl_cmd_pool {
+ /* DMA allocation */
+ dma_addr_t dma;
+ size_t size;
+ void *mem;
+
+ struct vsp1_dl_ext_cmd *cmds;
+ struct list_head free;
+
+ spinlock_t lock;
+
+ struct vsp1_device *vsp1;
+};
+
+/**
* struct vsp1_dl_list - Display list
* @list: entry in the display list manager lists
* @dlm: the display list manager
@@ -162,6 +186,7 @@ struct vsp1_dl_list {
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
* @pool: body pool for the display list bodies
+ * @autofld_cmds: command pool to support auto-fld interlaced mode
*/
struct vsp1_dl_manager {
unsigned int index;
@@ -175,6 +200,7 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *pending;
struct vsp1_dl_body_pool *pool;
+ struct vsp1_dl_cmd_pool *autofld_cmds;
};
/*
-
@@ -338,6 +364,140 @@ void vsp1_dl_body_write(struct vsp1_dl_body *dlb, u32
reg, u32 data)
}
/*
-
+ * Display List Extended Command Management
+ */
+
+enum vsp1_extcmd_type {
+ VSP1_EXTCMD_AUTODISP,
+ VSP1_EXTCMD_AUTOFLD,
+};
+
+struct vsp1_extended_command_info {
+ u16 opcode;
+ size_t body_size;
+} static const vsp1_extended_commands[] = {
+ [VSP1_EXTCMD_AUTODISP] = { 0x02, 96 },
+ [VSP1_EXTCMD_AUTOFLD] = { 0x03, 160 },
+};
+
+/**
+ * vsp1_dl_cmd_pool_create - Create a pool of commands from a single allocation
+ * @vsp1: The VSP1 device
+ * @type: The command pool type
+ * @num_commands: The quantity of commands to allocate
+ *
+ * Allocate a pool of commands each with enough memory to contain the private
+ * data of each command. The allocation sizes are dependent upon the command
+ * type.
+ *
+ * Return a pointer to a pool on success or NULL if memory can't be allocated.
+ */
+static struct vsp1_dl_cmd_pool *
+vsp1_dl_cmd_pool_create(struct vsp1_device *vsp1, enum vsp1_extcmd_type type,
+ unsigned int num_cmds)
+{
+ struct vsp1_dl_cmd_pool *pool;
+ unsigned int i;
+ size_t cmd_size;
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool)
+ return NULL;
+
+ pool->cmds = kcalloc(num_cmds, sizeof(*pool->cmds), GFP_KERNEL);
+ if (!pool->cmds) {
+ kfree(pool);
+ return NULL;
+ }
+
+ cmd_size = sizeof(struct vsp1_dl_ext_cmd_header) +
+ vsp1_extended_commands[type].body_size;
+ cmd_size = ALIGN(cmd_size, 16);
+
+ pool->size = cmd_size * num_cmds;
+ pool->mem = dma_alloc_wc(vsp1->bus_master, pool->size, >dma,
+GFP_KERNEL);
+ if (!pool->mem) {
+ kfree(pool->cmds);
+ kfree(pool);
+ return NULL;
+ }
+
+ spin_lock_init(>lock);
+ INIT_LIST_HEAD(>free);
+
+ for (i = 0; i < num_cmds; ++i) {
+ struct vsp1_dl_ext_cmd *cmd = >cmds[i];
+ size_t cmd_offset = i * cmd_size;
+ size_t data_offset = sizeof(struct vsp1_dl_ext_cmd_header) +
+cmd_offset;
+
+ cmd->pool = pool;
+ cmd->cmd_opcode = vsp1_extended_commands[type].opcode;
+
+ /* TOD
[PATCH v4 01/11] media: vsp1: drm: Fix minor grammar error
The pixel format is 'unsupported'. Fix the small debug message which
incorrectly declares this.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_drm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c
b/drivers/media/platform/vsp1/vsp1_drm.c
index ef0148082bf7..2c3db8b8adce 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -806,7 +806,7 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int
pipe_index,
*/
fmtinfo = vsp1_get_format_info(vsp1, cfg->pixelformat);
if (!fmtinfo) {
- dev_dbg(vsp1->dev, "Unsupport pixel format %08x for RPF\n",
+ dev_dbg(vsp1->dev, "Unsupported pixel format %08x for RPF\n",
cfg->pixelformat);
return -EINVAL;
}
--
git-series 0.9.1
[PATCH v4 05/11] media: vsp1: Clean up DLM objects on error
If there is an error allocating a display list within a DLM object
the existing display lists are not free'd, and neither is the DL body
pool.
Use the existing vsp1_dlm_destroy() function to clean up on error.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
drivers/media/platform/vsp1/vsp1_dl.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index b23e88cda49f..fbffbd407b29 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -851,8 +851,10 @@ struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device
*vsp1,
struct vsp1_dl_list *dl;
dl = vsp1_dl_list_alloc(dlm);
- if (!dl)
+ if (!dl) {
+ vsp1_dlm_destroy(dlm);
return NULL;
+ }
list_add_tail(>list, >free);
}
--
git-series 0.9.1
[PATCH v4 11/11] drm: rcar-du: Support interlaced video output through vsp1
Use the newly exposed VSP1 interface to enable interlaced frame support through the VSP1 lif pipelines. Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com> --- drivers/gpu/drm/rcar-du/rcar_du_crtc.c | 1 + drivers/gpu/drm/rcar-du/rcar_du_vsp.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c index d71d709fe3d9..206532959ec9 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c @@ -289,6 +289,7 @@ static void rcar_du_crtc_set_display_timing(struct rcar_du_crtc *rcrtc) /* Signal polarities */ value = ((mode->flags & DRM_MODE_FLAG_PVSYNC) ? DSMR_VSL : 0) | ((mode->flags & DRM_MODE_FLAG_PHSYNC) ? DSMR_HSL : 0) + | ((mode->flags & DRM_MODE_FLAG_INTERLACE) ? DSMR_ODEV : 0) | DSMR_DIPM_DISP | DSMR_CSPM; rcar_du_crtc_write(rcrtc, DSMR, value); diff --git a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c index af7822a66dee..c7b37232ee91 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c @@ -186,6 +186,9 @@ static void rcar_du_vsp_plane_setup(struct rcar_du_vsp_plane *plane) }; unsigned int i; + cfg.interlaced = !!(plane->plane.state->crtc->mode.flags + & DRM_MODE_FLAG_INTERLACE); + cfg.src.left = state->state.src.x1 >> 16; cfg.src.top = state->state.src.y1 >> 16; cfg.src.width = drm_rect_width(>state.src) >> 16; -- git-series 0.9.1
[PATCH v9 8/8] media: vsp1: Move video configuration to a cached dlb
We are now able to configure a pipeline directly into a local display
list body. Take advantage of this fact, and create a cacheable body to
store the configuration of the pipeline in the video object.
vsp1_video_pipeline_run() is now the last user of the pipe->dl object.
Convert this function to use the cached video->config body and obtain a
local display list reference.
Attach the video->config body to the display list when needed before
committing to hardware.
The pipe object is marked as un-configured when resuming from a suspend.
This ensures that when the hardware is reset - our cached configuration
will be re-attached to the next committed DL.
Our video DL usage now looks like the below output:
dl->body0 contains our disposable runtime configuration. Max 41.
dl_child->body0 is our partition specific configuration. Max 12.
dl->bodies shows our constant configuration and LUTs.
These two are LUT/CLU:
* dl->bodies[x]->num_entries 256 / max 256
* dl->bodies[x]->num_entries 4914 / max 4914
Which shows that our 'constant' configuration cache is currently
utilised to a maximum of 64 entries.
trace-cmd report | \
grep max | sed 's/.*vsp1_dl_list_commit://g' | sort | uniq;
dl->body0->num_entries 13 / max 128
dl->body0->num_entries 14 / max 128
dl->body0->num_entries 16 / max 128
dl->body0->num_entries 20 / max 128
dl->body0->num_entries 27 / max 128
dl->body0->num_entries 34 / max 128
dl->body0->num_entries 41 / max 128
dl_child->body0->num_entries 10 / max 128
dl_child->body0->num_entries 12 / max 128
dl->bodies[x]->num_entries 15 / max 128
dl->bodies[x]->num_entries 16 / max 128
dl->bodies[x]->num_entries 17 / max 128
dl->bodies[x]->num_entries 18 / max 128
dl->bodies[x]->num_entries 20 / max 128
dl->bodies[x]->num_entries 21 / max 128
dl->bodies[x]->num_entries 256 / max 256
dl->bodies[x]->num_entries 31 / max 128
dl->bodies[x]->num_entries 32 / max 128
dl->bodies[x]->num_entries 39 / max 128
dl->bodies[x]->num_entries 40 / max 128
dl->bodies[x]->num_entries 47 / max 128
dl->bodies[x]->num_entries 48 / max 128
dl->bodies[x]->num_entries 4914 / max 4914
dl->bodies[x]->num_entries 55 / max 128
dl->bodies[x]->num_entries 56 / max 128
dl->bodies[x]->num_entries 63 / max 128
dl->bodies[x]->num_entries 64 / max 128
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v8:
- Fix comments
- Rename video->pipe_config -> video->stream_config
v3:
- 's/fragment/body/', 's/fragments/bodies/'
- video dlb cache allocation increased from 2 to 3 dlbs
v4:
- Adjust pipe configured flag to be reset on resume rather than suspend
- rename dl_child, dl_next
---
drivers/media/platform/vsp1/vsp1_pipe.c | 9 +++-
drivers/media/platform/vsp1/vsp1_pipe.h | 5 +--
drivers/media/platform/vsp1/vsp1_video.c | 72 +++--
drivers/media/platform/vsp1/vsp1_video.h | 2 +-
4 files changed, 59 insertions(+), 29 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c
b/drivers/media/platform/vsp1/vsp1_pipe.c
index d06ffa01027c..70af6c9fc97f 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -230,6 +230,7 @@ void vsp1_pipeline_run(struct vsp1_pipeline *pipe)
vsp1_write(vsp1, VI6_CMD(pipe->output->entity.index),
VI6_CMD_STRCMD);
pipe->state = VSP1_PIPELINE_RUNNING;
+ pipe->configured = true;
}
pipe->buffers_ready = 0;
@@ -295,6 +296,8 @@ int vsp1_pipeline_stop(struct vsp1_pipeline *pipe)
v4l2_subdev_call(>output->entity.subdev, video, s_stream, 0);
+ pipe->configured = false;
+
return ret;
}
@@ -451,6 +454,12 @@ void vsp1_pipelines_resume(struct vsp1_device *vsp1)
continue;
spin_lock_irqsave(>irqlock, flags);
+ /*
+* The hardware may have been reset during a suspend and will
+* need a full reconfiguration.
+*/
+ pipe->configured = false;
+
if (vsp1_pipeline_ready(pipe))
vsp1_pipeline_run(pipe);
spin_unlock_irqrestore(>irqlock, flags);
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h
b/drivers/media/platform/vsp1/vsp1_pipe.h
index e00010693eef..7c8b30018aac 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -86,6 +86,7 @@ struct vsp1_partition {
* @irqlock: protects the pipeline state
* @state: current state
* @wq: wait queue to wait for state change completion
+ * @configured: flag determining if the hardware has run since reset
* @frame_end: frame end interrupt
[PATCH v9 5/8] media: vsp1: Use reference counting for bodies
Extend the display list body with a reference count, allowing bodies to
be kept as long as a reference is maintained. This provides the ability
to keep a cached copy of bodies which will not change, so that they can
be re-applied to multiple display lists.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
This could be squashed into the body update code, but it's not a
straightforward squash as the refcounts will affect both:
v4l: vsp1: Provide a body pool
and
v4l: vsp1: Convert display lists to use new body pool
therefore, I have kept this separate to prevent breaking bisectability
of the vsp-tests.
v3:
- 's/fragment/body/'
v4:
- Fix up reference handling comments.
Changes since v7:
- Fix comment style
---
drivers/media/platform/vsp1/vsp1_clu.c | 7 ++-
drivers/media/platform/vsp1/vsp1_dl.c | 16 ++--
drivers/media/platform/vsp1/vsp1_lut.c | 7 ++-
3 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index 8efa12f5e53f..ea83f1b7d125 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -212,8 +212,13 @@ static void clu_configure(struct vsp1_entity *entity,
clu->clu = NULL;
spin_unlock_irqrestore(>lock, flags);
- if (dlb)
+ if (dlb) {
vsp1_dl_list_add_body(dl, dlb);
+
+ /* Release our local reference. */
+ vsp1_dl_body_put(dlb);
+ }
+
break;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 617c46a03dec..1407c90c6880 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -10,6 +10,7 @@
#include
#include
#include
+#include
#include
#include
@@ -54,6 +55,8 @@ struct vsp1_dl_body {
struct list_head list;
struct list_head free;
+ refcount_t refcnt;
+
struct vsp1_dl_body_pool *pool;
struct vsp1_device *vsp1;
@@ -258,6 +261,7 @@ struct vsp1_dl_body *vsp1_dl_body_get(struct
vsp1_dl_body_pool *pool)
if (!list_empty(>free)) {
dlb = list_first_entry(>free, struct vsp1_dl_body, free);
list_del(>free);
+ refcount_set(>refcnt, 1);
}
spin_unlock_irqrestore(>lock, flags);
@@ -278,6 +282,9 @@ void vsp1_dl_body_put(struct vsp1_dl_body *dlb)
if (!dlb)
return;
+ if (!refcount_dec_and_test(>refcnt))
+ return;
+
dlb->num_entries = 0;
spin_lock_irqsave(>pool->lock, flags);
@@ -463,8 +470,11 @@ void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg,
u32 data)
* which bodies are added.
*
* Adding a body to a display list passes ownership of the body to the list.
The
- * caller must not touch the body after this call, and must not release it
- * explicitly with vsp1_dl_body_put().
+ * caller retains its reference to the fragment when adding it to the display
+ * list, but is not allowed to add new entries to the body.
+ *
+ * The reference must be explicitly released by a call to vsp1_dl_body_put()
+ * when the body isn't needed anymore.
*
* Additional bodies are only usable for display lists in header mode.
* Attempting to add a body to a header-less display list will return an error.
@@ -475,6 +485,8 @@ int vsp1_dl_list_add_body(struct vsp1_dl_list *dl, struct
vsp1_dl_body *dlb)
if (dl->dlm->mode != VSP1_DL_MODE_HEADER)
return -EINVAL;
+ refcount_inc(>refcnt);
+
list_add_tail(>list, >bodies);
return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c
b/drivers/media/platform/vsp1/vsp1_lut.c
index 6b358617ce15..b3ea90172439 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -168,8 +168,13 @@ static void lut_configure(struct vsp1_entity *entity,
lut->lut = NULL;
spin_unlock_irqrestore(>lock, flags);
- if (dlb)
+ if (dlb) {
vsp1_dl_list_add_body(dl, dlb);
+
+ /* Release our local reference. */
+ vsp1_dl_body_put(dlb);
+ }
+
break;
}
}
--
git-series 0.9.1
[PATCH v9 2/8] media: vsp1: Protect bodies against overflow
The body write function relies on the code never asking it to write more
than the entries available in the list.
Currently with each list body containing 256 entries, this is fine, but
we can reduce this number greatly saving memory. In preparation of this
add a level of protection to catch any buffer overflows.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinch...@ideasonboard.com>
---
v3:
- adapt for new 'body' terminology
- simplify WARN_ON macro usage
---
drivers/media/platform/vsp1/vsp1_dl.c | 7 +++
1 file changed, 7 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 083da4f05c20..51965c30dec2 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -46,6 +46,7 @@ struct vsp1_dl_entry {
* @dma: DMA address of the entries
* @size: size of the DMA memory in bytes
* @num_entries: number of stored entries
+ * @max_entries: number of entries available
*/
struct vsp1_dl_body {
struct list_head list;
@@ -56,6 +57,7 @@ struct vsp1_dl_body {
size_t size;
unsigned int num_entries;
+ unsigned int max_entries;
};
/**
@@ -138,6 +140,7 @@ static int vsp1_dl_body_init(struct vsp1_device *vsp1,
dlb->vsp1 = vsp1;
dlb->size = size;
+ dlb->max_entries = num_entries;
dlb->entries = dma_alloc_wc(vsp1->bus_master, dlb->size, >dma,
GFP_KERNEL);
@@ -219,6 +222,10 @@ void vsp1_dl_body_free(struct vsp1_dl_body *dlb)
*/
void vsp1_dl_body_write(struct vsp1_dl_body *dlb, u32 reg, u32 data)
{
+ if (WARN_ONCE(dlb->num_entries >= dlb->max_entries,
+ "DLB size exceeded (max %u)", dlb->max_entries))
+ return;
+
dlb->entries[dlb->num_entries].addr = reg;
dlb->entries[dlb->num_entries].data = data;
dlb->num_entries++;
--
git-series 0.9.1
[PATCH v9 7/8] media: vsp1: Adapt entities to configure into a body
Currently the entities store their configurations into a display list.
Adapt this such that the code can be configured into a body directly,
allowing greater flexibility and control of the content.
All users of vsp1_dl_list_write() are removed in this process, thus it
too is removed.
A helper, vsp1_dl_list_get_body0() is provided to access the internal body0
from the display list.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
v9:
- Pass the DL through configure_partition() calls
v8:
- Fixed comment style and indentation
- Supported UIF
- Supported new configure_partition() functionality
v7:
- Rebase
- s/prepare/configure_stream/
- s/configure/configure_frame/
---
drivers/media/platform/vsp1/vsp1_brx.c| 22 ++--
drivers/media/platform/vsp1/vsp1_clu.c| 23 ++---
drivers/media/platform/vsp1/vsp1_dl.c | 12 ++-
drivers/media/platform/vsp1/vsp1_dl.h | 2 +-
drivers/media/platform/vsp1/vsp1_drm.c| 12 ---
drivers/media/platform/vsp1/vsp1_entity.c | 22 ++--
drivers/media/platform/vsp1/vsp1_entity.h | 18 ++
drivers/media/platform/vsp1/vsp1_hgo.c| 16 -
drivers/media/platform/vsp1/vsp1_hgt.c| 18 +-
drivers/media/platform/vsp1/vsp1_hsit.c | 10 ++---
drivers/media/platform/vsp1/vsp1_lif.c| 15
drivers/media/platform/vsp1/vsp1_lut.c| 23 ++---
drivers/media/platform/vsp1/vsp1_pipe.c | 4 +-
drivers/media/platform/vsp1/vsp1_pipe.h | 3 +-
drivers/media/platform/vsp1/vsp1_rpf.c| 44
drivers/media/platform/vsp1/vsp1_sru.c| 14
drivers/media/platform/vsp1/vsp1_uds.c| 25 +++---
drivers/media/platform/vsp1/vsp1_uds.h| 2 +-
drivers/media/platform/vsp1/vsp1_uif.c| 21 +--
drivers/media/platform/vsp1/vsp1_video.c | 16 ++---
drivers/media/platform/vsp1/vsp1_wpf.c| 42 ---
21 files changed, 194 insertions(+), 170 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_brx.c
b/drivers/media/platform/vsp1/vsp1_brx.c
index 011edac5ebc1..359917b5d842 100644
--- a/drivers/media/platform/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/vsp1/vsp1_brx.c
@@ -26,10 +26,10 @@
* Device Access
*/
-static inline void vsp1_brx_write(struct vsp1_brx *brx, struct vsp1_dl_list
*dl,
- u32 reg, u32 data)
+static inline void vsp1_brx_write(struct vsp1_brx *brx,
+ struct vsp1_dl_body *dlb, u32 reg, u32 data)
{
- vsp1_dl_list_write(dl, brx->base + reg, data);
+ vsp1_dl_body_write(dlb, brx->base + reg, data);
}
/*
-
@@ -283,7 +283,7 @@ static const struct v4l2_subdev_ops brx_ops = {
static void brx_configure_stream(struct vsp1_entity *entity,
struct vsp1_pipeline *pipe,
-struct vsp1_dl_list *dl)
+struct vsp1_dl_body *dlb)
{
struct vsp1_brx *brx = to_brx(>subdev);
struct v4l2_mbus_framefmt *format;
@@ -305,7 +305,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
* format at the pipeline output is premultiplied.
*/
flags = pipe->output ? pipe->output->format.flags : 0;
- vsp1_brx_write(brx, dl, VI6_BRU_INCTRL,
+ vsp1_brx_write(brx, dlb, VI6_BRU_INCTRL,
flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
0 : VI6_BRU_INCTRL_NRM);
@@ -313,12 +313,12 @@ static void brx_configure_stream(struct vsp1_entity
*entity,
* Set the background position to cover the whole output image and
* configure its color.
*/
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_SIZE,
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_SIZE,
(format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
(format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_LOC, 0);
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_LOC, 0);
- vsp1_brx_write(brx, dl, VI6_BRU_VIRRPF_COL, brx->bgcolor |
+ vsp1_brx_write(brx, dlb, VI6_BRU_VIRRPF_COL, brx->bgcolor |
(0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
/*
@@ -328,7 +328,7 @@ static void brx_configure_stream(struct vsp1_entity *entity,
* unit.
*/
if (entity->type == VSP1_ENTITY_BRU)
- vsp1_brx_write(brx, dl, VI6_BRU_ROP,
+ vsp1_brx_write(brx, dlb, VI6_BRU_ROP,
VI6_BRU_ROP_DSTSEL_BRUIN(1) |
VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
VI6_BRU_ROP_AROP(VI6_ROP_NOP));
@@ -370,7 +370,7 @@ static void brx_config
[PATCH v9 3/8] media: vsp1: Provide a body pool
Each display list allocates a body to store register values in a dma
accessible buffer from a dma_alloc_wc() allocation. Each of these
results in an entry in the IOMMU TLB, and a large number of display list
allocations adds pressure to this resource.
Reduce TLB pressure on the IPMMUs by allocating multiple display list
bodies in a single allocation, and providing these to the display list
through a 'body pool'. A pool can be allocated by the display list
manager or entities which require their own body allocations.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v8:
- Update commit message
- Fix comments and descriptions
v4:
- Provide comment explaining extra allocation on body pool
highlighting area for optimisation later.
v3:
- s/fragment/body/, s/fragments/bodies/
- qty -> num_bodies
- indentation fix
- s/vsp1_dl_body_pool_{alloc,free}/vsp1_dl_body_pool_{create,destroy}/'
- Add kerneldoc to non-static functions
v2:
- assign dlb->dma correctly
---
drivers/media/platform/vsp1/vsp1_dl.c | 163 +++-
drivers/media/platform/vsp1/vsp1_dl.h | 8 +-
2 files changed, 171 insertions(+)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 51965c30dec2..41ace89a585b 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -41,6 +41,8 @@ struct vsp1_dl_entry {
/**
* struct vsp1_dl_body - Display list body
* @list: entry in the display list list of bodies
+ * @free: entry in the pool free body list
+ * @pool: pool to which this body belongs
* @vsp1: the VSP1 device
* @entries: array of entries
* @dma: DMA address of the entries
@@ -50,6 +52,9 @@ struct vsp1_dl_entry {
*/
struct vsp1_dl_body {
struct list_head list;
+ struct list_head free;
+
+ struct vsp1_dl_body_pool *pool;
struct vsp1_device *vsp1;
struct vsp1_dl_entry *entries;
@@ -61,6 +66,30 @@ struct vsp1_dl_body {
};
/**
+ * struct vsp1_dl_body_pool - display list body pool
+ * @dma: DMA address of the entries
+ * @size: size of the full DMA memory pool in bytes
+ * @mem: CPU memory pointer for the pool
+ * @bodies: Array of DLB structures for the pool
+ * @free: List of free DLB entries
+ * @lock: Protects the free list
+ * @vsp1: the VSP1 device
+ */
+struct vsp1_dl_body_pool {
+ /* DMA allocation */
+ dma_addr_t dma;
+ size_t size;
+ void *mem;
+
+ /* Body management */
+ struct vsp1_dl_body *bodies;
+ struct list_head free;
+ spinlock_t lock;
+
+ struct vsp1_device *vsp1;
+};
+
+/**
* struct vsp1_dl_list - Display list
* @list: entry in the display list manager lists
* @dlm: the display list manager
@@ -104,6 +133,7 @@ enum vsp1_dl_mode {
* @active: list currently being processed (loaded) by hardware
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
+ * @pool: body pool for the display list bodies
* @gc_work: bodies garbage collector work struct
* @gc_bodies: array of display list bodies waiting to be freed
*/
@@ -119,6 +149,8 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *queued;
struct vsp1_dl_list *pending;
+ struct vsp1_dl_body_pool *pool;
+
struct work_struct gc_work;
struct list_head gc_bodies;
};
@@ -127,6 +159,137 @@ struct vsp1_dl_manager {
* Display List Body Management
*/
+/**
+ * vsp1_dl_body_pool_create - Create a pool of bodies from a single allocation
+ * @vsp1: The VSP1 device
+ * @num_bodies: The number of bodies to allocate
+ * @num_entries: The maximum number of entries that a body can contain
+ * @extra_size: Extra allocation provided for the bodies
+ *
+ * Allocate a pool of display list bodies each with enough memory to contain
the
+ * requested number of entries plus the @extra_size.
+ *
+ * Return a pointer to a pool on success or NULL if memory can't be allocated.
+ */
+struct vsp1_dl_body_pool *
+vsp1_dl_body_pool_create(struct vsp1_device *vsp1, unsigned int num_bodies,
+unsigned int num_entries, size_t extra_size)
+{
+ struct vsp1_dl_body_pool *pool;
+ size_t dlb_size;
+ unsigned int i;
+
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool)
+ return NULL;
+
+ pool->vsp1 = vsp1;
+
+ /*
+* TODO: 'extra_size' is only used by vsp1_dlm_create(), to allocate
+* extra memory for the display list header. We need only one header per
+* display list, not per display list body, thus this allocation is
+* extraneous and should be reworked in the future.
+*/
+ dlb_size = num_entries * sizeof(struct vsp1_dl_entry) + extra_size;
+ pool->size = dlb_size * num_bodies;
+
+ pool->bodies = kcalloc(num_bodies, sizeof(*pool->bodies), GFP_KERNEL);
+ if (!pool->bodies) {
[PATCH v9 6/8] media: vsp1: Refactor display list configure operations
The entities provide a single .configure operation which configures the
object into the target display list, based on the vsp1_entity_params
selection.
Split the configure function into three parts, '.configure_stream()',
'.configure_frame()', and '.configure_partition()' to facilitate
splitting the configuration of each parameter class into separate
display list bodies.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
The checkpatch warning:
WARNING: function definition argument 'struct vsp1_dl_list *' should
also have an identifier name
has been ignored to match the existing code style.
v8:
- Add support for the UIF
- Remove unrelated whitespace change
- Fix comment location for clu_configure_stream()
- Update configure documentations
- Implement configure_partition separation.
v7
- Fix formatting and white space
- s/prepare/configure_stream/
- s/configure/configure_frame/
---
drivers/media/platform/vsp1/vsp1_brx.c| 12 +-
drivers/media/platform/vsp1/vsp1_clu.c| 77 ++
drivers/media/platform/vsp1/vsp1_drm.c| 12 +-
drivers/media/platform/vsp1/vsp1_entity.c | 24 ++-
drivers/media/platform/vsp1/vsp1_entity.h | 39 +--
drivers/media/platform/vsp1/vsp1_hgo.c| 12 +-
drivers/media/platform/vsp1/vsp1_hgt.c| 12 +-
drivers/media/platform/vsp1/vsp1_hsit.c | 12 +-
drivers/media/platform/vsp1/vsp1_lif.c| 12 +-
drivers/media/platform/vsp1/vsp1_lut.c| 47 +---
drivers/media/platform/vsp1/vsp1_rpf.c| 168 ++---
drivers/media/platform/vsp1/vsp1_sru.c| 12 +-
drivers/media/platform/vsp1/vsp1_uds.c| 56 ++--
drivers/media/platform/vsp1/vsp1_uif.c| 16 +-
drivers/media/platform/vsp1/vsp1_video.c | 28 +--
drivers/media/platform/vsp1/vsp1_wpf.c| 303 ---
16 files changed, 422 insertions(+), 420 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_brx.c
b/drivers/media/platform/vsp1/vsp1_brx.c
index 3beec18fd863..011edac5ebc1 100644
--- a/drivers/media/platform/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/vsp1/vsp1_brx.c
@@ -281,19 +281,15 @@ static const struct v4l2_subdev_ops brx_ops = {
* VSP1 Entity Operations
*/
-static void brx_configure(struct vsp1_entity *entity,
- struct vsp1_pipeline *pipe,
- struct vsp1_dl_list *dl,
- enum vsp1_entity_params params)
+static void brx_configure_stream(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl)
{
struct vsp1_brx *brx = to_brx(>subdev);
struct v4l2_mbus_framefmt *format;
unsigned int flags;
unsigned int i;
- if (params != VSP1_ENTITY_PARAMS_INIT)
- return;
-
format = vsp1_entity_get_pad_format(>entity, brx->entity.config,
brx->entity.source_pad);
@@ -400,7 +396,7 @@ static void brx_configure(struct vsp1_entity *entity,
}
static const struct vsp1_entity_operations brx_entity_ops = {
- .configure = brx_configure,
+ .configure_stream = brx_configure_stream,
};
/*
-
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index ea83f1b7d125..0a978980d447 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -168,58 +168,50 @@ static const struct v4l2_subdev_ops clu_ops = {
/*
-
* VSP1 Entity Operations
*/
+static void clu_configure_stream(struct vsp1_entity *entity,
+struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl)
+{
+ struct vsp1_clu *clu = to_clu(>subdev);
+ struct v4l2_mbus_framefmt *format;
-static void clu_configure(struct vsp1_entity *entity,
- struct vsp1_pipeline *pipe,
- struct vsp1_dl_list *dl,
- enum vsp1_entity_params params)
+ /*
+* The yuv_mode can't be changed during streaming. Cache it internally
+* for future runtime configuration calls.
+*/
+ format = vsp1_entity_get_pad_format(>entity,
+ clu->entity.config,
+ CLU_PAD_SINK);
+ clu->yuv_mode = format->code == MEDIA_BUS_FMT_AYUV8_1X32;
+}
+
+static void clu_configure_frame(struct vsp1_entity *entity,
+ struct vsp1_pipeline *pipe,
+ struct vsp1_dl_list *dl)
{
struct vsp1_clu *clu = to_clu(>subdev);
struct vsp1_dl_body *dlb;
unsigned long flags;
u32 ctrl = VI6_CLU_CTRL_AAI | VI6_CLU_CTRL_MVS | VI6_CLU_CTRL_EN;
-
[PATCH v9 4/8] media: vsp1: Convert display lists to use new body pool
Adapt the dl->body0 object to use an object from the body pool. This
greatly reduces the pressure on the TLB for IPMMU use cases, as all of
the lists use a single allocation for the main body.
The CLU and LUT objects pre-allocate a pool containing three bodies,
allowing a userspace update before the hardware has committed a previous
set of tables.
Bodies are no longer 'freed' in interrupt context, but instead released
back to their respective pools. This allows us to remove the garbage
collector in the DLM.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v9:
- Remove redundant reference to gc_bodies
v8:
- Don't pass dlm->pool through vsp1_dl_list_alloc() as it's already in the
dlm.
- Fix up comments
v4-v7:
- No changes (except rebases)
v3:
- 's/fragment/body', 's/fragments/bodies/'
- CLU/LUT now allocate 3 bodies
- vsp1_dl_list_fragments_free -> vsp1_dl_list_bodies_put
v2:
- Use dl->body0->max_entries to determine header offset, instead of the
global constant VSP1_DL_NUM_ENTRIES which is incorrect.
- squash updates for LUT, CLU, and fragment cleanup into single patch.
(Not fully bisectable when separated)
---
drivers/media/platform/vsp1/vsp1_clu.c | 27 ++-
drivers/media/platform/vsp1/vsp1_clu.h | 1 +-
drivers/media/platform/vsp1/vsp1_dl.c | 221 ++
drivers/media/platform/vsp1/vsp1_dl.h | 3 +-
drivers/media/platform/vsp1/vsp1_lut.c | 27 ++-
drivers/media/platform/vsp1/vsp1_lut.h | 1 +-
6 files changed, 100 insertions(+), 180 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index ebfbb915dcdc..8efa12f5e53f 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -19,6 +19,8 @@
#define CLU_MIN_SIZE 4U
#define CLU_MAX_SIZE 8190U
+#define CLU_SIZE (17 * 17 * 17)
+
/*
-
* Device Access
*/
@@ -43,19 +45,19 @@ static int clu_set_table(struct vsp1_clu *clu, struct
v4l2_ctrl *ctrl)
struct vsp1_dl_body *dlb;
unsigned int i;
- dlb = vsp1_dl_body_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
+ dlb = vsp1_dl_body_get(clu->pool);
if (!dlb)
return -ENOMEM;
vsp1_dl_body_write(dlb, VI6_CLU_ADDR, 0);
- for (i = 0; i < 17 * 17 * 17; ++i)
+ for (i = 0; i < CLU_SIZE; ++i)
vsp1_dl_body_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
spin_lock_irq(>lock);
swap(clu->clu, dlb);
spin_unlock_irq(>lock);
- vsp1_dl_body_free(dlb);
+ vsp1_dl_body_put(dlb);
return 0;
}
@@ -216,8 +218,16 @@ static void clu_configure(struct vsp1_entity *entity,
}
}
+static void clu_destroy(struct vsp1_entity *entity)
+{
+ struct vsp1_clu *clu = to_clu(>subdev);
+
+ vsp1_dl_body_pool_destroy(clu->pool);
+}
+
static const struct vsp1_entity_operations clu_entity_ops = {
.configure = clu_configure,
+ .destroy = clu_destroy,
};
/*
-
@@ -243,6 +253,17 @@ struct vsp1_clu *vsp1_clu_create(struct vsp1_device *vsp1)
if (ret < 0)
return ERR_PTR(ret);
+ /*
+* Pre-allocate a body pool, with 3 bodies allowing a userspace update
+* before the hardware has committed a previous set of tables, handling
+* both the queued and pending dl entries. One extra entry is added to
+* the CLU_SIZE to allow for the VI6_CLU_ADDR header.
+*/
+ clu->pool = vsp1_dl_body_pool_create(clu->entity.vsp1, 3, CLU_SIZE + 1,
+0);
+ if (!clu->pool)
+ return ERR_PTR(-ENOMEM);
+
/* Initialize the control handler. */
v4l2_ctrl_handler_init(>ctrls, 2);
v4l2_ctrl_new_custom(>ctrls, _table_control, NULL);
diff --git a/drivers/media/platform/vsp1/vsp1_clu.h
b/drivers/media/platform/vsp1/vsp1_clu.h
index c45e6e707592..cef2f44481ba 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.h
+++ b/drivers/media/platform/vsp1/vsp1_clu.h
@@ -32,6 +32,7 @@ struct vsp1_clu {
spinlock_t lock;
unsigned int mode;
struct vsp1_dl_body *clu;
+ struct vsp1_dl_body_pool *pool;
};
static inline struct vsp1_clu *to_clu(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 41ace89a585b..617c46a03dec 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -108,7 +108,7 @@ struct vsp1_dl_list {
struct vsp1_dl_header *header;
dma_addr_t dma;
- struct vsp1_dl_body body0;
+ struct vsp1_dl_body *body0;
[PATCH v9 1/8] media: vsp1: Reword uses of 'fragment' as 'body'
Throughout the codebase, the term 'fragment' is used to represent a
display list body. This term duplicates the 'body' which is already in
use.
The datasheet references these objects as a body, therefore replace all
mentions of a fragment with a body, along with the corresponding
pluralised terms.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+rene...@ideasonboard.com>
---
Changes since v7:
- Fix indentation
- Reword vsp1_dl_list_add_body() documentation
Changes since v6:
- Clean up the formatting of the vsp1_dl_list_add_body()
---
drivers/media/platform/vsp1/vsp1_clu.c | 10 +-
drivers/media/platform/vsp1/vsp1_dl.c | 111 --
drivers/media/platform/vsp1/vsp1_dl.h | 13 +--
drivers/media/platform/vsp1/vsp1_lut.c | 8 +-
4 files changed, 70 insertions(+), 72 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_clu.c
b/drivers/media/platform/vsp1/vsp1_clu.c
index 96a448e1504c..ebfbb915dcdc 100644
--- a/drivers/media/platform/vsp1/vsp1_clu.c
+++ b/drivers/media/platform/vsp1/vsp1_clu.c
@@ -43,19 +43,19 @@ static int clu_set_table(struct vsp1_clu *clu, struct
v4l2_ctrl *ctrl)
struct vsp1_dl_body *dlb;
unsigned int i;
- dlb = vsp1_dl_fragment_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
+ dlb = vsp1_dl_body_alloc(clu->entity.vsp1, 1 + 17 * 17 * 17);
if (!dlb)
return -ENOMEM;
- vsp1_dl_fragment_write(dlb, VI6_CLU_ADDR, 0);
+ vsp1_dl_body_write(dlb, VI6_CLU_ADDR, 0);
for (i = 0; i < 17 * 17 * 17; ++i)
- vsp1_dl_fragment_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
+ vsp1_dl_body_write(dlb, VI6_CLU_DATA, ctrl->p_new.p_u32[i]);
spin_lock_irq(>lock);
swap(clu->clu, dlb);
spin_unlock_irq(>lock);
- vsp1_dl_fragment_free(dlb);
+ vsp1_dl_body_free(dlb);
return 0;
}
@@ -211,7 +211,7 @@ static void clu_configure(struct vsp1_entity *entity,
spin_unlock_irqrestore(>lock, flags);
if (dlb)
- vsp1_dl_list_add_fragment(dl, dlb);
+ vsp1_dl_list_add_body(dl, dlb);
break;
}
}
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 801dea475740..083da4f05c20 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -65,7 +65,7 @@ struct vsp1_dl_body {
* @header: display list header, NULL for headerless lists
* @dma: DMA address for the header
* @body0: first display list body
- * @fragments: list of extra display list bodies
+ * @bodies: list of extra display list bodies
* @has_chain: if true, indicates that there's a partition chain
* @chain: entry in the display list partition chain
* @internal: whether the display list is used for internal purpose
@@ -78,7 +78,7 @@ struct vsp1_dl_list {
dma_addr_t dma;
struct vsp1_dl_body body0;
- struct list_head fragments;
+ struct list_head bodies;
bool has_chain;
struct list_head chain;
@@ -97,13 +97,13 @@ enum vsp1_dl_mode {
* @mode: display list operation mode (header or headerless)
* @singleshot: execute the display list in single-shot mode
* @vsp1: the VSP1 device
- * @lock: protects the free, active, queued, pending and gc_fragments lists
+ * @lock: protects the free, active, queued, pending and gc_bodies lists
* @free: array of all free display lists
* @active: list currently being processed (loaded) by hardware
* @queued: list queued to the hardware (written to the DL registers)
* @pending: list waiting to be queued to the hardware
- * @gc_work: fragments garbage collector work struct
- * @gc_fragments: array of display list fragments waiting to be freed
+ * @gc_work: bodies garbage collector work struct
+ * @gc_bodies: array of display list bodies waiting to be freed
*/
struct vsp1_dl_manager {
unsigned int index;
@@ -118,7 +118,7 @@ struct vsp1_dl_manager {
struct vsp1_dl_list *pending;
struct work_struct gc_work;
- struct list_head gc_fragments;
+ struct list_head gc_bodies;
};
/*
-
@@ -156,18 +156,17 @@ static void vsp1_dl_body_cleanup(struct vsp1_dl_body *dlb)
}
/**
- * vsp1_dl_fragment_alloc - Allocate a display list fragment
+ * vsp1_dl_body_alloc - Allocate a display list body
* @vsp1: The VSP1 device
- * @num_entries: The maximum number of entries that the fragment can contain
+ * @num_entries: The maximum number of entries that the body can contain
*
- * Allocate a display list fragment with enough memory to contain the requested
+ * Allocate a display list body with enough memory to contain the requested
* number of entries.
*
- * Return a pointer to a fragment on success o
[PATCH v9 0/8] vsp1: TLB optimisation and DL caching
ng in RGB565 during stress testing: pass Testing WPF packing in BGR24 during stress testing: pass Testing WPF packing in RGB24 during stress testing: pass Testing WPF packing in ABGR32 during stress testing: pass Testing WPF packing in ARGB32 during stress testing: pass Testing WPF packing in XBGR32 during stress testing: pass Testing WPF packing in XRGB32 during stress testing: pass ./vsp-unit-test-0021.sh: line 34: 14785 Killed stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M - vsp-unit-test-0022.sh Testing long duration pipelines under stress: pass ./vsp-unit-test-0022.sh: line 38: 16751 Killed stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M - vsp-unit-test-0023.sh Testing histogram HGT with hue areas 0,255,255,255,255,255,255,255,255,255,255,255: pass Testing histogram HGT with hue areas 0,40,40,80,80,120,120,160,160,200,200,255: pass Testing histogram HGT with hue areas 220,40,40,80,80,120,120,160,160,200,200,220: pass Testing histogram HGT with hue areas 0,10,50,60,100,110,150,160,200,210,250,255: pass Testing histogram HGT with hue areas 10,20,50,60,100,110,150,160,200,210,230,240: pass Testing histogram HGT with hue areas 240,20,60,80,100,120,140,160,180,200,210,220: pass - vsp-unit-test-0024.sh Test requires unavailable feature set `rpf.0 rpf.1 brs wpf.0': skipped 168 tests: 147 passed, 0 failed, 3 skipped Kieran Bingham (8): media: vsp1: Reword uses of 'fragment' as 'body' media: vsp1: Protect bodies against overflow media: vsp1: Provide a body pool media: vsp1: Convert display lists to use new body pool media: vsp1: Use reference counting for bodies media: vsp1: Refactor display list configure operations media: vsp1: Adapt entities to configure into a body media: vsp1: Move video configuration to a cached dlb drivers/media/platform/vsp1/vsp1_brx.c| 32 +-- drivers/media/platform/vsp1/vsp1_clu.c| 112 --- drivers/media/platform/vsp1/vsp1_clu.h| 1 +- drivers/media/platform/vsp1/vsp1_dl.c | 388 +-- drivers/media/platform/vsp1/vsp1_dl.h | 20 +- drivers/media/platform/vsp1/vsp1_drm.c| 18 +- drivers/media/platform/vsp1/vsp1_entity.c | 34 +- drivers/media/platform/vsp1/vsp1_entity.h | 45 +-- drivers/media/platform/vsp1/vsp1_hgo.c| 26 +-- drivers/media/platform/vsp1/vsp1_hgt.c| 28 +-- drivers/media/platform/vsp1/vsp1_hsit.c | 20 +- drivers/media/platform/vsp1/vsp1_lif.c| 25 +- drivers/media/platform/vsp1/vsp1_lut.c| 80 +++-- drivers/media/platform/vsp1/vsp1_lut.h| 1 +- drivers/media/platform/vsp1/vsp1_pipe.c | 13 +- drivers/media/platform/vsp1/vsp1_pipe.h | 8 +- drivers/media/platform/vsp1/vsp1_rpf.c| 188 +-- drivers/media/platform/vsp1/vsp1_sru.c| 24 +- drivers/media/platform/vsp1/vsp1_uds.c| 73 ++-- drivers/media/platform/vsp1/vsp1_uds.h| 2 +- drivers/media/platform/vsp1/vsp1_uif.c| 35 +-- drivers/media/platform/vsp1/vsp1_video.c | 92 ++--- drivers/media/platform/vsp1/vsp1_video.h | 2 +- drivers/media/platform/vsp1/vsp1_wpf.c| 327 ++- 24 files changed, 884 insertions(+), 710 deletions(-) base-commit: 22cf3c24e08f6ebff46c3e733991548a5dab25ed -- git-series 0.9.1
Re: [PATCH v3 10/11] media: vsp1: Support Interlaced display pipelines
Hi Reviewers ...
Comments inline ...
On 03/05/18 09:44, Kieran Bingham wrote:
> Calculate the top and bottom fields for the interlaced frames and
> utilise the extended display list command feature to implement the
> auto-field operations. This allows the DU to update the VSP2 registers
> dynamically based upon the currently processing field.
>
> Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
>
> ---
> v3:
> - Pass DL through partition calls to allow autocmd's to be retrieved
This change itself could actually be folded into the TLB-Optimise series, but I
posted this in v3 here to show why it is necessary (currently).
I'll take suggestions on alternative implementations here too ...
> - Document interlaced field in struct vsp1_du_atomic_config
>
> v2:
> - fix erroneous BIT value which enabled interlaced
> - fix field handling at frame_end interrupt
>
> drivers/media/platform/vsp1/vsp1_dl.c | 10 +++-
> drivers/media/platform/vsp1/vsp1_drm.c| 13 +++-
> drivers/media/platform/vsp1/vsp1_entity.c | 3 +-
> drivers/media/platform/vsp1/vsp1_entity.h | 2 +-
> drivers/media/platform/vsp1/vsp1_regs.h | 1 +-
> drivers/media/platform/vsp1/vsp1_rpf.c| 73 +++-
> drivers/media/platform/vsp1/vsp1_rwpf.h | 1 +-
> drivers/media/platform/vsp1/vsp1_uds.c| 1 +-
> drivers/media/platform/vsp1/vsp1_video.c | 2 +-
> drivers/media/platform/vsp1/vsp1_wpf.c| 1 +-
> include/media/vsp1.h | 2 +-
> 11 files changed, 103 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
> b/drivers/media/platform/vsp1/vsp1_dl.c
> index 6366a1fc92b9..f3e75cff5ab3 100644
> --- a/drivers/media/platform/vsp1/vsp1_dl.c
> +++ b/drivers/media/platform/vsp1/vsp1_dl.c
> @@ -906,6 +906,8 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl, bool
> internal)
> */
> unsigned int vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
> {
> + struct vsp1_device *vsp1 = dlm->vsp1;
> + u32 status = vsp1_read(vsp1, VI6_STATUS);
> unsigned int flags = 0;
>
> spin_lock(>lock);
> @@ -931,6 +933,14 @@ unsigned int vsp1_dlm_irq_frame_end(struct
> vsp1_dl_manager *dlm)
> goto done;
>
> /*
> + * Progressive streams report only TOP fields. If we have a BOTTOM
> + * field, we are interlaced, and expect the frame to complete on the
> + * next frame end interrupt.
> + */
> + if (status & VI6_STATUS_FLD_STD(dlm->index))
> + goto done;
> +
> + /*
>* The device starts processing the queued display list right after the
>* frame end interrupt. The display list thus becomes active.
>*/
> diff --git a/drivers/media/platform/vsp1/vsp1_drm.c
> b/drivers/media/platform/vsp1/vsp1_drm.c
> index 7714be7f50af..cc29c9d96bb7 100644
> --- a/drivers/media/platform/vsp1/vsp1_drm.c
> +++ b/drivers/media/platform/vsp1/vsp1_drm.c
> @@ -556,7 +556,7 @@ static void vsp1_du_pipeline_configure(struct
> vsp1_pipeline *pipe)
> vsp1_entity_route_setup(entity, pipe, dlb);
> vsp1_entity_configure_stream(entity, pipe, dlb);
> vsp1_entity_configure_frame(entity, pipe, dl, dlb);
> - vsp1_entity_configure_partition(entity, pipe, dlb);
> + vsp1_entity_configure_partition(entity, pipe, dl, dlb);
This change, and other changes to vsp1_entity_configure_partition() could be
performed in tlb-optimise when the corresponding change is made to
vsp1_entity_configure_frame()
> }
>
> vsp1_dl_list_commit(dl, drm_pipe->force_brx_release);
> @@ -811,6 +811,17 @@ int vsp1_du_atomic_update(struct device *dev, unsigned
> int pipe_index,
> return -EINVAL;
> }
>
> + if (!(vsp1_feature(vsp1, VSP1_HAS_EXT_DL)) && cfg->interlaced) {
> + /*
> + * Interlaced support requires extended display lists to
> + * provide the auto-fld feature with the DU.
> + */
> + dev_dbg(vsp1->dev, "Interlaced unsupported on this output\n");
> + return -EINVAL;
> + }
> +
> + rpf->interlaced = cfg->interlaced;
> +
> rpf->fmtinfo = fmtinfo;
> rpf->format.num_planes = fmtinfo->planes;
> rpf->format.plane_fmt[0].bytesperline = cfg->pitch;
> diff --git a/drivers/media/platform/vsp1/vsp1_entity.c
> b/drivers/media/platform/vsp1/vsp1_entity.c
> index 7b29ef3532fc..da276a85aa95 100644
> --- a/drivers/media/platform/vsp1/vsp1_entity.c
> +++ b/drivers/media/platform/vsp1/vsp1_entity.c
> @@ -88,10 +88,11 @@ void vsp1_entity_con
[PATCH v3 11/11] drm: rcar-du: Support interlaced video output through vsp1
Use the newly exposed VSP1 interface to enable interlaced frame support through the VSP1 lif pipelines. Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com> --- drivers/gpu/drm/rcar-du/rcar_du_crtc.c | 1 + drivers/gpu/drm/rcar-du/rcar_du_vsp.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c index d71d709fe3d9..206532959ec9 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c @@ -289,6 +289,7 @@ static void rcar_du_crtc_set_display_timing(struct rcar_du_crtc *rcrtc) /* Signal polarities */ value = ((mode->flags & DRM_MODE_FLAG_PVSYNC) ? DSMR_VSL : 0) | ((mode->flags & DRM_MODE_FLAG_PHSYNC) ? DSMR_HSL : 0) + | ((mode->flags & DRM_MODE_FLAG_INTERLACE) ? DSMR_ODEV : 0) | DSMR_DIPM_DISP | DSMR_CSPM; rcar_du_crtc_write(rcrtc, DSMR, value); diff --git a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c index af7822a66dee..c7b37232ee91 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_vsp.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_vsp.c @@ -186,6 +186,9 @@ static void rcar_du_vsp_plane_setup(struct rcar_du_vsp_plane *plane) }; unsigned int i; + cfg.interlaced = !!(plane->plane.state->crtc->mode.flags + & DRM_MODE_FLAG_INTERLACE); + cfg.src.left = state->state.src.x1 >> 16; cfg.src.top = state->state.src.y1 >> 16; cfg.src.width = drm_rect_width(>state.src) >> 16; -- git-series 0.9.1
[PATCH v3 10/11] media: vsp1: Support Interlaced display pipelines
Calculate the top and bottom fields for the interlaced frames and
utilise the extended display list command feature to implement the
auto-field operations. This allows the DU to update the VSP2 registers
dynamically based upon the currently processing field.
Signed-off-by: Kieran Bingham <kieran.bingham+rene...@ideasonboard.com>
---
v3:
- Pass DL through partition calls to allow autocmd's to be retrieved
- Document interlaced field in struct vsp1_du_atomic_config
v2:
- fix erroneous BIT value which enabled interlaced
- fix field handling at frame_end interrupt
drivers/media/platform/vsp1/vsp1_dl.c | 10 +++-
drivers/media/platform/vsp1/vsp1_drm.c| 13 +++-
drivers/media/platform/vsp1/vsp1_entity.c | 3 +-
drivers/media/platform/vsp1/vsp1_entity.h | 2 +-
drivers/media/platform/vsp1/vsp1_regs.h | 1 +-
drivers/media/platform/vsp1/vsp1_rpf.c| 73 +++-
drivers/media/platform/vsp1/vsp1_rwpf.h | 1 +-
drivers/media/platform/vsp1/vsp1_uds.c| 1 +-
drivers/media/platform/vsp1/vsp1_video.c | 2 +-
drivers/media/platform/vsp1/vsp1_wpf.c| 1 +-
include/media/vsp1.h | 2 +-
11 files changed, 103 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c
b/drivers/media/platform/vsp1/vsp1_dl.c
index 6366a1fc92b9..f3e75cff5ab3 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -906,6 +906,8 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl, bool
internal)
*/
unsigned int vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
{
+ struct vsp1_device *vsp1 = dlm->vsp1;
+ u32 status = vsp1_read(vsp1, VI6_STATUS);
unsigned int flags = 0;
spin_lock(>lock);
@@ -931,6 +933,14 @@ unsigned int vsp1_dlm_irq_frame_end(struct vsp1_dl_manager
*dlm)
goto done;
/*
+* Progressive streams report only TOP fields. If we have a BOTTOM
+* field, we are interlaced, and expect the frame to complete on the
+* next frame end interrupt.
+*/
+ if (status & VI6_STATUS_FLD_STD(dlm->index))
+ goto done;
+
+ /*
* The device starts processing the queued display list right after the
* frame end interrupt. The display list thus becomes active.
*/
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c
b/drivers/media/platform/vsp1/vsp1_drm.c
index 7714be7f50af..cc29c9d96bb7 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -556,7 +556,7 @@ static void vsp1_du_pipeline_configure(struct vsp1_pipeline
*pipe)
vsp1_entity_route_setup(entity, pipe, dlb);
vsp1_entity_configure_stream(entity, pipe, dlb);
vsp1_entity_configure_frame(entity, pipe, dl, dlb);
- vsp1_entity_configure_partition(entity, pipe, dlb);
+ vsp1_entity_configure_partition(entity, pipe, dl, dlb);
}
vsp1_dl_list_commit(dl, drm_pipe->force_brx_release);
@@ -811,6 +811,17 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int
pipe_index,
return -EINVAL;
}
+ if (!(vsp1_feature(vsp1, VSP1_HAS_EXT_DL)) && cfg->interlaced) {
+ /*
+* Interlaced support requires extended display lists to
+* provide the auto-fld feature with the DU.
+*/
+ dev_dbg(vsp1->dev, "Interlaced unsupported on this output\n");
+ return -EINVAL;
+ }
+
+ rpf->interlaced = cfg->interlaced;
+
rpf->fmtinfo = fmtinfo;
rpf->format.num_planes = fmtinfo->planes;
rpf->format.plane_fmt[0].bytesperline = cfg->pitch;
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c
b/drivers/media/platform/vsp1/vsp1_entity.c
index 7b29ef3532fc..da276a85aa95 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -88,10 +88,11 @@ void vsp1_entity_configure_frame(struct vsp1_entity *entity,
void vsp1_entity_configure_partition(struct vsp1_entity *entity,
struct vsp1_pipeline *pipe,
+struct vsp1_dl_list *dl,
struct vsp1_dl_body *dlb)
{
if (entity->ops->configure_partition)
- entity->ops->configure_partition(entity, pipe, dlb);
+ entity->ops->configure_partition(entity, pipe, dl, dlb);
}
/*
-
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h
b/drivers/media/platform/vsp1/vsp1_entity.h
index f329c69731d1..97acb7795cf1 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -83,6 +83,7 @@ struct vsp1_entity_operations {
