Re: [PATCH v10 13/16] vb2: add out-fence support to QBUF

[Brian Starkey]

Hi Ezequiel,

Not sure if this patch series is still relevant, but I spotted a
couple more things below.

On Mon, May 21, 2018 at 01:59:43PM -0300, Ezequiel Garcia wrote:

From: Gustavo Padovan 

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send its fd to userspace in the fence_fd field as a
return arg for the QBUF call.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v12: - Pass the fence_fd in vb2_qbuf for clarity.
- Increase fence seqno if fence context if shared
- Check get_unused_fd return

v11: - Return fence_fd to userpace only in the QBUF ioctl.
- Rework implementation to avoid storing the sync_file
  as state, which is not really needed.

v10: - use -EIO for fence error (Hans Verkuil)
- add comment around fence context creation (Hans Verkuil)

v9: - remove in-fences changes from this patch (Alex Courbot)
   - improve fence context creation (Hans Verkuil)
   - clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)

v8: - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
   - fix crash when checking not using fences in vb2_buffer_done()

v7: - merge patch that add the infrastructure to out-fences into
 this one (Alex Courbot)
   - Do not install the fd if there is no fence. (Alex Courbot)
   - do not report error on requeueing, just WARN_ON_ONCE() (Hans)

v6: - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
 ordering in vb2 for queueing in the driver, so the event is not
 necessary anymore and the out_fence_fd is sent back to userspace
 on QBUF call return arg
   - do not allow requeueing with out-fences, instead mark the buffer
 with an error and wake up to userspace.
   - send the out_fence_fd back to userspace on the fence_fd field

v5: - delay fd_install to DQ_EVENT (Hans)
   - if queue is fully ordered send OUT_FENCE event right away
 (Brian)
   - rename 'q->ordered' to 'q->ordered_in_driver'
   - merge change to implement OUT_FENCE event here

v4: - return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
   - set the OUT_FENCE flag if there is a fence pending (Hans)
   - call fd_install() after vb2_core_qbuf() (Hans)
   - clean up fence if vb2_core_qbuf() fails (Hans)
   - add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
drivers/media/common/videobuf2/videobuf2-core.c | 113 +++-
drivers/media/common/videobuf2/videobuf2-v4l2.c |  10 ++-
drivers/media/dvb-core/dvb_vb2.c|   2 +-
include/media/videobuf2-core.h  |  20 -
4 files changed, 136 insertions(+), 9 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 86b5ebe25263..edc2fdaf56de 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -25,6 +25,7 @@
#include 
#include 
#include 
+#include 

#include 
#include 
@@ -380,6 +381,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;

/* Allocate video buffer memory for the MMAP type */
@@ -976,10 +978,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
*vb, enum vb2_buffer_state
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /* Requeuing with explicit synchronization, spit warning */
+   WARN_ON_ONCE(vb->out_fence);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (vb->out_fence) {
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -EIO);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+   }
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1406,6 +1420,76 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);

+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+   return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+   return 

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Brian Starkey]

On Wed, May 09, 2018 at 01:03:15PM -0300, Ezequiel Garcia wrote:

On Wed, 2018-05-09 at 11:36 +0100, Brian Starkey wrote:

[..]

> @@ -203,9 +215,14 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, 
void *pb)
>b->timestamp = ns_to_timeval(vb->timestamp);
>b->timecode = vbuf->timecode;
>b->sequence = vbuf->sequence;
> -  b->fence_fd = 0;
>b->reserved = 0;
>
> +  b->fence_fd = 0;

I didn't understand why we're returning 0 instead of -1. Actually the
doc in patch 10 seems to say it will be -1 or 0 depending on whether
we set one of the fence flags? I'm not sure:

For all other ioctls V4L2 sets this field to -1 if
``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
otherwise this field is set to 0 for backward compatibility.



Well, I think that for backwards compatibility (userspace not knowing
about fence_fd field), we should return 0, unless the flags are explicitly
set.

That is what the doc says and it sounds sane.


On the line below where you snipped, is this:

+  if (vb->in_fence)
+  b->flags |= V4L2_BUF_FLAG_IN_FENCE;
+  else
+  b->flags &= ~V4L2_BUF_FLAG_IN_FENCE;

If the "if (vb->in_fence)" condition is true, then the flag is set,
and the fence_fd field is 0. I think that's the opposite of what the
doc says:

   For all other ioctls V4L2 sets this field to -1 if
   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
   otherwise this field is set to 0 for backward compatibility.

V4L2_BUF_FLAG_IN_FENCE is set, therefore the doc says V4L2 will set
this field to -1. (Or at least the comment should be made less
ambiguous).



The bits are implemented in patch 12, but as I mentioned in my reply to
patch 10, I will move it to patch 10, for consistency.


Yeah as you say, it looks like you change this behaviour in path 12,
so I'm not totally sure which is right or expected. But consistency is
good :-)

-Brian



Thanks,
Eze

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Brian Starkey]

On Wed, May 09, 2018 at 12:52:26PM -0300, Ezequiel Garcia wrote:

On Wed, 2018-05-09 at 11:33 +0100, Brian Starkey wrote:

Hi Ezequiel,

On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:
> From: Gustavo Padovan <gustavo.pado...@collabora.com>
>
> Turn the reserved2 field into fence_fd that we will use to send
> an in-fence to the kernel or return an out-fence from the kernel to
> userspace.
>
> Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> when sending an in-fence to the kernel to be waited on, and
> V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
>
> v7: minor fixes on the Documentation (Hans Verkuil)
>
> v6: big improvement on doc (Hans Verkuil)
>
> v5: - keep using reserved2 field for cpia2
>- set fence_fd to 0 for now, for compat with userspace(Mauro)
>
> v4: make it a union with reserved2 and fence_fd (Hans Verkuil)
>
> v3: make the out_fence refer to the current buffer (Hans Verkuil)
>
> v2: add documentation
>
> Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
> ---
> Documentation/media/uapi/v4l/buffer.rst | 45 +++--
> drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
> include/uapi/linux/videodev2.h  |  8 -
> 4 files changed, 52 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
> index e2c85ddc990b..be9719cf5745 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -301,10 +301,22 @@ struct v4l2_buffer
>elements in the ``planes`` array. The driver will fill in the
>actual number of valid elements in that array.
> * - __u32
> -  - ``reserved2``
> +  - ``fence_fd``
>   -
> -  - A place holder for future extensions. Drivers and applications
> -  must set this to 0.
> +  - Used to communicate a fence file descriptors from userspace to kernel
> +  and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
> +  an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag must
> +  be used and this field set to the fence file descriptor of the in-fence.
> +  If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
> +
> +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
> +  must be set, the kernel will return the out-fence file descriptor in
> +  this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
> +an error.
> +
> +  For all other ioctls V4L2 sets this field to -1 if
> +  ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
> +  otherwise this field is set to 0 for backward compatibility.
> * - __u32
>   - ``reserved``
>   -
> @@ -648,6 +660,33 @@ Buffer Flags
>   - Start Of Exposure. The buffer timestamp has been taken when the
>exposure of the frame has begun. This is only valid for the
>``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> +* .. _`V4L2-BUF-FLAG-IN-FENCE`:
> +
> +  - ``V4L2_BUF_FLAG_IN_FENCE``
> +  - 0x0020
> +  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
> +  buffer won't be queued to the driver until the fence signals. The order
> +  in which buffers are queued is guaranteed to be preserved, so any
> +  buffers queued after this buffer will also be blocked until this fence
> +  signals. This flag must be set before calling ``VIDIOC_QBUF``. For
> +  other ioctls the driver just reports the value of the flag.
> +
> +If the fence signals the flag is cleared and not reported anymore.
> +  If the fence is not valid ``VIDIOC_QBUF`` returns an error.
> +
> +
> +* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
> +
> +  - ``V4L2_BUF_FLAG_OUT_FENCE``
> +  - 0x0040
> +  - Request for a fence to be attached to the buffer. The driver will 
fill
> +  in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
> +  ` returns. This flag must be set before calling
> +  ``VIDIOC_QBUF``. For other ioctls the driver just reports the value of
> +  the flag.
> +
> +If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
> +  error.
>

I commented similarly on some of the old patch-sets, and it's a minor
thing, but I still think the ordering of this series is off. It's
strange/wrong to me document all this behaviour, and expose the flags
to userspace, when the functionality isn't implemented yet.

If I apply this patch to the kernel, then the kernel doesn't do what
the (newly added) kernel-doc says it will.



This has never been a problem, and it has always been the canonical
way 

Re: [PATCH v9 12/15] vb2: add out-fence support to QBUF

[Brian Starkey]

Hi,

On Fri, May 04, 2018 at 05:06:09PM -0300, Ezequiel Garcia wrote:

From: Gustavo Padovan 

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send its fd to userspace in the fence_fd field as a
return arg for the QBUF call.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v11: - Return fence_fd to userpace only in the QBUF ioctl.
- Rework implementation to avoid storing the sync_file
  as state, which is not really needed.

v10: - use -EIO for fence error (Hans Verkuil)
- add comment around fence context creation (Hans Verkuil)

v9: - remove in-fences changes from this patch (Alex Courbot)
   - improve fence context creation (Hans Verkuil)
   - clean up out fences if vb2_core_qbuf() fails (Hans Verkuil)

v8: - return 0 as fence_fd if OUT_FENCE flag not used (Mauro)
   - fix crash when checking not using fences in vb2_buffer_done()

v7: - merge patch that add the infrastructure to out-fences into
 this one (Alex Courbot)
   - Do not install the fd if there is no fence. (Alex Courbot)
   - do not report error on requeueing, just WARN_ON_ONCE() (Hans)

v6: - get rid of the V4L2_EVENT_OUT_FENCE event. We always keep the
 ordering in vb2 for queueing in the driver, so the event is not
 necessary anymore and the out_fence_fd is sent back to userspace
 on QBUF call return arg
   - do not allow requeueing with out-fences, instead mark the buffer
 with an error and wake up to userspace.
   - send the out_fence_fd back to userspace on the fence_fd field

v5: - delay fd_install to DQ_EVENT (Hans)
   - if queue is fully ordered send OUT_FENCE event right away
 (Brian)
   - rename 'q->ordered' to 'q->ordered_in_driver'
   - merge change to implement OUT_FENCE event here

v4: - return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
   - set the OUT_FENCE flag if there is a fence pending (Hans)
   - call fd_install() after vb2_core_qbuf() (Hans)
   - clean up fence if vb2_core_qbuf() fails (Hans)
   - add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan 
Signed-off-by: Ezequiel Garcia 
---
drivers/media/common/videobuf2/videobuf2-core.c | 95 +++--
drivers/media/common/videobuf2/videobuf2-v4l2.c | 22 +-
drivers/media/dvb-core/dvb_vb2.c|  2 +-
include/media/videobuf2-core.h  | 16 -
4 files changed, 127 insertions(+), 8 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 996b99497a98..0f7306a04db7 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -25,6 +25,7 @@
#include 
#include 
#include 
+#include 

#include 
#include 
@@ -357,6 +358,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;

/* Allocate video buffer memory for the MMAP type */
@@ -948,10 +950,22 @@ static void vb2_process_buffer_done(struct vb2_buffer 
*vb, enum vb2_buffer_state
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /* Requeuing with explicit synchronization, spit warning */
+   WARN_ON_ONCE(vb->out_fence);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (vb->out_fence) {
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -EIO);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+   }
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1367,6 +1381,68 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);

+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+   return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+   return "vb2_fence_timeline";
+}
+
+static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
+{
+   return true;
+}
+
+static const struct dma_fence_ops vb2_fence_ops = {
+   .get_driver_name = vb2_fence_get_driver_name,
+   

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Brian Starkey]

Hi Ezequiel,

On Fri, May 04, 2018 at 05:06:08PM -0300, Ezequiel Garcia wrote:

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers can't be queued to the
driver before its fences signal. And a buffer can't be queued to the driver
out of the order they were queued from userspace. That means that even if
its fence signals it must wait for all other buffers, ahead of it in the queue,
to signal first.

If the fence for some buffer fails we do not queue it to the driver,
instead we mark it as error and wait until the previous buffer is done
to notify userspace of the error. We wait here to deliver the buffers back
to userspace in order.

v12: fixed dvb_vb2.c usage of vb2_core_qbuf.

v11: - minor doc/comments fixes (Hans Verkuil)
- reviewed the in-fence path at __fill_v4l2_buffer()

v10: - rename fence to in_fence in many places
- handle fences signalling with error better (Hans Verkuil)

v9: - improve comments and docs (Hans Verkuil)
   - fix unlocking of vb->fence_cb_lock on vb2_core_qbuf (Hans Verkuil)
   - move in-fences code that was in the out-fences patch here (Alex)

v8: - improve comments about fences with errors

v7: - get rid of the fence array stuff for ordering and just use
 get_num_buffers_ready() (Hans)
   - fix issue of queuing the buffer twice (Hans)
   - avoid the dma_fence_wait() in core_qbuf() (Alex)
   - merge preparation commit in

v6: - With fences always keep the order userspace queues the buffers.
   - Protect in_fence manipulation with a lock (Brian Starkey)
   - check if fences have the same context before adding a fence array
   - Fix last_fence ref unbalance in __set_in_fence() (Brian Starkey)
   - Clean up fence if __set_in_fence() fails (Brian Starkey)
   - treat -EINVAL from dma_fence_add_callback() (Brian Starkey)

v5: - use fence_array to keep buffers ordered in vb2 core when
 needed (Brian Starkey)
   - keep backward compat on the reserved2 field (Brian Starkey)
   - protect fence callback removal with lock (Brian Starkey)

v4: - Add a comment about dma_fence_add_callback() not returning a
 error (Hans)
   - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
   - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
   - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
   - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
   - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
 vb2_start_streaming() (Hans)
   - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
   - Queue buffers to the driver as soon as they are ready (Hans)
   - call fill_user_buffer() after queuing the buffer (Hans)
   - add err: label to clean up fence
   - add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
   - remove ternary if at vb2_qbuf() return (Mauro)
   - do not change if conditions behaviour (Mauro)

v2: - fix vb2_queue_or_prepare_buf() ret check
   - remove check for VB2_MEMORY_DMABUF only (Javier)
   - check num of ready buffers to start streaming
   - when queueing, start from the first ready buffer
   - handle queue cancel

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequ...@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 197 
drivers/media/common/videobuf2/videobuf2-v4l2.c |  34 +++-
drivers/media/dvb-core/dvb_vb2.c|   2 +-
drivers/media/v4l2-core/Kconfig |  33 
include/media/videobuf2-core.h  |  14 +-
5 files changed, 249 insertions(+), 31 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c 
b/drivers/media/common/videobuf2/videobuf2-core.c
index 6b8e083893ad..996b99497a98 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -352,6 +352,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
+   spin_lock_init(>fence_cb_lock);
for (plane = 0; plane < num_planes; ++plane) {
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
@@ -905,20 +906,12 @@ void *vb2_plane_cookie(struct vb2_buffer *vb, unsigned 
int plane_no)
}
EXPORT_SYMBOL_GPL(vb2_plane_cookie);

-void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
+static void vb2_process_buffer_done(struct vb2_buffer *vb, enum 
vb2_buffer_state state)
{
struct vb2_queue *q = vb->vb2_queue;
unsigned long flags;
unsigned int plane;

-   if (WARN_ON(vb->state != VB2_BUF_STATE_ACTIVE))
-   return;
-
-   if (WARN

Re: [PATCH v9 11/15] vb2: add in-fence support to QBUF

[Brian Starkey]

Hi,

On Tue, May 08, 2018 at 08:18:06PM -0300, Gustavo Padovan wrote:


Hi Hans,

On Mon, 2018-05-07 at 14:07 +0200, Hans Verkuil wrote:

On 04/05/18 22:06, Ezequiel Garcia wrote:
> From: Gustavo Padovan 


[snip]


> diff --git a/include/media/videobuf2-core.h
> b/include/media/videobuf2-core.h
> index 364e4cb41b10..28ce8f66882e 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -17,6 +17,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>
>  #define VB2_MAX_FRAME (32)
>  #define VB2_MAX_PLANES(8)
> @@ -255,12 +256,21 @@ struct vb2_buffer {
> * done_entry:  entry on the list that
> stores all buffers ready
> *  to be dequeued to userspace
> * vb2_plane:   per-plane information; do not
> change
> +   * in_fence:fence received from vb2 client
> to wait on before
> +   *  using the buffer (queueing to
> the driver)
> +   * fence_cb:fence callback information
> +   * fence_cb_lock:   protect callback signal/remove
> */
>enum vb2_buffer_state   state;
>
>struct vb2_planeplanes[VB2_MAX_PLANES];
>struct list_headqueued_entry;
>struct list_headdone_entry;
> +
> +  struct dma_fence*in_fence;
> +  struct dma_fence_cb fence_cb;
> +  spinlock_t  fence_cb_lock;
> +

So for the _MPLANE formats this is one fence for all planes. Which
makes sense, but how
does drm handle that? Also one fence for all planes?


Yes, this is one fence for all planes.

The DRM concept for planes is a totally different concept and is
basically a representation of an user definable square on the screen,
and to that plane there in one framebuffer attached - display hw has no
such a multiplanar for the same image AFAICT. So you probably need some
blit to convert the v4l2 multiplanar to a DRM framebuffer.



Lots of display hardware can do multi-planar formats, and there's
space in struct drm_framebuffer for up to 4 buffer handles (e.g. 3
handles are passed for Luma, Cr, and Cb when the framebuffer format is
DRM_FORMAT_YUV420) - like V4L2 MPLANE.

The V4L2 code here matches with the DRM "explicit sync"
(IN_FENCE_FD/OUT_FENCE_PTR) stuff, which is probably what we want.
The main difference is that in DRM, explicit fences aren't associated
with framebuffers, they're associated with the things using the
framebuffers - but practically it doesn't make a difference.

There can be per-buffer "implicit sync" via dma-buf reservation
objects, but I don't think this series should attempt to deal with
that.

Cheers,
-Brian



I think there should be a comment about this somewhere.


Yes, we've been over this exact discussion a few times :)
Having entirely different things with the same name is quite confusing.

Regards,

Gustavo

--
Gustavo Padovan
Principal Software Engineer
Collabora Ltd.

Re: [PATCH v9 10/15] vb2: add explicit fence user API

[Brian Starkey]

Hi Ezequiel,

On Fri, May 04, 2018 at 05:06:07PM -0300, Ezequiel Garcia wrote:

From: Gustavo Padovan 

Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel or return an out-fence from the kernel to
userspace.

Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending an in-fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.

v7: minor fixes on the Documentation (Hans Verkuil)

v6: big improvement on doc (Hans Verkuil)

v5: - keep using reserved2 field for cpia2
   - set fence_fd to 0 for now, for compat with userspace(Mauro)

v4: make it a union with reserved2 and fence_fd (Hans Verkuil)

v3: make the out_fence refer to the current buffer (Hans Verkuil)

v2: add documentation

Signed-off-by: Gustavo Padovan 
---
Documentation/media/uapi/v4l/buffer.rst | 45 +++--
drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  4 +--
include/uapi/linux/videodev2.h  |  8 -
4 files changed, 52 insertions(+), 7 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..be9719cf5745 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -301,10 +301,22 @@ struct v4l2_buffer
elements in the ``planes`` array. The driver will fill in the
actual number of valid elements in that array.
* - __u32
-  - ``reserved2``
+  - ``fence_fd``
  -
-  - A place holder for future extensions. Drivers and applications
-   must set this to 0.
+  - Used to communicate a fence file descriptors from userspace to kernel
+   and vice-versa. On :ref:`VIDIOC_QBUF ` when sending
+   an in-fence for V4L2 to wait on, the ``V4L2_BUF_FLAG_IN_FENCE`` flag 
must
+   be used and this field set to the fence file descriptor of the in-fence.
+   If the in-fence is not valid ` VIDIOC_QBUF`` returns an error.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE``
+   must be set, the kernel will return the out-fence file descriptor in
+   this field. If it fails to create the out-fence ``VIDIOC_QBUF` returns
+an error.
+
+   For all other ioctls V4L2 sets this field to -1 if
+   ``V4L2_BUF_FLAG_IN_FENCE`` and/or ``V4L2_BUF_FLAG_OUT_FENCE`` are set,
+   otherwise this field is set to 0 for backward compatibility.
* - __u32
  - ``reserved``
  -
@@ -648,6 +660,33 @@ Buffer Flags
  - Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+* .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+  - ``V4L2_BUF_FLAG_IN_FENCE``
+  - 0x0020
+  - Ask V4L2 to wait on the fence passed in the ``fence_fd`` field. The
+   buffer won't be queued to the driver until the fence signals. The order
+   in which buffers are queued is guaranteed to be preserved, so any
+   buffers queued after this buffer will also be blocked until this fence
+   signals. This flag must be set before calling ``VIDIOC_QBUF``. For
+   other ioctls the driver just reports the value of the flag.
+
+If the fence signals the flag is cleared and not reported anymore.
+   If the fence is not valid ``VIDIOC_QBUF`` returns an error.
+
+
+* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+  - ``V4L2_BUF_FLAG_OUT_FENCE``
+  - 0x0040
+  - Request for a fence to be attached to the buffer. The driver will fill
+   in the out-fence fd in the ``fence_fd`` field when :ref:`VIDIOC_QBUF
+   ` returns. This flag must be set before calling
+   ``VIDIOC_QBUF``. For other ioctls the driver just reports the value of
+   the flag.
+
+If the creation of the out-fence fails ``VIDIOC_QBUF`` returns an
+   error.



I commented similarly on some of the old patch-sets, and it's a minor
thing, but I still think the ordering of this series is off. It's
strange/wrong to me document all this behaviour, and expose the flags
to userspace, when the functionality isn't implemented yet.

If I apply this patch to the kernel, then the kernel doesn't do what
the (newly added) kernel-doc says it will.

I'd use this sequence:

  * Remove all the stuff in patch 11 which checks/sets FLAG_IN_FENCE
(i.e. the code added in videobuf2-v4l2.c, except for adding the
extra argument to vb2_core_qbuf(), which will always be NULL).

  * Remove all the stuff in patch 12 which checks/sets FLAG_OUT_FENCE
(i.e. the code added in videobuf2-v4l2.c, except for adding the
extra argument to vb2_core_qbuf(), which will always be false).

  * Move this to be a "new" patch 13, and add back the code to
videobuf2-v4l2.c, which hooks up the 

Re: [RFC v5 07/11] [media] vb2: add in-fence support to QBUF

[Brian Starkey]

On Fri, Nov 17, 2017 at 11:19:05AM -0200, Mauro Carvalho Chehab wrote:

Em Fri, 17 Nov 2017 11:08:01 -0200
Gustavo Padovan  escreveu:


2017-11-17 Mauro Carvalho Chehab :

> Em Fri, 17 Nov 2017 15:49:23 +0900
> Alexandre Courbot  escreveu:
>
> > > @@ -178,6 +179,12 @@ static int vb2_queue_or_prepare_buf(struct
> > > vb2_queue *q, struct v4l2_buffer *b,
> > >  return -EINVAL;
> > >  }
> > >
> > > +if ((b->fence_fd != 0 && b->fence_fd != -1) &&
> >
> > Why do we need to consider both values invalid? Can 0 ever be a valid fence
> > fd?
>
> Programs that don't use fences will initialize reserved2/fence_fd field
> at the uAPI call to zero.
>
> So, I guess using fd=0 here could be a problem. Anyway, I would, instead,
> do:
>
>if ((b->fence_fd < 1) &&
>...
>
> as other negative values are likely invalid as well.

We are checking when the fence_fd is set but the flag wasn't. Checking
for < 1 is exactly the opposite. so we keep as is or do it fence_fd > 0.


Ah, yes. Anyway, I would stick with:
if ((b->fence_fd > 0) &&
...



0 is a valid fence_fd right? If I close stdin, and create a sync_file,
couldn't I get a fence with fd zero?

-Brian



Gustavo



--
Thanks,
Mauro

Re: [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour

[Brian Starkey]

On Fri, Oct 20, 2017 at 07:50:12PM -0200, Gustavo Padovan wrote:

From: Gustavo Padovan 

Add section to VIDIOC_QBUF about it

v3:
- make the out_fence refer to the current buffer (Hans)
- Note what happens when the IN_FENCE is not set (Hans)

v2:
- mention that fences are files (Hans)
- rework for the new API

Signed-off-by: Gustavo Padovan 
---
Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 
1 file changed, 31 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst 
b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..a65a50578bad 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -118,6 +118,37 @@ immediately with an ``EAGAIN`` error code when no buffer 
is available.
The struct :c:type:`v4l2_buffer` structure is specified in
:ref:`buffer`.

+Explicit Synchronization
+
+
+Explicit Synchronization allows us to control the synchronization of
+shared buffers from userspace by passing fences to the kernel and/or
+receiving them from it. Fences passed to the kernel are named in-fences and
+the kernel should wait on them to signal before using the buffer, i.e., 
queueing
+it to the driver. On the other side, the kernel can create out-fences for the
+buffers it queues to the drivers. Out-fences signal when the driver is
+finished with buffer, i.e., the buffer is ready. The fences are represented
+as a file and passed as a file descriptor to userspace.
+
+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer
+flags and the `fence_fd` field. If an in-fence needs to be passed to the 
kernel,
+`fence_fd` should be set to the fence file descriptor number and the
+``V4L2_BUF_FLAG_IN_FENCE`` should be set as well Setting one but not the other
+will cause ``VIDIOC_QBUF`` to return with error.


nit: full-stop before "Setting".

Also, depending what is decided about passing in a -1 in-fence, the
wording might want to include that behaviour.


+
+The fence_fd field (formely the reserved2 field) will be ignored if the
+``V4L2_BUF_FLAG_IN_FENCE`` is not set.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
+be set to ask for a fence to be attached to the buffer. To become aware of
+the out-fence created one should listen for the ``V4L2_EVENT_OUT_FENCE`` event.
+An event will be triggered for every buffer queued to the V4L2 driver with the
+``V4L2_BUF_FLAG_OUT_FENCE``.


Is it worth mentioning the immediate out-fence behaviour for ordered
queues? And/or clarifying that otherwise the V4L2_EVENT_OUT_FENCE
event will not be sent until all in-fence(s) on the buffer (or
previously queued ones) have signalled.

Cheers,
-Brian


+
+At streamoff the out-fences will either signal normally if the drivers waits
+for the operations on the buffers to finish or signal with error if the
+driver cancels the pending operations.

Return Value

--
2.13.6

Re: [RFC v4 16/17] [media] vb2: add out-fence support to QBUF

[Brian Starkey]

Hi Gustavo,

On Fri, Oct 20, 2017 at 07:50:11PM -0200, Gustavo Padovan wrote:

From: Gustavo Padovan 

If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send to userspace on the V4L2_EVENT_OUT_FENCE when
the buffer is queued to the driver, or right away if the queue is ordered
both in VB2 and in the driver.

The fence is signaled on buffer_done(), when the job on the buffer is
finished.

v5:
- delay fd_install to DQ_EVENT (Hans)
- if queue is fully ordered send OUT_FENCE event right away
(Brian)
- rename 'q->ordered' to 'q->ordered_in_driver'
- merge change to implement OUT_FENCE event here

v4:
- return the out_fence_fd in the BUF_QUEUED event(Hans)

v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
- set the OUT_FENCE flag if there is a fence pending (Hans)
- call fd_install() after vb2_core_qbuf() (Hans)
- clean up fence if vb2_core_qbuf() fails (Hans)
- add list to store sync_file and fence for the next queued buffer

v2: check if the queue is ordered.

Signed-off-by: Gustavo Padovan 
---
drivers/media/v4l2-core/v4l2-event.c |  2 ++
drivers/media/v4l2-core/videobuf2-core.c | 25 +++
drivers/media/v4l2-core/videobuf2-v4l2.c | 55 
3 files changed, 82 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-event.c 
b/drivers/media/v4l2-core/v4l2-event.c
index 6274e3e174e0..275da224ace4 100644
--- a/drivers/media/v4l2-core/v4l2-event.c
+++ b/drivers/media/v4l2-core/v4l2-event.c
@@ -385,6 +385,8 @@ int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
switch (sub->type) {
case V4L2_EVENT_CTRL:
return v4l2_ctrl_subscribe_event(fh, sub);
+   case V4L2_EVENT_OUT_FENCE:
+   return v4l2_event_subscribe(fh, sub, VIDEO_MAX_FRAME, NULL);
}
return -EINVAL;
}
diff --git a/drivers/media/v4l2-core/videobuf2-core.c 
b/drivers/media/v4l2-core/videobuf2-core.c
index c7ba67bda5ac..21e2052776c1 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -354,6 +354,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+   vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;

/* Allocate video buffer memory for the MMAP type */
@@ -934,10 +935,24 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum 
vb2_buffer_state state)
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+   /*
+* Explicit synchronization requires ordered queues for now,
+* so WARN_ON if we are requeuing on an ordered queue.
+*/
+   if (vb->out_fence)
+   WARN_ON_ONCE(q->ordered_in_driver);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+   if (state == VB2_BUF_STATE_ERROR)
+   dma_fence_set_error(vb->out_fence, -ENOENT);
+   dma_fence_signal(vb->out_fence);
+   dma_fence_put(vb->out_fence);
+   vb->out_fence = NULL;
+   vb->out_fence_fd = -1;
+
/* Inform any processes that may be waiting for buffers */
wake_up(>done_wq);
break;
@@ -1235,6 +1250,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
trace_vb2_buf_queue(q, vb);

call_void_vb_qop(vb, buf_queue, vb);
+
+   if (!(q->is_output || q->ordered_in_vb2))
+   call_void_bufop(q, send_out_fence, vb);
}

static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
@@ -1451,6 +1469,7 @@ static struct dma_fence *__set_in_fence(struct vb2_queue 
*q,
}

q->last_fence = dma_fence_get(fence);
+   call_void_bufop(q, send_out_fence, vb);
}

return fence;
@@ -1840,6 +1859,11 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
}

/*
+* Renew out-fence context.
+*/


Why is that? I don't think I understand the nuances of fence contexts.


+   q->out_fence_context = dma_fence_context_alloc(1);
+
+   /*
 * Remove all buffers from videobuf's list...
 */
INIT_LIST_HEAD(>queued_list);
@@ -2171,6 +2195,7 @@ int vb2_core_queue_init(struct vb2_queue *q)
spin_lock_init(>done_lock);
mutex_init(>mmap_lock);
init_waitqueue_head(>done_wq);
+   q->out_fence_context = dma_fence_context_alloc(1);

if (q->buf_struct_size == 0)
q->buf_struct_size = sizeof(struct vb2_buffer);
diff --git 

Re: [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences

[Brian Starkey]

Hi Gustavo,

Sorry for the sporadic review... I'm struggling to get time to look at
these.

On Fri, Oct 20, 2017 at 07:50:10PM -0200, Gustavo Padovan wrote:

From: Gustavo Padovan 

Add vb2_setup_out_fence() and the needed members to struct vb2_buffer.

v2: - change it to reflect fd_install at DQEVENT
- add fence context for out-fences

Signed-off-by: Gustavo Padovan 
---
drivers/media/v4l2-core/videobuf2-core.c | 31 +++
include/media/videobuf2-core.h   | 20 
2 files changed, 51 insertions(+)

diff --git a/drivers/media/v4l2-core/videobuf2-core.c 
b/drivers/media/v4l2-core/videobuf2-core.c
index 78f369dba3e3..c7ba67bda5ac 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -23,8 +23,11 @@
#include 
#include 
#include 
+#include 
+#include 

#include 
+#include 
#include 

#include 
@@ -1316,6 +1319,34 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned 
int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);

+int vb2_setup_out_fence(struct vb2_queue *q, unsigned int index)
+{
+   struct vb2_buffer *vb;
+   struct dma_fence *fence;
+
+   vb = q->bufs[index];
+
+   vb->out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
+
+   fence = vb2_fence_alloc(q->out_fence_context);
+   if (!fence) {
+   put_unused_fd(vb->out_fence_fd);
+   return -ENOMEM;
+   }
+
+   vb->sync_file = sync_file_create(fence);
+   if (!vb->sync_file) {
+   dma_fence_put(fence);
+   put_unused_fd(vb->out_fence_fd);
+   return -ENOMEM;
+   }


sync_file_create() takes an extra reference of its own, so I think you
don't need the dma_fence_get() below, or you should drop a reference
before returning.

-Brian


+
+   vb->out_fence = dma_fence_get(fence);
+
+   return 0;
+}
+EXPORT_SYMBOL_GPL(vb2_setup_out_fence);
+
/**
 * vb2_start_streaming() - Attempt to start streaming.
 * @q:  videobuf2 queue
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 624ca2dce9ea..1925ede73804 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -259,6 +259,10 @@ struct vb2_buffer {
 *  using the buffer (queueing to the driver)
 * fence_cb:fence callback information
 * fence_cb_lock:   protect callback signal/remove
+* out_fence_fd:the out_fence_fd to be shared with userspace.
+* out_fence:   the out-fence associated with the buffer once
+*  it is queued to the driver.
+* sync_file:   the sync file to wrap the out fence
 */
enum vb2_buffer_state   state;

@@ -269,6 +273,10 @@ struct vb2_buffer {
struct dma_fence_cb fence_cb;
spinlock_t  fence_cb_lock;

+   int out_fence_fd;
+   struct dma_fence*out_fence;
+   struct sync_file*sync_file;
+
#ifdef CONFIG_VIDEO_ADV_DEBUG
/*
 * Counters for how often these buffer-related ops are
@@ -518,6 +526,7 @@ struct vb2_buf_ops {
 * @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
 *  of buffer queue from userspace with QBUF() until they are
 *  queued to the driver.
+ * @out_fence_context: the fence context for the out fences
 * @last_fence: last in-fence received. Used to keep ordering.
 * @fileio: file io emulator internal data, used only if emulator is active
 * @threadio:   thread io internal data, used only if thread is active
@@ -574,6 +583,7 @@ struct vb2_queue {
unsigned intordered_in_driver:1;
unsigned intordered_in_vb2:1;

+   u64 out_fence_context;
struct dma_fence*last_fence;

struct vb2_fileio_data  *fileio;
@@ -745,6 +755,16 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum 
vb2_memory memory,
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);

/**
+ * vb2_setup_out_fence() - setup new out-fence
+ * @q: The vb2_queue where to setup it
+ * @index: index of the buffer
+ *
+ * Setup the file descriptor, the fence and the sync_file for the next
+ * buffer to be queued and add everything to the tail of the q->out_fence_list.
+ */
+int vb2_setup_out_fence(struct vb2_queue *q, unsigned int index);
+
+/**
 * vb2_core_qbuf() - Queue a buffer from userspace
 *
 * @q:  videobuf2 queue
--
2.13.6

Re: [RFC v4 13/17] [media] vb2: add in-fence support to QBUF

[Brian Starkey]

Hi Gustavo,

On Fri, Oct 20, 2017 at 07:50:08PM -0200, Gustavo Padovan wrote:

From: Gustavo Padovan <gustavo.pado...@collabora.com>

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers are only queued
to the driver once they are ready. A buffer is ready when its
in-fence signals.

For queues that require vb2 to queue buffers to the v4l2 driver in same
order they are received from userspace we use fence_array to keep that
ordering. Basically we create a fence_array that contains both the current
fence and the fence from the previous buffer (which might be a fence array
as well). The base fence class for the fence_array becomes the new buffer
fence, waiting on that one guarantees that it won't be queued out of
order.


The API sounds/looks good to me, makes sense to let driver opt in/out
via the ordered_in_vb2 thing. Thanks for implementing it!



v5: - use fence_array to keep buffers ordered in vb2 core when
needed (Brian Stark)
- keep backward compatibility on the reserved2 field (Brian Stark)
- protect fence callback removal with lock (Brian Stark)


Brian Starkey, but close ;-)



v4:
- Add a comment about dma_fence_add_callback() not returning a
error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
-  Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)

v2:
- fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel

Signed-off-by: Gustavo Padovan <gustavo.pado...@collabora.com>
---
drivers/media/v4l2-core/Kconfig  |   1 +
drivers/media/v4l2-core/videobuf2-core.c | 179 ---
drivers/media/v4l2-core/videobuf2-v4l2.c |  29 -
include/media/videobuf2-core.h   |  17 ++-
4 files changed, 208 insertions(+), 18 deletions(-)

diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index a35c33686abf..3f988c407c80 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -83,6 +83,7 @@ config VIDEOBUF_DVB
# Used by drivers that need Videobuf2 modules
config VIDEOBUF2_CORE
select DMA_SHARED_BUFFER
+   select SYNC_FILE
tristate

config VIDEOBUF2_MEMOPS
diff --git a/drivers/media/v4l2-core/videobuf2-core.c 
b/drivers/media/v4l2-core/videobuf2-core.c
index 60f8b582396a..78f369dba3e3 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -346,6 +346,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum 
vb2_memory memory,
vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
+   spin_lock_init(>fence_cb_lock);
for (plane = 0; plane < num_planes; ++plane) {
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
@@ -1222,6 +1223,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
{
struct vb2_queue *q = vb->vb2_queue;

+   if (vb->in_fence && !dma_fence_is_signaled(vb->in_fence))
+   return;
+
vb->state = VB2_BUF_STATE_ACTIVE;
atomic_inc(>owned_by_drv_count);

@@ -1273,6 +1277,20 @@ static int __buf_prepare(struct vb2_buffer *vb, const 
void *pb)
return 0;
}

+static int __get_num_ready_buffers(struct vb2_queue *q)
+{
+   struct vb2_buffer *vb;
+   int ready_count = 0;
+
+   /* count num of buffers ready in front of the queued_list */
+   list_for_each_entry(vb, >queued_list, queued_entry) {
+   if (!vb->in_fence || dma_fence_is_signaled(vb->in_fence))
+   ready_count++;


I think there's still some races on vb->in_fence. Couldn't the
callback on any of the buffers in the queued_list have their callback
run at any moment?

That could make this loop do bad things, and similarly the loop in
vb2_start_streaming() which calls __enqueue_in_driver().


+   }

Re: [PATCH v3 00/15] V4L2 Explicit Synchronization support

[Brian Starkey]

Hi,

On Wed, Oct 04, 2017 at 05:08:55PM -0300, Gustavo Padovan wrote:

Hi Brian,

On Mon, 2017-10-02 at 14:41 +0100, Brian Starkey wrote:

Hi Gustavo,

On Thu, Sep 07, 2017 at 03:42:11PM -0300, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.pado...@collabora.com>
>
> Hi,
>
> Refer to the documentation on the first patch for the details. The
> previous
> iteration is here: https://www.mail-archive.com/linux-media@vger.ke
> rnel.org/msg118077.html
>
> The 2nd patch proposes an userspace API for fences, then on patch 3
> we
> prepare to the addition of in-fences in patch 4, by introducing the
> infrastructure on vb2 to wait on an in-fence signal before queueing
> the
> buffer in the driver.
>
> Patch 5 fix uvc v4l2 event handling and patch 6 configure q->dev
> for
> vivid drivers to enable to subscribe and dequeue events on it.
>
> Patches 7-9 enables support to notify BUF_QUEUED events, the event
> send
> to userspace the out-fence fd and the index of the buffer that was
> queued.
>
> Patches 10-11 add support to mark queues as ordered. Finally
> patches 12
> and 13 add more fence infrastructure to support out-fences, patch
> 13 exposes
> close_fd() and patch 14 adds support to out-fences.
>
> It only works for ordered queues for now, see open question at the
> end
> of the letter.
>
> Test tool can be found at:
> https://git.collabora.com/cgit/user/padovan/v4l2-test.git/
>
> Main Changes
> 
>
> * out-fences: change in behavior: the out-fence fd now comes out of
> the
> BUF_QUEUED event along with the buffer id.

The more I think about this, the more unfortunate it seems.
Especially
for our use-case (m2m engine which sits in front of the display
processor to convert the format of some buffers), having to wait for
the in-fence to signal before we can get an out-fence removes a lot
of
the advantages of having fences at all.


Does your m2m driver ensures ordering between the buffer queued to it?



I'm not so familiar with the code, how can I check that?



Ideally, we'd like to queue up our m2m work (while the GPU is still
rendering that buffer, holding the in-fence), immediately get the
out-fence for the m2m work, and pass that to DRM as the in-fence for
display. With the current behaviour we need to wait in userspace
before we can pass the buffer to display.

Wouldn't it be possible to enforce that the buffers aren't queued
out-of-order in VB2? An easy way might be to (in qbuf) set a buffer's
->in_fence to be a fence_array of all the ->in_fences from the
buffers
before it in the queue (and its own). That would then naturally order
the enqueue-ing in the driver, and allow you to return the out-fence
immediately.

This would also solve your output devices question from below - a
buffer can never get queued in the driver until all of the buffers
which were QBUF'd before it are queued in the driver.


What you say makes sense, what this proposal lacks the most now is
feedback regarding its usecases. We can create a control setting to
enforce ordering in the queue, if it's set we create the fence arrays.
For output devices this should be set by default.


Yeah that could work. I can see that in some cases queueing
out-of-order as the fences signal would be the right thing to do, so
makes sense to allow both.

Thanks,
-Brian



Gustavo

--
Gustavo Padovan
Principal Software Engineer
Collabora Ltd.

Re: [PATCH v3 10/15] [media] vb2: add 'ordered' property to queues

[Brian Starkey]

Hi,

On Thu, Sep 07, 2017 at 03:42:21PM -0300, Gustavo Padovan wrote:

From: Gustavo Padovan 

For explicit synchronization (and soon for HAL3/Request API) we need
the v4l2-driver to guarantee the ordering in which the buffers were queued
by userspace. This is already true for many drivers, but we never needed
to say it.

Signed-off-by: Gustavo Padovan 
---
include/media/videobuf2-core.h | 4 
1 file changed, 4 insertions(+)

diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 5ed8d3402474..20099dc22f26 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -508,6 +508,9 @@ struct vb2_buf_ops {
 * @last_buffer_dequeued: used in poll() and DQBUF to immediately return if the
 *  last decoded buffer was already dequeued. Set for capture queues
 *  when a buffer with the V4L2_BUF_FLAG_LAST is dequeued.
+ * @ordered: if the driver can guarantee that the queue will be ordered or not.
+ * The default is not ordered unless the driver sets this flag. It
+ * is mandatory for using explicit fences.


If it's mandatory for fences (why is that?), then I guess this patch
should come before any of the fence implementation?

But it's not entirely clear to me what this flag means - ordered with
respect to what? Ordered such that the order in which the buffers are
queued in the driver are the same order that they will be dequeued by
userspace?

I think the order they are queued from userspace can still be
different from both the order they are queued in the driver (because
the in-fences can signal in any order), and dequeued again in
userspace, so "ordered" seems a bit ambiguous.

I think it should be more clear.

Cheers
-Brian


 * @fileio: file io emulator internal data, used only if emulator is active
 * @threadio:   thread io internal data, used only if thread is active
 */
@@ -560,6 +563,7 @@ struct vb2_queue {
unsigned intis_output:1;
unsigned intcopy_timestamp:1;
unsigned intlast_buffer_dequeued:1;
+   unsigned intordered:1;

struct vb2_fileio_data  *fileio;
struct vb2_threadio_data*threadio;
--
2.13.5

Re: [PATCH v3 04/15] [media] vb2: add in-fence support to QBUF

[Brian Starkey]

Hi,

On Thu, Sep 07, 2017 at 03:42:15PM -0300, Gustavo Padovan wrote:

From: Gustavo Padovan 

Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers are only queued
to the driver once they are ready. A buffer is ready when its
in-fence signals.

v4:
- Add a comment about dma_fence_add_callback() not returning a
error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
-  Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()

v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)

v2:
- fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel

Signed-off-by: Gustavo Padovan 
---
drivers/media/v4l2-core/Kconfig  |   1 +
drivers/media/v4l2-core/videobuf2-core.c | 103 +++
drivers/media/v4l2-core/videobuf2-v4l2.c |  27 +++-
include/media/videobuf2-core.h   |  11 +++-
4 files changed, 127 insertions(+), 15 deletions(-)

diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index a35c33686abf..3f988c407c80 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -83,6 +83,7 @@ config VIDEOBUF_DVB
# Used by drivers that need Videobuf2 modules
config VIDEOBUF2_CORE
select DMA_SHARED_BUFFER
+   select SYNC_FILE
tristate

config VIDEOBUF2_MEMOPS
diff --git a/drivers/media/v4l2-core/videobuf2-core.c 
b/drivers/media/v4l2-core/videobuf2-core.c
index 60f8b582396a..b19c1bc4b083 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -1222,6 +1222,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
{
struct vb2_queue *q = vb->vb2_queue;

+   if (vb->in_fence && !dma_fence_is_signaled(vb->in_fence))
+   return;
+
vb->state = VB2_BUF_STATE_ACTIVE;
atomic_inc(>owned_by_drv_count);

@@ -1273,6 +1276,20 @@ static int __buf_prepare(struct vb2_buffer *vb, const 
void *pb)
return 0;
}

+static int __get_num_ready_buffers(struct vb2_queue *q)
+{
+   struct vb2_buffer *vb;
+   int ready_count = 0;
+
+   /* count num of buffers ready in front of the queued_list */
+   list_for_each_entry(vb, >queued_list, queued_entry) {
+   if (!vb->in_fence || dma_fence_is_signaled(vb->in_fence))
+   ready_count++;
+   }
+
+   return ready_count;
+}
+
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
{
struct vb2_buffer *vb;
@@ -1361,7 +1378,19 @@ static int vb2_start_streaming(struct vb2_queue *q)
return ret;
}

-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
+static void vb2_qbuf_fence_cb(struct dma_fence *f, struct dma_fence_cb *cb)
+{
+   struct vb2_buffer *vb = container_of(cb, struct vb2_buffer, fence_cb);
+
+   dma_fence_put(vb->in_fence);
+   vb->in_fence = NULL;
+
+   if (vb->vb2_queue->start_streaming_called)
+   __enqueue_in_driver(vb);
+}
+
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct dma_fence *fence)
{
struct vb2_buffer *vb;
int ret;
@@ -1372,16 +1401,18 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int 
index, void *pb)
case VB2_BUF_STATE_DEQUEUED:
ret = __buf_prepare(vb, pb);
if (ret)
-   return ret;
+   goto err;
break;
case VB2_BUF_STATE_PREPARED:
break;
case VB2_BUF_STATE_PREPARING:
dprintk(1, "buffer still being prepared\n");
-   return -EINVAL;
+   ret = -EINVAL;
+   goto err;
default:
dprintk(1, "invalid buffer state %d\n", vb->state);
-   return -EINVAL;
+   ret = -EINVAL;
+   goto err;
}

/*
@@ -1392,6 +1423,7 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int 
index, void *pb)
q->queued_count++;
q->waiting_for_buffers = 

Re: [PATCH v3 02/15] [media] vb2: add explicit fence user API

[Brian Starkey]

Hi,

On Thu, Sep 07, 2017 at 03:42:13PM -0300, Gustavo Padovan wrote:

From: Gustavo Padovan 

Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel and return an out-fence from the kernel to
userspace.

Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending a fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.


It seems a bit off to me to add this to the uapi, and document it,
before any of the implementation is present in the kernel.

Wouldn't it be better to move this patch to be the last one, after all
of the implementation is done?

Cheers,
-Brian



v2: add documentation

Signed-off-by: Gustavo Padovan 
---
Documentation/media/uapi/v4l/buffer.rst   | 19 +++
drivers/media/usb/cpia2/cpia2_v4l.c   |  2 +-
drivers/media/v4l2-core/v4l2-compat-ioctl32.c |  4 ++--
drivers/media/v4l2-core/videobuf2-v4l2.c  |  2 +-
include/uapi/linux/videodev2.h|  4 +++-
5 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst 
b/Documentation/media/uapi/v4l/buffer.rst
index ae6ee73f151c..664507ad06c6 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -648,6 +648,25 @@ Buffer Flags
  - Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+* .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+  - ``V4L2_BUF_FLAG_IN_FENCE``
+  - 0x0020
+  - Ask V4L2 to wait on fence passed in ``fence_fd`` field. The buffer
+   won't be queued to the driver until the fence signals.
+
+* .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+  - ``V4L2_BUF_FLAG_OUT_FENCE``
+  - 0x0040
+  - Request a fence for the next buffer to be queued to V4L2 driver.
+   The fence received back through the ``fence_fd`` field  doesn't
+   necessarily relate to the current buffer in the
+   :ref:`VIDIOC_QBUF ` ioctl. Although, most of the time
+   the fence will relate to the current buffer it can't be guaranteed.
+   So to tell userspace which buffer is associated to the out_fence,
+   one should listen for the ``V4L2_EVENT_BUF_QUEUED`` event that
+   provide the id of the buffer when it is queued to the V4L2 driver.



diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c 
b/drivers/media/usb/cpia2/cpia2_v4l.c
index 3dedd83f0b19..6cde686bf44c 100644
--- a/drivers/media/usb/cpia2/cpia2_v4l.c
+++ b/drivers/media/usb/cpia2/cpia2_v4l.c
@@ -948,7 +948,7 @@ static int cpia2_dqbuf(struct file *file, void *fh, struct 
v4l2_buffer *buf)
buf->sequence = cam->buffers[buf->index].seq;
buf->m.offset = cam->buffers[buf->index].data - cam->frame_buffer;
buf->length = cam->frame_size;
-   buf->reserved2 = 0;
+   buf->fence_fd = -1;
buf->reserved = 0;
memset(>timecode, 0, sizeof(buf->timecode));

diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c 
b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 821f2aa299ae..d624fb5df130 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -370,7 +370,7 @@ struct v4l2_buffer32 {
__s32   fd;
} m;
__u32   length;
-   __u32   reserved2;
+   __s32   fence_fd;
__u32   reserved;
};

@@ -533,8 +533,8 @@ static int put_v4l2_buffer32(struct v4l2_buffer *kp, struct 
v4l2_buffer32 __user
put_user(kp->timestamp.tv_usec, >timestamp.tv_usec) ||
copy_to_user(>timecode, >timecode, sizeof(struct 
v4l2_timecode)) ||
put_user(kp->sequence, >sequence) ||
-   put_user(kp->reserved2, >reserved2) ||
put_user(kp->reserved, >reserved) ||
+   put_user(kp->fence_fd, >fence_fd) ||
put_user(kp->length, >length))
return -EFAULT;

diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c 
b/drivers/media/v4l2-core/videobuf2-v4l2.c
index 0c0669976bdc..110fb45fef6f 100644
--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
@@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void 
*pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
-   b->reserved2 = 0;
+   b->fence_fd = -1;
b->reserved = 0;

if (q->is_multiplanar) {
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 185d6a0acc06..e5abab9a908c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -924,7 +924,7 @@ struct v4l2_buffer {

Re: [PATCH v3 00/15] V4L2 Explicit Synchronization support

[Brian Starkey]

Hi Gustavo,

On Thu, Sep 07, 2017 at 03:42:11PM -0300, Gustavo Padovan wrote:

From: Gustavo Padovan 

Hi,

Refer to the documentation on the first patch for the details. The previous
iteration is here: 
https://www.mail-archive.com/linux-media@vger.kernel.org/msg118077.html

The 2nd patch proposes an userspace API for fences, then on patch 3 we
prepare to the addition of in-fences in patch 4, by introducing the
infrastructure on vb2 to wait on an in-fence signal before queueing the
buffer in the driver.

Patch 5 fix uvc v4l2 event handling and patch 6 configure q->dev for
vivid drivers to enable to subscribe and dequeue events on it.

Patches 7-9 enables support to notify BUF_QUEUED events, the event send
to userspace the out-fence fd and the index of the buffer that was queued.

Patches 10-11 add support to mark queues as ordered. Finally patches 12
and 13 add more fence infrastructure to support out-fences, patch 13 exposes
close_fd() and patch 14 adds support to out-fences.

It only works for ordered queues for now, see open question at the end
of the letter.

Test tool can be found at:
https://git.collabora.com/cgit/user/padovan/v4l2-test.git/

Main Changes


* out-fences: change in behavior: the out-fence fd now comes out of the
BUF_QUEUED event along with the buffer id.


The more I think about this, the more unfortunate it seems. Especially
for our use-case (m2m engine which sits in front of the display
processor to convert the format of some buffers), having to wait for
the in-fence to signal before we can get an out-fence removes a lot of
the advantages of having fences at all.

Ideally, we'd like to queue up our m2m work (while the GPU is still
rendering that buffer, holding the in-fence), immediately get the
out-fence for the m2m work, and pass that to DRM as the in-fence for
display. With the current behaviour we need to wait in userspace
before we can pass the buffer to display.

Wouldn't it be possible to enforce that the buffers aren't queued
out-of-order in VB2? An easy way might be to (in qbuf) set a buffer's
->in_fence to be a fence_array of all the ->in_fences from the buffers
before it in the queue (and its own). That would then naturally order
the enqueue-ing in the driver, and allow you to return the out-fence
immediately.

This would also solve your output devices question from below - a
buffer can never get queued in the driver until all of the buffers
which were QBUF'd before it are queued in the driver.

Cheers,
-Brian



All other changes are recorded on the patches' commit messages.

Open Questions
--

* non-ordered devices, like m2m: I've been thinking a lot about those
 and one possibility is to have a way to tell userspace that the queue
 is not ordered and then associate the fence with the current buffer in
 QBUF instead of the next one to be queued. Of course, there won't be
 any ordering between the fences. But it may be enough for userspace to
 take advantage of Explicit Synchronization in such cases. Any
 thoughts?

* OUTPUT devices and in-fence. If I understood OUTPUT devices correctly
 it is desirable to queue the buffers to the driver in the same order
 we received them from userspace. If that is correct, shouldn't we add
 some mechanism to prevent buffer whose fence signaled to jump ahead of
 others?

Gustavo Padovan (14):
 [media] v4l: Document explicit synchronization behaviour
 [media] vb2: add explicit fence user API
 [media] vb2: check earlier if stream can be started
 [media] vb2: add in-fence support to QBUF
 [media] uvc: enable subscriptions to other events
 [media] vivid: assign the specific device to the vb2_queue->dev
 [media] v4l: add V4L2_EVENT_BUF_QUEUED event
 [media] vb2: add .buffer_queued() to notify queueing in the driver
 [media] v4l: add support to BUF_QUEUED event
 [media] vb2: add 'ordered' property to queues
 [media] vivid: mark vivid queues as ordered
 [media] vb2: add infrastructure to support out-fences
 fs/files: export close_fd() symbol
 [media] vb2: add out-fence support to QBUF

Javier Martinez Canillas (1):
 [media] vb2: add videobuf2 dma-buf fence helpers

Documentation/media/uapi/v4l/buffer.rst |  19 ++
Documentation/media/uapi/v4l/vidioc-dqevent.rst |  23 +++
Documentation/media/uapi/v4l/vidioc-qbuf.rst|  31 
Documentation/media/videodev2.h.rst.exceptions  |   1 +
drivers/android/binder.c|   2 +-
drivers/media/platform/vivid/vivid-core.c   |  15 +-
drivers/media/usb/cpia2/cpia2_v4l.c |   2 +-
drivers/media/usb/uvc/uvc_v4l2.c|   2 +-
drivers/media/v4l2-core/Kconfig |   1 +
drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |   4 +-
drivers/media/v4l2-core/v4l2-ctrls.c|   6 +-
drivers/media/v4l2-core/videobuf2-core.c| 221 ++--
drivers/media/v4l2-core/videobuf2-v4l2.c|  63 ++-
fs/file.c   |   5 +-
fs/open.c   

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

Hi Laurent,

On Thu, Aug 31, 2017 at 05:51:33PM +0300, Laurent Pinchart wrote:

Hi Brian,

On Wednesday, 30 August 2017 13:32:01 EEST Brian Starkey wrote:

On Wed, Aug 30, 2017 at 11:53:58AM +0200, Hans Verkuil wrote:
> On 30/08/17 11:36, Brian Starkey wrote:
>> On Wed, Aug 30, 2017 at 10:10:01AM +0200, Hans Verkuil wrote:
>>> On 30/08/17 09:50, Daniel Vetter wrote:
>>>> On Tue, Aug 29, 2017 at 10:47:01AM +0100, Brian Starkey wrote:
>>>>> The fact is, adding special formats for each combination is
>>>>> unmanageable - we're talking dozens in the case of our hardware.
>>>>
>>>> Hm right, we can just remap the special combos to the drm-fourcc +
>>>> modifier style. Bonus point if v4l does that in the core so not
>>>> everyone has to reinvent that wheel :-)
>>>
>>> Probably not something we'll do: there are I believe only two drivers
>>> that are affected (exynos & mediatek), so they can do that in their
>>> driver.
>>>
>>> Question: how many modifiers will typically apply to a format? I ask
>>> because I realized that V4L2 could use VIDIOC_ENUMFMT to make the link
>>> between a fourcc and modifiers:
>>>
>>> https://hverkuil.home.xs4all.nl/spec/uapi/v4l/vidioc-enum-fmt.html
>>>
>>> The __u32 reserved[4] array can be used to provide a bitmask to modifier
>>> indices (for the integer menu control). It's similar to what drm does,
>>> except instead of modifiers mapping to fourccs it is the other way
>>> around.
>>>
>>> This would avoid having to change the modifiers control whenever a new
>>> format is set and it makes it easy to enumerate all combinations.
>>>
>>> But this only works if the total number of modifiers used by a single
>>> driver is expected to remain small (let's say no more than 64).
>>
>> In our current (yet to be submitted) description, we've got around a
>> dozen modifiers for any one format to describe our compression
>> variants. We have a lot of on/off toggles which leads to combinatorial
>> expansion, so it can grow pretty quickly (though I am trying to limit
>> the valid combinations as much as possible).
>>
>> How about if the mask fills up then VIDIOC_ENUM_FMT can return another
>> fmtdsc with the same FourCC and different modifier bitmask, where the
>> second one's modifier bitmask is for the next "N" modifiers?
>
> I was thinking along similar lines, but it could cause some problems with
> the ABI since applications currently assume that no fourcc will appear
> twice when enumerating formats. Admittedly, we never explicitly said in
> the spec that that can't happen, but it is kind of expected.
>
> There are ways around that, but if possible I'd like to avoid that.
>
> In theory there are up to 128 bits available but I can't help thinking
> that if you create more than, say, 64 modifiers for a HW platform you
> have a big mess anyway.
>
> If I am wrong, then I need to know because then I can prepare for it
> (or whoever is going to actually implement this...)

You're probably right, but I can't speak for everyone. From the
current state of drm_fourcc.h it looks like 64 would be plenty (there
aren't anywhere near 64 modifiers even defined right now). Adding in
the Arm compression formats will expand it a lot, but still not to 64
(yet).


Do all those modifiers make sense on the V4L2 side ? I expect that some
modifiers will mostly be used for buffers shared between the GPU and the
display engine, while others will be used by codecs. The sets will likely
overlap, but might not be identical.



All of the ones from drm_fourcc.h - I expect not. In the case of Arm's
framebuffer compression though, it's used in all our media IPs;
Display, GPU and Video, and ideally on all data exchanged between
them. For the most part the modifiers describing it could apply to
all three.

There's also the question of what you're calling a codec - mem2mem
rotation blocks/scalers/etc. exposed via V4L2 would likely want the
same set as any display device which consumes their output (which I
think puts them firmly in the shared between the XXX and display
engine camp).

Cheers,
-Brian


> If the number of modifiers is expected to be limited then making 64 bits
> available would be good enough, at least for now.
>
> BTW, is a modifier always optional? I.e. for all fourccs, is the
> unmodified format always available? Or are there fourccs that require the
> use of a modifier?

We do actually have one or two formats which are only supported with a
modifier (on our HW).


--
Regards,

Laurent Pinchart

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

On Wed, Aug 30, 2017 at 11:53:58AM +0200, Hans Verkuil wrote:

On 30/08/17 11:36, Brian Starkey wrote:

On Wed, Aug 30, 2017 at 10:10:01AM +0200, Hans Verkuil wrote:

On 30/08/17 09:50, Daniel Vetter wrote:

On Tue, Aug 29, 2017 at 10:47:01AM +0100, Brian Starkey wrote:

The fact is, adding special formats for each combination is
unmanageable - we're talking dozens in the case of our hardware.


Hm right, we can just remap the special combos to the drm-fourcc +
modifier style. Bonus point if v4l does that in the core so not everyone
has to reinvent that wheel :-)


Probably not something we'll do: there are I believe only two drivers that
are affected (exynos & mediatek), so they can do that in their driver.

Question: how many modifiers will typically apply to a format? I ask
because I realized that V4L2 could use VIDIOC_ENUMFMT to make the link
between a fourcc and modifiers:

https://hverkuil.home.xs4all.nl/spec/uapi/v4l/vidioc-enum-fmt.html

The __u32 reserved[4] array can be used to provide a bitmask to modifier
indices (for the integer menu control). It's similar to what drm does,
except instead of modifiers mapping to fourccs it is the other way around.

This would avoid having to change the modifiers control whenever a new
format is set and it makes it easy to enumerate all combinations.

But this only works if the total number of modifiers used by a single driver
is expected to remain small (let's say no more than 64).


In our current (yet to be submitted) description, we've got around a
dozen modifiers for any one format to describe our compression
variants. We have a lot of on/off toggles which leads to combinatorial
expansion, so it can grow pretty quickly (though I am trying to limit
the valid combinations as much as possible).

How about if the mask fills up then VIDIOC_ENUM_FMT can return another
fmtdsc with the same FourCC and different modifier bitmask, where the
second one's modifier bitmask is for the next "N" modifiers?


I was thinking along similar lines, but it could cause some problems with
the ABI since applications currently assume that no fourcc will appear
twice when enumerating formats. Admittedly, we never explicitly said in
the spec that that can't happen, but it is kind of expected.

There are ways around that, but if possible I'd like to avoid that.

In theory there are up to 128 bits available but I can't help thinking
that if you create more than, say, 64 modifiers for a HW platform you
have a big mess anyway.

If I am wrong, then I need to know because then I can prepare for it
(or whoever is going to actually implement this...)


You're probably right, but I can't speak for everyone. From the
current state of drm_fourcc.h it looks like 64 would be plenty (there
aren't anywhere near 64 modifiers even defined right now). Adding in
the Arm compression formats will expand it a lot, but still not to 64
(yet).



If the number of modifiers is expected to be limited then making 64 bits
available would be good enough, at least for now.

BTW, is a modifier always optional? I.e. for all fourccs, is the unmodified
format always available? Or are there fourccs that require the use of a
modifier?


We do actually have one or two formats which are only supported with a
modifier (on our HW).

-Brian



Regards,

Hans

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

On Wed, Aug 30, 2017 at 10:10:01AM +0200, Hans Verkuil wrote:

On 30/08/17 09:50, Daniel Vetter wrote:

On Tue, Aug 29, 2017 at 10:47:01AM +0100, Brian Starkey wrote:

The fact is, adding special formats for each combination is
unmanageable - we're talking dozens in the case of our hardware.


Hm right, we can just remap the special combos to the drm-fourcc +
modifier style. Bonus point if v4l does that in the core so not everyone
has to reinvent that wheel :-)


Probably not something we'll do: there are I believe only two drivers that
are affected (exynos & mediatek), so they can do that in their driver.

Question: how many modifiers will typically apply to a format? I ask
because I realized that V4L2 could use VIDIOC_ENUMFMT to make the link
between a fourcc and modifiers:

https://hverkuil.home.xs4all.nl/spec/uapi/v4l/vidioc-enum-fmt.html

The __u32 reserved[4] array can be used to provide a bitmask to modifier
indices (for the integer menu control). It's similar to what drm does,
except instead of modifiers mapping to fourccs it is the other way around.

This would avoid having to change the modifiers control whenever a new
format is set and it makes it easy to enumerate all combinations.

But this only works if the total number of modifiers used by a single driver
is expected to remain small (let's say no more than 64).


In our current (yet to be submitted) description, we've got around a
dozen modifiers for any one format to describe our compression
variants. We have a lot of on/off toggles which leads to combinatorial
expansion, so it can grow pretty quickly (though I am trying to limit
the valid combinations as much as possible).

How about if the mask fills up then VIDIOC_ENUM_FMT can return another
fmtdsc with the same FourCC and different modifier bitmask, where the
second one's modifier bitmask is for the next "N" modifiers?

-Brian


Regards,

Hans

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

On Mon, Aug 28, 2017 at 10:49:07PM +0200, Daniel Vetter wrote:

On Mon, Aug 28, 2017 at 8:07 PM, Nicolas Dufresne <nico...@ndufresne.ca> wrote:

Le jeudi 24 ao??t 2017 ?? 13:26 +0100, Brian Starkey a ??crit :

> What I mean was: an application can use the modifier to give buffers from
> one device to another without needing to understand it.
>
> But a generic video capture application that processes the video itself
> cannot be expected to know about the modifiers. It's a custom HW specific
> format that you only use between two HW devices or with software written
> for that hardware.
>

Yes, makes sense.

> >
> > However, in DRM the API lets you get the supported formats for each
> > modifier as-well-as the modifier list itself. I'm not sure how exactly
> > to provide that in a control.
>
> We have support for a 'menu' of 64 bit integers: V4L2_CTRL_TYPE_INTEGER_MENU.
> You use VIDIOC_QUERYMENU to enumerate the available modifiers.
>
> So enumerating these modifiers would work out-of-the-box.

Right. So I guess the supported set of formats could be somehow
enumerated in the menu item string. In DRM the pairs are (modifier +
bitmask) where bits represent formats in the supported formats list
(commit db1689aa61bd in drm-next). Printing a hex representation of
the bitmask would be functional but I concede not very pretty.


The problem is that the list of modifiers depends on the format
selected. Having to call S_FMT to obtain this list is quite
inefficient.

Also, be aware that DRM_FORMAT_MOD_SAMSUNG_64_32_TILE modifier has been
implemented in V4L2 with a direct format (V4L2_PIX_FMT_NV12MT). I think
an other one made it the same way recently, something from Mediatek if
I remember. Though, unlike the Intel one, the same modifier does not
have various result depending on the hardware revision.


Note on the intel modifers: On most recent platforms (iirc gen9) the
modifier is well defined and always describes the same byte layout. We
simply didn't want to rewrite our entire software stack for all the
old gunk platforms, hence the language. I guess we could/should
describe the layout in detail, but atm we're the only ones using it.

On your topic of v4l2 encoding the drm fourcc+modifier combo into a
special v4l fourcc: That's exactly the mismatch I was thinking of.
There's other examples of v4l2 fourcc being more specific than their
drm counters (e.g. specific way the different planes are laid out).


I'm not entirely clear on the v4l2 fourccs being more specific than
DRM ones - do you mean e.g. NV12 vs NV12M? Specifically in the case of
multi-planar formats I think it's a non-issue because modifiers are
allowed to alter the number of planes and the meanings of them. Also
V4L2 NV12M is a superset of NV12 - so NV12M would always be able to
describe a DRM NV12 buffer.

I don't see the "special v4l2 format already exists" case as a problem
either. It would be up to any drivers that already have special
formats to decide if they want to also support it via a more generic
modifiers API or not.

The fact is, adding special formats for each combination is
unmanageable - we're talking dozens in the case of our hardware.

Cheers,
-Brian


-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

On Fri, Aug 25, 2017 at 10:14:03AM +0200, Hans Verkuil wrote:

On 24/08/17 14:26, Brian Starkey wrote:

On Thu, Aug 24, 2017 at 01:37:35PM +0200, Hans Verkuil wrote:

On 08/24/17 13:14, Brian Starkey wrote:

Hi Hans,

On Mon, Aug 21, 2017 at 06:36:29PM +0200, Hans Verkuil wrote:

On 08/21/2017 06:01 PM, Daniel Vetter wrote:

On Mon, Aug 21, 2017 at 5:52 PM, Brian Starkey <brian.star...@arm.com> wrote:

Hi all,

I couldn't find this topic talked about elsewhere, but apologies if
it's a duplicate - I'll be glad to be steered in the direction of a
thread.

We'd like to support DRM format modifiers in v4l2 in order to share
the description of different (mostly proprietary) buffer formats
between e.g. a v4l2 device and a DRM device.

DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
are a vendor-namespaced 64-bit value used to describe various
vendor-specific buffer layouts. They are combined with a (DRM) FourCC
code to give a complete description of the data contained in a buffer.

The same modifier definition is used in the Khronos EGL extension
EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
Wayland linux-dmabuf protocol.


This buffer information could of course be described in the
vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
information already defined in drm_fourcc.h. Additionally, there
would be quite a format explosion where a device supports a dozen or
more formats, all of which can use one or more different
layouts/compression schemes.

So, I'm wondering if anyone has views on how/whether this could be
incorporated?

I spoke briefly about this to Laurent at LPC last year, and he
suggested v4l2_control as one approach.

I also wondered if could be added in v4l2_pix_format_mplane - looks
like there's 8 bytes left before it exceeds the 200 bytes, or could go
in the reserved portion of v4l2_plane_pix_format.

Thanks for any thoughts,


One problem is that the modifers sometimes reference the DRM fourcc
codes. v4l has a different (and incompatible set) of fourcc codes,
whereas all the protocols and specs (you can add DRI3.1 for Xorg to
that list btw) use both drm fourcc and drm modifiers.

This might or might not make this proposal unworkable, but it's
something I'd at least review carefully.

Otherwise I think it'd be great if we could have one namespace for all
modifiers, that's pretty much why we have them. Please also note that
for drm_fourcc.h we don't require an in-kernel user for a new modifier
since a bunch of them might need to be allocated just for
userspace-to-userspace buffer sharing (e.g. in EGL/vk). One example
for this would be compressed surfaces with fast-clearing, which is
planned for i915 (but current hw can't scan it out). And we really
want to have one namespace for everything.


Who sets these modifiers? Kernel or userspace? Or can it be set by both?
I assume any userspace code that sets/reads this is code specific for that
hardware?


I think normally the modifier would be set by userspace. However it
might not necessarily be device-specific code. In DRM the intention is
for userspace to query the set of modifiers which are supported, and
then use them without necessarily knowing exactly what they mean
(insofar as that is possible).

e.g. if I have two devices which support MODIFIER_FOO, I could attempt
to share a buffer between them which uses MODIFIER_FOO without
necessarily knowing exactly what it is/does.



I think Laurent's suggestion of using a 64 bit V4L2 control for this makes
the most sense.

Especially if you can assume that whoever sets this knows the hardware.

I think this only makes sense if you pass buffers from one HW device to another.

Because you cannot expect generic video capture code to be able to interpret
all the zillion different combinations of modifiers.


I don't quite follow this last bit. The control could report the set
of supported modifiers.


What I mean was: an application can use the modifier to give buffers from
one device to another without needing to understand it.

But a generic video capture application that processes the video itself
cannot be expected to know about the modifiers. It's a custom HW specific
format that you only use between two HW devices or with software written
for that hardware.



Yes, makes sense.



However, in DRM the API lets you get the supported formats for each
modifier as-well-as the modifier list itself. I'm not sure how exactly
to provide that in a control.


We have support for a 'menu' of 64 bit integers: V4L2_CTRL_TYPE_INTEGER_MENU.
You use VIDIOC_QUERYMENU to enumerate the available modifiers.

So enumerating these modifiers would work out-of-the-box.


Right. So I guess the supported set of formats could be somehow
enumerated in the menu item string. In DRM the pairs are (modifier +
bitmask) where bits represent formats in the supported formats list
(commit db1689aa61bd in drm-next). Printing a hex representation of
the bitmask would be func

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

On Thu, Aug 24, 2017 at 01:37:35PM +0200, Hans Verkuil wrote:

On 08/24/17 13:14, Brian Starkey wrote:

Hi Hans,

On Mon, Aug 21, 2017 at 06:36:29PM +0200, Hans Verkuil wrote:

On 08/21/2017 06:01 PM, Daniel Vetter wrote:

On Mon, Aug 21, 2017 at 5:52 PM, Brian Starkey <brian.star...@arm.com> wrote:

Hi all,

I couldn't find this topic talked about elsewhere, but apologies if
it's a duplicate - I'll be glad to be steered in the direction of a
thread.

We'd like to support DRM format modifiers in v4l2 in order to share
the description of different (mostly proprietary) buffer formats
between e.g. a v4l2 device and a DRM device.

DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
are a vendor-namespaced 64-bit value used to describe various
vendor-specific buffer layouts. They are combined with a (DRM) FourCC
code to give a complete description of the data contained in a buffer.

The same modifier definition is used in the Khronos EGL extension
EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
Wayland linux-dmabuf protocol.


This buffer information could of course be described in the
vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
information already defined in drm_fourcc.h. Additionally, there
would be quite a format explosion where a device supports a dozen or
more formats, all of which can use one or more different
layouts/compression schemes.

So, I'm wondering if anyone has views on how/whether this could be
incorporated?

I spoke briefly about this to Laurent at LPC last year, and he
suggested v4l2_control as one approach.

I also wondered if could be added in v4l2_pix_format_mplane - looks
like there's 8 bytes left before it exceeds the 200 bytes, or could go
in the reserved portion of v4l2_plane_pix_format.

Thanks for any thoughts,


One problem is that the modifers sometimes reference the DRM fourcc
codes. v4l has a different (and incompatible set) of fourcc codes,
whereas all the protocols and specs (you can add DRI3.1 for Xorg to
that list btw) use both drm fourcc and drm modifiers.

This might or might not make this proposal unworkable, but it's
something I'd at least review carefully.

Otherwise I think it'd be great if we could have one namespace for all
modifiers, that's pretty much why we have them. Please also note that
for drm_fourcc.h we don't require an in-kernel user for a new modifier
since a bunch of them might need to be allocated just for
userspace-to-userspace buffer sharing (e.g. in EGL/vk). One example
for this would be compressed surfaces with fast-clearing, which is
planned for i915 (but current hw can't scan it out). And we really
want to have one namespace for everything.


Who sets these modifiers? Kernel or userspace? Or can it be set by both?
I assume any userspace code that sets/reads this is code specific for that
hardware?


I think normally the modifier would be set by userspace. However it
might not necessarily be device-specific code. In DRM the intention is
for userspace to query the set of modifiers which are supported, and
then use them without necessarily knowing exactly what they mean
(insofar as that is possible).

e.g. if I have two devices which support MODIFIER_FOO, I could attempt
to share a buffer between them which uses MODIFIER_FOO without
necessarily knowing exactly what it is/does.



I think Laurent's suggestion of using a 64 bit V4L2 control for this makes
the most sense.

Especially if you can assume that whoever sets this knows the hardware.

I think this only makes sense if you pass buffers from one HW device to another.

Because you cannot expect generic video capture code to be able to interpret
all the zillion different combinations of modifiers.


I don't quite follow this last bit. The control could report the set
of supported modifiers.


What I mean was: an application can use the modifier to give buffers from
one device to another without needing to understand it.

But a generic video capture application that processes the video itself
cannot be expected to know about the modifiers. It's a custom HW specific
format that you only use between two HW devices or with software written
for that hardware.



Yes, makes sense.



However, in DRM the API lets you get the supported formats for each
modifier as-well-as the modifier list itself. I'm not sure how exactly
to provide that in a control.


We have support for a 'menu' of 64 bit integers: V4L2_CTRL_TYPE_INTEGER_MENU.
You use VIDIOC_QUERYMENU to enumerate the available modifiers.

So enumerating these modifiers would work out-of-the-box.


Right. So I guess the supported set of formats could be somehow
enumerated in the menu item string. In DRM the pairs are (modifier +
bitmask) where bits represent formats in the supported formats list
(commit db1689aa61bd in drm-next). Printing a hex representation of
the bitmask would be functional but I concede not very pretty.

Cheers,
-Brian



Regards,

Hans

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

Hi Hans,

On Mon, Aug 21, 2017 at 06:36:29PM +0200, Hans Verkuil wrote:

On 08/21/2017 06:01 PM, Daniel Vetter wrote:

On Mon, Aug 21, 2017 at 5:52 PM, Brian Starkey <brian.star...@arm.com> wrote:

Hi all,

I couldn't find this topic talked about elsewhere, but apologies if
it's a duplicate - I'll be glad to be steered in the direction of a
thread.

We'd like to support DRM format modifiers in v4l2 in order to share
the description of different (mostly proprietary) buffer formats
between e.g. a v4l2 device and a DRM device.

DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
are a vendor-namespaced 64-bit value used to describe various
vendor-specific buffer layouts. They are combined with a (DRM) FourCC
code to give a complete description of the data contained in a buffer.

The same modifier definition is used in the Khronos EGL extension
EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
Wayland linux-dmabuf protocol.


This buffer information could of course be described in the
vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
information already defined in drm_fourcc.h. Additionally, there
would be quite a format explosion where a device supports a dozen or
more formats, all of which can use one or more different
layouts/compression schemes.

So, I'm wondering if anyone has views on how/whether this could be
incorporated?

I spoke briefly about this to Laurent at LPC last year, and he
suggested v4l2_control as one approach.

I also wondered if could be added in v4l2_pix_format_mplane - looks
like there's 8 bytes left before it exceeds the 200 bytes, or could go
in the reserved portion of v4l2_plane_pix_format.

Thanks for any thoughts,


One problem is that the modifers sometimes reference the DRM fourcc
codes. v4l has a different (and incompatible set) of fourcc codes,
whereas all the protocols and specs (you can add DRI3.1 for Xorg to
that list btw) use both drm fourcc and drm modifiers.

This might or might not make this proposal unworkable, but it's
something I'd at least review carefully.

Otherwise I think it'd be great if we could have one namespace for all
modifiers, that's pretty much why we have them. Please also note that
for drm_fourcc.h we don't require an in-kernel user for a new modifier
since a bunch of them might need to be allocated just for
userspace-to-userspace buffer sharing (e.g. in EGL/vk). One example
for this would be compressed surfaces with fast-clearing, which is
planned for i915 (but current hw can't scan it out). And we really
want to have one namespace for everything.


Who sets these modifiers? Kernel or userspace? Or can it be set by both?
I assume any userspace code that sets/reads this is code specific for that
hardware?


I think normally the modifier would be set by userspace. However it
might not necessarily be device-specific code. In DRM the intention is
for userspace to query the set of modifiers which are supported, and
then use them without necessarily knowing exactly what they mean
(insofar as that is possible).

e.g. if I have two devices which support MODIFIER_FOO, I could attempt
to share a buffer between them which uses MODIFIER_FOO without
necessarily knowing exactly what it is/does.



I think Laurent's suggestion of using a 64 bit V4L2 control for this makes
the most sense.

Especially if you can assume that whoever sets this knows the hardware.

I think this only makes sense if you pass buffers from one HW device to another.

Because you cannot expect generic video capture code to be able to interpret
all the zillion different combinations of modifiers.


I don't quite follow this last bit. The control could report the set
of supported modifiers.

However, in DRM the API lets you get the supported formats for each
modifier as-well-as the modifier list itself. I'm not sure how exactly
to provide that in a control.

Thanks,
-Brian



Regards,

Hans

Re: DRM Format Modifiers in v4l2

[Brian Starkey]

On Mon, Aug 21, 2017 at 06:01:24PM +0200, Daniel Vetter wrote:

On Mon, Aug 21, 2017 at 5:52 PM, Brian Starkey <brian.star...@arm.com> wrote:

Hi all,

I couldn't find this topic talked about elsewhere, but apologies if
it's a duplicate - I'll be glad to be steered in the direction of a
thread.

We'd like to support DRM format modifiers in v4l2 in order to share
the description of different (mostly proprietary) buffer formats
between e.g. a v4l2 device and a DRM device.

DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
are a vendor-namespaced 64-bit value used to describe various
vendor-specific buffer layouts. They are combined with a (DRM) FourCC
code to give a complete description of the data contained in a buffer.

The same modifier definition is used in the Khronos EGL extension
EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
Wayland linux-dmabuf protocol.


This buffer information could of course be described in the
vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
information already defined in drm_fourcc.h. Additionally, there
would be quite a format explosion where a device supports a dozen or
more formats, all of which can use one or more different
layouts/compression schemes.

So, I'm wondering if anyone has views on how/whether this could be
incorporated?

I spoke briefly about this to Laurent at LPC last year, and he
suggested v4l2_control as one approach.

I also wondered if could be added in v4l2_pix_format_mplane - looks
like there's 8 bytes left before it exceeds the 200 bytes, or could go
in the reserved portion of v4l2_plane_pix_format.

Thanks for any thoughts,


One problem is that the modifers sometimes reference the DRM fourcc
codes. v4l has a different (and incompatible set) of fourcc codes,
whereas all the protocols and specs (you can add DRI3.1 for Xorg to
that list btw) use both drm fourcc and drm modifiers.



This problem already exists (ignoring modifiers) in the case of any
v4l2 <-> DRM buffer sharing (direct video scanout, for instance).

I was hoping it would be possible to draw enough equivalency between
the different definitions to manage a useful subset through a 1:1
lookup table. If that's not possible then this gets a whole lot more
tricky. Are you already aware of incompatibilities which would prevent
it?

-Brian


This might or might not make this proposal unworkable, but it's
something I'd at least review carefully.

Otherwise I think it'd be great if we could have one namespace for all
modifiers, that's pretty much why we have them. Please also note that
for drm_fourcc.h we don't require an in-kernel user for a new modifier
since a bunch of them might need to be allocated just for
userspace-to-userspace buffer sharing (e.g. in EGL/vk). One example
for this would be compressed surfaces with fast-clearing, which is
planned for i915 (but current hw can't scan it out). And we really
want to have one namespace for everything.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

DRM Format Modifiers in v4l2

[Brian Starkey]

Hi all,

I couldn't find this topic talked about elsewhere, but apologies if
it's a duplicate - I'll be glad to be steered in the direction of a
thread.

We'd like to support DRM format modifiers in v4l2 in order to share
the description of different (mostly proprietary) buffer formats
between e.g. a v4l2 device and a DRM device.

DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
are a vendor-namespaced 64-bit value used to describe various
vendor-specific buffer layouts. They are combined with a (DRM) FourCC
code to give a complete description of the data contained in a buffer.

The same modifier definition is used in the Khronos EGL extension
EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
Wayland linux-dmabuf protocol.


This buffer information could of course be described in the
vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
information already defined in drm_fourcc.h. Additionally, there
would be quite a format explosion where a device supports a dozen or
more formats, all of which can use one or more different
layouts/compression schemes.

So, I'm wondering if anyone has views on how/whether this could be
incorporated?

I spoke briefly about this to Laurent at LPC last year, and he
suggested v4l2_control as one approach.

I also wondered if could be added in v4l2_pix_format_mplane - looks
like there's 8 bytes left before it exceeds the 200 bytes, or could go
in the reserved portion of v4l2_plane_pix_format.

Thanks for any thoughts,
-Brian

Re: [PATCH 1/6] drm: Add writeback connector type

[Brian Starkey]

On Wed, Apr 19, 2017 at 01:34:34PM +0200, Boris Brezillon wrote:

On Wed, 19 Apr 2017 10:51:23 +0100
Brian Starkey <brian.star...@arm.com> wrote:


[snip]


Could you expand a bit on how you think planes fit better?


Just had the impression that the writeback feature was conceptually
closer to a plane object (which is attached buffers and expose ways to
specify which portion of the buffer should be used, provides way to
atomically switch 2 buffers, ...).


Yeah sort-of, except that SRC_X/Y/W/H doesn't mean the same for an
"output" plane as an "input" plane (and CRTC_X/Y/W/H similarly,
probably other properties too).

In atomic land, the swapping of buffers is really just the swapping of
object IDs via properties - I don't think planes actually have
anything special in terms of buffer handling, except for all the
legacy state handling cruft.




It is
something we've previously talked about internally, but so far I'm not
convinced :-)


Okay, as I said, I don't know all the history, hence my questions ;-).



I think that history was here in our office rather than on the list
anyway.



>By doing that, we would also get rid of these fake connector/encoder
>objects as well as the fake modes we are returning in
>connector->get_modes().

What makes the connector/encoder fake? They represent a real piece of
hardware just the same as a drm_plane would.


Well, that's probably subject to interpretation, but I don't consider
these writeback encoders/connectors as real encoders/connectors. They
are definitely real HW blocks, but not what we usually consider as an
encoder/connector.



This is true



I don't mind dropping the mode list and letting userspace just make
up whatever timing it wants - it'd need to do the same if writeback
was a plane - but in some respects giving it a list of modes the same
way we normally do seems nicer for userspace.

>
>As far as I can tell, the VC4 and Atmel HLCDC IP should fit pretty well
>in this model, not sure about the mali-dp though.
>
>Brian, did you consider this approach, and if you did, can you detail
>why you decided to expose this feature as a connector?
>
>Daniel (or anyone else), please step in if you think this is a stupid
>idea :-).

Ville originally suggested using a connector, which Eric followed up
by saying that's what he was thinking of for VC4 writeback too[1].


Thanks for the pointer.


That was my initial reason for focussing on a connector-based
approach.

I prefer connector over plane conceptually because it keeps with the
established data flow: planes are sources, connectors are sinks.


Okay, it's a valid point.



In some respects the plane _object_ looks like it would fit - it has a
pixel format list and an FB_ID. But everything else would be acting
the exact opposite to a normal plane, and I think there's a bunch of
baked-in assumptions in the kernel and userspace around CRTCs always
having at least one connector.


Yep, but writeback connectors are already different enough to not be
considered as regular connectors, so userspace programs will have to
handle them differently anyway (pass a framebuffer and pixel format to
it before adding them to the display pipeline).

Anyway, I see this approach has already been suggested in [1], and you
all agreed that the writeback feature should be exposed as a connector,
so I'll just stop here :-).

Thanks for taking the time to explain the rationale behind this
decision.



No problem, now is the right time to be discussing the decision before
we merge something wrong.

Are you happy enough with the connector approach then? Any concerns
with going ahead with it?

Cheers,
-Brian

Re: [PATCH 1/6] drm: Add writeback connector type

[Brian Starkey]

On Tue, Apr 18, 2017 at 09:57:17PM +0200, Boris Brezillon wrote:

Hi Brian,

On Tue, 18 Apr 2017 18:34:43 +0100
Brian Starkey <brian.star...@arm.com> wrote:


>> @@ -214,6 +214,19 @@ struct drm_connector_state {
>>struct drm_encoder *best_encoder;
>>
>>struct drm_atomic_state *state;
>> +
>> +  /**
>> +   * @writeback_job: Writeback job for writeback connectors
>> +   *
>> +   * Holds the framebuffer for a writeback connector. As the writeback
>> +   * completion may be asynchronous to the normal commit cycle, the
>> +   * writeback job lifetime is managed separately from the normal atomic
>> +   * state by this object.
>> +   *
>> +   * See also: drm_writeback_queue_job() and
>> +   * drm_writeback_signal_completion()
>> +   */
>> +  struct drm_writeback_job *writeback_job;
>
>Maybe I'm wrong, but is feels weird to have the writeback_job field
>directly embedded in drm_connector_state, while drm_writeback_connector
>inherits from drm_connector.
>
>IMO, either you decide to directly put the drm_writeback_connector's
>job_xxx fields in drm_connector and keep the drm_connector_state as is,
>or you create a drm_writeback_connector_state which inherits from
>drm_connector_state and embeds the writeback_job field.

I did spend a decent amount of time looking at tracking the writeback
state along with the normal connector state. I couldn't come up with
anything I liked.

As the comment mentions, one of the problems is that you have to make
sure the relevant parts of the connector_state stay around until the
writeback is finished. That means you've got to block before
"swap_state()" until the previous writeback is done, and that
effectively limits your frame rate to refresh/2.

The Mali-DP HW doesn't have that limitation - we can queue up a new
commit while the current writeback is ongoing. For that reason I
didn't want to impose such a limitation in the framework.

In v1 I allowed that by making the Mali-DP driver hold its own
references to the relevant bits of the state for as long as it needed
them. In v3 I moved most of that code back to the core (in part
because Gustavo didn't like me signalling the DRM-"owned" fence from
my driver code directly). I think the new approach of "queue_job()"
and "signal_job()" reduces the amount of tricky code in drivers, and
is generally more clear (also familiar, when compared to vsync
events).

I'm certain there's other ways to do it (refcount atomic states?), but
it seemed like a biggish overhaul to achieve what would basically be
the same thing.

I was expecting each driver supporting writeback to have its own
different requirements around writeback lifetime/duration. For example
I think VC4 specifically came up, in that its writeback could take
several frames, whereas on Mali-DP we either finish within the frame
or we fail.

Letting the driver manage its writeback_job lifetime seemed like a
reasonable way to handle all that, with the documentation stating the
only behaviour which is guaranteed to work on all drivers:

   * Userspace should wait for this fence to signal before making another
   * commit affecting any of the same CRTCs, Planes or Connectors.
   * **Failure to do so will result in undefined behaviour.**
   * For this reason it is strongly recommended that all userspace
   * applications making use of writeback connectors *always* retrieve an
   * out-fence for the commit and use it appropriately.



... so all of that is why the _job fields don't live in a *_state
structure directly, and instead have to live in the separately-managed
structure pointed to by ->writeback_job.

Now, I did look at creating drm_writeback_connector_state, but as it
would only be holding the job pointer (see above) it didn't seem worth
scattering around the

if (conn_state->connector->connector_type ==
DRM_MODE_CONNECTOR_WRITEBACK)

checks everywhere before up-casting - {clear,reset,duplicate}_state(),
prepare_signalling(), complete_signalling(), etc. It just touched a
lot of code for the sake of an extra pointer field in each connector
state.

I can easily revisit that part if you like.


I think there's a misunderstanding. I was just suggesting to be
consistent in the inheritance vs 'one object to handle everything'
approach.


doh.. right yeah I misread. Sorry for the tangent. :-)



I'm perfectly fine with embedding the writeback_job pointer directly in
drm_connector_state, but then it would IMO make more sense to do the
same for the drm_connector object (embed drm_writeback_connector fields
into drm_connector instead of making drm_writeback_connector inherit
from drm_connector).



I agree that it's inconsistent. I guess I did it out of pragmatism -
there's quite a lot of new fields in drm_writ

Re: [PATCH 6/6] drm: mali-dp: Add writeback connector

[Brian Starkey]

On Fri, Apr 14, 2017 at 11:47:00AM +0200, Boris Brezillon wrote:

On Fri, 25 Nov 2016 16:49:04 +
Brian Starkey <brian.star...@arm.com> wrote:


+static int
+malidp_mw_encoder_atomic_check(struct drm_encoder *encoder,
+  struct drm_crtc_state *crtc_state,
+  struct drm_connector_state *conn_state)
+{
+   struct malidp_mw_connector_state *mw_state = to_mw_state(conn_state);
+   struct malidp_drm *malidp = encoder->dev->dev_private;
+   struct drm_framebuffer *fb;
+   int i, n_planes;
+
+   if (!conn_state->writeback_job || !conn_state->writeback_job->fb)
+   return 0;
+
+   fb = conn_state->writeback_job->fb;
+   if ((fb->width != crtc_state->mode.hdisplay) ||
+   (fb->height != crtc_state->mode.vdisplay)) {
+   DRM_DEBUG_KMS("Invalid framebuffer size %ux%u\n",
+   fb->width, fb->height);
+   return -EINVAL;
+   }


These checks look pretty generic to me. Shouldn't we have a default
helper doing that?



Yeah makes sense. These should be common to everyone until
cropping/scaling support is added.


+
+   mw_state->format =
+   malidp_hw_get_format_id(>dev->map, SE_MEMWRITE,
+   fb->pixel_format);
+   if (mw_state->format == MALIDP_INVALID_FORMAT_ID) {


Same goes here. By adding a format_types table similar to what is
exposed in drm_plane [1], we could do this check in the core. The only
thing left to the driver is the 4CC -> driver-specific-id conversion.



Yeah could do, but given our driver requires us to run through the
whole table to get the HW ID anyway it seemed like totally wasted
effort to do the same thing in the core.

It's probably a negligible overhead, but it's also unnecessary for
100% of the current writeback implementations ;-)

If a different driver is implemented such that the HW ID lookup isn't
an exhaustive list search then we could add a helper for them to use
which checks the blob.

Cheers,
-Brian


+   struct drm_format_name_buf format_name;
+
+   DRM_DEBUG_KMS("Invalid pixel format %s\n",
+ drm_get_format_name(fb->pixel_format, 
_name));
+   return -EINVAL;
+   }
+
+   n_planes = drm_format_num_planes(fb->pixel_format);
+   for (i = 0; i < n_planes; i++) {
+   struct drm_gem_cma_object *obj = drm_fb_cma_get_gem_obj(fb, i);
+   if (!malidp_hw_pitch_valid(malidp->dev, fb->pitches[i])) {
+   DRM_DEBUG_KMS("Invalid pitch %u for plane %d\n",
+ fb->pitches[i], i);
+   return -EINVAL;
+   }
+   mw_state->pitches[i] = fb->pitches[i];
+   mw_state->addrs[i] = obj->paddr + fb->offsets[i];
+   }
+   mw_state->n_planes = n_planes;
+
+   return 0;
+}



[1]http://lxr.free-electrons.com/source/include/drm/drm_plane.h#L482

Re: [PATCH 2/6] drm: writeback: Add out-fences for writeback connectors

[Brian Starkey]

On Fri, Apr 14, 2017 at 12:11:14PM +0200, Boris Brezillon wrote:

On Fri, 25 Nov 2016 16:49:00 +
Brian Starkey <brian.star...@arm.com> wrote:


Add the OUT_FENCE_PTR property to writeback connectors, to enable
userspace to get a fence which will signal once the writeback is
complete. It is not allowed to request an out-fence without a
framebuffer attached to the connector.

A timeline is added to drm_writeback_connector for use by the writeback
out-fences.

In the case of a commit failure or DRM_MODE_ATOMIC_TEST_ONLY, the fence
is set to -1.

Changes from v2:
 - Rebase onto Gustavo Padovan's v9 explicit sync series
 - Change out_fence_ptr type to s32 __user *


Don't know what happened, but I still see s32 __user * types in this
patch (I had to patch it to make in work on top of 4.11-rc1).


Yeah this really confused me too when rebasing. Given that this patch
predates Gustavo's change to s32 I can only assume I typo'd and meant
s64 in this commit message.

-Brian

Re: [PATCH 1/6] drm: Add writeback connector type

[Brian Starkey]

Hi Boris,

On Fri, Apr 14, 2017 at 12:08:23PM +0200, Boris Brezillon wrote:

On Fri, 25 Nov 2016 16:48:59 +
Brian Starkey <brian.star...@arm.com> wrote:




diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
index b5c6a8e..6bbd93f 100644
--- a/drivers/gpu/drm/drm_connector.c
+++ b/drivers/gpu/drm/drm_connector.c
@@ -86,6 +86,7 @@ struct drm_conn_prop_enum_list {
{ DRM_MODE_CONNECTOR_VIRTUAL, "Virtual" },
{ DRM_MODE_CONNECTOR_DSI, "DSI" },
{ DRM_MODE_CONNECTOR_DPI, "DPI" },
+   { DRM_MODE_CONNECTOR_WRITEBACK, "Writeback" },


Is there a reason we have a Writeback connector, but keep using a
Virtual encoder to connect it to the CRTC? Wouldn't it make more sense
to also add a Writeback encoder?



Only that a writeback connector is functionally and conceptually quite
different from the existing connector types, whereas the "encoder"
(which realistically only exists because the framework forces it to)
acts pretty much like any other.


 };

 void drm_connector_ida_init(void)
@@ -235,7 +236,8 @@ int drm_connector_init(struct drm_device *dev,
list_add_tail(>head, >connector_list);
config->num_connector++;

-   if (connector_type != DRM_MODE_CONNECTOR_VIRTUAL)
+   if ((connector_type != DRM_MODE_CONNECTOR_VIRTUAL) &&
+   (connector_type != DRM_MODE_CONNECTOR_WRITEBACK))


Nitpick: you don't need the extra parenthesis:

if (connector_type != DRM_MODE_CONNECTOR_VIRTUAL &&
connector_type != DRM_MODE_CONNECTOR_WRITEBACK)



Yeah fair enough, I can drop them.


drm_object_attach_property(>base,
  config->edid_property,
  0);





diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h
index 34f9741..dc4910d6 100644
--- a/include/drm/drm_connector.h
+++ b/include/drm/drm_connector.h
@@ -214,6 +214,19 @@ struct drm_connector_state {
struct drm_encoder *best_encoder;

struct drm_atomic_state *state;
+
+   /**
+* @writeback_job: Writeback job for writeback connectors
+*
+* Holds the framebuffer for a writeback connector. As the writeback
+* completion may be asynchronous to the normal commit cycle, the
+* writeback job lifetime is managed separately from the normal atomic
+* state by this object.
+*
+* See also: drm_writeback_queue_job() and
+* drm_writeback_signal_completion()
+*/
+   struct drm_writeback_job *writeback_job;


Maybe I'm wrong, but is feels weird to have the writeback_job field
directly embedded in drm_connector_state, while drm_writeback_connector
inherits from drm_connector.

IMO, either you decide to directly put the drm_writeback_connector's
job_xxx fields in drm_connector and keep the drm_connector_state as is,
or you create a drm_writeback_connector_state which inherits from
drm_connector_state and embeds the writeback_job field.


I did spend a decent amount of time looking at tracking the writeback
state along with the normal connector state. I couldn't come up with
anything I liked.

As the comment mentions, one of the problems is that you have to make
sure the relevant parts of the connector_state stay around until the
writeback is finished. That means you've got to block before
"swap_state()" until the previous writeback is done, and that
effectively limits your frame rate to refresh/2.

The Mali-DP HW doesn't have that limitation - we can queue up a new
commit while the current writeback is ongoing. For that reason I
didn't want to impose such a limitation in the framework.

In v1 I allowed that by making the Mali-DP driver hold its own
references to the relevant bits of the state for as long as it needed
them. In v3 I moved most of that code back to the core (in part
because Gustavo didn't like me signalling the DRM-"owned" fence from
my driver code directly). I think the new approach of "queue_job()"
and "signal_job()" reduces the amount of tricky code in drivers, and
is generally more clear (also familiar, when compared to vsync
events).

I'm certain there's other ways to do it (refcount atomic states?), but
it seemed like a biggish overhaul to achieve what would basically be
the same thing.

I was expecting each driver supporting writeback to have its own
different requirements around writeback lifetime/duration. For example
I think VC4 specifically came up, in that its writeback could take
several frames, whereas on Mali-DP we either finish within the frame
or we fail.

Letting the driver manage its writeback_job lifetime seemed like a
reasonable way to handle all that, with the documentation stating the
only behaviour which is guaranteed to work on all drivers:

  * Userspace should wait for this fence to signal before making an

Re: [RFC PATCH v3 0/6] Introduce writeback connectors

[Brian Starkey]

Hi Boris,

On Fri, Apr 14, 2017 at 11:35:17AM +0200, Boris Brezillon wrote:

Hi Brian,

On Fri, 25 Nov 2016 16:48:58 +
Brian Starkey <brian.star...@arm.com> wrote:


Hi,

This is v3 of my series introducing a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
See v1 and v2 here: [1] [2]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Writeback connectors are given a WRITEBACK_FB_ID property (which acts
slightly differently to FB_ID, so gets a new name), as well as
a PIXEL_FORMATS blob to list the supported writeback formats, and
OUT_FENCE_PTR to be used for out-fences.

The changes since v2 are in the commit messages of each commit.

The main differences are:
 - Subclass drm_connector as drm_writeback_connector
 - Slight relaxation of core checks, to allow
   (connector->crtc && !connector->fb)
 - Dropped PIXEL_FORMATS_SIZE, which was redundant
 - Reworked the event interface, drivers don't need to deal with the
   fence directly
 - Re-ordered the commits to introduce writeback out-fences up-front.

I've kept RFC on this series because the event reporting (introduction
of drm_writeback_job) is probably up for debate.

v4 will be accompanied by igt tests.


I plan to add writeback support to the VC4 driver and wanted to know if
anything has changed since this v3 (IOW, do you have a v4 + igt tests
ready)?



Oh that's good to hear. I've got a v4 (just rebased for the most
part), but was holding off sending it until having some "proper"
userspace to support it. Unfortunately in the meantime we've had some
team changes which mean I'm not really able to work on it to move
things forward - Liviu might be able to pick this up.

I'll collect together what I have and send it out anyway. It includes
some functional tests in igt, but I'm not sure if that meets the "new
uapi needs userspace" bar.



As always, I look forward to any comments.


I'll try to review these patches. Keep in mind that I didn't follow the
initial discussions and might suggest things or ask questions that have
already been answered in previous versions of this series or on IRC.


Thanks,

-Brian



Regards,

Boris

Re: DRM Atomic property for color-space conversion

[Brian Starkey]

Hi Ville,

On Thu, Mar 16, 2017 at 07:36:56PM +0200, Ville Syrjälä wrote:

On Thu, Mar 16, 2017 at 07:05:12PM +0200, Sharma, Shashank wrote:

On 3/16/2017 5:55 PM, Brian Starkey wrote:
> On Thu, Mar 16, 2017 at 05:14:07PM +0200, Sharma Shashank wrote:
>> On 3/16/2017 4:37 PM, Local user for Liviu Dudau wrote:
>>> On Thu, Mar 16, 2017 at 04:30:59PM +0200, Ville Syrjälä wrote:
>>>> On Thu, Mar 16, 2017 at 04:20:29PM +0200, Sharma, Shashank wrote:
>>>>> On 3/16/2017 4:07 PM, Ville Syrjälä wrote:


[snip]


>>>>>>
>>>>>> So what we might need is something like:
>>>>>> enum YCBCR_TO_RGB_CSC
>>>>>>   * YCbCr BT.601 limited to RGB BT.709 full
>>>>>>   * YCbCr BT.709 limited to RGB BT.709 full >>>>> likely default value IMO>
>>>>>>   * YCbCr BT.601 limited to RGB BT.2020 full
>>>>>>   * YCbCr BT.709 limited to RGB BT.2020 full
>>>>>>   * YCbCr BT.2020 limited to RGB BT.2020 full
>>>>>>
>>>>>> And thanks to BT.2020 we'll need a RGB->RGB CSC property as well.
>>>>>> Eg:
>>>>>> enum RGB_TO_RGB_CSC
>>>>>>   * bypass (or separate 709->709, 2020->2020?) >>>>> default>
>>>>>>   * RGB BT.709 full to RGB BT.2020 full
>
> I like this approach, from a point of view of being explicit and
> discoverable by userspace. It also happens to map quite nicely to our
> hardware... we have a matrix before degamma, so we could do a
> CSC + Gamut conversion there in one go, which is apparently not 100%
> mathematically correct, but in general is good enough.
>
> ... however having talked this over a bit with someone who understands
> the detail a lot better than me, it sounds like the "correct" thing to
> do as per the spec is:
>
> CSC -> DEGAMMA -> GAMUT
>
> e.g.
>
> YCbCr bt.601 limited to RGB bt.601 full -> degamma ->
> RGB bt.601 full to RGB bt.709 full
>
> So that sounds like what we need to support in the API, and also
> sounds more like the "separate properties" approach.
I agree.
>
>>>>>>
>>>>>> Alternatives would involve two properties to define the input and
>>>>>> output
>>>>>> from the CSC separately, but then you lose the capability to see
>>>>>> which
>>>>>> combinations are actually supoorted.
>>>>> I was thinking about this too, or would it make more sense to
>>>>> create two
>>>>> properties:
>>>>> - one for gamut mapping (cases like RGB709->RGB2020)
>>>>> - other one for Color space conversion (cases lile YUV 709 -> RGB
>>>>> 709)
>>>>>
>>>>> Gamut mapping can represent any of the fix function mapping,
>>>>> wereas CSC
>>>>> can bring up any programmable matrix
>>>>>
>>>>> Internally these properties can use the same HW unit or even same
>>>>> function.
>>>>> Does it sound any good ?
>
> It seems to me that actually the two approaches can be combined into
> the same thing:
>  * We definitely need a YCbCr-to-RGB conversion before degamma
>(for converting YUV data to RGB, in some flavour)
>  * We definitely need an RGB-to-RGB conversion after gamma to handle
>709 layers blended with Rec.2020.
> The exact conversion each of those properties represents (CSC + gamut,
> CSC only, gamut only) can be implicit in the enum name.
>
> For hardware which has a fixed-function CSC before DEGAMMA with a
> matrix after DEGAMMA, I'd expect to see something like below. None of
> the YCBCR_TO_RGB_CSC values include a gamut conversion, because that
> is instead exposed with the RGB_TO_RGB_CSC property (which represents
> the hardware matrix)
>
> YCBCR_TO_RGB_CSC (before DEGAMMA):
> YCbCr BT.601 limited to RGB BT.601 full
> YCbCr BT.709 limited to RGB BT.709 full
> YCbCr BT.2020 limited to RGB BT.2020 full
>
> RGB_TO_RGB_CSC (after DEGAMMA):
> RGB BT.601 full to RGB BT.709 full
> RGB BT.709 full to RGB BT.2020 full
>
>
> On the other hand, on hardware which does a CSC + Gamut conversion in
> one go, before DEGAMMA (like ours), you might have:
>
> YCBCR_TO_RGB_CSC (before DEGAMMA):
> YCbCr BT.601 limited to RGB BT.601 full
> YCbCr BT.601 limited to RGB BT.709 full
> YCbCr BT.709 limited to RGB BT.709 full
> YCbCr BT.2020 limited to RGB BT.2020 full
>
> RGB_TO_RGB_CSC (after DEGAMMA):
> Not supported
>
> Userspace can parse the

Re: DRM Atomic property for color-space conversion

[Brian Starkey]

Hi,

On Thu, Mar 16, 2017 at 05:14:07PM +0200, Sharma Shashank wrote:

Regards

Shashank


On 3/16/2017 4:37 PM, Local user for Liviu Dudau wrote:

On Thu, Mar 16, 2017 at 04:30:59PM +0200, Ville Syrjälä wrote:

On Thu, Mar 16, 2017 at 04:20:29PM +0200, Sharma, Shashank wrote:

Regards

Shashank


On 3/16/2017 4:07 PM, Ville Syrjälä wrote:

On Tue, Jan 31, 2017 at 03:55:41PM +, Brian Starkey wrote:

On Tue, Jan 31, 2017 at 05:15:46PM +0200, Ville Syrjälä wrote:

On Tue, Jan 31, 2017 at 12:33:29PM +, Brian Starkey wrote:

Hi,

On Mon, Jan 30, 2017 at 03:35:13PM +0200, Ville Syrjälä wrote:

On Fri, Jan 27, 2017 at 05:23:24PM +, Brian Starkey wrote:

Hi,

We're looking to enable the per-plane color management hardware in
Mali-DP with atomic properties, which has sparked some conversation
around how to handle YCbCr formats.

As it stands today, it's assumed that a driver will implicitly "do the
right thing" to display a YCbCr buffer.

YCbCr data often uses different gamma curves and signal ranges (e.g.
BT.609, BT.701, BT.2020, studio range, full-range), so its desirable
to be able to explicitly control the YCbCr to RGB conversion process
from userspace.

We're proposing adding a "CSC" (color-space conversion) property to
control this - primarily per-plane for framebuffer->pipeline CSC, but
perhaps one per CRTC too for devices which have an RGB pipeline and
want to output in YUV to the display:

Name: "CSC"
Type: ENUM | ATOMIC;
Enum values (representative):
"default":
Same behaviour as now. "Some kind" of YCbCr->RGB conversion
for YCbCr buffers, bypass for RGB buffers
"disable":
Explicitly disable all colorspace conversion (i.e. use an
identity matrix).
"YCbCr to RGB: BT.709":
Only valid for YCbCr formats. CSC in accordance with BT.709
using [16..235] for (8-bit) luma values, and [16..240] for
8-bit chroma values. For 10-bit formats, the range limits are
multiplied by 4.
"YCbCr to RGB: BT.709 full-swing":
Only valid for YCbCr formats. CSC in accordance with BT.709,
but using the full range of each channel.
"YCbCr to RGB: Use CTM":*
Only valid for YCbCr formats. Use the matrix applied via the
plane's CTM property
"RGB to RGB: Use CTM":*
Only valid for RGB formats. Use the matrix applied via the
plane's CTM property
"Use CTM":*
Valid for any format. Use the matrix applied via the plane's
CTM property
... any other values for BT.601, BT.2020, RGB to YCbCr etc. etc. as
they are required.

Having some RGB2RGB and YCBCR2RGB things in the same property seems
weird. I would just go with something very simple like:

YCBCR_TO_RGB_CSC:
* BT.601
* BT.709
* custom matrix


I think we've agreed in #dri-devel that this CSC property
can't/shouldn't be mapped on-to the existing (hardware implementing
the) CTM property - even in the case of per-plane color management -
because CSC needs to be done before DEGAMMA.

So, I'm in favour of going with what you suggested in the first place:

A new YCBCR_TO_RGB_CSC property, enum type, with a list of fixed
conversions. I'd drop the custom matrix for now, as we'd need to add
another property to attach the custom matrix blob too.

I still think we need a way to specify whether the source data range
is broadcast/full-range, so perhaps the enum list should be expanded
to all combinations of BT.601/BT.709 + broadcast/full-range.

Sounds reasonable. Not that much full range YCbCr stuff out there
perhaps. Well, apart from jpegs I suppose. But no harm in being able
to deal with it.


(I'm not sure what the canonical naming for broadcast/full-range is,
we call them narrow and wide)

We tend to call them full vs. limited range. That's how our
"Broadcast RGB" property is defined as well.


OK, using the same ones sounds sensible.


And trying to use the same thing for the crtc stuff is probably not
going to end well. Like Daniel said we already have the
'Broadcast RGB' property muddying the waters there, and that stuff
also ties in with what colorspace we signal to the sink via
infoframes/whatever the DP thing was called. So my gut feeling is
that trying to use the same property everywhere will just end up
messy.

Yeah, agreed. If/when someone wants to add CSC on the output of a CRTC
(after GAMMA), we can add a new property.

That makes me wonder about calling this one SOURCE_YCBCR_TO_RGB_CSC to
be explicit that it describes the source data. Then we can later add
SINK_RGB_TO_YCBCR_CSC, and it will be reasonably obvious that its
value describes the output data rather than the input data.

Source and sink have a slight connotation in my mind wrt. the box that
produces the display signal and the box that eats the signal. So trying
to use the same terms to describe the internals of the pipeline inside
the "source box" 

Re: [RFC PATCH 00/12] Ion cleanup in preparation for moving out of staging

[Brian Starkey]

On Sun, Mar 12, 2017 at 02:34:14PM +0100, Benjamin Gaignard wrote:

2017-03-09 18:38 GMT+01:00 Laura Abbott :

On 03/09/2017 02:00 AM, Benjamin Gaignard wrote:

2017-03-06 17:04 GMT+01:00 Daniel Vetter :

On Mon, Mar 06, 2017 at 11:58:05AM +0100, Mark Brown wrote:

On Mon, Mar 06, 2017 at 11:40:41AM +0100, Daniel Vetter wrote:


No one gave a thing about android in upstream, so Greg KH just dumped it
all into staging/android/. We've discussed ION a bunch of times, recorded
anything we'd like to fix in staging/android/TODO, and Laura's patch
series here addresses a big chunk of that.



This is pretty much the same approach we (gpu folks) used to de-stage the
syncpt stuff.


Well, there's also the fact that quite a few people have issues with the
design (like Laurent).  It seems like a lot of them have either got more
comfortable with it over time, or at least not managed to come up with
any better ideas in the meantime.


See the TODO, it has everything a really big group (look at the patch for
the full Cc: list) figured needs to be improved at LPC 2015. We don't just
merge stuff because merging stuff is fun :-)

Laurent was even in that group ...
-Daniel


For me those patches are going in the right direction.

I still have few questions:
- since alignment management has been remove from ion-core, should it
be also removed from ioctl structure ?


Yes, I think I'm going to go with the suggestion to fixup the ABI
so we don't need the compat layer and as part of that I'm also
dropping the align argument.


- can you we ride off ion_handle (at least in userland) and only
export a dma-buf descriptor ?


Yes, I think this is the right direction given we're breaking
everything anyway. I was debating trying to keep the two but
moving to only dma bufs is probably cleaner. The only reason
I could see for keeping the handles is running out of file
descriptors for dma-bufs but that seems unlikely.


In the future how can we add new heaps ?
Some platforms have very specific memory allocation
requirements (just have a look in the number of gem custom allocator in drm)
Do you plan to add heap type/mask for each ?


Yes, that was my thinking.


My concern is about the policy to adding heaps, will you accept
"customs" heap per
platforms ? per devices ? or only generic ones ?
If you are too strict, we will have lot of out-of-tree heaps and if
you accept of of them
it will be a nightmare to maintain



Are you concerned about actual heaps (e.g. a carveout at 0x8000 vs
a carveout at 0x6000) or heap types?

For heap types, I think the policy can be strict - if it's generally
useful then it should live in-tree in ion. Otherwise, it would be
out-of-tree. I'd expect most "custom" heaps to be parameterisable to
the point of being generally useful.

For actual heap instances, I would expect them to be communicated via
reserved-memory regions or something similar, and so the maintenance
burden is pretty low.

The existing query ioctl can allow heap IDs to get assigned
dynamically at runtime, so there's no need to reserve "bit 6" for
"CUSTOM_ACME_HEAP_1"


Another point is how can we put secure rules (like selinux policy) on
heaps since all the allocations
go to the same device (/dev/ion) ? For example, until now, in Android
we have to give the same
access rights to all the process that use ION.
It will become problem when we will add secure heaps because we won't
be able to distinguish secure
processes to standard ones or set specific policy per heaps.
Maybe I'm wrong here but I have never see selinux policy checking an
ioctl field but if that
exist it could be a solution.



I might be thinking of a different type of "secure", but...

Should the security of secure heaps be enforced by OS-level
permissions? I don't know about other architectures, but at least on
arm/arm64 this is enforced in hardware; it doesn't matter who has
access to the ion heap, because only secure devices (or the CPU
running a secure process) is physically able to access the memory
backing the buffer.

In fact, in the use-cases I know of, the process asking for the ion
allocation is not a secure process, and so we wouldn't *want* to
restrict the secure heap to be allocated from only by secure
processes.

-Brian





Benjamin



Thanks,
Laura

Re: [RFC PATCH 00/12] Ion cleanup in preparation for moving out of staging

[Brian Starkey]

On Fri, Mar 10, 2017 at 11:46:42AM +, Robin Murphy wrote:

On 10/03/17 10:31, Brian Starkey wrote:

Hi,

On Thu, Mar 09, 2017 at 09:38:49AM -0800, Laura Abbott wrote:

On 03/09/2017 02:00 AM, Benjamin Gaignard wrote:


[snip]



For me those patches are going in the right direction.

I still have few questions:
- since alignment management has been remove from ion-core, should it
be also removed from ioctl structure ?


Yes, I think I'm going to go with the suggestion to fixup the ABI
so we don't need the compat layer and as part of that I'm also
dropping the align argument.



Is the only motivation for removing the alignment parameter that
no-one got around to using it for something useful yet?
The original comment was true - different devices do have different
alignment requirements.

Better alignment can help SMMUs use larger blocks when mapping,
reducing TLB pressure and the chance of a page table walk causing
display underruns.


For that use-case, though, alignment alone doesn't necessarily help -
you need the whole allocation granularity to match your block size (i.e.
given a 1MB block size, asking for 17KB and getting back 17KB starting
at a 1MB boundary doesn't help much - that whole 1MB needs to be
allocated and everyone needs to know it to ensure that the whole lot can
be mapped safely). Now, whether it's down to the callers or the heap
implementations to decide and enforce that granularity is another
question, but provided allocations are at least naturally aligned to
whatever the granularity is (which is a reasonable assumption to bake
in) then it's all good.

Robin.


Agreed, alignment alone isn't enough. But lets assume that an app
knows what a "good" granularity is, and always asks for allocation
sizes which are suitably rounded to allow blocks to be used. Currently
it looks like a "standard" ION_HEAP_TYPE_CARVEOUT heap would give me
back just a PAGE_SIZE aligned buffer. So even *if* the caller knows
its desired block size, there's no way for it to get guaranteed better
alignment, which wouldn't be a bad feature to have.

Anyway as Daniel and Rob say, if the interface is designed properly
this kind of extension would be possible later, or you can have a
special heap with a larger granule.

I suppose it makes sense to remove it while there's no-one actually
implementing it, in case an alternate method proves more usable.

-Brian





-Brian

___
linux-arm-kernel mailing list
linux-arm-ker...@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH 00/12] Ion cleanup in preparation for moving out of staging

[Brian Starkey]

Hi,

On Thu, Mar 09, 2017 at 09:38:49AM -0800, Laura Abbott wrote:

On 03/09/2017 02:00 AM, Benjamin Gaignard wrote:


[snip]



For me those patches are going in the right direction.

I still have few questions:
- since alignment management has been remove from ion-core, should it
be also removed from ioctl structure ?


Yes, I think I'm going to go with the suggestion to fixup the ABI
so we don't need the compat layer and as part of that I'm also
dropping the align argument.



Is the only motivation for removing the alignment parameter that
no-one got around to using it for something useful yet?
The original comment was true - different devices do have different
alignment requirements.

Better alignment can help SMMUs use larger blocks when mapping,
reducing TLB pressure and the chance of a page table walk causing
display underruns.

-Brian

Re: DRM Atomic property for color-space conversion

[Brian Starkey]

On Tue, Jan 31, 2017 at 05:15:46PM +0200, Ville Syrjälä wrote:

On Tue, Jan 31, 2017 at 12:33:29PM +, Brian Starkey wrote:

Hi,

On Mon, Jan 30, 2017 at 03:35:13PM +0200, Ville Syrjälä wrote:
>On Fri, Jan 27, 2017 at 05:23:24PM +0000, Brian Starkey wrote:
>> Hi,
>>
>> We're looking to enable the per-plane color management hardware in
>> Mali-DP with atomic properties, which has sparked some conversation
>> around how to handle YCbCr formats.
>>
>> As it stands today, it's assumed that a driver will implicitly "do the
>> right thing" to display a YCbCr buffer.
>>
>> YCbCr data often uses different gamma curves and signal ranges (e.g.
>> BT.609, BT.701, BT.2020, studio range, full-range), so its desirable
>> to be able to explicitly control the YCbCr to RGB conversion process
>> from userspace.
>>
>> We're proposing adding a "CSC" (color-space conversion) property to
>> control this - primarily per-plane for framebuffer->pipeline CSC, but
>> perhaps one per CRTC too for devices which have an RGB pipeline and
>> want to output in YUV to the display:
>>
>> Name: "CSC"
>> Type: ENUM | ATOMIC;
>> Enum values (representative):
>> "default":
>>Same behaviour as now. "Some kind" of YCbCr->RGB conversion
>>for YCbCr buffers, bypass for RGB buffers
>> "disable":
>>Explicitly disable all colorspace conversion (i.e. use an
>>identity matrix).
>> "YCbCr to RGB: BT.709":
>>Only valid for YCbCr formats. CSC in accordance with BT.709
>>using [16..235] for (8-bit) luma values, and [16..240] for
>>8-bit chroma values. For 10-bit formats, the range limits are
>>multiplied by 4.
>> "YCbCr to RGB: BT.709 full-swing":
>>Only valid for YCbCr formats. CSC in accordance with BT.709,
>>but using the full range of each channel.
>> "YCbCr to RGB: Use CTM":*
>>Only valid for YCbCr formats. Use the matrix applied via the
>>plane's CTM property
>> "RGB to RGB: Use CTM":*
>>Only valid for RGB formats. Use the matrix applied via the
>>plane's CTM property
>> "Use CTM":*
>>Valid for any format. Use the matrix applied via the plane's
>>CTM property
>> ... any other values for BT.601, BT.2020, RGB to YCbCr etc. etc. as
>> they are required.
>
>Having some RGB2RGB and YCBCR2RGB things in the same property seems
>weird. I would just go with something very simple like:
>
>YCBCR_TO_RGB_CSC:
>* BT.601
>* BT.709
>* custom matrix
>

I think we've agreed in #dri-devel that this CSC property
can't/shouldn't be mapped on-to the existing (hardware implementing
the) CTM property - even in the case of per-plane color management -
because CSC needs to be done before DEGAMMA.

So, I'm in favour of going with what you suggested in the first place:

A new YCBCR_TO_RGB_CSC property, enum type, with a list of fixed
conversions. I'd drop the custom matrix for now, as we'd need to add
another property to attach the custom matrix blob too.

I still think we need a way to specify whether the source data range
is broadcast/full-range, so perhaps the enum list should be expanded
to all combinations of BT.601/BT.709 + broadcast/full-range.


Sounds reasonable. Not that much full range YCbCr stuff out there
perhaps. Well, apart from jpegs I suppose. But no harm in being able
to deal with it.



(I'm not sure what the canonical naming for broadcast/full-range is,
we call them narrow and wide)


We tend to call them full vs. limited range. That's how our
"Broadcast RGB" property is defined as well.



OK, using the same ones sounds sensible.



>And trying to use the same thing for the crtc stuff is probably not
>going to end well. Like Daniel said we already have the
>'Broadcast RGB' property muddying the waters there, and that stuff
>also ties in with what colorspace we signal to the sink via
>infoframes/whatever the DP thing was called. So my gut feeling is
>that trying to use the same property everywhere will just end up
>messy.

Yeah, agreed. If/when someone wants to add CSC on the output of a CRTC
(after GAMMA), we can add a new property.

That makes me wonder about calling this one SOURCE_YCBCR_TO_RGB_CSC to
be explicit that it describes the source data. Then we can later add
SINK_RGB_TO_YCBCR_CSC, and it will be reasonably obvious that its
value describes the output data rather than the input data.


Source and sink have a slight connotation in my mind wrt. the box that
produces the display signal and the box that eats the signal. So trying
to use the same terms to describe the 

Re: DRM Atomic property for color-space conversion

[Brian Starkey]

Hi,

On Mon, Jan 30, 2017 at 03:35:13PM +0200, Ville Syrjälä wrote:

On Fri, Jan 27, 2017 at 05:23:24PM +, Brian Starkey wrote:

Hi,

We're looking to enable the per-plane color management hardware in
Mali-DP with atomic properties, which has sparked some conversation
around how to handle YCbCr formats.

As it stands today, it's assumed that a driver will implicitly "do the
right thing" to display a YCbCr buffer.

YCbCr data often uses different gamma curves and signal ranges (e.g.
BT.609, BT.701, BT.2020, studio range, full-range), so its desirable
to be able to explicitly control the YCbCr to RGB conversion process
from userspace.

We're proposing adding a "CSC" (color-space conversion) property to
control this - primarily per-plane for framebuffer->pipeline CSC, but
perhaps one per CRTC too for devices which have an RGB pipeline and
want to output in YUV to the display:

Name: "CSC"
Type: ENUM | ATOMIC;
Enum values (representative):
"default":
Same behaviour as now. "Some kind" of YCbCr->RGB conversion
for YCbCr buffers, bypass for RGB buffers
"disable":
Explicitly disable all colorspace conversion (i.e. use an
identity matrix).
"YCbCr to RGB: BT.709":
Only valid for YCbCr formats. CSC in accordance with BT.709
using [16..235] for (8-bit) luma values, and [16..240] for
8-bit chroma values. For 10-bit formats, the range limits are
multiplied by 4.
"YCbCr to RGB: BT.709 full-swing":
Only valid for YCbCr formats. CSC in accordance with BT.709,
but using the full range of each channel.
"YCbCr to RGB: Use CTM":*
Only valid for YCbCr formats. Use the matrix applied via the
plane's CTM property
"RGB to RGB: Use CTM":*
Only valid for RGB formats. Use the matrix applied via the
plane's CTM property
"Use CTM":*
Valid for any format. Use the matrix applied via the plane's
CTM property
... any other values for BT.601, BT.2020, RGB to YCbCr etc. etc. as
they are required.


Having some RGB2RGB and YCBCR2RGB things in the same property seems
weird. I would just go with something very simple like:

YCBCR_TO_RGB_CSC:
* BT.601
* BT.709
* custom matrix



I think we've agreed in #dri-devel that this CSC property
can't/shouldn't be mapped on-to the existing (hardware implementing
the) CTM property - even in the case of per-plane color management -
because CSC needs to be done before DEGAMMA.

So, I'm in favour of going with what you suggested in the first place:

A new YCBCR_TO_RGB_CSC property, enum type, with a list of fixed
conversions. I'd drop the custom matrix for now, as we'd need to add
another property to attach the custom matrix blob too.

I still think we need a way to specify whether the source data range
is broadcast/full-range, so perhaps the enum list should be expanded
to all combinations of BT.601/BT.709 + broadcast/full-range.

(I'm not sure what the canonical naming for broadcast/full-range is,
we call them narrow and wide)


And trying to use the same thing for the crtc stuff is probably not
going to end well. Like Daniel said we already have the
'Broadcast RGB' property muddying the waters there, and that stuff
also ties in with what colorspace we signal to the sink via
infoframes/whatever the DP thing was called. So my gut feeling is
that trying to use the same property everywhere will just end up
messy.


Yeah, agreed. If/when someone wants to add CSC on the output of a CRTC
(after GAMMA), we can add a new property.

That makes me wonder about calling this one SOURCE_YCBCR_TO_RGB_CSC to
be explicit that it describes the source data. Then we can later add
SINK_RGB_TO_YCBCR_CSC, and it will be reasonably obvious that its
value describes the output data rather than the input data.

I want to avoid confusion caused by ending up with two
{CS}_TO_{CS}_CSC properties, where one is describing the data to the
left of it, and the other describing the data to the right of it, with
no real way of telling which way around it is.

Cheers,
Brian



--
Ville Syrjälä
Intel OTC

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

DRM Atomic property for color-space conversion

[Brian Starkey]

Hi,

We're looking to enable the per-plane color management hardware in
Mali-DP with atomic properties, which has sparked some conversation
around how to handle YCbCr formats.

As it stands today, it's assumed that a driver will implicitly "do the
right thing" to display a YCbCr buffer.

YCbCr data often uses different gamma curves and signal ranges (e.g.
BT.609, BT.701, BT.2020, studio range, full-range), so its desirable
to be able to explicitly control the YCbCr to RGB conversion process
from userspace.

We're proposing adding a "CSC" (color-space conversion) property to
control this - primarily per-plane for framebuffer->pipeline CSC, but
perhaps one per CRTC too for devices which have an RGB pipeline and
want to output in YUV to the display:

Name: "CSC"
Type: ENUM | ATOMIC;
Enum values (representative):
"default":
Same behaviour as now. "Some kind" of YCbCr->RGB conversion
for YCbCr buffers, bypass for RGB buffers
"disable":
Explicitly disable all colorspace conversion (i.e. use an
identity matrix).
"YCbCr to RGB: BT.709":
Only valid for YCbCr formats. CSC in accordance with BT.709
using [16..235] for (8-bit) luma values, and [16..240] for
8-bit chroma values. For 10-bit formats, the range limits are
multiplied by 4.
"YCbCr to RGB: BT.709 full-swing":
Only valid for YCbCr formats. CSC in accordance with BT.709,
but using the full range of each channel.
"YCbCr to RGB: Use CTM":*
Only valid for YCbCr formats. Use the matrix applied via the
plane's CTM property
"RGB to RGB: Use CTM":*
Only valid for RGB formats. Use the matrix applied via the
plane's CTM property
"Use CTM":*
Valid for any format. Use the matrix applied via the plane's
CTM property
... any other values for BT.601, BT.2020, RGB to YCbCr etc. etc. as
they are required.

*This assumes color-management is enabled per-plane. We're currently
working on patches to add this mostly to be able to use per-plane
degamma, but it would be analogous to the CRTC color-management code
and so also be able to expose a per-plane CTM property.

Our hardware implements the color-space conversion as a 3x3 matrix, so
we can implement a helper to convert a CSC enum value to a CTM matrix
for use by any hardware which has a programmable CSC matrix. For any
other hardware, the driver simply needs to map the enum value to
whatever selector bits are available.

It's expected that the "Use CTM" value(s) are *not* the common case,
and most of the time userspace will use one of the provided "standard"
enum values. The three different flavours of "Use CTM" allow us to
support hardware whose CSC hardware can only be used on e.g. YCbCr
data.

Drivers can of course filter the enum list to expose whichever subset
the hardware can support.

Having thrashed this out a bit on IRC with Ville, I think the above
approach is flexible enough to support at least Mali-DP and i915,
without burdening userspace any more than necessary. It also provides
the "default" behaviour which is backwards compatible, and allows for
fully custom CSC matrices where that is supported.

We can obviously implement this as a Mali-DP driver private property,
but it would be good to come up with something to go into the core if
possible.

I'd be happy to hear any feedback,

Thanks,
Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/6] drm: writeback: Add out-fences for writeback connectors

[Brian Starkey]

Add the OUT_FENCE_PTR property to writeback connectors, to enable
userspace to get a fence which will signal once the writeback is
complete. It is not allowed to request an out-fence without a
framebuffer attached to the connector.

A timeline is added to drm_writeback_connector for use by the writeback
out-fences.

In the case of a commit failure or DRM_MODE_ATOMIC_TEST_ONLY, the fence
is set to -1.

Changes from v2:
 - Rebase onto Gustavo Padovan's v9 explicit sync series
 - Change out_fence_ptr type to s32 __user *
 - Set *out_fence_ptr to -1 in drm_atomic_connector_set_property
 - Store fence in drm_writeback_job
 Gustavo Padovan:
 - Move out_fence_ptr out of connector_state
 - Signal fence from drm_writeback_signal_completion instead of
   in driver directly

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_atomic.c|   96 +
 drivers/gpu/drm/drm_writeback.c |   99 ++-
 include/drm/drm_atomic.h|8 
 include/drm/drm_connector.h |8 ++--
 include/drm/drm_writeback.h |   39 ++-
 5 files changed, 234 insertions(+), 16 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 343e2b7..0e3c900 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -308,6 +308,32 @@ static s64 __user *get_out_fence_for_crtc(struct 
drm_atomic_state *state,
return fence_ptr;
 }
 
+static int set_out_fence_for_connector(struct drm_atomic_state *state,
+   struct drm_connector *connector,
+   s64 __user *fence_ptr)
+{
+   unsigned int index = drm_connector_index(connector);
+
+   if (fence_ptr && put_user(-1, fence_ptr))
+   return -EFAULT;
+
+   state->connectors[index].out_fence_ptr = fence_ptr;
+
+   return 0;
+}
+
+static s64 __user *get_out_fence_for_connector(struct drm_atomic_state *state,
+  struct drm_connector *connector)
+{
+   unsigned int index = drm_connector_index(connector);
+   s64 __user *fence_ptr;
+
+   fence_ptr = state->connectors[index].out_fence_ptr;
+   state->connectors[index].out_fence_ptr = NULL;
+
+   return fence_ptr;
+}
+
 /**
  * drm_atomic_set_mode_for_crtc - set mode for CRTC
  * @state: the CRTC whose incoming state to update
@@ -696,6 +722,12 @@ static int drm_atomic_connector_check(struct drm_connector 
*connector,
return -EINVAL;
}
 
+   if (writeback_job->out_fence && !writeback_job->fb) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] requesting out-fence 
without framebuffer\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
return 0;
 }
 
@@ -1134,6 +1166,11 @@ int drm_atomic_connector_set_property(struct 
drm_connector *connector,
if (fb)
drm_framebuffer_unreference(fb);
return ret;
+   } else if (property == config->prop_out_fence_ptr) {
+   s64 __user *fence_ptr = u64_to_user_ptr(val);
+
+   return set_out_fence_for_connector(state->state, connector,
+  fence_ptr);
} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
@@ -1185,6 +1222,8 @@ static void drm_atomic_connector_print_state(struct 
drm_printer *p,
} else if (property == config->writeback_fb_id_property) {
/* Writeback framebuffer is one-shot, write and forget */
*val = 0;
+   } else if (property == config->prop_out_fence_ptr) {
+   *val = 0;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
@@ -1976,7 +2015,7 @@ static int setup_out_fence(struct drm_out_fence_state 
*fence_state,
return 0;
 }
 
-static int prepare_crtc_signaling(struct drm_device *dev,
+static int prepare_signaling(struct drm_device *dev,
  struct drm_atomic_state *state,
  struct drm_mode_atomic *arg,
  struct drm_file *file_priv,
@@ -1985,6 +2024,8 @@ static int prepare_crtc_signaling(struct drm_device *dev,
 {
struct drm_crtc *crtc;
struct drm_crtc_state *crtc_state;
+   struct drm_connector *conn;
+   struct drm_connector_state *conn_state;
int i, ret;
 
if (arg->flags & DRM_MODE_ATOMIC_TEST_ONLY)
@@ -2048,14 +2089,51 @@ static int prepare_crtc_signaling(struct drm_device 
*d

[PATCH 1/6] drm: Add writeback connector type

[Brian Starkey]

Writeback connectors represent writeback engines which can write the
CRTC output to a memory framebuffer. Add a writeback connector type and
related support functions.

Drivers should initialize a writeback connector with
drm_writeback_connector_init() which takes care of setting up all the
writeback-specific details on top of the normal functionality of
drm_connector_init().

Writeback connectors have a WRITEBACK_FB_ID property, used to set the
output framebuffer, and a PIXEL_FORMATS blob used to expose the
supported writeback formats to userspace.

When a framebuffer is attached to a writeback connector with the
WRITEBACK_FB_ID property, it is used only once (for the commit in which
it was included), and userspace can never read back the value of
WRITEBACK_FB_ID. WRITEBACK_FB_ID can only be set if the connector is
attached to a CRTC.

Changes since v1:
 - Added drm_writeback.c + documentation
 - Added helper to initialize writeback connector in one go
 - Added core checks
 - Squashed into a single commit
 - Dropped the client cap
 - Writeback framebuffers are no longer persistent

Changes since v2:
 Daniel Vetter:
 - Subclass drm_connector to drm_writeback_connector
 - Relax check to allow CRTC to be set without an FB
 - Add some writeback_ prefixes
 - Drop PIXEL_FORMATS_SIZE property, as it was unnecessary
 Gustavo Padovan:
 - Add drm_writeback_job to handle writeback signalling centrally

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 Documentation/gpu/drm-kms.rst   |9 ++
 drivers/gpu/drm/Makefile|2 +-
 drivers/gpu/drm/drm_atomic.c|  130 
 drivers/gpu/drm/drm_atomic_helper.c |6 +
 drivers/gpu/drm/drm_connector.c |4 +-
 drivers/gpu/drm/drm_writeback.c |  230 +++
 include/drm/drm_atomic.h|3 +
 include/drm/drm_connector.h |   13 ++
 include/drm/drm_mode_config.h   |   14 +++
 include/drm/drm_writeback.h |   78 
 include/uapi/drm/drm_mode.h |1 +
 11 files changed, 488 insertions(+), 2 deletions(-)
 create mode 100644 drivers/gpu/drm/drm_writeback.c
 create mode 100644 include/drm/drm_writeback.h

diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 568f3c2..3a4f35b 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -122,6 +122,15 @@ Connector Functions Reference
 .. kernel-doc:: drivers/gpu/drm/drm_connector.c
:export:
 
+Writeback Connectors
+
+
+.. kernel-doc:: drivers/gpu/drm/drm_writeback.c
+  :doc: overview
+
+.. kernel-doc:: drivers/gpu/drm/drm_writeback.c
+  :export:
+
 Encoder Abstraction
 ===
 
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 883f3e7..3209aa4 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -16,7 +16,7 @@ drm-y   :=drm_auth.o drm_bufs.o drm_cache.o \
drm_framebuffer.o drm_connector.o drm_blend.o \
drm_encoder.o drm_mode_object.o drm_property.o \
drm_plane.o drm_color_mgmt.o drm_print.o \
-   drm_dumb_buffers.o drm_mode_config.o
+   drm_dumb_buffers.o drm_mode_config.o drm_writeback.o
 
 drm-$(CONFIG_COMPAT) += drm_ioc32.o
 drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index b476ec5..343e2b7 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -31,6 +31,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #include "drm_crtc_internal.h"
@@ -659,6 +660,46 @@ static void drm_atomic_crtc_print_state(struct drm_printer 
*p,
 }
 
 /**
+ * drm_atomic_connector_check - check connector state
+ * @connector: connector to check
+ * @state: connector state to check
+ *
+ * Provides core sanity checks for connector state.
+ *
+ * RETURNS:
+ * Zero on success, error code on failure
+ */
+static int drm_atomic_connector_check(struct drm_connector *connector,
+   struct drm_connector_state *state)
+{
+   struct drm_crtc_state *crtc_state;
+   struct drm_writeback_job *writeback_job = state->writeback_job;
+
+   if ((connector->connector_type != DRM_MODE_CONNECTOR_WRITEBACK) ||
+   !writeback_job)
+   return 0;
+
+   if (writeback_job->fb && !state->crtc) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] framebuffer without CRTC\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
+   if (state->crtc)
+   crtc_state = drm_atomic_get_existing_crtc_state(state->state,
+   state->crtc);
+
+   if (writeback_job->fb && !crtc_state->active) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] has framebuffer, but 
[CR

[PATCH 3/6] drm: mali-dp: Rename malidp_input_format

[Brian Starkey]

We're going to use the same format list for output formats, so rename
everything related to input formats to avoid confusion.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
Reviewed-by: Liviu Dudau <liviu.du...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |   24 
 drivers/gpu/drm/arm/malidp_hw.h |8 
 drivers/gpu/drm/arm/malidp_planes.c |8 
 3 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 4bdf531..9ec6d69 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -21,7 +21,7 @@
 #include "malidp_drv.h"
 #include "malidp_hw.h"
 
-static const struct malidp_input_format malidp500_de_formats[] = {
+static const struct malidp_format_id malidp500_de_formats[] = {
/*fourcc,   layers supporting the format, internal id  */
{ DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2,  0 },
{ DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2,  1 },
@@ -69,7 +69,7 @@
{ DRM_FORMAT_NV12, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 6) },\
{ DRM_FORMAT_YUV420, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 7) }
 
-static const struct malidp_input_format malidp550_de_formats[] = {
+static const struct malidp_format_id malidp550_de_formats[] = {
MALIDP_COMMON_FORMATS,
 };
 
@@ -436,8 +436,8 @@ static int malidp650_query_hw(struct malidp_hw_device 
*hwdev)
.irq_mask = MALIDP500_DE_IRQ_CONF_VALID,
.vsync_irq = MALIDP500_DE_IRQ_CONF_VALID,
},
-   .input_formats = malidp500_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp500_de_formats),
+   .pixel_formats = malidp500_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp500_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp500_query_hw,
@@ -469,8 +469,8 @@ static int malidp650_query_hw(struct malidp_hw_device 
*hwdev)
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
-   .input_formats = malidp550_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+   .pixel_formats = malidp550_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp550_query_hw,
@@ -503,8 +503,8 @@ static int malidp650_query_hw(struct malidp_hw_device 
*hwdev)
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
-   .input_formats = malidp550_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+   .pixel_formats = malidp550_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 16,
},
.query_hw = malidp650_query_hw,
@@ -522,10 +522,10 @@ u8 malidp_hw_get_format_id(const struct malidp_hw_regmap 
*map,
 {
unsigned int i;
 
-   for (i = 0; i < map->n_input_formats; i++) {
-   if (((map->input_formats[i].layer & layer_id) == layer_id) &&
-   (map->input_formats[i].format == format))
-   return map->input_formats[i].id;
+   for (i = 0; i < map->n_pixel_formats; i++) {
+   if (((map->pixel_formats[i].layer & layer_id) == layer_id) &&
+   (map->pixel_formats[i].format == format))
+   return map->pixel_formats[i].id;
}
 
return MALIDP_INVALID_FORMAT_ID;
diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h
index 087e1202..4f8c884 100644
--- a/drivers/gpu/drm/arm/malidp_hw.h
+++ b/drivers/gpu/drm/arm/malidp_hw.h
@@ -35,7 +35,7 @@ enum {
DE_SMART = BIT(4),
 };
 
-struct malidp_input_format {
+struct malidp_format_id {
u32 format; /* DRM fourcc */
u8 layer;   /* bitmask of layers supporting it */
u8 id;  /* used internally */
@@ -85,9 +85,9 @@ struct malidp_hw_regmap {
const struct malidp_irq_map se_irq_map;
const struct malidp_irq_map dc_irq_map;
 
-   /* list of supported input formats for each layer */
-   const struct malidp_input_format *input_formats;
-   const u8 n_input_formats;
+   /* list of supported pixel formats for each layer */
+   const struct malidp_format_id *pix

[PATCH 5/6] drm: mali-dp: Add RGB writeback formats for DP550/DP650

[Brian Starkey]

Add a layer bit for the SE memory-write, and add it to the pixel format
matrix for DP550/DP650.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |   28 ++--
 drivers/gpu/drm/arm/malidp_hw.h |1 +
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index c696e67..be17631 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -46,20 +46,20 @@
 
 #define MALIDP_COMMON_FORMATS \
/*fourcc,   layers supporting the format,  internal id   */ \
-   { DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 0) }, \
-   { DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 1) }, \
-   { DRM_FORMAT_RGBA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 2) }, \
-   { DRM_FORMAT_BGRA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 3) }, \
-   { DRM_FORMAT_ARGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 0) }, \
-   { DRM_FORMAT_ABGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 1) }, \
-   { DRM_FORMAT_RGBA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 2) }, \
-   { DRM_FORMAT_BGRA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 3) }, \
-   { DRM_FORMAT_XRGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 0) }, \
-   { DRM_FORMAT_XBGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 1) }, \
-   { DRM_FORMAT_RGBX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 2) }, \
-   { DRM_FORMAT_BGRX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 3) }, \
-   { DRM_FORMAT_RGB888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(3, 
0) }, \
-   { DRM_FORMAT_BGR888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(3, 
1) }, \
+   { DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 0) }, \
+   { DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 1) }, \
+   { DRM_FORMAT_RGBA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 2) }, \
+   { DRM_FORMAT_BGRA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 3) }, \
+   { DRM_FORMAT_ARGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 0) }, \
+   { DRM_FORMAT_ABGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 1) }, \
+   { DRM_FORMAT_RGBA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 2) }, \
+   { DRM_FORMAT_BGRA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 3) }, \
+   { DRM_FORMAT_XRGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 0) }, \
+   { DRM_FORMAT_XBGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 1) }, \
+   { DRM_FORMAT_RGBX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 2) }, \
+   { DRM_FORMAT_BGRX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 3) }, \
+   { DRM_FORMAT_RGB888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(3, 0) }, \
+   { DRM_FORMAT_BGR888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(3, 1) }, \
{ DRM_FORMAT_RGBA5551, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(4, 0) }, \
{ DRM_FORMAT_ABGR1555, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(4, 1) }, \
{ DRM_FORMAT_RGB565, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(4, 
2) }, \
diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h
index 091171d..8056efa 100644
--- a/drivers/gpu/drm/arm/malidp_hw.h
+++ b/drivers/gpu/drm/arm/malidp_hw.h
@@ -33,6 +33,7 @@ enum {
DE_GRAPHICS2 = BIT(2), /* used only in DP500 */
DE_VIDEO2 = BIT(3),
DE_SMART = BIT(4),
+   SE_MEMWRITE = BIT(5),
 };
 
 struct malidp_format_id {
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH v3 0/6] Introduce writeback connectors

[Brian Starkey]

Hi,

This is v3 of my series introducing a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
See v1 and v2 here: [1] [2]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Writeback connectors are given a WRITEBACK_FB_ID property (which acts
slightly differently to FB_ID, so gets a new name), as well as
a PIXEL_FORMATS blob to list the supported writeback formats, and
OUT_FENCE_PTR to be used for out-fences.

The changes since v2 are in the commit messages of each commit.

The main differences are:
 - Subclass drm_connector as drm_writeback_connector
 - Slight relaxation of core checks, to allow
   (connector->crtc && !connector->fb)
 - Dropped PIXEL_FORMATS_SIZE, which was redundant
 - Reworked the event interface, drivers don't need to deal with the
   fence directly
 - Re-ordered the commits to introduce writeback out-fences up-front.

I've kept RFC on this series because the event reporting (introduction
of drm_writeback_job) is probably up for debate.

v4 will be accompanied by igt tests.

As always, I look forward to any comments.

Thanks,
Brian

[1] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1247574.html
[2] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1258017.html

---

Brian Starkey (5):
  drm: Add writeback connector type
  drm: writeback: Add out-fences for writeback connectors
  drm: mali-dp: Rename malidp_input_format
  drm: mali-dp: Add RGB writeback formats for DP550/DP650
  drm: mali-dp: Add writeback connector

Liviu Dudau (1):
  drm: mali-dp: Add support for writeback on DP550/DP650

 Documentation/gpu/drm-kms.rst   |9 +
 drivers/gpu/drm/Makefile|2 +-
 drivers/gpu/drm/arm/Makefile|1 +
 drivers/gpu/drm/arm/malidp_crtc.c   |   21 +++
 drivers/gpu/drm/arm/malidp_drv.c|   25 ++-
 drivers/gpu/drm/arm/malidp_drv.h|3 +
 drivers/gpu/drm/arm/malidp_hw.c |  111 
 drivers/gpu/drm/arm/malidp_hw.h |   27 ++-
 drivers/gpu/drm/arm/malidp_mw.c |  278 ++
 drivers/gpu/drm/arm/malidp_mw.h |   18 ++
 drivers/gpu/drm/arm/malidp_planes.c |8 +-
 drivers/gpu/drm/arm/malidp_regs.h   |   15 ++
 drivers/gpu/drm/drm_atomic.c|  226 +++-
 drivers/gpu/drm/drm_atomic_helper.c |6 +
 drivers/gpu/drm/drm_connector.c |4 +-
 drivers/gpu/drm/drm_writeback.c |  325 +++
 include/drm/drm_atomic.h|   11 ++
 include/drm/drm_connector.h |   13 ++
 include/drm/drm_mode_config.h   |   14 ++
 include/drm/drm_writeback.h |  115 +
 include/uapi/drm/drm_mode.h |1 +
 21 files changed, 1181 insertions(+), 52 deletions(-)
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.c
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.h
 create mode 100644 drivers/gpu/drm/drm_writeback.c
 create mode 100644 include/drm/drm_writeback.h

-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/6] drm: mali-dp: Add writeback connector

[Brian Starkey]

Mali-DP has a memory writeback engine which can be used to write the
composition result to a memory buffer. Expose this functionality as a
DRM writeback connector on supported hardware.

Changes since v1:
 Daniel Vetter:
 - Don't require a modeset when writeback routing changes
 - Make writeback connector always disconnected

Changes since v2:
 - Rebase onto new drm_writeback_connector
 - Add reset callback, allocating subclassed state
 Daniel Vetter:
 - Squash out-fence support into this commit
 Gustavo Padovan:
 - Don't signal fence directly from driver (and drop malidp_mw_job)

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/Makefile  |1 +
 drivers/gpu/drm/arm/malidp_crtc.c |   21 +++
 drivers/gpu/drm/arm/malidp_drv.c  |   25 +++-
 drivers/gpu/drm/arm/malidp_drv.h  |3 +
 drivers/gpu/drm/arm/malidp_hw.c   |7 +-
 drivers/gpu/drm/arm/malidp_mw.c   |  278 +
 drivers/gpu/drm/arm/malidp_mw.h   |   18 +++
 7 files changed, 348 insertions(+), 5 deletions(-)
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.c
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.h

diff --git a/drivers/gpu/drm/arm/Makefile b/drivers/gpu/drm/arm/Makefile
index bb8b158..3bf31d1 100644
--- a/drivers/gpu/drm/arm/Makefile
+++ b/drivers/gpu/drm/arm/Makefile
@@ -1,4 +1,5 @@
 hdlcd-y := hdlcd_drv.o hdlcd_crtc.o
 obj-$(CONFIG_DRM_HDLCD)+= hdlcd.o
 mali-dp-y := malidp_drv.o malidp_hw.o malidp_planes.o malidp_crtc.o
+mali-dp-y += malidp_mw.o
 obj-$(CONFIG_DRM_MALI_DISPLAY) += mali-dp.o
diff --git a/drivers/gpu/drm/arm/malidp_crtc.c 
b/drivers/gpu/drm/arm/malidp_crtc.c
index 08e6a71..5413a5a 100644
--- a/drivers/gpu/drm/arm/malidp_crtc.c
+++ b/drivers/gpu/drm/arm/malidp_crtc.c
@@ -68,6 +68,18 @@ static void malidp_crtc_enable(struct drm_crtc *crtc)
clk_set_rate(hwdev->pxlclk, crtc->state->adjusted_mode.crtc_clock * 
1000);
 
hwdev->modeset(hwdev, );
+   /*
+* We should always disable the memory write when leaving config mode,
+* otherwise the hardware will start writing right away - possibly with
+* a stale config, and definitely before we've had a chance to configure
+* the planes.
+* If the memory write needs to be enabled, that will get taken care
+* of later during the atomic commit
+*/
+   if (hwdev->disable_memwrite) {
+   DRM_DEV_DEBUG_DRIVER(crtc->dev->dev, "Disable memwrite\n");
+   hwdev->disable_memwrite(hwdev);
+   }
hwdev->leave_config_mode(hwdev);
drm_crtc_vblank_on(crtc);
 }
@@ -157,6 +169,15 @@ static int malidp_crtc_atomic_check(struct drm_crtc *crtc,
}
}
 
+   /* If only the writeback routing has changed, we don't need a modeset */
+   if (state->connectors_changed) {
+   u32 old_mask = crtc->state->connector_mask;
+   u32 new_mask = state->connector_mask;
+   if ((old_mask ^ new_mask) ==
+   (1 << drm_connector_index(>mw_connector.base)))
+   state->connectors_changed = false;
+   }
+
return 0;
 }
 
diff --git a/drivers/gpu/drm/arm/malidp_drv.c b/drivers/gpu/drm/arm/malidp_drv.c
index 32f746e..2d0465b 100644
--- a/drivers/gpu/drm/arm/malidp_drv.c
+++ b/drivers/gpu/drm/arm/malidp_drv.c
@@ -28,6 +28,7 @@
 #include 
 
 #include "malidp_drv.h"
+#include "malidp_mw.h"
 #include "malidp_regs.h"
 #include "malidp_hw.h"
 
@@ -92,6 +93,14 @@ static void malidp_atomic_commit_tail(struct 
drm_atomic_state *state)
 
drm_atomic_helper_commit_modeset_disables(drm, state);
drm_atomic_helper_commit_modeset_enables(drm, state);
+
+   /*
+* The order here is important. We must configure memory-write after
+* the CRTC is already enabled, so that its configuration update is
+* gated on the next CVAL.
+*/
+   malidp_mw_atomic_commit(drm, state);
+
drm_atomic_helper_commit_planes(drm, state, 0);
 
malidp_atomic_commit_hw_done(state);
@@ -147,12 +156,20 @@ static int malidp_init(struct drm_device *drm)
drm->mode_config.helper_private = _mode_config_helpers;
 
ret = malidp_crtc_init(drm);
-   if (ret) {
-   drm_mode_config_cleanup(drm);
-   return ret;
-   }
+   if (ret)
+   goto crtc_fail;
+
+   ret = malidp_mw_connector_init(drm);
+   if (ret)
+   goto mw_fail;
 
return 0;
+
+mw_fail:
+   malidp_de_planes_destroy(drm);
+crtc_fail:
+   drm_mode_config_cleanup(drm);
+   return ret;
 }
 
 static void malidp_fini(struct drm_device *drm)
diff --git a/drivers/gpu/drm/arm/malidp_drv.h b/drivers/gpu/drm/arm/malidp_drv.h
index 9fc8a2e..8abfa8a 100644
--- a/drivers/gpu/drm/arm/malidp_drv.h
+++ b/drivers/gpu/drm/arm/malidp_drv.h
@@ -13,6 +13,7 @@

Re: [RFC PATCH v2 8/9] drm: writeback: Add out-fences for writeback connectors

[Brian Starkey]

Hi Gustavo,

On Wed, Oct 26, 2016 at 07:40:29PM -0200, Gustavo Padovan wrote:

2016-10-26 Brian Starkey <brian.star...@arm.com>:


Add the OUT_FENCE_PTR property to writeback connectors, to enable
userspace to get a fence which will signal once the writeback is
complete.

A timeline is added to drm_connector for use by the writeback
out-fences. It is up to drivers to check for a fence in the
connector_state and signal the it appropriately when their writeback has
finished.

It is not allowed to request an out-fence without a framebuffer attached
to the connector.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_atomic.c|   60 +++---
 drivers/gpu/drm/drm_atomic_helper.c |5 ++-
 drivers/gpu/drm/drm_writeback.c |   80 +++
 include/drm/drm_connector.h |   14 ++
 include/drm/drm_writeback.h |2 +
 5 files changed, 155 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 3f8fc97..061ea13 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -30,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 

 #include "drm_crtc_internal.h"
@@ -646,6 +647,12 @@ static int drm_atomic_connector_check(struct drm_connector 
*connector,
return -EINVAL;
}

+   if (state->out_fence && !state->fb) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] requesting out-fence without 
framebuffer\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
return 0;
 }

@@ -1047,6 +1054,8 @@ int drm_atomic_connector_set_property(struct 
drm_connector *connector,
drm_atomic_set_fb_for_connector(state, fb);
if (fb)
drm_framebuffer_unreference(fb);
+   } else if (property == config->prop_out_fence_ptr) {
+   state->out_fence_ptr = u64_to_user_ptr(val);


We need to move this out of the conn_state. For my v6 on CRTC out fences
I added drm_atomic_set_out_fence_for_crtc() and
drm_atomic_get_out_fence_for_crtc(). See padovan/fences.



Yep, will do. I was expecting this to change


} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
@@ -1088,6 +1097,8 @@ drm_atomic_connector_get_property(struct drm_connector 
*connector,
} else if (property == config->writeback_fb_id_property) {
/* Writeback framebuffer is one-shot, write and forget */
*val = 0;
+   } else if (property == config->prop_out_fence_ptr) {
+   *val = 0;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
@@ -1736,6 +1747,39 @@ static int setup_out_fence(struct drm_out_fence_state 
*fence_state,
return 0;
 }

+static int setup_connector_out_fences(struct drm_atomic_state *state,
+ struct drm_out_fence_state *fence_state,
+ int *fence_idx)
+{
+   struct drm_connector *conn;
+   struct drm_connector_state *conn_state;
+   int i, ret;
+
+   for_each_connector_in_state(state, conn, conn_state, i) {
+   struct fence *fence;
+
+   if (!conn_state->out_fence_ptr)
+   continue;
+
+   fence = drm_writeback_get_out_fence(conn, conn_state);
+   if (!fence)
+   return -ENOMEM;
+
+   ret = setup_out_fence(_state[(*fence_idx)++],
+ conn_state->out_fence_ptr,
+ fence);
+   if (ret) {
+   fence_put(fence);
+   return ret;
+   }
+
+   /* One-time usage only */
+   conn_state->out_fence_ptr = NULL;
+   }
+
+   return 0;
+}
+
 int drm_mode_atomic_ioctl(struct drm_device *dev,
  void *data, struct drm_file *file_priv)
 {
@@ -1868,8 +1912,8 @@ retry:
drm_mode_object_unreference(obj);
}

-   fence_state = kcalloc(dev->mode_config.num_crtc, sizeof(*fence_state),
- GFP_KERNEL);
+   fence_state = kcalloc(dev->mode_config.num_crtc + state->num_connector,
+ sizeof(*fence_state), GFP_KERNEL);
if (!fence_state) {
ret = -ENOMEM;
goto out;
@@ -1929,10 +1973,16 @@ retry:
 * Below we call drm_atomic_state_free for it.
 */
ret = drm

Re: [RFC PATCH v2 7/9] drm: atomic: factor out common out-fence operations

[Brian Starkey]

On Wed, Oct 26, 2016 at 07:45:14PM -0200, Gustavo Padovan wrote:


%p should be kept for your internal debug only. Make sure to remove
anything that exposes kernel address when sending patches.

Gustavo



Noted, thanks!

-Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH v2 9/9] drm: mali-dp: Add writeback out-fence support

[Brian Starkey]

On Wed, Oct 26, 2016 at 07:43:57PM -0200, Gustavo Padovan wrote:

2016-10-26 Brian Starkey <brian.star...@arm.com>:


If userspace has asked for an out-fence for the writeback, we add a
fence to malidp_mw_job, to be signaled when the writeback job has
completed.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |5 -
 drivers/gpu/drm/arm/malidp_mw.c |   18 +-
 drivers/gpu/drm/arm/malidp_mw.h |3 +++
 3 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 1689547..3032226 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -707,8 +707,11 @@ static irqreturn_t malidp_se_irq(int irq, void *arg)
unsigned long irqflags;
/*
 * We can't unreference the framebuffer here, so we queue it
-* up on our threaded handler.
+* up on our threaded handler. However, signal the fence
+* as soon as possible
 */
+   malidp_mw_job_signal(drm, malidp->current_mw, 0);


Drivers should not deal with fences directly. We need some sort of
drm_writeback_finished() that will do the signalling for you.



With a signature like this?
drm_writeback_finished(struct drm_connector_state *state);

I'll have to think about how to achieve that. The state isn't
refcounted and the driver isn't in charge of it's lifetime. I'm not
sure how/where to ensure the state doesn't get destroyed before its
been signaled.

-Brian


Gustavo


--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH v2 8/9] drm: writeback: Add out-fences for writeback connectors

[Brian Starkey]

Hi Gustavo,

It would be great if you could cast your eye over this one as-well.
My intention was to be as similar to the CRTC out-fences as possible.

Thanks,
Brian

On Wed, Oct 26, 2016 at 09:55:07AM +0100, Brian Starkey wrote:

Add the OUT_FENCE_PTR property to writeback connectors, to enable
userspace to get a fence which will signal once the writeback is
complete.

A timeline is added to drm_connector for use by the writeback
out-fences. It is up to drivers to check for a fence in the
connector_state and signal the it appropriately when their writeback has
finished.

It is not allowed to request an out-fence without a framebuffer attached
to the connector.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
drivers/gpu/drm/drm_atomic.c|   60 +++---
drivers/gpu/drm/drm_atomic_helper.c |5 ++-
drivers/gpu/drm/drm_writeback.c |   80 +++
include/drm/drm_connector.h |   14 ++
include/drm/drm_writeback.h |2 +
5 files changed, 155 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 3f8fc97..061ea13 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -30,6 +30,7 @@
#include 
#include 
#include 
+#include 
#include 

#include "drm_crtc_internal.h"
@@ -646,6 +647,12 @@ static int drm_atomic_connector_check(struct drm_connector 
*connector,
return -EINVAL;
}

+   if (state->out_fence && !state->fb) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] requesting out-fence without 
framebuffer\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
return 0;
}

@@ -1047,6 +1054,8 @@ int drm_atomic_connector_set_property(struct 
drm_connector *connector,
drm_atomic_set_fb_for_connector(state, fb);
if (fb)
drm_framebuffer_unreference(fb);
+   } else if (property == config->prop_out_fence_ptr) {
+   state->out_fence_ptr = u64_to_user_ptr(val);
} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
@@ -1088,6 +1097,8 @@ drm_atomic_connector_get_property(struct drm_connector 
*connector,
} else if (property == config->writeback_fb_id_property) {
/* Writeback framebuffer is one-shot, write and forget */
*val = 0;
+   } else if (property == config->prop_out_fence_ptr) {
+   *val = 0;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
@@ -1736,6 +1747,39 @@ static int setup_out_fence(struct drm_out_fence_state 
*fence_state,
return 0;
}

+static int setup_connector_out_fences(struct drm_atomic_state *state,
+ struct drm_out_fence_state *fence_state,
+ int *fence_idx)
+{
+   struct drm_connector *conn;
+   struct drm_connector_state *conn_state;
+   int i, ret;
+
+   for_each_connector_in_state(state, conn, conn_state, i) {
+   struct fence *fence;
+
+   if (!conn_state->out_fence_ptr)
+   continue;
+
+   fence = drm_writeback_get_out_fence(conn, conn_state);
+   if (!fence)
+   return -ENOMEM;
+
+   ret = setup_out_fence(_state[(*fence_idx)++],
+ conn_state->out_fence_ptr,
+ fence);
+   if (ret) {
+   fence_put(fence);
+   return ret;
+   }
+
+   /* One-time usage only */
+   conn_state->out_fence_ptr = NULL;
+   }
+
+   return 0;
+}
+
int drm_mode_atomic_ioctl(struct drm_device *dev,
  void *data, struct drm_file *file_priv)
{
@@ -1868,8 +1912,8 @@ retry:
drm_mode_object_unreference(obj);
}

-   fence_state = kcalloc(dev->mode_config.num_crtc, sizeof(*fence_state),
- GFP_KERNEL);
+   fence_state = kcalloc(dev->mode_config.num_crtc + state->num_connector,
+ sizeof(*fence_state), GFP_KERNEL);
if (!fence_state) {
ret = -ENOMEM;
goto out;
@@ -1929,10 +1973,16 @@ retry:
 * Below we call drm_atomic_state_free for it.
 */
ret = drm_atomic_check_only(state);
-   } else if (arg->flags & DRM_MODE_ATOMIC_NONBLOCK) {
-   ret = drm_atomic_nonblocking_commit(state);
} else {
-   

Re: [RFC PATCH v2 1/9] drm: Add writeback connector type

[Brian Starkey]

On Wed, Oct 26, 2016 at 01:00:21PM +0200, Daniel Vetter wrote:

On Wed, Oct 26, 2016 at 09:55:00AM +0100, Brian Starkey wrote:

Writeback connectors represent writeback engines which can write the
CRTC output to a memory framebuffer. Add a writeback connector type and
related support functions.

Drivers should initialize a writeback connector with
drm_writeback_connector_init() which takes care of setting up all the
writeback-specific details on top of the normal functionality of
drm_connector_init().

Writeback connectors have a WRITEBACK_FB_ID property, used to set the
output framebuffer, PIXEL_FORMATS and PIXEL_FORMATS_SIZE used to expose
the supported writeback formats to userspace.

The drm_atomic core takes care of enforcing fairly strict semantics on
the use of writeback connectors. In short, a writeback connector can
only be included in a commit if it has both a framebuffer and a CRTC.
Conversely, you may not attach a framebuffer if the connector is not
attached to a CRTC, or if the CRTC is disabled.

When a framebuffer is attached to a writeback connector with the
WRITEBACK_FB_ID property, it is used only once (for the commit in which
it was included), and userspace can never read back the value of
WRITEBACK_FB_ID.

Changes since v1:
 - Added drm_writeback.c + documentation
 - Added helper to initialize writeback connector in one go
 - Added core checks
 - Squashed into a single commit
 - Dropped the client cap
 - Writeback framebuffers are no longer persistent

Signed-off-by: Brian Starkey <brian.star...@arm.com>


I think this is looking pretty good. Bunch of nitpicks below, but
otherwise I think we just need the userspace for this, plus an ack from
other people interested in this, and it's good to get merged.
-Daniel



Thanks for having another look,


---
 Documentation/gpu/drm-kms.rst   |9 ++
 drivers/gpu/drm/Makefile|2 +-
 drivers/gpu/drm/drm_atomic.c|   95 +
 drivers/gpu/drm/drm_atomic_helper.c |5 ++
 drivers/gpu/drm/drm_connector.c |4 +-
 drivers/gpu/drm/drm_writeback.c |  157 +++
 include/drm/drm_atomic.h|3 +
 include/drm/drm_connector.h |   12 +++
 include/drm/drm_crtc.h  |   20 +
 include/drm/drm_writeback.h |   19 +
 include/uapi/drm/drm_mode.h |1 +
 11 files changed, 325 insertions(+), 2 deletions(-)
 create mode 100644 drivers/gpu/drm/drm_writeback.c
 create mode 100644 include/drm/drm_writeback.h

diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 53b872c..c3d0370 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -149,6 +149,15 @@ Connector Functions Reference
 .. kernel-doc:: drivers/gpu/drm/drm_connector.c
:export:

+Writeback Connectors
+
+
+.. kernel-doc:: drivers/gpu/drm/drm_writeback.c
+  :doc: overview
+
+.. kernel-doc:: drivers/gpu/drm/drm_writeback.c
+  :export:
+
 Encoder Abstraction
 ===

diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 25c7204..2dc4a48 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -15,7 +15,7 @@ drm-y   :=drm_auth.o drm_bufs.o drm_cache.o \
drm_modeset_lock.o drm_atomic.o drm_bridge.o \
drm_framebuffer.o drm_connector.o drm_blend.o \
drm_encoder.o drm_mode_object.o drm_property.o \
-   drm_plane.o drm_color_mgmt.o
+   drm_plane.o drm_color_mgmt.o drm_writeback.o

 drm-$(CONFIG_COMPAT) += drm_ioc32.o
 drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index fe37987..f434f34 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -612,6 +612,44 @@ static int drm_atomic_crtc_check(struct drm_crtc *crtc,
 }

 /**
+ * drm_atomic_connector_check - check connector state
+ * @connector: connector to check
+ * @state: connector state to check
+ *
+ * Provides core sanity checks for connector state.
+ *
+ * RETURNS:
+ * Zero on success, error code on failure
+ */
+static int drm_atomic_connector_check(struct drm_connector *connector,
+   struct drm_connector_state *state)
+{
+   struct drm_crtc_state *crtc_state;
+
+   if (connector->connector_type != DRM_MODE_CONNECTOR_WRITEBACK)
+   return 0;
+
+   if (!state->fb != !state->crtc) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] framebuffer/CRTC 
mismatch\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
+   if (state->crtc)
+   crtc_state = drm_atomic_get_existing_crtc_state(state->state,
+   state->crtc);
+
+   if (state->fb && !crtc_state->active) {
+   DRM_DEBUG_A

Re: [RFC PATCH v2 7/9] drm: atomic: factor out common out-fence operations

[Brian Starkey]

Hi Gustavo,

As Daniel rightly pointed out you would likely be interested in this
patch.

This is on-top of your v5 patch-set, with the bug-fixes I mentioned
before. I haven't dropped the fence_get(), as I figured you're
probably going to rebase your patches on top of the fence_get() in
sync_file_create(), and then I will do the same.

Thanks,
Brian

On Wed, Oct 26, 2016 at 09:55:06AM +0100, Brian Starkey wrote:

Some parts of setting up the CRTC out-fence can be re-used for
writeback out-fences. Factor this out into a separate function.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
drivers/gpu/drm/drm_atomic.c |   64 +++---
1 file changed, 42 insertions(+), 22 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index f434f34..3f8fc97 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -1693,37 +1693,46 @@ void drm_atomic_clean_old_fb(struct drm_device *dev,
}
EXPORT_SYMBOL(drm_atomic_clean_old_fb);

-static int crtc_setup_out_fence(struct drm_crtc *crtc,
-   struct drm_crtc_state *crtc_state,
-   struct drm_out_fence_state *fence_state)
+static struct fence *get_crtc_fence(struct drm_crtc *crtc,
+   struct drm_crtc_state *crtc_state)
{
struct fence *fence;

-   fence_state->fd = get_unused_fd_flags(O_CLOEXEC);
-   if (fence_state->fd < 0) {
-   return fence_state->fd;
-   }
-
-   fence_state->out_fence_ptr = crtc_state->out_fence_ptr;
-   crtc_state->out_fence_ptr = NULL;
-
-   if (put_user(fence_state->fd, fence_state->out_fence_ptr))
-   return -EFAULT;
-
fence = kzalloc(sizeof(*fence), GFP_KERNEL);
if (!fence)
-   return -ENOMEM;
+   return NULL;

fence_init(fence, _crtc_fence_ops, >fence_lock,
   crtc->fence_context, ++crtc->fence_seqno);

+   crtc_state->event->base.fence = fence_get(fence);
+
+   return fence;
+}
+
+static int setup_out_fence(struct drm_out_fence_state *fence_state,
+  u64 __user *out_fence_ptr,
+  struct fence *fence)
+{
+   int ret;
+
+   DRM_DEBUG_ATOMIC("Putting out-fence %p into user ptr: %p\n",
+fence, out_fence_ptr);
+
+   fence_state->fd = get_unused_fd_flags(O_CLOEXEC);
+   if (fence_state->fd < 0)
+   return fence_state->fd;
+
+   ret = put_user(fence_state->fd, out_fence_ptr);
+   if (ret)
+   return ret;
+
+   fence_state->out_fence_ptr = out_fence_ptr;
+
fence_state->sync_file = sync_file_create(fence);
-   if(!fence_state->sync_file) {
-   fence_put(fence);
+   if (!fence_state->sync_file)
return -ENOMEM;
-   }

-   crtc_state->event->base.fence = fence_get(fence);
return 0;
}

@@ -1896,10 +1905,21 @@ retry:
}

if (crtc_state->out_fence_ptr) {
-   ret = crtc_setup_out_fence(crtc, crtc_state,
-  _state[fence_idx++]);
-   if (ret)
+   struct fence *fence = get_crtc_fence(crtc, crtc_state);
+   if (!fence) {
+   ret = -ENOMEM;
+   goto out;
+   }
+
+   ret = setup_out_fence(_state[fence_idx++],
+ crtc_state->out_fence_ptr,
+ fence);
+   if (ret) {
+   fence_put(fence);
goto out;
+   }
+
+   crtc_state->out_fence_ptr = NULL;
}
}

--
1.7.9.5


--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH v2 9/9] drm: mali-dp: Add writeback out-fence support

[Brian Starkey]

If userspace has asked for an out-fence for the writeback, we add a
fence to malidp_mw_job, to be signaled when the writeback job has
completed.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |5 -
 drivers/gpu/drm/arm/malidp_mw.c |   18 +-
 drivers/gpu/drm/arm/malidp_mw.h |3 +++
 3 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 1689547..3032226 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -707,8 +707,11 @@ static irqreturn_t malidp_se_irq(int irq, void *arg)
unsigned long irqflags;
/*
 * We can't unreference the framebuffer here, so we queue it
-* up on our threaded handler.
+* up on our threaded handler. However, signal the fence
+* as soon as possible
 */
+   malidp_mw_job_signal(drm, malidp->current_mw, 0);
+
spin_lock_irqsave(>mw_lock, irqflags);
list_add_tail(>current_mw->list,
  >finished_mw_jobs);
diff --git a/drivers/gpu/drm/arm/malidp_mw.c b/drivers/gpu/drm/arm/malidp_mw.c
index 69e423c..05880b1 100644
--- a/drivers/gpu/drm/arm/malidp_mw.c
+++ b/drivers/gpu/drm/arm/malidp_mw.c
@@ -34,11 +34,24 @@ struct malidp_mw_connector_state {
u8 n_planes;
 };
 
+void malidp_mw_job_signal(struct drm_device *drm,
+ struct malidp_mw_job *job, int status)
+{
+   DRM_DEV_DEBUG_DRIVER(drm->dev, "MW job signal %p\n", job);
+
+   if (!job->fence)
+   return;
+
+   job->fence->status = status;
+   fence_signal(job->fence);
+}
+
 void malidp_mw_job_cleanup(struct drm_device *drm,
  struct malidp_mw_job *job)
 {
DRM_DEV_DEBUG_DRIVER(drm->dev, "MW job cleanup %p\n", job);
drm_framebuffer_unreference(job->fb);
+   fence_put(job->fence);
kfree(job);
 }
 
@@ -107,8 +120,10 @@ static void malidp_mw_connector_destroy_state(struct 
drm_connector *connector,
struct malidp_mw_connector_state *mw_state = to_mw_state(state);
 
__drm_atomic_helper_connector_destroy_state(_state->base);
-   if (mw_state->job)
+   if (mw_state->job) {
+   malidp_mw_job_signal(connector->dev, mw_state->job, -EPIPE);
malidp_mw_job_cleanup(connector->dev, mw_state->job);
+   }
kfree(mw_state);
 }
 
@@ -177,6 +192,7 @@ malidp_mw_encoder_atomic_check(struct drm_encoder *encoder,
/* We can take ownership of the framebuffer reference in the job. */
mw_state->job->fb = conn_state->fb;
conn_state->fb = NULL;
+   mw_state->job->fence = fence_get(conn_state->out_fence);
 
return 0;
 }
diff --git a/drivers/gpu/drm/arm/malidp_mw.h b/drivers/gpu/drm/arm/malidp_mw.h
index db7b2b0..5d0bd1d 100644
--- a/drivers/gpu/drm/arm/malidp_mw.h
+++ b/drivers/gpu/drm/arm/malidp_mw.h
@@ -15,11 +15,14 @@
 struct malidp_mw_job {
struct list_head list;
struct drm_framebuffer *fb;
+   struct fence *fence;
 };
 
 int malidp_mw_connector_init(struct drm_device *drm);
 void malidp_mw_atomic_commit(struct drm_device *drm,
 struct drm_atomic_state *old_state);
+void malidp_mw_job_signal(struct drm_device *drm,
+ struct malidp_mw_job *job, int status);
 void malidp_mw_job_cleanup(struct drm_device *drm,
  struct malidp_mw_job *job);
 #endif
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH v2 1/9] drm: Add writeback connector type

[Brian Starkey]

Writeback connectors represent writeback engines which can write the
CRTC output to a memory framebuffer. Add a writeback connector type and
related support functions.

Drivers should initialize a writeback connector with
drm_writeback_connector_init() which takes care of setting up all the
writeback-specific details on top of the normal functionality of
drm_connector_init().

Writeback connectors have a WRITEBACK_FB_ID property, used to set the
output framebuffer, PIXEL_FORMATS and PIXEL_FORMATS_SIZE used to expose
the supported writeback formats to userspace.

The drm_atomic core takes care of enforcing fairly strict semantics on
the use of writeback connectors. In short, a writeback connector can
only be included in a commit if it has both a framebuffer and a CRTC.
Conversely, you may not attach a framebuffer if the connector is not
attached to a CRTC, or if the CRTC is disabled.

When a framebuffer is attached to a writeback connector with the
WRITEBACK_FB_ID property, it is used only once (for the commit in which
it was included), and userspace can never read back the value of
WRITEBACK_FB_ID.

Changes since v1:
 - Added drm_writeback.c + documentation
 - Added helper to initialize writeback connector in one go
 - Added core checks
 - Squashed into a single commit
 - Dropped the client cap
 - Writeback framebuffers are no longer persistent

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 Documentation/gpu/drm-kms.rst   |9 ++
 drivers/gpu/drm/Makefile|2 +-
 drivers/gpu/drm/drm_atomic.c|   95 +
 drivers/gpu/drm/drm_atomic_helper.c |5 ++
 drivers/gpu/drm/drm_connector.c |4 +-
 drivers/gpu/drm/drm_writeback.c |  157 +++
 include/drm/drm_atomic.h|3 +
 include/drm/drm_connector.h |   12 +++
 include/drm/drm_crtc.h  |   20 +
 include/drm/drm_writeback.h |   19 +
 include/uapi/drm/drm_mode.h |1 +
 11 files changed, 325 insertions(+), 2 deletions(-)
 create mode 100644 drivers/gpu/drm/drm_writeback.c
 create mode 100644 include/drm/drm_writeback.h

diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 53b872c..c3d0370 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -149,6 +149,15 @@ Connector Functions Reference
 .. kernel-doc:: drivers/gpu/drm/drm_connector.c
:export:
 
+Writeback Connectors
+
+
+.. kernel-doc:: drivers/gpu/drm/drm_writeback.c
+  :doc: overview
+
+.. kernel-doc:: drivers/gpu/drm/drm_writeback.c
+  :export:
+
 Encoder Abstraction
 ===
 
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 25c7204..2dc4a48 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -15,7 +15,7 @@ drm-y   :=drm_auth.o drm_bufs.o drm_cache.o \
drm_modeset_lock.o drm_atomic.o drm_bridge.o \
drm_framebuffer.o drm_connector.o drm_blend.o \
drm_encoder.o drm_mode_object.o drm_property.o \
-   drm_plane.o drm_color_mgmt.o
+   drm_plane.o drm_color_mgmt.o drm_writeback.o
 
 drm-$(CONFIG_COMPAT) += drm_ioc32.o
 drm-$(CONFIG_DRM_GEM_CMA_HELPER) += drm_gem_cma_helper.o
diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index fe37987..f434f34 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -612,6 +612,44 @@ static int drm_atomic_crtc_check(struct drm_crtc *crtc,
 }
 
 /**
+ * drm_atomic_connector_check - check connector state
+ * @connector: connector to check
+ * @state: connector state to check
+ *
+ * Provides core sanity checks for connector state.
+ *
+ * RETURNS:
+ * Zero on success, error code on failure
+ */
+static int drm_atomic_connector_check(struct drm_connector *connector,
+   struct drm_connector_state *state)
+{
+   struct drm_crtc_state *crtc_state;
+
+   if (connector->connector_type != DRM_MODE_CONNECTOR_WRITEBACK)
+   return 0;
+
+   if (!state->fb != !state->crtc) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] framebuffer/CRTC 
mismatch\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
+   if (state->crtc)
+   crtc_state = drm_atomic_get_existing_crtc_state(state->state,
+   state->crtc);
+
+   if (state->fb && !crtc_state->active) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] has framebuffer, but 
[CRTC:%d] is off\n",
+connector->base.id, connector->name,
+state->crtc->base.id);
+   return -EINVAL;
+   }
+
+   return 0;
+}
+
+/**
  * drm_atomic_get_plane_state - get plane state
  * @state: global atomic state 

[RFC PATCH v2 8/9] drm: writeback: Add out-fences for writeback connectors

[Brian Starkey]

Add the OUT_FENCE_PTR property to writeback connectors, to enable
userspace to get a fence which will signal once the writeback is
complete.

A timeline is added to drm_connector for use by the writeback
out-fences. It is up to drivers to check for a fence in the
connector_state and signal the it appropriately when their writeback has
finished.

It is not allowed to request an out-fence without a framebuffer attached
to the connector.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_atomic.c|   60 +++---
 drivers/gpu/drm/drm_atomic_helper.c |5 ++-
 drivers/gpu/drm/drm_writeback.c |   80 +++
 include/drm/drm_connector.h |   14 ++
 include/drm/drm_writeback.h |2 +
 5 files changed, 155 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 3f8fc97..061ea13 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -30,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #include "drm_crtc_internal.h"
@@ -646,6 +647,12 @@ static int drm_atomic_connector_check(struct drm_connector 
*connector,
return -EINVAL;
}
 
+   if (state->out_fence && !state->fb) {
+   DRM_DEBUG_ATOMIC("[CONNECTOR:%d:%s] requesting out-fence 
without framebuffer\n",
+connector->base.id, connector->name);
+   return -EINVAL;
+   }
+
return 0;
 }
 
@@ -1047,6 +1054,8 @@ int drm_atomic_connector_set_property(struct 
drm_connector *connector,
drm_atomic_set_fb_for_connector(state, fb);
if (fb)
drm_framebuffer_unreference(fb);
+   } else if (property == config->prop_out_fence_ptr) {
+   state->out_fence_ptr = u64_to_user_ptr(val);
} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
@@ -1088,6 +1097,8 @@ drm_atomic_connector_get_property(struct drm_connector 
*connector,
} else if (property == config->writeback_fb_id_property) {
/* Writeback framebuffer is one-shot, write and forget */
*val = 0;
+   } else if (property == config->prop_out_fence_ptr) {
+   *val = 0;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
@@ -1736,6 +1747,39 @@ static int setup_out_fence(struct drm_out_fence_state 
*fence_state,
return 0;
 }
 
+static int setup_connector_out_fences(struct drm_atomic_state *state,
+ struct drm_out_fence_state *fence_state,
+ int *fence_idx)
+{
+   struct drm_connector *conn;
+   struct drm_connector_state *conn_state;
+   int i, ret;
+
+   for_each_connector_in_state(state, conn, conn_state, i) {
+   struct fence *fence;
+
+   if (!conn_state->out_fence_ptr)
+   continue;
+
+   fence = drm_writeback_get_out_fence(conn, conn_state);
+   if (!fence)
+   return -ENOMEM;
+
+   ret = setup_out_fence(_state[(*fence_idx)++],
+ conn_state->out_fence_ptr,
+ fence);
+   if (ret) {
+   fence_put(fence);
+   return ret;
+   }
+
+   /* One-time usage only */
+   conn_state->out_fence_ptr = NULL;
+   }
+
+   return 0;
+}
+
 int drm_mode_atomic_ioctl(struct drm_device *dev,
  void *data, struct drm_file *file_priv)
 {
@@ -1868,8 +1912,8 @@ retry:
drm_mode_object_unreference(obj);
}
 
-   fence_state = kcalloc(dev->mode_config.num_crtc, sizeof(*fence_state),
- GFP_KERNEL);
+   fence_state = kcalloc(dev->mode_config.num_crtc + state->num_connector,
+ sizeof(*fence_state), GFP_KERNEL);
if (!fence_state) {
ret = -ENOMEM;
goto out;
@@ -1929,10 +1973,16 @@ retry:
 * Below we call drm_atomic_state_free for it.
 */
ret = drm_atomic_check_only(state);
-   } else if (arg->flags & DRM_MODE_ATOMIC_NONBLOCK) {
-   ret = drm_atomic_nonblocking_commit(state);
} else {
-   ret = drm_atomic_commit(state);
+   ret = setup_connector_out_fences(state, fence_state,
+_idx);
+   if (ret)
+  

[RFC PATCH v2 7/9] drm: atomic: factor out common out-fence operations

[Brian Starkey]

Some parts of setting up the CRTC out-fence can be re-used for
writeback out-fences. Factor this out into a separate function.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_atomic.c |   64 +++---
 1 file changed, 42 insertions(+), 22 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index f434f34..3f8fc97 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -1693,37 +1693,46 @@ void drm_atomic_clean_old_fb(struct drm_device *dev,
 }
 EXPORT_SYMBOL(drm_atomic_clean_old_fb);
 
-static int crtc_setup_out_fence(struct drm_crtc *crtc,
-   struct drm_crtc_state *crtc_state,
-   struct drm_out_fence_state *fence_state)
+static struct fence *get_crtc_fence(struct drm_crtc *crtc,
+   struct drm_crtc_state *crtc_state)
 {
struct fence *fence;
 
-   fence_state->fd = get_unused_fd_flags(O_CLOEXEC);
-   if (fence_state->fd < 0) {
-   return fence_state->fd;
-   }
-
-   fence_state->out_fence_ptr = crtc_state->out_fence_ptr;
-   crtc_state->out_fence_ptr = NULL;
-
-   if (put_user(fence_state->fd, fence_state->out_fence_ptr))
-   return -EFAULT;
-
fence = kzalloc(sizeof(*fence), GFP_KERNEL);
if (!fence)
-   return -ENOMEM;
+   return NULL;
 
fence_init(fence, _crtc_fence_ops, >fence_lock,
   crtc->fence_context, ++crtc->fence_seqno);
 
+   crtc_state->event->base.fence = fence_get(fence);
+
+   return fence;
+}
+
+static int setup_out_fence(struct drm_out_fence_state *fence_state,
+  u64 __user *out_fence_ptr,
+  struct fence *fence)
+{
+   int ret;
+
+   DRM_DEBUG_ATOMIC("Putting out-fence %p into user ptr: %p\n",
+fence, out_fence_ptr);
+
+   fence_state->fd = get_unused_fd_flags(O_CLOEXEC);
+   if (fence_state->fd < 0)
+   return fence_state->fd;
+
+   ret = put_user(fence_state->fd, out_fence_ptr);
+   if (ret)
+   return ret;
+
+   fence_state->out_fence_ptr = out_fence_ptr;
+
fence_state->sync_file = sync_file_create(fence);
-   if(!fence_state->sync_file) {
-   fence_put(fence);
+   if (!fence_state->sync_file)
return -ENOMEM;
-   }
 
-   crtc_state->event->base.fence = fence_get(fence);
return 0;
 }
 
@@ -1896,10 +1905,21 @@ retry:
}
 
if (crtc_state->out_fence_ptr) {
-   ret = crtc_setup_out_fence(crtc, crtc_state,
-  _state[fence_idx++]);
-   if (ret)
+   struct fence *fence = get_crtc_fence(crtc, crtc_state);
+   if (!fence) {
+   ret = -ENOMEM;
+   goto out;
+   }
+
+   ret = setup_out_fence(_state[fence_idx++],
+ crtc_state->out_fence_ptr,
+ fence);
+   if (ret) {
+   fence_put(fence);
goto out;
+   }
+
+   crtc_state->out_fence_ptr = NULL;
}
}
 
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH v2 5/9] drm: mali-dp: Add support for writeback on DP550/DP650

[Brian Starkey]

From: Liviu Dudau <liviu.du...@arm.com>

Mali-DP display processors are able to write the composition result to a
memory buffer via the SE.

Add entry points in the HAL for enabling/disabling this feature, and
implement support for it on DP650 and DP550. DP500 acts differently and
so is omitted from this change.

Signed-off-by: Liviu Dudau <liviu.du...@arm.com>
Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c   |   52 +++--
 drivers/gpu/drm/arm/malidp_hw.h   |   18 +
 drivers/gpu/drm/arm/malidp_regs.h |   15 +++
 3 files changed, 83 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index df1fce0..5004988 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -389,6 +389,48 @@ static int malidp550_rotmem_required(struct 
malidp_hw_device *hwdev, u16 w, u16
return w * bytes_per_col;
 }
 
+static int malidp550_enable_memwrite(struct malidp_hw_device *hwdev,
+dma_addr_t *addrs, s32 *pitches,
+int num_planes, u16 w, u16 h, u32 fmt_id)
+{
+   u32 base = MALIDP550_SE_MEMWRITE_BASE;
+   u32 de_base = malidp_get_block_base(hwdev, MALIDP_DE_BLOCK);
+
+   /* enable the scaling engine block */
+   malidp_hw_setbits(hwdev, MALIDP_SCALE_ENGINE_EN, de_base + 
MALIDP_DE_DISPLAY_FUNC);
+
+   malidp_hw_write(hwdev, fmt_id, base + MALIDP_MW_FORMAT);
+   switch (num_planes) {
+   case 2:
+   malidp_hw_write(hwdev, lower_32_bits(addrs[1]), base + 
MALIDP_MW_P2_PTR_LOW);
+   malidp_hw_write(hwdev, upper_32_bits(addrs[1]), base + 
MALIDP_MW_P2_PTR_HIGH);
+   malidp_hw_write(hwdev, pitches[1], base + MALIDP_MW_P2_STRIDE);
+   /* fall through */
+   case 1:
+   malidp_hw_write(hwdev, lower_32_bits(addrs[0]), base + 
MALIDP_MW_P1_PTR_LOW);
+   malidp_hw_write(hwdev, upper_32_bits(addrs[0]), base + 
MALIDP_MW_P1_PTR_HIGH);
+   malidp_hw_write(hwdev, pitches[0], base + MALIDP_MW_P1_STRIDE);
+   break;
+   default:
+   WARN(1, "Invalid number of planes");
+   }
+
+   malidp_hw_write(hwdev, MALIDP_DE_H_ACTIVE(w) | MALIDP_DE_V_ACTIVE(h),
+   MALIDP550_SE_MEMWRITE_OUT_SIZE);
+   malidp_hw_setbits(hwdev, MALIDP550_SE_MEMWRITE_ONESHOT | 
MALIDP_SE_MEMWRITE_EN,
+ MALIDP550_SE_CONTROL);
+
+   return 0;
+}
+
+static void malidp550_disable_memwrite(struct malidp_hw_device *hwdev)
+{
+   u32 base = malidp_get_block_base(hwdev, MALIDP_DE_BLOCK);
+   malidp_hw_clearbits(hwdev, MALIDP550_SE_MEMWRITE_ONESHOT | 
MALIDP_SE_MEMWRITE_EN,
+   MALIDP550_SE_CONTROL);
+   malidp_hw_clearbits(hwdev, MALIDP_SCALE_ENGINE_EN, base + 
MALIDP_DE_DISPLAY_FUNC);
+}
+
 static int malidp650_query_hw(struct malidp_hw_device *hwdev)
 {
u32 conf = malidp_hw_read(hwdev, MALIDP550_CONFIG_ID);
@@ -471,7 +513,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
MALIDP550_SE_IRQ_AXI_ERR,
},
.dc_irq_map = {
-   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
+   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID |
+   MALIDP550_DC_IRQ_SE,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
.pixel_formats = malidp550_de_formats,
@@ -485,6 +528,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.set_config_valid = malidp550_set_config_valid,
.modeset = malidp550_modeset,
.rotmem_required = malidp550_rotmem_required,
+   .enable_memwrite = malidp550_enable_memwrite,
+   .disable_memwrite = malidp550_disable_memwrite,
},
[MALIDP_650] = {
.map = {
@@ -505,7 +550,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
MALIDP550_SE_IRQ_AXI_ERR,
},
.dc_irq_map = {
-   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
+   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID |
+   MALIDP550_DC_IRQ_SE,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
.pixel_formats = malidp550_de_formats,
@@ -519,6 +565,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.set_config_valid = malidp550_set_config_valid,
.modeset = malidp550_modeset,
.rotmem_required

[RFC PATCH v2 3/9] drm: mali-dp: Rename malidp_input_format

[Brian Starkey]

We're going to use the same format list for output formats, so rename
everything related to input formats to avoid confusion.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
Reviewed-by: Liviu Dudau <liviu.du...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |   24 
 drivers/gpu/drm/arm/malidp_hw.h |8 
 drivers/gpu/drm/arm/malidp_planes.c |8 
 3 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 65e667b..eb8a4bf 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -21,7 +21,7 @@
 #include "malidp_drv.h"
 #include "malidp_hw.h"
 
-static const struct malidp_input_format malidp500_de_formats[] = {
+static const struct malidp_format_id malidp500_de_formats[] = {
/*fourcc,   layers supporting the format, internal id  */
{ DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2,  0 },
{ DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2,  1 },
@@ -69,7 +69,7 @@ static const struct malidp_input_format 
malidp500_de_formats[] = {
{ DRM_FORMAT_NV12, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 6) },\
{ DRM_FORMAT_YUV420, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 7) }
 
-static const struct malidp_input_format malidp550_de_formats[] = {
+static const struct malidp_format_id malidp550_de_formats[] = {
MALIDP_COMMON_FORMATS,
 };
 
@@ -441,8 +441,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.irq_mask = MALIDP500_DE_IRQ_CONF_VALID,
.vsync_irq = MALIDP500_DE_IRQ_CONF_VALID,
},
-   .input_formats = malidp500_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp500_de_formats),
+   .pixel_formats = malidp500_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp500_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp500_query_hw,
@@ -474,8 +474,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
-   .input_formats = malidp550_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+   .pixel_formats = malidp550_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp550_query_hw,
@@ -508,8 +508,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
-   .input_formats = malidp550_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+   .pixel_formats = malidp550_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 16,
},
.query_hw = malidp650_query_hw,
@@ -527,10 +527,10 @@ u8 malidp_hw_get_format_id(const struct malidp_hw_regmap 
*map,
 {
unsigned int i;
 
-   for (i = 0; i < map->n_input_formats; i++) {
-   if (((map->input_formats[i].layer & layer_id) == layer_id) &&
-   (map->input_formats[i].format == format))
-   return map->input_formats[i].id;
+   for (i = 0; i < map->n_pixel_formats; i++) {
+   if (((map->pixel_formats[i].layer & layer_id) == layer_id) &&
+   (map->pixel_formats[i].format == format))
+   return map->pixel_formats[i].id;
}
 
return MALIDP_INVALID_FORMAT_ID;
diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h
index 087e1202..4f8c884 100644
--- a/drivers/gpu/drm/arm/malidp_hw.h
+++ b/drivers/gpu/drm/arm/malidp_hw.h
@@ -35,7 +35,7 @@ enum {
DE_SMART = BIT(4),
 };
 
-struct malidp_input_format {
+struct malidp_format_id {
u32 format; /* DRM fourcc */
u8 layer;   /* bitmask of layers supporting it */
u8 id;  /* used internally */
@@ -85,9 +85,9 @@ struct malidp_hw_regmap {
const struct malidp_irq_map se_irq_map;
const struct malidp_irq_map dc_irq_map;
 
-   /* list of supported input formats for each layer */
-   const struct malidp_input_format *input_formats;
-   const u8 n_input_formats;
+   /* l

[RFC PATCH v2 6/9] drm: mali-dp: Add writeback connector

[Brian Starkey]

Mali-DP has a memory writeback engine which can be used to write the
composition result to a memory buffer.
Expose this functionality as a DRM writeback connector on supported
hardware.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/Makefile  |1 +
 drivers/gpu/drm/arm/malidp_crtc.c |   21 +++
 drivers/gpu/drm/arm/malidp_drv.c  |   28 +++-
 drivers/gpu/drm/arm/malidp_drv.h  |7 +
 drivers/gpu/drm/arm/malidp_hw.c   |   40 -
 drivers/gpu/drm/arm/malidp_mw.c   |  297 +
 drivers/gpu/drm/arm/malidp_mw.h   |   25 
 7 files changed, 413 insertions(+), 6 deletions(-)
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.c
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.h

diff --git a/drivers/gpu/drm/arm/Makefile b/drivers/gpu/drm/arm/Makefile
index bb8b158..3bf31d1 100644
--- a/drivers/gpu/drm/arm/Makefile
+++ b/drivers/gpu/drm/arm/Makefile
@@ -1,4 +1,5 @@
 hdlcd-y := hdlcd_drv.o hdlcd_crtc.o
 obj-$(CONFIG_DRM_HDLCD)+= hdlcd.o
 mali-dp-y := malidp_drv.o malidp_hw.o malidp_planes.o malidp_crtc.o
+mali-dp-y += malidp_mw.o
 obj-$(CONFIG_DRM_MALI_DISPLAY) += mali-dp.o
diff --git a/drivers/gpu/drm/arm/malidp_crtc.c 
b/drivers/gpu/drm/arm/malidp_crtc.c
index 08e6a71..aadc223 100644
--- a/drivers/gpu/drm/arm/malidp_crtc.c
+++ b/drivers/gpu/drm/arm/malidp_crtc.c
@@ -68,6 +68,18 @@ static void malidp_crtc_enable(struct drm_crtc *crtc)
clk_set_rate(hwdev->pxlclk, crtc->state->adjusted_mode.crtc_clock * 
1000);
 
hwdev->modeset(hwdev, );
+   /*
+* We should always disable the memory write when leaving config mode,
+* otherwise the hardware will start writing right away - possibly with
+* a stale config, and definitely before we've had a chance to configure
+* the planes.
+* If the memory write needs to be enabled, that will get taken care
+* of later during the atomic commit
+*/
+   if (hwdev->disable_memwrite) {
+   DRM_DEV_DEBUG_DRIVER(crtc->dev->dev, "Disable memwrite\n");
+   hwdev->disable_memwrite(hwdev);
+   }
hwdev->leave_config_mode(hwdev);
drm_crtc_vblank_on(crtc);
 }
@@ -157,6 +169,15 @@ static int malidp_crtc_atomic_check(struct drm_crtc *crtc,
}
}
 
+   /* If only the writeback routing has changed, we don't need a modeset */
+   if (state->connectors_changed) {
+   u32 old_mask = crtc->state->connector_mask;
+   u32 new_mask = state->connector_mask;
+   if ((old_mask ^ new_mask) ==
+   (1 << drm_connector_index(>mw_connector)))
+   state->connectors_changed = false;
+   }
+
return 0;
 }
 
diff --git a/drivers/gpu/drm/arm/malidp_drv.c b/drivers/gpu/drm/arm/malidp_drv.c
index f63a4df..cdefbab 100644
--- a/drivers/gpu/drm/arm/malidp_drv.c
+++ b/drivers/gpu/drm/arm/malidp_drv.c
@@ -28,6 +28,7 @@
 #include 
 
 #include "malidp_drv.h"
+#include "malidp_mw.h"
 #include "malidp_regs.h"
 #include "malidp_hw.h"
 
@@ -92,6 +93,14 @@ static void malidp_atomic_commit_tail(struct 
drm_atomic_state *state)
 
drm_atomic_helper_commit_modeset_disables(drm, state);
drm_atomic_helper_commit_modeset_enables(drm, state);
+
+   /*
+* The order here is important. We must configure memory-write after
+* the CRTC is already enabled, so that its configuration update is
+* gated on the next CVAL.
+*/
+   malidp_mw_atomic_commit(drm, state);
+
drm_atomic_helper_commit_planes(drm, state, 0);
 
malidp_atomic_commit_hw_done(state);
@@ -147,12 +156,20 @@ static int malidp_init(struct drm_device *drm)
drm->mode_config.helper_private = _mode_config_helpers;
 
ret = malidp_crtc_init(drm);
-   if (ret) {
-   drm_mode_config_cleanup(drm);
-   return ret;
-   }
+   if (ret)
+   goto crtc_fail;
+
+   ret = malidp_mw_connector_init(drm);
+   if (ret)
+   goto mw_fail;
 
return 0;
+
+mw_fail:
+   malidp_de_planes_destroy(drm);
+crtc_fail:
+   drm_mode_config_cleanup(drm);
+   return ret;
 }
 
 static void malidp_fini(struct drm_device *drm)
@@ -357,6 +374,9 @@ static int malidp_bind(struct device *dev)
atomic_set(>config_valid, 0);
init_waitqueue_head(>wq);
 
+   INIT_LIST_HEAD(>finished_mw_jobs);
+   spin_lock_init(>mw_lock);
+
ret = malidp_init(drm);
if (ret < 0)
goto init_fail;
diff --git a/drivers/gpu/drm/arm/malidp_drv.h b/drivers/gpu/drm/arm/malidp_drv.h
index 9fc8a2e..0dce0cf 100644
--- a/drivers/gpu/drm/arm/malidp_drv.h
+++ b/drivers/gpu/drm/arm/malidp_drv.h
@@ -22,8 +22,15 @@ struct malidp_drm {
struct drm_fbdev_cma *fbdev;
struct list_he

[RFC PATCH v2 2/9] drm: mali-dp: Clear CVAL when leaving config mode

[Brian Starkey]

It's possible for CVAL to get set whilst we are in config mode. If this
happens, afer we leave config mode the HW will latch whatever
configuration is in the registers at the next vsync. Most likely this
will be a partial configuration, as we'll be racing against the ongoing
atomic_commit.

To avoid this, clear CVAL before leaving config mode.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 7f4a0bd..65e667b 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -125,6 +125,7 @@ static void malidp500_leave_config_mode(struct 
malidp_hw_device *hwdev)
 {
u32 status, count = 100;
 
+   malidp_hw_clearbits(hwdev, MALIDP_CFG_VALID, MALIDP500_CONFIG_VALID);
malidp_hw_clearbits(hwdev, MALIDP500_DC_CONFIG_REQ, 
MALIDP500_DC_CONTROL);
while (count) {
status = malidp_hw_read(hwdev, hwdev->map.dc_base + 
MALIDP_REG_STATUS);
@@ -271,6 +272,7 @@ static void malidp550_leave_config_mode(struct 
malidp_hw_device *hwdev)
 {
u32 status, count = 100;
 
+   malidp_hw_clearbits(hwdev, MALIDP_CFG_VALID, MALIDP550_CONFIG_VALID);
malidp_hw_clearbits(hwdev, MALIDP550_DC_CONFIG_REQ, 
MALIDP550_DC_CONTROL);
while (count) {
status = malidp_hw_read(hwdev, hwdev->map.dc_base + 
MALIDP_REG_STATUS);
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH v2 4/9] drm: mali-dp: Add RGB writeback formats for DP550/DP650

[Brian Starkey]

Add a layer bit for the SE memory-write, and add it to the pixel format
matrix for DP550/DP650.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |   28 ++--
 drivers/gpu/drm/arm/malidp_hw.h |1 +
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index eb8a4bf..df1fce0 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -46,20 +46,20 @@ static const struct malidp_format_id malidp500_de_formats[] 
= {
 
 #define MALIDP_COMMON_FORMATS \
/*fourcc,   layers supporting the format,  internal id   */ \
-   { DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 0) }, \
-   { DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 1) }, \
-   { DRM_FORMAT_RGBA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 2) }, \
-   { DRM_FORMAT_BGRA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 3) }, \
-   { DRM_FORMAT_ARGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 0) }, \
-   { DRM_FORMAT_ABGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 1) }, \
-   { DRM_FORMAT_RGBA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 2) }, \
-   { DRM_FORMAT_BGRA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 3) }, \
-   { DRM_FORMAT_XRGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 0) }, \
-   { DRM_FORMAT_XBGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 1) }, \
-   { DRM_FORMAT_RGBX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 2) }, \
-   { DRM_FORMAT_BGRX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 3) }, \
-   { DRM_FORMAT_RGB888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(3, 
0) }, \
-   { DRM_FORMAT_BGR888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(3, 
1) }, \
+   { DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 0) }, \
+   { DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 1) }, \
+   { DRM_FORMAT_RGBA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 2) }, \
+   { DRM_FORMAT_BGRA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 3) }, \
+   { DRM_FORMAT_ARGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 0) }, \
+   { DRM_FORMAT_ABGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 1) }, \
+   { DRM_FORMAT_RGBA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 2) }, \
+   { DRM_FORMAT_BGRA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 3) }, \
+   { DRM_FORMAT_XRGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 0) }, \
+   { DRM_FORMAT_XBGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 1) }, \
+   { DRM_FORMAT_RGBX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 2) }, \
+   { DRM_FORMAT_BGRX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 3) }, \
+   { DRM_FORMAT_RGB888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(3, 0) }, \
+   { DRM_FORMAT_BGR888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(3, 1) }, \
{ DRM_FORMAT_RGBA5551, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(4, 0) }, \
{ DRM_FORMAT_ABGR1555, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(4, 1) }, \
{ DRM_FORMAT_RGB565, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(4, 
2) }, \
diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h
index 4f8c884..ce4ea55 100644
--- a/drivers/gpu/drm/arm/malidp_hw.h
+++ b/drivers/gpu/drm/arm/malidp_hw.h
@@ -33,6 +33,7 @@ enum {
DE_GRAPHICS2 = BIT(2), /* used only in DP500 */
DE_VIDEO2 = BIT(3),
DE_SMART = BIT(4),
+   SE_MEMWRITE = BIT(5),
 };
 
 struct malidp_format_id {
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH v2 0/9] Introduce writeback connectors

[Brian Starkey]

Hi,

This is an updated RFC series introducing a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
See v1 here: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Writeback connectors are given a WRITEBACK_FB_ID property (which acts
slightly differently to FB_ID, so gets a new name), as well as
PIXEL_FORMATS and PIXEL_FORMATS_SIZE to list the supported writeback
formats, and OUT_FENCE_PTR to be used for out-fences.

The semantics of writeback connectors have been changed significantly
since v1, based largely on Daniel's feedback. Now, a writeback
connector can only be attached to a CRTC if it has a framebuffer
attached and vice-versa. The writeback framebuffer applies only to a
single atomic commit, and userspace can never read back the value of
WRITEBACK_FB_ID. This makes writeback a "one-shot" operation, it must
be re-armed every time it is to be used.

Patch 1 introduces the actual connector type and the infrastructure
around it. Patches 2-6 add a writeback connector for mali-dp.

Patches 7-9 add support for writeback out-fences, based on Gustavo
Padovan's v5 series [2] for adding explicit fencing.

As always, I look forward to any comments.

Thanks,
Brian

[1] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1247574.html
[2] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1253822.html

Changes since v1, based on Daniel and Eric's comments:
 - The writeback framebuffer is no longer persistent across commits
 - Removed the client cap, made the connector report disconnected
   instead
 - Added drm_writeback.c for central connector initialization and
   documentation
 - Added support for out-fences
 - Added core checks for writeback connectors, e.g. disallowing
   a framebuffer with no CRTC
 - Mali-DP doesn't require a full modeset to enable/disable the
   writeback connector

---

Brian Starkey (8):
  drm: Add writeback connector type
  drm: mali-dp: Clear CVAL when leaving config mode
  drm: mali-dp: Rename malidp_input_format
  drm: mali-dp: Add RGB writeback formats for DP550/DP650
  drm: mali-dp: Add writeback connector
  drm: atomic: factor out common out-fence operations
  drm: writeback: Add out-fences for writeback connectors
  drm: mali-dp: Add writeback out-fence support

Liviu Dudau (1):
  drm: mali-dp: Add support for writeback on DP550/DP650

 Documentation/gpu/drm-kms.rst   |9 +
 drivers/gpu/drm/Makefile|2 +-
 drivers/gpu/drm/arm/Makefile|1 +
 drivers/gpu/drm/arm/malidp_crtc.c   |   21 +++
 drivers/gpu/drm/arm/malidp_drv.c|   28 +++-
 drivers/gpu/drm/arm/malidp_drv.h|7 +
 drivers/gpu/drm/arm/malidp_hw.c |  149 +
 drivers/gpu/drm/arm/malidp_hw.h |   27 ++-
 drivers/gpu/drm/arm/malidp_mw.c |  313 +++
 drivers/gpu/drm/arm/malidp_mw.h |   28 
 drivers/gpu/drm/arm/malidp_planes.c |8 +-
 drivers/gpu/drm/arm/malidp_regs.h   |   15 ++
 drivers/gpu/drm/drm_atomic.c|  218 +---
 drivers/gpu/drm/drm_atomic_helper.c |8 +
 drivers/gpu/drm/drm_connector.c |4 +-
 drivers/gpu/drm/drm_writeback.c |  237 ++
 include/drm/drm_atomic.h|3 +
 include/drm/drm_connector.h |   26 +++
 include/drm/drm_crtc.h  |   20 +++
 include/drm/drm_writeback.h |   21 +++
 include/uapi/drm/drm_mode.h |1 +
 21 files changed, 1076 insertions(+), 70 deletions(-)
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.c
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.h
 create mode 100644 drivers/gpu/drm/drm_writeback.c
 create mode 100644 include/drm/drm_writeback.h

-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 00/11] Introduce writeback connectors

[Brian Starkey]

Hi Archit,

On Fri, Oct 14, 2016 at 04:20:14PM +0530, Archit Taneja wrote:

Hi Brian,

On 10/11/2016 08:23 PM, Brian Starkey wrote:

Hi,

This RFC series introduces a new connector type:
DRM_MODE_CONNECTOR_WRITEBACK
It is a follow-on from a previous discussion: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Patches 1-7 include the core framework changes required, and patches
8-11 implement a writeback connector for the Mali-DP writeback engine.
The Mali-DP patches depend on this other series: [2].

The connector is given the FB_ID property for the output framebuffer,
and two new read-only properties: PIXEL_FORMATS and
PIXEL_FORMATS_SIZE, which expose the supported framebuffer pixel
formats of the engine.

The EDID property is not exposed for writeback connectors.

Writeback connector usage:
--
Due to connector routing changes being treated as "full modeset"
operations, any client which wishes to use a writeback connector
should include the connector in every modeset. The writeback will not
actually become active until a framebuffer is attached.

The writeback itself is enabled by attaching a framebuffer to the
FB_ID property of the connector. The driver must then ensure that the
CRTC content of that atomic commit is written into the framebuffer.

The writeback works in a one-shot mode with each atomic commit. This
prevents the same content from being written multiple times.
In some cases (front-buffer rendering) there might be a desire for
continuous operation - I think a property could be added later for
this kind of control.

Writeback can be disabled by setting FB_ID to zero.

Known issues:
-
* I'm not sure what "DPMS" should mean for writeback connectors.
  It could be used to disable writeback (even when a framebuffer is
  attached), or it could be hidden entirely (which would break the
  legacy DPMS call for writeback connectors).
* With Daniel's recent re-iteration of the userspace API rules, I
  fully expect to provide some userspace code to support this. The
  question is what, and where? We want to use writeback for testing,
  so perhaps some tests in igt is suitable.
* Documentation. Probably some portion of this cover letter needs to
  make it into Documentation/
* Synchronisation. Our hardware will finish the writeback by the next
  vsync. I've not implemented fence support here, but it would be an
  obvious addition.

See Also:
-
[1] https://lists.freedesktop.org/archives/dri-devel/2016-July/113197.html
[2] https://lists.freedesktop.org/archives/dri-devel/2016-October/120486.html

I welcome any comments, especially if this approach does/doesn't fit
well with anyone else's hardware.


Thanks for working on this! Some points below.

- Writeback hardware generally allows us to specify the region within
the framebuffer we want to write to. It's analogous to the SRC_X/Y/W/H
plane properties. We could have similar props for the writeback
connectors, and maybe set them to the FB_ID dimensions if they aren't
configured by userspace.

- Besides the above property, writeback hardware can have provisions
for scaling, color space conversion and rotation. This would mean that
we'd eventually add more writeback specific props/params in
drm_connector/drm_connector_state. Would we be okay adding more such
props for connectors?


I've wondered the same thing about bloating non-writeback connectors
with writeback-specific stuff. If it does become significant, maybe
we should subclass drm_connector and add a drm_writeback_state pointer
to drm_connector_state.

Ville touched on scaling support previously, suggesting adding a
fixed_mode property (for all types of connectors) - on writeback this
would represent scaling the framebuffer, and on normal connectors it
could control output scaling (like panel-fitting).

Certainly destination coords, color-space converstion etc. are things
that are worth adding, but IMO I'd rather keep this initial
implementation small so we can enable the basic case right away. For
the most part, the additional things are "just properties" which
should be easily added later without impacting the overall interface.

Cheers,
Brian


Thanks,
Archit



Thanks,

-Brian

---

Brian Starkey (10):
 drm: add writeback connector type
 drm/fb-helper: skip writeback connectors
 drm: extract CRTC/plane disable from drm_framebuffer_remove
 drm: add __drm_framebuffer_remove_atomic
 drm: add fb to connector state
 drm: expose fb_id property for writeback connectors
 drm: add writeback-connector pixel format properties
 drm: mali-dp: rename malidp_input_format
 drm: mali-dp: add RGB writeback formats for DP550/DP650
 drm: mali-dp: add writeback connector

Liviu Dudau (

Re: [RFC PATCH 00/11] Introduce writeback connectors

[Brian Starkey]

Hi Eric,

On Tue, Oct 11, 2016 at 12:01:14PM -0700, Eric Anholt wrote:

Brian Starkey <brian.star...@arm.com> writes:


Hi,

This RFC series introduces a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
It is a follow-on from a previous discussion: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Patches 1-7 include the core framework changes required, and patches
8-11 implement a writeback connector for the Mali-DP writeback engine.
The Mali-DP patches depend on this other series: [2].

The connector is given the FB_ID property for the output framebuffer,
and two new read-only properties: PIXEL_FORMATS and
PIXEL_FORMATS_SIZE, which expose the supported framebuffer pixel
formats of the engine.

The EDID property is not exposed for writeback connectors.

Writeback connector usage:
--
Due to connector routing changes being treated as "full modeset"
operations, any client which wishes to use a writeback connector
should include the connector in every modeset. The writeback will not
actually become active until a framebuffer is attached.

The writeback itself is enabled by attaching a framebuffer to the
FB_ID property of the connector. The driver must then ensure that the
CRTC content of that atomic commit is written into the framebuffer.

The writeback works in a one-shot mode with each atomic commit. This
prevents the same content from being written multiple times.
In some cases (front-buffer rendering) there might be a desire for
continuous operation - I think a property could be added later for
this kind of control.

Writeback can be disabled by setting FB_ID to zero.


I think this sounds great, and the interface is just right IMO.



Thanks, glad you like it! Hopefully you're equally agreeable with the
changes Daniel has been suggesting.


I don't really see a use for continuous mode -- a sequence of one-shots
makes a lot more sense because then you can know what data has changed,
which anyone trying to use the writeback buffer would need to know.



Agreed - we've never found a use for it.


Known issues:
-
 * I'm not sure what "DPMS" should mean for writeback connectors.
   It could be used to disable writeback (even when a framebuffer is
   attached), or it could be hidden entirely (which would break the
   legacy DPMS call for writeback connectors).
 * With Daniel's recent re-iteration of the userspace API rules, I
   fully expect to provide some userspace code to support this. The
   question is what, and where? We want to use writeback for testing,
   so perhaps some tests in igt is suitable.
 * Documentation. Probably some portion of this cover letter needs to
   make it into Documentation/
 * Synchronisation. Our hardware will finish the writeback by the next
   vsync. I've not implemented fence support here, but it would be an
   obvious addition.


My hardware won't necessarily finish by the next vsync -- it trickles
out at whatever rate it can find memory bandwidth to get the job done,
and fires an interrupt when it's finished.



Is it bounded? You presumably have to finish the write-out before you
can change any input buffers?


So I would like some definition for how syncing works.  One answer would
be that these flips don't trigger their pageflip events until the
writeback is done (so I need to collect both the vsync irq and the
writeback irq before sending).  Another would be that manage an
independent fence for the writeback fb, so that you still immediately
know when framebuffers from the previous scanout-only frame are idle.



I much prefer the sound of the explicit fence approach.

Hopefully we can agree that a new atomic commit can't be completed
whilst there's a writeback ongoing, otherwise managing the fence and
framebuffer lifetime sounds really tricky - they'd need to be decoupled
from the atomic_state and outlive the commit that spawned them.

Cheers,
-Brian


Also, tests for this in igt, please.  Writeback in igt will give us so
much more ability to cover KMS functionality on non-Intel hardware.

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 00/11] Introduce writeback connectors

[Brian Starkey]

On Tue, Oct 11, 2016 at 10:02:43PM +0200, Daniel Vetter wrote:

On Tue, Oct 11, 2016 at 9:44 PM, Brian Starkey <brian.star...@arm.com> wrote:

Hi,


On Tue, Oct 11, 2016 at 07:01:33PM +0200, Daniel Vetter wrote:


On Tue, Oct 11, 2016 at 6:43 PM, Brian Starkey <brian.star...@arm.com>
wrote:


Hi Daniel,

Firstly thanks very much for having a look.


On Tue, Oct 11, 2016 at 05:43:59PM +0200, Daniel Vetter wrote:



On Tue, Oct 11, 2016 at 03:53:57PM +0100, Brian Starkey wrote:



Hi,

This RFC series introduces a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
It is a follow-on from a previous discussion: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Patches 1-7 include the core framework changes required, and patches
8-11 implement a writeback connector for the Mali-DP writeback engine.
The Mali-DP patches depend on this other series: [2].

The connector is given the FB_ID property for the output framebuffer,
and two new read-only properties: PIXEL_FORMATS and
PIXEL_FORMATS_SIZE, which expose the supported framebuffer pixel
formats of the engine.

The EDID property is not exposed for writeback connectors.

Writeback connector usage:
--
Due to connector routing changes being treated as "full modeset"
operations, any client which wishes to use a writeback connector
should include the connector in every modeset. The writeback will not
actually become active until a framebuffer is attached.




Erhm, this is just the default, drivers can override this. And we could
change the atomic helpers to not mark a modeset as a modeset if the
connector that changed is a writeback one.



Hmm, maybe. I don't think it's ideal - the driver would need to
re-implement drm_atomic_helper_check_modeset, which is quite a chunk
of code (along with exposing update_connector_routing, mode_fixup,
maybe others), and even after that it would have to lie and set
crtc_state->connectors_changed to false so that
drm_crtc_needs_modeset returns false to drm_atomic_check_only.



You only need to update the property in your encoders's ->atomic_check
function. No need for more, and I think being consistent with
computing when you need a modeset is really a crucial part of the
atomic ioctl that we should imo try to implement correctly as much as
possible.



Sorry I really don't follow. Which property? CRTC_ID?

Userspace changing CRTC_ID will change connector_state->crtc (before
we even get to a driver callback).

After that, drm_atomic_helper_check_modeset derives connectors_changed
based on the ->crtc pointers.

After that, my encoder ->atomic_check *could* clear
connectors_changed (or I could achieve the same thing by wrapping
drm_atomic_helper_check), but it seems wrong to do so, considering
that the connector routing *has* changed.

If you think changing CRTC_ID shouldn't require a full modeset, I'd
rather give drivers a ->needs_modeset callback to override the default
drm_atomic_crtc_needs_modeset behaviour, instead of "tricking" it into
returning false.


The problem with just that is that there's lots of different things
that can feed into the overall needs_modeset variable. That's why we
split it up into multiple booleans.

So yes you're supposed to clear connectors_changed if there is some
change that you can handle without a full modeset. If you want, think
of connectors_changed as
needs_modeset_due_to_change_in_connnector_state, but that's cumbersome
to type and too long ;-)



All right, got it :-). This intention wasn't clear to me from the
comments in the code.


I can imagine some hardware will need a full modeset to changed the
writeback CRTC binding anyway.


Yup, and then they can upgrade this again. With all these flow-control
booleans the idea is very much that helpers give a default that works
for 90% of all cases, and driver callbacks can then change it for the
other 10%.

I tried to keep special-casing of writeback connectors in the 
core to

a bare minimum, because I think it will quickly get messy and fragile
otherwise.



Please always make the disdinction between core and shared drm
helpers. Special cases in core == probably not good. Special cases in
helpers == perfectly fine imo.


Honestly, I don't see modesetting the writeback connectors at
start-of-day as a big problem.



It's inconsistent. Claiming it needs a modeset when it doesn't isn't
great. Making that more discoverable to userspace is the entire point
of atomic. And there might be hw where reconfiguring for writeback
might need a full modeset.



I'm a little confused - what bit exactly is inconsistent?


Not being truthful for when you need a modeset and when not.


My implementation here is consistent with other co

Re: [RFC PATCH 00/11] Introduce writeback connectors

[Brian Starkey]

Hi,

On Tue, Oct 11, 2016 at 07:01:33PM +0200, Daniel Vetter wrote:

On Tue, Oct 11, 2016 at 6:43 PM, Brian Starkey <brian.star...@arm.com> wrote:

Hi Daniel,

Firstly thanks very much for having a look.


On Tue, Oct 11, 2016 at 05:43:59PM +0200, Daniel Vetter wrote:


On Tue, Oct 11, 2016 at 03:53:57PM +0100, Brian Starkey wrote:


Hi,

This RFC series introduces a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
It is a follow-on from a previous discussion: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Patches 1-7 include the core framework changes required, and patches
8-11 implement a writeback connector for the Mali-DP writeback engine.
The Mali-DP patches depend on this other series: [2].

The connector is given the FB_ID property for the output framebuffer,
and two new read-only properties: PIXEL_FORMATS and
PIXEL_FORMATS_SIZE, which expose the supported framebuffer pixel
formats of the engine.

The EDID property is not exposed for writeback connectors.

Writeback connector usage:
--
Due to connector routing changes being treated as "full modeset"
operations, any client which wishes to use a writeback connector
should include the connector in every modeset. The writeback will not
actually become active until a framebuffer is attached.



Erhm, this is just the default, drivers can override this. And we could
change the atomic helpers to not mark a modeset as a modeset if the
connector that changed is a writeback one.



Hmm, maybe. I don't think it's ideal - the driver would need to
re-implement drm_atomic_helper_check_modeset, which is quite a chunk
of code (along with exposing update_connector_routing, mode_fixup,
maybe others), and even after that it would have to lie and set
crtc_state->connectors_changed to false so that
drm_crtc_needs_modeset returns false to drm_atomic_check_only.


You only need to update the property in your encoders's ->atomic_check
function. No need for more, and I think being consistent with
computing when you need a modeset is really a crucial part of the
atomic ioctl that we should imo try to implement correctly as much as
possible.



Sorry I really don't follow. Which property? CRTC_ID?

Userspace changing CRTC_ID will change connector_state->crtc (before
we even get to a driver callback).

After that, drm_atomic_helper_check_modeset derives connectors_changed
based on the ->crtc pointers.

After that, my encoder ->atomic_check *could* clear
connectors_changed (or I could achieve the same thing by wrapping
drm_atomic_helper_check), but it seems wrong to do so, considering
that the connector routing *has* changed.

If you think changing CRTC_ID shouldn't require a full modeset, I'd
rather give drivers a ->needs_modeset callback to override the default
drm_atomic_crtc_needs_modeset behaviour, instead of "tricking" it into
returning false.

I can imagine some hardware will need a full modeset to changed the
writeback CRTC binding anyway.

I tried to keep special-casing of writeback connectors in the core 
to

a bare minimum, because I think it will quickly get messy and fragile
otherwise.


Please always make the disdinction between core and shared drm
helpers. Special cases in core == probably not good. Special cases in
helpers == perfectly fine imo.


Honestly, I don't see modesetting the writeback connectors at
start-of-day as a big problem.


It's inconsistent. Claiming it needs a modeset when it doesn't isn't
great. Making that more discoverable to userspace is the entire point
of atomic. And there might be hw where reconfiguring for writeback
might need a full modeset.



I'm a little confused - what bit exactly is inconsistent?

My implementation here is consistent with other connectors.
Updating the writeback connector CRTC_ID property requires a full
modeset, the same as other connectors.

Changing the FB_ID does *not* require a full modeset, because our
hardware has no such restriction. This is analogous to updating the
FB_ID on our planes, and is consistent with the other instances of the
FB_ID property.

If there is hardware which does have a restriction on changing FB_ID, 
I think that driver must be responsible for handling it in the same

way as drivers which can't handle plane updates without a full
modeset.

Are you saying that because setting CRTC_ID on Mali-DP is a no-op, it
shouldn't require a full modeset? I'd rather somehow hard-code the
CRTC_ID for our writeback connector to have it always attached to
the CRTC in that case.


The writeback itself is enabled by attaching a framebuffer to the
FB_ID property of the connector. The driver must then ensure that the
CRTC content of that atomic commit is written into the framebuf

Re: [RFC PATCH 02/11] drm/fb-helper: Skip writeback connectors

[Brian Starkey]

On Tue, Oct 11, 2016 at 05:44:48PM +0200, Daniel Vetter wrote:

On Tue, Oct 11, 2016 at 03:53:59PM +0100, Brian Starkey wrote:

Writeback connectors aren't much use to the fbdev helpers, as they won't
show anything to the user. Skip them when looking for candidate output
configurations.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_fb_helper.c |4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 03414bd..dedf6e7 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -2016,6 +2016,10 @@ static int drm_pick_crtcs(struct drm_fb_helper 
*fb_helper,
if (modes[n] == NULL)
return best_score;

+   /* Writeback connectors aren't much use for fbdev */
+   if (connector->connector_type == DRM_MODE_CONNECTOR_WRITEBACK)
+   return best_score;


I think we could handle this by always marking writeback connectors as
disconnected. Userspace and fbdev emulation should then avoid them,
always.
-Daniel



Good idea; I'll need to take a closer look at how it would interact
with the probe helper (connector->force etc).

Are you thinking instead-of or in-addition-to the client cap? I'd be
worried about apps doing strange things and trying to use even
disconnected connectors.


+
crtcs = kzalloc(fb_helper->connector_count *
sizeof(struct drm_fb_helper_crtc *), GFP_KERNEL);
if (!crtcs)
--
1.7.9.5



--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch


--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 00/11] Introduce writeback connectors

[Brian Starkey]

Hi Daniel,

Firstly thanks very much for having a look.

On Tue, Oct 11, 2016 at 05:43:59PM +0200, Daniel Vetter wrote:

On Tue, Oct 11, 2016 at 03:53:57PM +0100, Brian Starkey wrote:

Hi,

This RFC series introduces a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
It is a follow-on from a previous discussion: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Patches 1-7 include the core framework changes required, and patches
8-11 implement a writeback connector for the Mali-DP writeback engine.
The Mali-DP patches depend on this other series: [2].

The connector is given the FB_ID property for the output framebuffer,
and two new read-only properties: PIXEL_FORMATS and
PIXEL_FORMATS_SIZE, which expose the supported framebuffer pixel
formats of the engine.

The EDID property is not exposed for writeback connectors.

Writeback connector usage:
--
Due to connector routing changes being treated as "full modeset"
operations, any client which wishes to use a writeback connector
should include the connector in every modeset. The writeback will not
actually become active until a framebuffer is attached.


Erhm, this is just the default, drivers can override this. And we could
change the atomic helpers to not mark a modeset as a modeset if the
connector that changed is a writeback one.



Hmm, maybe. I don't think it's ideal - the driver would need to
re-implement drm_atomic_helper_check_modeset, which is quite a chunk
of code (along with exposing update_connector_routing, mode_fixup,
maybe others), and even after that it would have to lie and set
crtc_state->connectors_changed to false so that
drm_crtc_needs_modeset returns false to drm_atomic_check_only.

I tried to keep special-casing of writeback connectors in the core to
a bare minimum, because I think it will quickly get messy and fragile
otherwise.

Honestly, I don't see modesetting the writeback connectors at
start-of-day as a big problem.


The writeback itself is enabled by attaching a framebuffer to the
FB_ID property of the connector. The driver must then ensure that the
CRTC content of that atomic commit is written into the framebuffer.

The writeback works in a one-shot mode with each atomic commit. This
prevents the same content from being written multiple times.
In some cases (front-buffer rendering) there might be a desire for
continuous operation - I think a property could be added later for
this kind of control.

Writeback can be disabled by setting FB_ID to zero.


This seems to contradict itself: If it's one-shot, there's no need to
disable it - it will auto-disable.


I should have explained one-shot more clearly. What I mean is, one
drmModeAtomicCommit == one write to memory. This is as opposed to
writing the same thing to memory every vsync until it is stopped
(which our HW is capable of doing).

A subsequent drmModeAtomicCommit which doesn't touch the writeback 
FB_ID will write (again - but with whatever scene updates) to the same

framebuffer.

This continues for every drmModeAtomicCommit until FB_ID is set to
zero - to disable writing - or changed to a different framebuffer, in
which case we write to the new one.

IMO this behaviour makes sense in the context of the rest of Atomic,
and as the FB_ID is indeed persistent across atomic commits, I think
it should be read-able.



In other cases where we write a property as a one-shot thing (fences for
android). In that case when you read that property it's always 0 (well, -1
for fences since file descriptor). That also avoids the issues when
userspace unconditionally saves/restores all properties (this is needed
for generic compositor switching).

I think a better behaviour would be to do the same trick, with FB_ID on
the connector always returning 0 as the current value. That encodes the
one-shot behaviour directly.

For one-shot vs continuous: Maybe we want to simply have a separate
writeback property for continues, e.g. FB_WRITEBACK_ONE_SHOT_ID and
FB_WRITEBACK_CONTINUOUS_ID.


Known issues:
-
 * I'm not sure what "DPMS" should mean for writeback connectors.
   It could be used to disable writeback (even when a framebuffer is
   attached), or it could be hidden entirely (which would break the
   legacy DPMS call for writeback connectors).


dpms is legacy, in atomic land the only thing you have is "ACTIVE" on the
crtc. it disables everything, i.e. also writeback.



So removing the DPMS property is a viable option for writeback 
connectors in your opinion?



 * With Daniel's recent re-iteration of the userspace API rules, I
   fully expect to provide some userspace code to support this. The
   question is what, and where? We want to use writeback for testing

[RFC PATCH 04/11] drm: Add __drm_framebuffer_remove_atomic

[Brian Starkey]

Implement the CRTC/Plane disable functionality of drm_framebuffer_remove
using the atomic API, and use it if possible.

For atomic drivers, this removes the possibility of several commits when
a framebuffer is in use by more than one CRTC/plane.

Additionally, this will provide a suitable place to support the removal
of a framebuffer from a writeback connector, in the case that a
writeback connector is still actively using a framebuffer when it is
removed by userspace.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_framebuffer.c |  154 -
 1 file changed, 152 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_framebuffer.c 
b/drivers/gpu/drm/drm_framebuffer.c
index 528f75d..b02cf73 100644
--- a/drivers/gpu/drm/drm_framebuffer.c
+++ b/drivers/gpu/drm/drm_framebuffer.c
@@ -22,6 +22,7 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -795,6 +796,148 @@ void drm_framebuffer_cleanup(struct drm_framebuffer *fb)
 EXPORT_SYMBOL(drm_framebuffer_cleanup);
 
 /**
+ * __drm_framebuffer_remove_atomic - atomic version of __drm_framebuffer_remove
+ * @dev: drm device
+ * @fb: framebuffer to remove
+ *
+ * If the driver implements the atomic API, we can handle the disabling of all
+ * CRTCs/planes which use a framebuffer which is going away in a single atomic
+ * commit.
+ *
+ * This scans all CRTCs and planes in @dev's mode_config. If they're using @fb,
+ * it is removed and the CRTC/plane disabled.
+ * The legacy references are dropped and the ->fb pointers set to NULL
+ * accordingly.
+ *
+ * Returns:
+ * true if the framebuffer was successfully removed from use
+ */
+static bool __drm_framebuffer_remove_atomic(struct drm_device *dev,
+   struct drm_framebuffer *fb)
+{
+   struct drm_modeset_acquire_ctx ctx;
+   struct drm_atomic_state *state;
+   struct drm_connector_state *conn_state;
+   struct drm_connector *connector;
+   struct drm_plane *plane;
+   struct drm_crtc *crtc;
+   unsigned plane_mask;
+   int i, ret;
+
+   drm_modeset_acquire_init(, 0);
+
+   state = drm_atomic_state_alloc(dev);
+   if (!state)
+   return false;
+
+   state->acquire_ctx = 
+
+retry:
+   drm_for_each_crtc(crtc, dev) {
+   struct drm_plane_state *primary_state;
+   struct drm_crtc_state *crtc_state;
+
+   primary_state = drm_atomic_get_plane_state(state, 
crtc->primary);
+   if (IS_ERR(primary_state)) {
+   ret = PTR_ERR(primary_state);
+   goto fail;
+   }
+
+   if (primary_state->fb != fb)
+   continue;
+
+   crtc_state = drm_atomic_get_crtc_state(state, crtc);
+   if (IS_ERR(crtc_state)) {
+   ret = PTR_ERR(crtc_state);
+   goto fail;
+   }
+
+   /* Only handle the CRTC itself here, handle the plane later */
+   ret = drm_atomic_set_mode_for_crtc(crtc_state, NULL);
+   if (ret != 0)
+   goto fail;
+
+   crtc_state->active = false;
+
+   /* Get the connectors in order to disable them */
+   ret = drm_atomic_add_affected_connectors(state, crtc);
+   if (ret)
+   goto fail;
+   }
+
+   plane_mask = 0;
+   drm_for_each_plane(plane, dev) {
+   struct drm_plane_state *plane_state;
+
+   plane_state = drm_atomic_get_plane_state(state, plane);
+   if (IS_ERR(plane_state)) {
+   ret = PTR_ERR(plane_state);
+   goto fail;
+   }
+
+   if (plane_state->fb != fb)
+   continue;
+
+   plane->old_fb = plane->fb;
+   plane_mask |= 1 << drm_plane_index(plane);
+
+   /*
+* Open-coded copy of __drm_atomic_helper_disable_plane to avoid
+* a dependency on atomic-helper
+*/
+   ret = drm_atomic_set_crtc_for_plane(plane_state, NULL);
+   if (ret != 0)
+   goto fail;
+
+   drm_atomic_set_fb_for_plane(plane_state, NULL);
+   plane_state->crtc_x = 0;
+   plane_state->crtc_y = 0;
+   plane_state->crtc_w = 0;
+   plane_state->crtc_h = 0;
+   plane_state->src_x = 0;
+   plane_state->src_y = 0;
+   plane_state->src_w = 0;
+   plane_state->src_h = 0;
+   }
+
+   /* All of the connectors in state need disabling */
+   for_each_connector_in_state(state, connector, conn_state, i) {
+   ret = drm_atomic_set_crtc_for_connector(conn_state,
+   NULL);
+   if (ret)
+ 

[RFC PATCH 01/11] drm: Add writeback connector type

[Brian Starkey]

Writeback connectors represent writeback engines which can write the
CRTC output to a memory framebuffer.

Add a writeback connector type, hidden from userspace behind a client
cap. They are hidden from non-aware clients so that they do not attempt
to use writeback connectors to provide visual output to the user.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_connector.c |4 +++-
 drivers/gpu/drm/drm_crtc.c  |   14 +-
 drivers/gpu/drm/drm_ioctl.c |7 +++
 include/drm/drmP.h  |2 ++
 include/uapi/drm/drm.h  |   10 ++
 include/uapi/drm/drm_mode.h |1 +
 6 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
index 26bb78c7..027d7a9 100644
--- a/drivers/gpu/drm/drm_connector.c
+++ b/drivers/gpu/drm/drm_connector.c
@@ -86,6 +86,7 @@ static struct drm_conn_prop_enum_list 
drm_connector_enum_list[] = {
{ DRM_MODE_CONNECTOR_VIRTUAL, "Virtual" },
{ DRM_MODE_CONNECTOR_DSI, "DSI" },
{ DRM_MODE_CONNECTOR_DPI, "DPI" },
+   { DRM_MODE_CONNECTOR_WRITEBACK, "Writeback" },
 };
 
 void drm_connector_ida_init(void)
@@ -235,7 +236,8 @@ int drm_connector_init(struct drm_device *dev,
list_add_tail(>head, >connector_list);
config->num_connector++;
 
-   if (connector_type != DRM_MODE_CONNECTOR_VIRTUAL)
+   if ((connector_type != DRM_MODE_CONNECTOR_VIRTUAL) &&
+   (connector_type != DRM_MODE_CONNECTOR_WRITEBACK))
drm_object_attach_property(>base,
  config->edid_property,
  0);
diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index 2d7bedf..33f66e2 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -422,6 +422,14 @@ static int drm_mode_create_standard_properties(struct 
drm_device *dev)
return 0;
 }
 
+static bool
+drm_connector_expose_to_userspace(const struct drm_connector *conn,
+ const struct drm_file *file_priv)
+{
+   return (file_priv->writeback_connectors) ||
+  (conn->connector_type != DRM_MODE_CONNECTOR_WRITEBACK);
+}
+
 /**
  * drm_mode_getresources - get graphics configuration
  * @dev: drm device for the ioctl
@@ -491,7 +499,8 @@ int drm_mode_getresources(struct drm_device *dev, void 
*data,
crtc_count++;
 
drm_for_each_connector(connector, dev)
-   connector_count++;
+   if (drm_connector_expose_to_userspace(connector, file_priv))
+   connector_count++;
 
drm_for_each_encoder(encoder, dev)
encoder_count++;
@@ -535,6 +544,9 @@ int drm_mode_getresources(struct drm_device *dev, void 
*data,
copied = 0;
connector_id = (uint32_t __user *)(unsigned 
long)card_res->connector_id_ptr;
drm_for_each_connector(connector, dev) {
+   if (!drm_connector_expose_to_userspace(connector, 
file_priv))
+   continue;
+
if (put_user(connector->base.id,
 connector_id + copied)) {
ret = -EFAULT;
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index 0ad2c47..838a6e8 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -308,6 +308,13 @@ drm_setclientcap(struct drm_device *dev, void *data, 
struct drm_file *file_priv)
file_priv->atomic = req->value;
file_priv->universal_planes = req->value;
break;
+   case DRM_CLIENT_CAP_WRITEBACK_CONNECTORS:
+   if (!drm_core_check_feature(dev, DRIVER_ATOMIC))
+   return -EINVAL;
+   if (req->value > 1)
+   return -EINVAL;
+   file_priv->writeback_connectors = req->value;
+   break;
default:
return -EINVAL;
}
diff --git a/include/drm/drmP.h b/include/drm/drmP.h
index 0e99669..222d5dc 100644
--- a/include/drm/drmP.h
+++ b/include/drm/drmP.h
@@ -388,6 +388,8 @@ struct drm_file {
unsigned universal_planes:1;
/* true if client understands atomic properties */
unsigned atomic:1;
+   /* true if client understands writeback connectors */
+   unsigned writeback_connectors:1;
/*
 * This client is the creator of @master.
 * Protected by struct drm_device::master_mutex.
diff --git a/include/uapi/drm/drm.h b/include/uapi/drm/drm.h
index b2c5284..d2b4543 100644
--- a/include/uapi/drm/drm.h
+++ b/include/uapi/drm/drm.h
@@ -678,6 +678,16 @@ struct drm_get_cap {
  */
 #define DRM_CLIENT_CAP_ATOMIC  3
 
+/**
+ * DRM_CLIENT_CAP_WRITEBACK_

[RFC PATCH 05/11] drm: Add fb to connector state

[Brian Starkey]

Add a framebuffer to the connector state, for use as the output target
by writeback connectors.

If a framebuffer is in use by a writeback connector when userspace
removes it, it is handled by removing the framebuffer from the connector.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_atomic.c|   31 +++
 drivers/gpu/drm/drm_atomic_helper.c |4 
 drivers/gpu/drm/drm_framebuffer.c   |   24 
 include/drm/drm_atomic.h|3 +++
 include/drm/drm_connector.h |3 +++
 5 files changed, 61 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index 2373960..b16b4fc 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -1205,6 +1205,37 @@ drm_atomic_set_crtc_for_connector(struct 
drm_connector_state *conn_state,
 EXPORT_SYMBOL(drm_atomic_set_crtc_for_connector);
 
 /**
+ * drm_atomic_set_fb_for_connector - set framebuffer for (writeback) connector
+ * @connector_state: atomic state object for the connector
+ * @fb: fb to use for the connector
+ *
+ * This is used to set the framebuffer for a writeback connector, which outputs
+ * to a buffer instead of an actual physical connector.
+ * Changing the assigned framebuffer requires us to grab a reference to the new
+ * fb and drop the reference to the old fb, if there is one. This function
+ * takes care of all these details besides updating the pointer in the
+ * state object itself.
+ */
+void
+drm_atomic_set_fb_for_connector(struct drm_connector_state *conn_state,
+   struct drm_framebuffer *fb)
+{
+   if (conn_state->fb)
+   drm_framebuffer_unreference(conn_state->fb);
+   if (fb)
+   drm_framebuffer_reference(fb);
+   conn_state->fb = fb;
+
+   if (fb)
+   DRM_DEBUG_ATOMIC("Set [FB:%d] for connector state %p\n",
+fb->base.id, conn_state);
+   else
+   DRM_DEBUG_ATOMIC("Set [NOFB] for connector state %p\n",
+conn_state);
+}
+EXPORT_SYMBOL(drm_atomic_set_fb_for_connector);
+
+/**
  * drm_atomic_add_affected_connectors - add connectors for crtc
  * @state: atomic state
  * @crtc: DRM crtc
diff --git a/drivers/gpu/drm/drm_atomic_helper.c 
b/drivers/gpu/drm/drm_atomic_helper.c
index 3eecfc1..78ea735 100644
--- a/drivers/gpu/drm/drm_atomic_helper.c
+++ b/drivers/gpu/drm/drm_atomic_helper.c
@@ -3234,6 +3234,8 @@ __drm_atomic_helper_connector_duplicate_state(struct 
drm_connector *connector,
memcpy(state, connector->state, sizeof(*state));
if (state->crtc)
drm_connector_reference(connector);
+   if (state->fb)
+   drm_framebuffer_reference(state->fb);
 }
 EXPORT_SYMBOL(__drm_atomic_helper_connector_duplicate_state);
 
@@ -3361,6 +3363,8 @@ __drm_atomic_helper_connector_destroy_state(struct 
drm_connector_state *state)
 */
if (state->crtc)
drm_connector_unreference(state->connector);
+   if (state->fb)
+   drm_framebuffer_unreference(state->fb);
 }
 EXPORT_SYMBOL(__drm_atomic_helper_connector_destroy_state);
 
diff --git a/drivers/gpu/drm/drm_framebuffer.c 
b/drivers/gpu/drm/drm_framebuffer.c
index b02cf73..f66908b1 100644
--- a/drivers/gpu/drm/drm_framebuffer.c
+++ b/drivers/gpu/drm/drm_framebuffer.c
@@ -24,6 +24,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #include "drm_crtc_internal.h"
@@ -808,6 +809,8 @@ EXPORT_SYMBOL(drm_framebuffer_cleanup);
  * it is removed and the CRTC/plane disabled.
  * The legacy references are dropped and the ->fb pointers set to NULL
  * accordingly.
+ * It also checks for (writeback) connectors which are using @fb, and removes
+ * it if found.
  *
  * Returns:
  * true if the framebuffer was successfully removed from use
@@ -900,7 +903,7 @@ retry:
plane_state->src_h = 0;
}
 
-   /* All of the connectors in state need disabling */
+   /* All of the connectors currently in state need disabling */
for_each_connector_in_state(state, connector, conn_state, i) {
ret = drm_atomic_set_crtc_for_connector(conn_state,
NULL);
@@ -908,10 +911,23 @@ retry:
goto fail;
}
 
-   if (WARN_ON(!plane_mask)) {
-   DRM_ERROR("Couldn't find any usage of [FB:%d]\n", fb->base.id);
-   ret = -ENOENT;
+   /* Now find any writeback connectors that need handling */
+   ret = drm_modeset_lock(>dev->mode_config.connection_mutex,
+  state->acquire_ctx);
+   if (ret)
goto fail;
+
+   drm_for_each_connector(connector, dev) {
+   conn_state = drm_atomic_get_connector_stat

[RFC PATCH 06/11] drm: Expose fb_id property for writeback connectors

[Brian Starkey]

Expose the framebuffer for writeback connectors to userspace by
attaching the fb_id property to them.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_atomic.c|9 +
 drivers/gpu/drm/drm_connector.c |4 
 2 files changed, 13 insertions(+)

diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c
index b16b4fc..82e8e3a 100644
--- a/drivers/gpu/drm/drm_atomic.c
+++ b/drivers/gpu/drm/drm_atomic.c
@@ -986,12 +986,19 @@ int drm_atomic_connector_set_property(struct 
drm_connector *connector,
 * now?) atomic writes to DPMS property:
 */
return -EINVAL;
+   } else if (property == config->prop_fb_id) {
+   struct drm_framebuffer *fb = drm_framebuffer_lookup(dev, val);
+   drm_atomic_set_fb_for_connector(state, fb);
+   if (fb)
+   drm_framebuffer_unreference(fb);
} else if (connector->funcs->atomic_set_property) {
return connector->funcs->atomic_set_property(connector,
state, property, val);
} else {
return -EINVAL;
}
+
+   return 0;
 }
 EXPORT_SYMBOL(drm_atomic_connector_set_property);
 
@@ -1022,6 +1029,8 @@ drm_atomic_connector_get_property(struct drm_connector 
*connector,
*val = (state->crtc) ? state->crtc->base.id : 0;
} else if (property == config->dpms_property) {
*val = connector->dpms;
+   } else if (property == config->prop_fb_id) {
+   *val = (state->fb) ? state->fb->base.id : 0;
} else if (connector->funcs->atomic_get_property) {
return connector->funcs->atomic_get_property(connector,
state, property, val);
diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
index 027d7a9..fb83870 100644
--- a/drivers/gpu/drm/drm_connector.c
+++ b/drivers/gpu/drm/drm_connector.c
@@ -249,6 +249,10 @@ int drm_connector_init(struct drm_device *dev,
drm_object_attach_property(>base, 
config->prop_crtc_id, 0);
}
 
+   if (connector_type == DRM_MODE_CONNECTOR_WRITEBACK)
+   drm_object_attach_property(>base,
+  config->prop_fb_id, 0);
+
connector->debugfs_entry = NULL;
 out_put_type_id:
if (ret)
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 07/11] drm: Add writeback-connector pixel format properties

[Brian Starkey]

So that userspace can determine what pixel formats are supported for a
writeback connector's framebuffer, add a pixel format list to writeback
connectors. This is in the form of an immutable blob containing an array
of formats, and an immutable uint holding the array size.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_connector.c |   73 ++-
 include/drm/drm_connector.h |   12 +++
 include/drm/drm_crtc.h  |   12 +++
 3 files changed, 96 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/drm_connector.c b/drivers/gpu/drm/drm_connector.c
index fb83870..2f1f61d 100644
--- a/drivers/gpu/drm/drm_connector.c
+++ b/drivers/gpu/drm/drm_connector.c
@@ -249,9 +249,14 @@ int drm_connector_init(struct drm_device *dev,
drm_object_attach_property(>base, 
config->prop_crtc_id, 0);
}
 
-   if (connector_type == DRM_MODE_CONNECTOR_WRITEBACK)
+   if (connector_type == DRM_MODE_CONNECTOR_WRITEBACK) {
drm_object_attach_property(>base,
   config->prop_fb_id, 0);
+   drm_object_attach_property(>base,
+  config->pixel_formats_property, 0);
+   drm_object_attach_property(>base,
+  config->pixel_formats_size_property, 
0);
+   }
 
connector->debugfs_entry = NULL;
 out_put_type_id:
@@ -851,6 +856,45 @@ int drm_mode_create_suggested_offset_properties(struct 
drm_device *dev)
 EXPORT_SYMBOL(drm_mode_create_suggested_offset_properties);
 
 /**
+ * drm_mode_create_writeback_connector_properties - create writeback connector 
properties
+ * @dev: DRM device
+ *
+ * Create the properties specific to writeback connectors. These will be 
attached
+ * to writeback connectors by drm_connector_init. Drivers can set these
+ * properties using drm_mode_connector_set_writeback_formats().
+ *
+ *  "PIXEL_FORMATS":
+ * Immutable blob property to store the supported pixel formats table. The
+ * data is an array of u32 DRM_FORMAT_* fourcc values.
+ * Userspace can use this blob to find out what pixel formats are supported
+ * by the connector's writeback engine.
+ *
+ *  "PIXEL_FORMATS_SIZE":
+ *  Immutable unsigned range property storing the number of entries in the
+ *  PIXEL_FORMATS array.
+ */
+int drm_mode_create_writeback_connector_properties(struct drm_device *dev)
+{
+   if (dev->mode_config.pixel_formats_property &&
+   dev->mode_config.pixel_formats_size_property)
+   return 0;
+
+   dev->mode_config.pixel_formats_property =
+   drm_property_create(dev, DRM_MODE_PROP_BLOB | 
DRM_MODE_PROP_IMMUTABLE,
+   "PIXEL_FORMATS", 0);
+
+   dev->mode_config.pixel_formats_size_property =
+   drm_property_create_range(dev, DRM_MODE_PROP_IMMUTABLE,
+   "PIXEL_FORMATS_SIZE", 0, UINT_MAX);
+
+   if (dev->mode_config.pixel_formats_property == NULL ||
+   dev->mode_config.pixel_formats_size_property == NULL)
+   return -ENOMEM;
+   return 0;
+}
+EXPORT_SYMBOL(drm_mode_create_writeback_connector_properties);
+
+/**
  * drm_mode_connector_set_path_property - set tile property on connector
  * @connector: connector to set property on.
  * @path: path to use for property; must not be NULL.
@@ -957,6 +1001,33 @@ int drm_mode_connector_update_edid_property(struct 
drm_connector *connector,
 }
 EXPORT_SYMBOL(drm_mode_connector_update_edid_property);
 
+int drm_mode_connector_set_writeback_formats(struct drm_connector *connector,
+u32 *formats,
+unsigned int n_formats)
+{
+   struct drm_device *dev = connector->dev;
+   size_t size = n_formats * sizeof(*formats);
+   int ret;
+
+   if (connector->connector_type != DRM_MODE_CONNECTOR_WRITEBACK)
+   return -EINVAL;
+
+   ret = drm_property_replace_global_blob(dev,
+  
>pixel_formats_blob_ptr,
+  size,
+  formats,
+  >base,
+  
dev->mode_config.pixel_formats_property);
+
+   if (!ret)
+   drm_object_property_set_value(>base,
+ 
dev->mode_config.pixel_formats_size_property,
+ n_formats);
+
+   return 0;
+}
+EXPORT_SYMBOL(drm_mode_connector_set_writeback_formats);
+
 int drm_mode_connector_set_obj_prop(struct drm_mode_object *obj,
struct drm_property *property,
 

[RFC PATCH 08/11] drm: mali-dp: Rename malidp_input_format

[Brian Starkey]

We're going to use the same format list for output formats, so rename
everything related to input formats to avoid confusion.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
Reviewed-by: Liviu Dudau <liviu.du...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |   24 
 drivers/gpu/drm/arm/malidp_hw.h |8 
 drivers/gpu/drm/arm/malidp_planes.c |8 
 3 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 7f4a0bd..44a9d10 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -21,7 +21,7 @@
 #include "malidp_drv.h"
 #include "malidp_hw.h"
 
-static const struct malidp_input_format malidp500_de_formats[] = {
+static const struct malidp_format_id malidp500_de_formats[] = {
/*fourcc,   layers supporting the format, internal id  */
{ DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2,  0 },
{ DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_GRAPHICS2,  1 },
@@ -69,7 +69,7 @@ static const struct malidp_input_format 
malidp500_de_formats[] = {
{ DRM_FORMAT_NV12, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 6) },\
{ DRM_FORMAT_YUV420, DE_VIDEO1 | DE_VIDEO2, MALIDP_ID(5, 7) }
 
-static const struct malidp_input_format malidp550_de_formats[] = {
+static const struct malidp_format_id malidp550_de_formats[] = {
MALIDP_COMMON_FORMATS,
 };
 
@@ -439,8 +439,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.irq_mask = MALIDP500_DE_IRQ_CONF_VALID,
.vsync_irq = MALIDP500_DE_IRQ_CONF_VALID,
},
-   .input_formats = malidp500_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp500_de_formats),
+   .pixel_formats = malidp500_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp500_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp500_query_hw,
@@ -472,8 +472,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
-   .input_formats = malidp550_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+   .pixel_formats = malidp550_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 8,
},
.query_hw = malidp550_query_hw,
@@ -506,8 +506,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
-   .input_formats = malidp550_de_formats,
-   .n_input_formats = ARRAY_SIZE(malidp550_de_formats),
+   .pixel_formats = malidp550_de_formats,
+   .n_pixel_formats = ARRAY_SIZE(malidp550_de_formats),
.bus_align_bytes = 16,
},
.query_hw = malidp650_query_hw,
@@ -525,10 +525,10 @@ u8 malidp_hw_get_format_id(const struct malidp_hw_regmap 
*map,
 {
unsigned int i;
 
-   for (i = 0; i < map->n_input_formats; i++) {
-   if (((map->input_formats[i].layer & layer_id) == layer_id) &&
-   (map->input_formats[i].format == format))
-   return map->input_formats[i].id;
+   for (i = 0; i < map->n_pixel_formats; i++) {
+   if (((map->pixel_formats[i].layer & layer_id) == layer_id) &&
+   (map->pixel_formats[i].format == format))
+   return map->pixel_formats[i].id;
}
 
return MALIDP_INVALID_FORMAT_ID;
diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h
index 087e1202..4f8c884 100644
--- a/drivers/gpu/drm/arm/malidp_hw.h
+++ b/drivers/gpu/drm/arm/malidp_hw.h
@@ -35,7 +35,7 @@ enum {
DE_SMART = BIT(4),
 };
 
-struct malidp_input_format {
+struct malidp_format_id {
u32 format; /* DRM fourcc */
u8 layer;   /* bitmask of layers supporting it */
u8 id;  /* used internally */
@@ -85,9 +85,9 @@ struct malidp_hw_regmap {
const struct malidp_irq_map se_irq_map;
const struct malidp_irq_map dc_irq_map;
 
-   /* list of supported input formats for each layer */
-   const struct malidp_input_format *input_formats;
-   const u8 n_input_formats;
+   /* l

[RFC PATCH 09/11] drm: mali-dp: Add RGB writeback formats for DP550/DP650

[Brian Starkey]

Add a layer bit for the SE memory-write, and add it to the pixel format
matrix for DP550/DP650.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c |   28 ++--
 drivers/gpu/drm/arm/malidp_hw.h |1 +
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 44a9d10..5235d0b 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -46,20 +46,20 @@ static const struct malidp_format_id malidp500_de_formats[] 
= {
 
 #define MALIDP_COMMON_FORMATS \
/*fourcc,   layers supporting the format,  internal id   */ \
-   { DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 0) }, \
-   { DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 1) }, \
-   { DRM_FORMAT_RGBA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 2) }, \
-   { DRM_FORMAT_BGRA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(0, 3) }, \
-   { DRM_FORMAT_ARGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 0) }, \
-   { DRM_FORMAT_ABGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 1) }, \
-   { DRM_FORMAT_RGBA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 2) }, \
-   { DRM_FORMAT_BGRA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(1, 3) }, \
-   { DRM_FORMAT_XRGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 0) }, \
-   { DRM_FORMAT_XBGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 1) }, \
-   { DRM_FORMAT_RGBX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 2) }, \
-   { DRM_FORMAT_BGRX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART, 
MALIDP_ID(2, 3) }, \
-   { DRM_FORMAT_RGB888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(3, 
0) }, \
-   { DRM_FORMAT_BGR888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(3, 
1) }, \
+   { DRM_FORMAT_ARGB2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 0) }, \
+   { DRM_FORMAT_ABGR2101010, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 1) }, \
+   { DRM_FORMAT_RGBA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 2) }, \
+   { DRM_FORMAT_BGRA1010102, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(0, 3) }, \
+   { DRM_FORMAT_ARGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 0) }, \
+   { DRM_FORMAT_ABGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 1) }, \
+   { DRM_FORMAT_RGBA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 2) }, \
+   { DRM_FORMAT_BGRA, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(1, 3) }, \
+   { DRM_FORMAT_XRGB, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 0) }, \
+   { DRM_FORMAT_XBGR, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 1) }, \
+   { DRM_FORMAT_RGBX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 2) }, \
+   { DRM_FORMAT_BGRX, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | DE_SMART 
| SE_MEMWRITE, MALIDP_ID(2, 3) }, \
+   { DRM_FORMAT_RGB888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(3, 0) }, \
+   { DRM_FORMAT_BGR888, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2 | 
SE_MEMWRITE, MALIDP_ID(3, 1) }, \
{ DRM_FORMAT_RGBA5551, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(4, 0) }, \
{ DRM_FORMAT_ABGR1555, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, 
MALIDP_ID(4, 1) }, \
{ DRM_FORMAT_RGB565, DE_VIDEO1 | DE_GRAPHICS1 | DE_VIDEO2, MALIDP_ID(4, 
2) }, \
diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h
index 4f8c884..ce4ea55 100644
--- a/drivers/gpu/drm/arm/malidp_hw.h
+++ b/drivers/gpu/drm/arm/malidp_hw.h
@@ -33,6 +33,7 @@ enum {
DE_GRAPHICS2 = BIT(2), /* used only in DP500 */
DE_VIDEO2 = BIT(3),
DE_SMART = BIT(4),
+   SE_MEMWRITE = BIT(5),
 };
 
 struct malidp_format_id {
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 10/11] drm: mali-dp: Add support for writeback on DP550/DP650

[Brian Starkey]

From: Liviu Dudau <liviu.du...@arm.com>

Mali-DP display processors are able to write the composition result to a
memory buffer via the SE.

Add entry points in the HAL for enabling/disabling this feature, and
implement support for it on DP650 and DP550. DP500 acts differently and
so is omitted from this change.

Signed-off-by: Liviu Dudau <liviu.du...@arm.com>
Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/malidp_hw.c   |   52 +++--
 drivers/gpu/drm/arm/malidp_hw.h   |   18 +
 drivers/gpu/drm/arm/malidp_regs.h |   15 +++
 3 files changed, 83 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c
index 5235d0b..dee7605 100644
--- a/drivers/gpu/drm/arm/malidp_hw.c
+++ b/drivers/gpu/drm/arm/malidp_hw.c
@@ -387,6 +387,48 @@ static int malidp550_rotmem_required(struct 
malidp_hw_device *hwdev, u16 w, u16
return w * bytes_per_col;
 }
 
+static int malidp550_enable_memwrite(struct malidp_hw_device *hwdev,
+dma_addr_t *addrs, s32 *pitches,
+int num_planes, u16 w, u16 h, u32 fmt_id)
+{
+   u32 base = MALIDP550_SE_MEMWRITE_BASE;
+   u32 de_base = malidp_get_block_base(hwdev, MALIDP_DE_BLOCK);
+
+   /* enable the scaling engine block */
+   malidp_hw_setbits(hwdev, MALIDP_SCALE_ENGINE_EN, de_base + 
MALIDP_DE_DISPLAY_FUNC);
+
+   malidp_hw_write(hwdev, fmt_id, base + MALIDP_MW_FORMAT);
+   switch (num_planes) {
+   case 2:
+   malidp_hw_write(hwdev, lower_32_bits(addrs[1]), base + 
MALIDP_MW_P2_PTR_LOW);
+   malidp_hw_write(hwdev, upper_32_bits(addrs[1]), base + 
MALIDP_MW_P2_PTR_HIGH);
+   malidp_hw_write(hwdev, pitches[1], base + MALIDP_MW_P2_STRIDE);
+   /* fall through */
+   case 1:
+   malidp_hw_write(hwdev, lower_32_bits(addrs[0]), base + 
MALIDP_MW_P1_PTR_LOW);
+   malidp_hw_write(hwdev, upper_32_bits(addrs[0]), base + 
MALIDP_MW_P1_PTR_HIGH);
+   malidp_hw_write(hwdev, pitches[0], base + MALIDP_MW_P1_STRIDE);
+   break;
+   default:
+   WARN(1, "Invalid number of planes");
+   }
+
+   malidp_hw_write(hwdev, MALIDP_DE_H_ACTIVE(w) | MALIDP_DE_V_ACTIVE(h),
+   MALIDP550_SE_MEMWRITE_OUT_SIZE);
+   malidp_hw_setbits(hwdev, MALIDP550_SE_MEMWRITE_ONESHOT | 
MALIDP_SE_MEMWRITE_EN,
+ MALIDP550_SE_CONTROL);
+
+   return 0;
+}
+
+static void malidp550_disable_memwrite(struct malidp_hw_device *hwdev)
+{
+   u32 base = malidp_get_block_base(hwdev, MALIDP_DE_BLOCK);
+   malidp_hw_clearbits(hwdev, MALIDP550_SE_MEMWRITE_ONESHOT | 
MALIDP_SE_MEMWRITE_EN,
+   MALIDP550_SE_CONTROL);
+   malidp_hw_clearbits(hwdev, MALIDP_SCALE_ENGINE_EN, base + 
MALIDP_DE_DISPLAY_FUNC);
+}
+
 static int malidp650_query_hw(struct malidp_hw_device *hwdev)
 {
u32 conf = malidp_hw_read(hwdev, MALIDP550_CONFIG_ID);
@@ -469,7 +511,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
MALIDP550_SE_IRQ_AXI_ERR,
},
.dc_irq_map = {
-   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
+   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID |
+   MALIDP550_DC_IRQ_SE,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
.pixel_formats = malidp550_de_formats,
@@ -483,6 +526,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.set_config_valid = malidp550_set_config_valid,
.modeset = malidp550_modeset,
.rotmem_required = malidp550_rotmem_required,
+   .enable_memwrite = malidp550_enable_memwrite,
+   .disable_memwrite = malidp550_disable_memwrite,
},
[MALIDP_650] = {
.map = {
@@ -503,7 +548,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
MALIDP550_SE_IRQ_AXI_ERR,
},
.dc_irq_map = {
-   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID,
+   .irq_mask = MALIDP550_DC_IRQ_CONF_VALID |
+   MALIDP550_DC_IRQ_SE,
.vsync_irq = MALIDP550_DC_IRQ_CONF_VALID,
},
.pixel_formats = malidp550_de_formats,
@@ -517,6 +563,8 @@ const struct malidp_hw_device 
malidp_device[MALIDP_MAX_DEVICES] = {
.set_config_valid = malidp550_set_config_valid,
.modeset = malidp550_modeset,
.rotmem_required

[RFC PATCH 11/11] drm: mali-dp: Add writeback connector

[Brian Starkey]

Mali-DP has a memory writeback engine which can be used to write the
composition result to a memory buffer.
Expose this functionality as a DRM writeback connector on supported
hardware.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/arm/Makefile  |1 +
 drivers/gpu/drm/arm/malidp_crtc.c |   10 ++
 drivers/gpu/drm/arm/malidp_drv.c  |   25 +++-
 drivers/gpu/drm/arm/malidp_drv.h  |5 +
 drivers/gpu/drm/arm/malidp_mw.c   |  268 +
 5 files changed, 305 insertions(+), 4 deletions(-)
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.c

diff --git a/drivers/gpu/drm/arm/Makefile b/drivers/gpu/drm/arm/Makefile
index bb8b158..3bf31d1 100644
--- a/drivers/gpu/drm/arm/Makefile
+++ b/drivers/gpu/drm/arm/Makefile
@@ -1,4 +1,5 @@
 hdlcd-y := hdlcd_drv.o hdlcd_crtc.o
 obj-$(CONFIG_DRM_HDLCD)+= hdlcd.o
 mali-dp-y := malidp_drv.o malidp_hw.o malidp_planes.o malidp_crtc.o
+mali-dp-y += malidp_mw.o
 obj-$(CONFIG_DRM_MALI_DISPLAY) += mali-dp.o
diff --git a/drivers/gpu/drm/arm/malidp_crtc.c 
b/drivers/gpu/drm/arm/malidp_crtc.c
index 08e6a71..98ddcea 100644
--- a/drivers/gpu/drm/arm/malidp_crtc.c
+++ b/drivers/gpu/drm/arm/malidp_crtc.c
@@ -68,6 +68,16 @@ static void malidp_crtc_enable(struct drm_crtc *crtc)
clk_set_rate(hwdev->pxlclk, crtc->state->adjusted_mode.crtc_clock * 
1000);
 
hwdev->modeset(hwdev, );
+   /*
+* We should always disable the memory write when leaving config mode,
+* otherwise the hardware will start writing right away - possibly with
+* a stale config, and definitely before we've had a chance to configure
+* the planes.
+* If the memory write needs to be enabled, that will get taken care
+* of later during the atomic commit
+*/
+   if (hwdev->disable_memwrite)
+   hwdev->disable_memwrite(hwdev);
hwdev->leave_config_mode(hwdev);
drm_crtc_vblank_on(crtc);
 }
diff --git a/drivers/gpu/drm/arm/malidp_drv.c b/drivers/gpu/drm/arm/malidp_drv.c
index 62a29f6..e20266e 100644
--- a/drivers/gpu/drm/arm/malidp_drv.c
+++ b/drivers/gpu/drm/arm/malidp_drv.c
@@ -91,7 +91,16 @@ static void malidp_atomic_commit_tail(struct 
drm_atomic_state *state)
struct drm_device *drm = state->dev;
 
drm_atomic_helper_commit_modeset_disables(drm, state);
+
drm_atomic_helper_commit_modeset_enables(drm, state);
+
+   /*
+* The order here is important. We must configure memory-write after
+* the CRTC is already enabled, so that its configuration update is
+* gated on the next CVAL.
+*/
+   malidp_mw_atomic_commit(drm, state);
+
drm_atomic_helper_commit_planes(drm, state,
DRM_PLANE_COMMIT_ACTIVE_ONLY);
 
@@ -148,12 +157,20 @@ static int malidp_init(struct drm_device *drm)
drm->mode_config.helper_private = _mode_config_helpers;
 
ret = malidp_crtc_init(drm);
-   if (ret) {
-   drm_mode_config_cleanup(drm);
-   return ret;
-   }
+   if (ret)
+   goto crtc_fail;
+
+   ret = malidp_mw_connector_init(drm);
+   if (ret)
+   goto mw_fail;
 
return 0;
+
+mw_fail:
+   malidp_de_planes_destroy(drm);
+crtc_fail:
+   drm_mode_config_cleanup(drm);
+   return ret;
 }
 
 static void malidp_fini(struct drm_device *drm)
diff --git a/drivers/gpu/drm/arm/malidp_drv.h b/drivers/gpu/drm/arm/malidp_drv.h
index 9fc8a2e..905c104 100644
--- a/drivers/gpu/drm/arm/malidp_drv.h
+++ b/drivers/gpu/drm/arm/malidp_drv.h
@@ -22,6 +22,8 @@ struct malidp_drm {
struct drm_fbdev_cma *fbdev;
struct list_head event_list;
struct drm_crtc crtc;
+   struct drm_encoder mw_encoder;
+   struct drm_connector mw_connector;
wait_queue_head_t wq;
atomic_t config_valid;
 };
@@ -50,6 +52,9 @@ struct malidp_plane_state {
 int malidp_de_planes_init(struct drm_device *drm);
 void malidp_de_planes_destroy(struct drm_device *drm);
 int malidp_crtc_init(struct drm_device *drm);
+int malidp_mw_connector_init(struct drm_device *drm);
+void malidp_mw_atomic_commit(struct drm_device *drm,
+struct drm_atomic_state *old_state);
 
 /* often used combination of rotational bits */
 #define MALIDP_ROTATED_MASK(DRM_ROTATE_90 | DRM_ROTATE_270)
diff --git a/drivers/gpu/drm/arm/malidp_mw.c b/drivers/gpu/drm/arm/malidp_mw.c
new file mode 100644
index 000..72df3fd
--- /dev/null
+++ b/drivers/gpu/drm/arm/malidp_mw.c
@@ -0,0 +1,268 @@
+/*
+ * (C) COPYRIGHT 2016 ARM Limited. All rights reserved.
+ * Author: Brian Starkey <brian.star...@arm.com>
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.

[RFC PATCH 00/11] Introduce writeback connectors

[Brian Starkey]

Hi,

This RFC series introduces a new connector type:
 DRM_MODE_CONNECTOR_WRITEBACK
It is a follow-on from a previous discussion: [1]

Writeback connectors are used to expose the memory writeback engines
found in some display controllers, which can write a CRTC's
composition result to a memory buffer.
This is useful e.g. for testing, screen-recording, screenshots,
wireless display, display cloning, memory-to-memory composition.

Patches 1-7 include the core framework changes required, and patches
8-11 implement a writeback connector for the Mali-DP writeback engine.
The Mali-DP patches depend on this other series: [2].

The connector is given the FB_ID property for the output framebuffer,
and two new read-only properties: PIXEL_FORMATS and
PIXEL_FORMATS_SIZE, which expose the supported framebuffer pixel
formats of the engine.

The EDID property is not exposed for writeback connectors.

Writeback connector usage:
--
Due to connector routing changes being treated as "full modeset"
operations, any client which wishes to use a writeback connector
should include the connector in every modeset. The writeback will not
actually become active until a framebuffer is attached.

The writeback itself is enabled by attaching a framebuffer to the
FB_ID property of the connector. The driver must then ensure that the
CRTC content of that atomic commit is written into the framebuffer.

The writeback works in a one-shot mode with each atomic commit. This
prevents the same content from being written multiple times.
In some cases (front-buffer rendering) there might be a desire for
continuous operation - I think a property could be added later for
this kind of control.

Writeback can be disabled by setting FB_ID to zero.

Known issues:
-
 * I'm not sure what "DPMS" should mean for writeback connectors.
   It could be used to disable writeback (even when a framebuffer is
   attached), or it could be hidden entirely (which would break the
   legacy DPMS call for writeback connectors).
 * With Daniel's recent re-iteration of the userspace API rules, I
   fully expect to provide some userspace code to support this. The
   question is what, and where? We want to use writeback for testing,
   so perhaps some tests in igt is suitable.
 * Documentation. Probably some portion of this cover letter needs to
   make it into Documentation/
 * Synchronisation. Our hardware will finish the writeback by the next
   vsync. I've not implemented fence support here, but it would be an
   obvious addition.

See Also:
-
[1] https://lists.freedesktop.org/archives/dri-devel/2016-July/113197.html
[2] https://lists.freedesktop.org/archives/dri-devel/2016-October/120486.html

I welcome any comments, especially if this approach does/doesn't fit
well with anyone else's hardware.

Thanks,

-Brian

---

Brian Starkey (10):
  drm: add writeback connector type
  drm/fb-helper: skip writeback connectors
  drm: extract CRTC/plane disable from drm_framebuffer_remove
  drm: add __drm_framebuffer_remove_atomic
  drm: add fb to connector state
  drm: expose fb_id property for writeback connectors
  drm: add writeback-connector pixel format properties
  drm: mali-dp: rename malidp_input_format
  drm: mali-dp: add RGB writeback formats for DP550/DP650
  drm: mali-dp: add writeback connector

Liviu Dudau (1):
  drm: mali-dp: Add support for writeback on DP550/DP650

 drivers/gpu/drm/arm/Makefile|1 +
 drivers/gpu/drm/arm/malidp_crtc.c   |   10 ++
 drivers/gpu/drm/arm/malidp_drv.c|   25 +++-
 drivers/gpu/drm/arm/malidp_drv.h|5 +
 drivers/gpu/drm/arm/malidp_hw.c |  104 ++
 drivers/gpu/drm/arm/malidp_hw.h |   27 +++-
 drivers/gpu/drm/arm/malidp_mw.c |  268 +++
 drivers/gpu/drm/arm/malidp_planes.c |8 +-
 drivers/gpu/drm/arm/malidp_regs.h   |   15 ++
 drivers/gpu/drm/drm_atomic.c|   40 ++
 drivers/gpu/drm/drm_atomic_helper.c |4 +
 drivers/gpu/drm/drm_connector.c |   79 ++-
 drivers/gpu/drm/drm_crtc.c  |   14 +-
 drivers/gpu/drm/drm_fb_helper.c |4 +
 drivers/gpu/drm/drm_framebuffer.c   |  249 
 drivers/gpu/drm/drm_ioctl.c |7 +
 include/drm/drmP.h  |2 +
 include/drm/drm_atomic.h|3 +
 include/drm/drm_connector.h |   15 ++
 include/drm/drm_crtc.h  |   12 ++
 include/uapi/drm/drm.h  |   10 ++
 include/uapi/drm/drm_mode.h |1 +
 22 files changed, 830 insertions(+), 73 deletions(-)
 create mode 100644 drivers/gpu/drm/arm/malidp_mw.c

-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 03/11] drm: Extract CRTC/plane disable from drm_framebuffer_remove

[Brian Starkey]

In preparation for adding an atomic version of the disable code, extract
the actual disable operation into a separate function.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_framebuffer.c |   87 +++--
 1 file changed, 54 insertions(+), 33 deletions(-)

diff --git a/drivers/gpu/drm/drm_framebuffer.c 
b/drivers/gpu/drm/drm_framebuffer.c
index 398efd6..528f75d 100644
--- a/drivers/gpu/drm/drm_framebuffer.c
+++ b/drivers/gpu/drm/drm_framebuffer.c
@@ -795,22 +795,61 @@ void drm_framebuffer_cleanup(struct drm_framebuffer *fb)
 EXPORT_SYMBOL(drm_framebuffer_cleanup);
 
 /**
- * drm_framebuffer_remove - remove and unreference a framebuffer object
+ * __drm_framebuffer_remove - remove all usage of a framebuffer object
+ * @dev: drm device
  * @fb: framebuffer to remove
  *
  * Scans all the CRTCs and planes in @dev's mode_config.  If they're
- * using @fb, removes it, setting it to NULL. Then drops the reference to the
- * passed-in framebuffer. Might take the modeset locks.
+ * using @fb, removes it, setting it to NULL. Takes the modeset locks.
  *
- * Note that this function optimizes the cleanup away if the caller holds the
- * last reference to the framebuffer. It is also guaranteed to not take the
- * modeset locks in this case.
+ * Returns:
+ * true if the framebuffer was successfully removed from use
  */
-void drm_framebuffer_remove(struct drm_framebuffer *fb)
+static bool __drm_framebuffer_remove(struct drm_device *dev, struct 
drm_framebuffer *fb)
 {
-   struct drm_device *dev;
struct drm_crtc *crtc;
struct drm_plane *plane;
+   bool ret = true;
+
+   drm_modeset_lock_all(dev);
+   /* remove from any CRTC */
+   drm_for_each_crtc(crtc, dev) {
+   if (crtc->primary->fb == fb) {
+   /* should turn off the crtc */
+   if (drm_crtc_force_disable(crtc))
+   ret = false;
+   }
+   }
+
+   drm_for_each_plane(plane, dev) {
+   if (plane->fb == fb)
+   /* TODO: Propagate error here? */
+   drm_plane_force_disable(plane);
+   }
+   drm_modeset_unlock_all(dev);
+
+   return ret;
+}
+
+/**
+ * drm_framebuffer_remove - remove and unreference a framebuffer object
+ * @fb: framebuffer to remove
+ *
+ * drm ABI mandates that we remove any deleted framebuffers from active usage.
+ * This function takes care of this detail, disabling any CRTCs/Planes which
+ * are using the framebuffer being removed.
+ *
+ * Since most sane clients only remove framebuffers they no longer need, we
+ * skip the disable step if the caller holds the last reference to the
+ * framebuffer. It is also guaranteed to not take the modeset locks in
+ * this case.
+ *
+ * Before returning this function drops (what should be) the last reference
+ * on the framebuffer.
+ */
+void drm_framebuffer_remove(struct drm_framebuffer *fb)
+{
+   struct drm_device *dev;
 
if (!fb)
return;
@@ -820,37 +859,19 @@ void drm_framebuffer_remove(struct drm_framebuffer *fb)
WARN_ON(!list_empty(>filp_head));
 
/*
-* drm ABI mandates that we remove any deleted framebuffers from active
-* useage. But since most sane clients only remove framebuffers they no
-* longer need, try to optimize this away.
-*
 * Since we're holding a reference ourselves, observing a refcount of 1
-* means that we're the last holder and can skip it. Also, the refcount
-* can never increase from 1 again, so we don't need any barriers or
-* locks.
+* means that we're the last holder and can skip the disable. Also, the
+* refcount can never increase from 1 again, so we don't need any
+* barriers or locks.
 *
-* Note that userspace could try to race with use and instate a new
+* Note that userspace could try to race with us and instate a new
 * usage _after_ we've cleared all current ones. End result will be an
 * in-use fb with fb-id == 0. Userspace is allowed to shoot its own foot
 * in this manner.
 */
-   if (drm_framebuffer_read_refcount(fb) > 1) {
-   drm_modeset_lock_all(dev);
-   /* remove from any CRTC */
-   drm_for_each_crtc(crtc, dev) {
-   if (crtc->primary->fb == fb) {
-   /* should turn off the crtc */
-   if (drm_crtc_force_disable(crtc))
-   DRM_ERROR("failed to reset crtc %p when 
fb was deleted\n", crtc);
-   }
-   }
-
-   drm_for_each_plane(plane, dev) {
-   if (plane->fb == fb)
-   drm_plane_force_disable(plane);
-   }
- 

[RFC PATCH 02/11] drm/fb-helper: Skip writeback connectors

[Brian Starkey]

Writeback connectors aren't much use to the fbdev helpers, as they won't
show anything to the user. Skip them when looking for candidate output
configurations.

Signed-off-by: Brian Starkey <brian.star...@arm.com>
---
 drivers/gpu/drm/drm_fb_helper.c |4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c
index 03414bd..dedf6e7 100644
--- a/drivers/gpu/drm/drm_fb_helper.c
+++ b/drivers/gpu/drm/drm_fb_helper.c
@@ -2016,6 +2016,10 @@ static int drm_pick_crtcs(struct drm_fb_helper 
*fb_helper,
if (modes[n] == NULL)
return best_score;
 
+   /* Writeback connectors aren't much use for fbdev */
+   if (connector->connector_type == DRM_MODE_CONNECTOR_WRITEBACK)
+   return best_score;
+
crtcs = kzalloc(fb_helper->connector_count *
sizeof(struct drm_fb_helper_crtc *), GFP_KERNEL);
if (!crtcs)
-- 
1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: DRM device memory writeback (Mali-DP)

[Brian Starkey]

Hi,

On Fri, Jul 15, 2016 at 10:42:01AM -0700, Eric Anholt wrote:

Ville Syrjälä <ville.syrj...@linux.intel.com> writes:


On Fri, Jul 15, 2016 at 10:09:19AM +0100, Brian Starkey wrote:

Hi Daniel,

Thanks for taking a look.

(+Cc Laurent)

On Fri, Jul 15, 2016 at 09:33:34AM +0200, Daniel Vetter wrote:
>On Thu, Jul 14, 2016 at 06:03:40PM +0100, Brian Starkey wrote:
>> Hi,
>>
>> The Mali-DP display processors have a memory-writeback engine which
>> can write the result of the composition (CRTC output) to a memory
>> buffer in a variety of formats.
>>
>> We're looking for feedback/suggestions on how to expose this in the
>> mali-dp DRM kernel driver - possibly via V4L2.
>>
>> We've got a few use cases where writeback is useful:
>>- testing, to check the displayed image
>
>This might or might not need a separate interface. There are efforts to
>make the intel kms validation tests in i-g-t generic (well under way
>already), and part of that is creating a generic infrastructure to capture
>display CRCs for functional tests (still in progress).
>
>But it might be better if userspace abstracts between full readback and
>display CRC, assuming we can make full writeback cross-vendor enough for
>that use-case.
>

I'd lean towards the userspace abstraction.
Encumbering a simple CRC interface with all the complexity of
full-writeback (size, scaling, pixel format, multi-planar etc.) sounds
a bit unnecessary.

Of course, if v4l2 isn't going to be the cross-vendor full-writeback
implementation, then we need to be aiming to use whatever _is_ in
the mali-dp driver.

>>- screen recording
>>- wireless display (e.g. Miracast)
>>- dual-display clone mode
>>- memory-to-memory composition
>> Note that the HW is capable of writing one of the input planes instead
>> of the CRTC output, but we've no good use-case for wanting to expose
>> that.
>>
>> In our Android ADF driver[1] we exposed the memory write engine as
>> part of the ADF device using ADF's "MEMORY" interface type. DRM/KMS
>> doesn't have any similar support for memory output from CRTCs, but we
>> want to expose the functionality in the mainline Mali-DP DRM driver.
>>
>> A previous discussion on the topic went towards exposing the
>> memory-write engine via V4L2[2].
>>
>> I'm thinking to userspace this would look like two distinct devices:
>>- A DRM KMS display controller.
>>- A V4L2 video source.
>> They'd both exist in the same kernel driver.
>> A V4L2 client can queue up (CAPTURE) buffers in the normal way, and
>> the DRM driver would see if there's a buffer to dequeue every time a
>> new modeset is received via the DRM API - if so, it can configure the
>> HW to dump into it (one-shot operation).
>>
>> An implication of this is that if the screen is actively displaying a
>> static scene and the V4L2 client queues up a buffer, it won't get
>> filled until the DRM scene changes. This seems best, otherwise the
>> V4L2 driver has to change the HW configuration out-of-band from the
>> DRM device which sounds horribly racy.
>>
>> One further complication is scaling. Our HW has a scaler which can
>> tasked with either scaling an input plane or the buffer being written
>> to memory, but not both at the same time. This means we need to
>> arbitrate the scaler between the DRM device (scaling input planes) and
>> the V4L2 device (scaling output buffers).
>>
>> I think the simplest approach here is to allow V4L2 to "claim" the
>> scaler by setting the image size (VIDIOC_S_FMT) to something other
>> than the CRTC's current resolution. After that, any attempt to use the
>> scaler on an input plane via DRM should fail atomic_check().
>
>That's perfectly fine atomic_check behaviour. Only trouble is that the v4l
>locking must integrate into the drm locking, but that should be doable.
>Worst case you must shadow all v4l locks with a wait/wound
>drm_modeset_lock to avoid deadlocks (since you could try to grab locks
>from either end).
>

Yes, I haven't looked at the details of the locking but I'm hoping
it's manageable.

>> If the V4L2 client goes away or sets the image size to the CRTC's
>> native resolution, then the DRM device is allowed to use the scaler.
>> I don't know if/how the DRM device should communicate to userspace
>> that the scaler is or isn't available for use.
>>
>> Any thoughts on this approach?
>> Is it acceptable to both V4L2 and DRM folks?
>
>For streaming a V4L2 capture device seems like the right interface. But if
>you want to use writeback in your compositor you must know which atomic
>kms u

Re: DRM device memory writeback (Mali-DP)

[Brian Starkey]

Hi Daniel,

Thanks for taking a look.

(+Cc Laurent)

On Fri, Jul 15, 2016 at 09:33:34AM +0200, Daniel Vetter wrote:

On Thu, Jul 14, 2016 at 06:03:40PM +0100, Brian Starkey wrote:

Hi,

The Mali-DP display processors have a memory-writeback engine which
can write the result of the composition (CRTC output) to a memory
buffer in a variety of formats.

We're looking for feedback/suggestions on how to expose this in the
mali-dp DRM kernel driver - possibly via V4L2.

We've got a few use cases where writeback is useful:
   - testing, to check the displayed image


This might or might not need a separate interface. There are efforts to
make the intel kms validation tests in i-g-t generic (well under way
already), and part of that is creating a generic infrastructure to capture
display CRCs for functional tests (still in progress).

But it might be better if userspace abstracts between full readback and
display CRC, assuming we can make full writeback cross-vendor enough for
that use-case.



I'd lean towards the userspace abstraction.
Encumbering a simple CRC interface with all the complexity of
full-writeback (size, scaling, pixel format, multi-planar etc.) sounds
a bit unnecessary.

Of course, if v4l2 isn't going to be the cross-vendor full-writeback
implementation, then we need to be aiming to use whatever _is_ in
the mali-dp driver.


   - screen recording
   - wireless display (e.g. Miracast)
   - dual-display clone mode
   - memory-to-memory composition
Note that the HW is capable of writing one of the input planes instead
of the CRTC output, but we've no good use-case for wanting to expose
that.

In our Android ADF driver[1] we exposed the memory write engine as
part of the ADF device using ADF's "MEMORY" interface type. DRM/KMS
doesn't have any similar support for memory output from CRTCs, but we
want to expose the functionality in the mainline Mali-DP DRM driver.

A previous discussion on the topic went towards exposing the
memory-write engine via V4L2[2].

I'm thinking to userspace this would look like two distinct devices:
   - A DRM KMS display controller.
   - A V4L2 video source.
They'd both exist in the same kernel driver.
A V4L2 client can queue up (CAPTURE) buffers in the normal way, and
the DRM driver would see if there's a buffer to dequeue every time a
new modeset is received via the DRM API - if so, it can configure the
HW to dump into it (one-shot operation).

An implication of this is that if the screen is actively displaying a
static scene and the V4L2 client queues up a buffer, it won't get
filled until the DRM scene changes. This seems best, otherwise the
V4L2 driver has to change the HW configuration out-of-band from the
DRM device which sounds horribly racy.

One further complication is scaling. Our HW has a scaler which can
tasked with either scaling an input plane or the buffer being written
to memory, but not both at the same time. This means we need to
arbitrate the scaler between the DRM device (scaling input planes) and
the V4L2 device (scaling output buffers).

I think the simplest approach here is to allow V4L2 to "claim" the
scaler by setting the image size (VIDIOC_S_FMT) to something other
than the CRTC's current resolution. After that, any attempt to use the
scaler on an input plane via DRM should fail atomic_check().


That's perfectly fine atomic_check behaviour. Only trouble is that the v4l
locking must integrate into the drm locking, but that should be doable.
Worst case you must shadow all v4l locks with a wait/wound
drm_modeset_lock to avoid deadlocks (since you could try to grab locks
from either end).



Yes, I haven't looked at the details of the locking but I'm hoping
it's manageable.


If the V4L2 client goes away or sets the image size to the CRTC's
native resolution, then the DRM device is allowed to use the scaler.
I don't know if/how the DRM device should communicate to userspace
that the scaler is or isn't available for use.

Any thoughts on this approach?
Is it acceptable to both V4L2 and DRM folks?


For streaming a V4L2 capture device seems like the right interface. But if
you want to use writeback in your compositor you must know which atomic
kms update results in which frame, since if you don't you can't use that
composited buffer for the next frame reliable.

For that case I think a drm-only solution would be better, to make sure
you can do an atomic update and writeback in one step. v4l seems to grow
an atomic api of its own, but I don't think we'll have one spanning
subsystems anytime soon.



I've been thinking about this from the point of view of a HWComposer
implementation and I think the hybrid DRM-V4L2 device would work OK.
However it depends on the behaviour I mentioned above:


if the screen is actively displaying a
static scene and the V4L2 client queues up a buffer, it won't get
filled until the DRM scene changes.


V4L2 buffer queues are FIFO, so as long as the compositor queues only
one V4L2 buffer per at

DRM device memory writeback (Mali-DP)

[Brian Starkey]

Hi,

The Mali-DP display processors have a memory-writeback engine which
can write the result of the composition (CRTC output) to a memory
buffer in a variety of formats.

We're looking for feedback/suggestions on how to expose this in the
mali-dp DRM kernel driver - possibly via V4L2.

We've got a few use cases where writeback is useful:
   - testing, to check the displayed image
   - screen recording
   - wireless display (e.g. Miracast)
   - dual-display clone mode
   - memory-to-memory composition
Note that the HW is capable of writing one of the input planes instead
of the CRTC output, but we've no good use-case for wanting to expose
that.

In our Android ADF driver[1] we exposed the memory write engine as
part of the ADF device using ADF's "MEMORY" interface type. DRM/KMS
doesn't have any similar support for memory output from CRTCs, but we
want to expose the functionality in the mainline Mali-DP DRM driver.

A previous discussion on the topic went towards exposing the
memory-write engine via V4L2[2].

I'm thinking to userspace this would look like two distinct devices:
   - A DRM KMS display controller.
   - A V4L2 video source.
They'd both exist in the same kernel driver.
A V4L2 client can queue up (CAPTURE) buffers in the normal way, and
the DRM driver would see if there's a buffer to dequeue every time a
new modeset is received via the DRM API - if so, it can configure the
HW to dump into it (one-shot operation).

An implication of this is that if the screen is actively displaying a
static scene and the V4L2 client queues up a buffer, it won't get
filled until the DRM scene changes. This seems best, otherwise the
V4L2 driver has to change the HW configuration out-of-band from the
DRM device which sounds horribly racy.

One further complication is scaling. Our HW has a scaler which can
tasked with either scaling an input plane or the buffer being written
to memory, but not both at the same time. This means we need to
arbitrate the scaler between the DRM device (scaling input planes) and
the V4L2 device (scaling output buffers).

I think the simplest approach here is to allow V4L2 to "claim" the
scaler by setting the image size (VIDIOC_S_FMT) to something other
than the CRTC's current resolution. After that, any attempt to use the
scaler on an input plane via DRM should fail atomic_check().

If the V4L2 client goes away or sets the image size to the CRTC's
native resolution, then the DRM device is allowed to use the scaler.
I don't know if/how the DRM device should communicate to userspace
that the scaler is or isn't available for use.

Any thoughts on this approach?
Is it acceptable to both V4L2 and DRM folks?

Thanks for your time,

-Brian

[1] 
http://malideveloper.arm.com/resources/drivers/open-source-mali-dp-adf-kernel-device-drivers/
[2] 
https://people.freedesktop.org/~cbrill/dri-log/?channel=dri-devel=2016-05-04
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html