Re: [RFCv3 15/17] v4l2: document the request API interface
Hi, A few comments below... On 02/06/2018 05:48 PM, Alexandre Courbot wrote: > Document how the request API can be used along with the existing V4L2 > interface. > > Signed-off-by: Alexandre Courbot> --- > Documentation/media/uapi/v4l/buffer.rst| 10 +- > Documentation/media/uapi/v4l/common.rst| 1 + > Documentation/media/uapi/v4l/request-api.rst | 236 > + > .../media/uapi/v4l/vidioc-g-ext-ctrls.rst | 16 +- > Documentation/media/uapi/v4l/vidioc-qbuf.rst | 21 ++ > 5 files changed, 280 insertions(+), 4 deletions(-) > create mode 100644 Documentation/media/uapi/v4l/request-api.rst > diff --git a/Documentation/media/uapi/v4l/request-api.rst > b/Documentation/media/uapi/v4l/request-api.rst > new file mode 100644 > index ..4c61a0dbe3a9 > --- /dev/null > +++ b/Documentation/media/uapi/v4l/request-api.rst > @@ -0,0 +1,236 @@ > +.. -*- coding: utf-8; mode: rst -*- > + > +.. _media-request-api: > + > +Request API > +=== > + > +The Request API has been designed to allow V4L2 to deal with requirements of > +modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec > +v2). One such requirement is the ability for devices belonging to the same > +pipeline to reconfigure and collaborate closely on a per-frame basis. > Another is > +efficient support of stateless codecs, which need per-frame controls to be > set > +asynchronously in order to be efficiently used. to be used efficiently. > + > +Supporting these features without the Request API is possible but terribly > +inefficient: user-space would have to flush all activity on the media > pipeline, > +reconfigure it for the next frame, queue the buffers to be processed with > that > +configuration, and wait until they are all available for dequeing before "dequeing" should be dequeueing or dequeuing. > +considering the next frame. This defeats the purpose of having buffer queues > +since in practice only one buffer would be queued at a time. > + > +The Request API allows a specific configuration of the pipeline (media > +controller topology + controls for each device) to be associated with > specific > +buffers. The parameters are applied by each participating device as buffers > +associated to a request flow in. This allows user-space to schedule several > +tasks ("requests") with different parameters in advance, knowing that the > +parameters will be applied when needed to get the expected result. Controls > +values at the time of request completion are also available for reading. > + > +Usage > += > + > +The Request API is used on top of standard media controller and V4L2 calls, > +which are augmented with an extra ``request_fd`` parameter. All operations on > +requests themselves are performed using the command parameter of the > +:c:func:`MEDIA_IOC_REQUEST_CMD` ioctl. > + > +Request Allocation > +-- > + > +User-space allocates requests using the ``MEDIA_REQ_CMD_ALLOC`` command on > +an opened media device. This returns a file descriptor representing the > +request. Typically, several such requests will be allocated. > + > +Request Preparation > +--- > + > +Standard V4L2 ioctls can then receive a request file descriptor to express > the > +fact that the ioctl is part of said request, and is not to be applied > +immediately. V4L2 ioctls supporting this are :c:func:`VIDIOC_S_EXT_CTRLS` and > +:c:func:`VIDIOC_QBUF`. Controls set with a request parameter are stored > instead > +of being immediately applied, and queued buffers will block the buffer queue > +until the request becomes active. > + > +RFC Note: currently several buffers can be queued to the same queue with the > +same request. The request parameters will be only be applied when processing > +the first buffer. Does it make more sense to allow at most one buffer per > +request per queue instead? > + > +Request Submission > +-- > + > +Once the parameters and buffers of the request are specified, it can be > +submitted with the ``MEDIA_REQ_CMD_SUBMIT`` command. This will make the > buffers > +associated to the request available to their driver, which can then apply the > +saved controls as buffers are processed. A submitted request cannot be > modified > +anymore. > + > +If several devices are part of the request, individual drivers may > synchronize > +so the requested pipeline's topology is applied before the buffers are > +processed. This is at the discretion of the drivers and is not a requirement. > + > +Buffers queued without an associated request after a request-bound buffer > will > +be processed using the state of the hardware at the time of the request > +completion. All the same, controls set without a request are applied > +immediately, regardless of whether a request is in use or not. > + > +User-space can ``poll()`` a request FD in order to wait until the request
[RFCv3 15/17] v4l2: document the request API interface
Document how the request API can be used along with the existing V4L2 interface. Signed-off-by: Alexandre Courbot--- Documentation/media/uapi/v4l/buffer.rst| 10 +- Documentation/media/uapi/v4l/common.rst| 1 + Documentation/media/uapi/v4l/request-api.rst | 236 + .../media/uapi/v4l/vidioc-g-ext-ctrls.rst | 16 +- Documentation/media/uapi/v4l/vidioc-qbuf.rst | 21 ++ 5 files changed, 280 insertions(+), 4 deletions(-) create mode 100644 Documentation/media/uapi/v4l/request-api.rst diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst index ae6ee73f151c..9d082784081d 100644 --- a/Documentation/media/uapi/v4l/buffer.rst +++ b/Documentation/media/uapi/v4l/buffer.rst @@ -301,10 +301,14 @@ struct v4l2_buffer elements in the ``planes`` array. The driver will fill in the actual number of valid elements in that array. * - __u32 - - ``reserved2`` + - ``request_fd`` - - - A place holder for future extensions. Drivers and applications - must set this to 0. + - The file descriptor of the request associated with this buffer. + user-space can set this when calling :ref:`VIDIOC_QBUF`, and drivers + will return the request used when processing a buffer (if any) upon + :ref:`VIDIOC_DQBUF`. + + A value of 0 means the buffer is not associated with any request. * - __u32 - ``reserved`` - diff --git a/Documentation/media/uapi/v4l/common.rst b/Documentation/media/uapi/v4l/common.rst index 13f2ed3fc5a6..a4aa0059d45a 100644 --- a/Documentation/media/uapi/v4l/common.rst +++ b/Documentation/media/uapi/v4l/common.rst @@ -44,3 +44,4 @@ applicable to all devices. crop selection-api streaming-par +request-api diff --git a/Documentation/media/uapi/v4l/request-api.rst b/Documentation/media/uapi/v4l/request-api.rst new file mode 100644 index ..4c61a0dbe3a9 --- /dev/null +++ b/Documentation/media/uapi/v4l/request-api.rst @@ -0,0 +1,236 @@ +.. -*- coding: utf-8; mode: rst -*- + +.. _media-request-api: + +Request API +=== + +The Request API has been designed to allow V4L2 to deal with requirements of +modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec +v2). One such requirement is the ability for devices belonging to the same +pipeline to reconfigure and collaborate closely on a per-frame basis. Another is +efficient support of stateless codecs, which need per-frame controls to be set +asynchronously in order to be efficiently used. + +Supporting these features without the Request API is possible but terribly +inefficient: user-space would have to flush all activity on the media pipeline, +reconfigure it for the next frame, queue the buffers to be processed with that +configuration, and wait until they are all available for dequeing before +considering the next frame. This defeats the purpose of having buffer queues +since in practice only one buffer would be queued at a time. + +The Request API allows a specific configuration of the pipeline (media +controller topology + controls for each device) to be associated with specific +buffers. The parameters are applied by each participating device as buffers +associated to a request flow in. This allows user-space to schedule several +tasks ("requests") with different parameters in advance, knowing that the +parameters will be applied when needed to get the expected result. Controls +values at the time of request completion are also available for reading. + +Usage += + +The Request API is used on top of standard media controller and V4L2 calls, +which are augmented with an extra ``request_fd`` parameter. All operations on +requests themselves are performed using the command parameter of the +:c:func:`MEDIA_IOC_REQUEST_CMD` ioctl. + +Request Allocation +-- + +User-space allocates requests using the ``MEDIA_REQ_CMD_ALLOC`` command on +an opened media device. This returns a file descriptor representing the +request. Typically, several such requests will be allocated. + +Request Preparation +--- + +Standard V4L2 ioctls can then receive a request file descriptor to express the +fact that the ioctl is part of said request, and is not to be applied +immediately. V4L2 ioctls supporting this are :c:func:`VIDIOC_S_EXT_CTRLS` and +:c:func:`VIDIOC_QBUF`. Controls set with a request parameter are stored instead +of being immediately applied, and queued buffers will block the buffer queue +until the request becomes active. + +RFC Note: currently several buffers can be queued to the same queue with the +same request. The request parameters will be only be applied when processing +the first buffer. Does it make more sense to allow at most one buffer per +request per queue instead? + +Request Submission +-- + +Once the parameters and buffers of the request are