Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Marek Szyprowski]

Hi Daniel,

On 2017-05-10 09:55, Daniel Vetter wrote:

On Wed, May 10, 2017 at 03:27:02PM +0900, Inki Dae wrote:

Hi Tomasz,

2017년 05월 10일 14:38에 Tomasz Figa 이(가) 쓴 글:

Hi Everyone,

On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:


2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:

Hi Marek,

On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:

Hi Laurent,

On 2017-04-20 12:25, Laurent Pinchart wrote:

Hi Marek,

(CC'ing Sakari Ailus)

Thank you for the patches.

On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:

Dear all,

This is an updated proposal for extending EXYNOS DRM API with generic
support for hardware modules, which can be used for processing image data

>from the one memory buffer to another. Typical memory-to-memory operations

are: rotation, scaling, colour space conversion or mix of them. This is a
follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
processors", which has been rejected as "not really needed in the DRM
core":
http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html

In this proposal I moved all the code to Exynos DRM driver, so now this
will be specific only to Exynos DRM. I've also changed the name from
framebuffer processor (fbproc) to picture processor (pp) to avoid confusion
with fbdev API.

Here is a bit more information what picture processors are:

Embedded SoCs are known to have a number of hardware blocks, which perform
such operations. They can be used in paralel to the main GPU module to
offload CPU from processing grapics or video data. One of example use of
such modules is implementing video overlay, which usually requires color
space conversion from NV12 (or similar) to RGB32 color space and scaling to
target window size.

The proposed API is heavily inspired by atomic KMS approach - it is also
based on DRM objects and their properties. A new DRM object is introduced:
picture processor (called pp for convenience). Such objects have a set of
standard DRM properties, which describes the operation to be performed by
respective hardware module. In typical case those properties are a source
fb id and rectangle (x, y, width, height) and destination fb id and
rectangle. Optionally a rotation property can be also specified if
supported by the given hardware. To perform an operation on image data,
userspace provides a set of properties and their values for given fbproc
object in a similar way as object and properties are provided for
performing atomic page flip / mode setting.

The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
   processors,
- DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
   processor,
- DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
   property set.

The proposed API is extensible. Drivers can attach their own, custom
properties to add support for more advanced picture processing (for example
blending).

This proposal aims to replace Exynos DRM IPP (Image Post Processing)
subsystem. IPP API is over-engineered in general, but not really extensible
on the other side. It is also buggy, with significant design flaws - the
biggest issue is the fact that the API covers memory-2-memory picture
operations together with CRTC writeback and duplicating features, which
belongs to video plane. Comparing with IPP subsystem, the PP framework is
smaller (1807 vs 778 lines) and allows driver simplification (Exynos
rotator driver smaller by over 200 lines).

This seems to be the kind of hardware that is typically supported by V4L2.
Stupid question, why DRM ?

Let me elaborate a bit on the reasons for implementing it in Exynos DRM:

1. we want to replace existing Exynos IPP subsystem:
  - it is used only in some internal/vendor trees, not in open-source
  - we want it to have sane and potentially extensible userspace API
  - but we don't want to loose its functionality

2. we want to have simple API for performing single image processing
operation:
  - typically it will be used by compositing window manager, this means that
some parameters of the processing might change on each vblank (like
destination rectangle for example). This api allows such change on each
operation without any additional cost. V4L2 requires to reinitialize
queues with new configuration on such change, what means that a bunch of
ioctls has to be called.

What do you mean by re-initialising the queue? Format, buffers or something
else?

If you need a larger buffer than what you have already allocated, you'll
need to re-allocate, V4L2 or not.

We also do lack a way to destroy individual buffers in V4L2. It'd be up to
implementing that and some work in videobuf2.

Another thing is that V4L2 is very stream oriented. For most devices that's
fine as a lot of the parameters are not changeable during streaming,
especially if the pipeline is handled by multiple drivers. That said, for
devices that process 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Daniel Vetter]

On Wed, May 10, 2017 at 12:29 PM, Inki Dae  wrote:
>> This kind of contradicts with response Marek received from DRM
>> community about his proposal. Which drivers in particular you have in
>> mind?
>
> You can check vmw_overlay_ioctl of vmwgfx driver and 
> intel_overlay_put_image_ioctl of i915 driver. These was all I could find in 
> mainline.
> Seems the boundaries of whether we have to implement pre/post post processing 
> mem2mem driver in V4L2 or DRM are really vague.

These aren't picture processors, but overlay plane support merged
before we had the core drm overlay support. Please do not emulate them
at all, your patch will be rejected :-)
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Inki Dae]

2017년 05월 10일 16:55에 Daniel Vetter 이(가) 쓴 글:
> On Wed, May 10, 2017 at 03:27:02PM +0900, Inki Dae wrote:
>> Hi Tomasz,
>>
>> 2017년 05월 10일 14:38에 Tomasz Figa 이(가) 쓴 글:
>>> Hi Everyone,
>>>
>>> On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:


 2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
> Hi Marek,
>
> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>> Hi Laurent,
>>
>> On 2017-04-20 12:25, Laurent Pinchart wrote:
>>> Hi Marek,
>>>
>>> (CC'ing Sakari Ailus)
>>>
>>> Thank you for the patches.
>>>
>>> On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
 Dear all,

 This is an updated proposal for extending EXYNOS DRM API with generic
 support for hardware modules, which can be used for processing image 
 data
>>> >from the one memory buffer to another. Typical memory-to-memory 
>>> >operations
 are: rotation, scaling, colour space conversion or mix of them. This 
 is a
 follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
 processors", which has been rejected as "not really needed in the DRM
 core":
 http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html

 In this proposal I moved all the code to Exynos DRM driver, so now this
 will be specific only to Exynos DRM. I've also changed the name from
 framebuffer processor (fbproc) to picture processor (pp) to avoid 
 confusion
 with fbdev API.

 Here is a bit more information what picture processors are:

 Embedded SoCs are known to have a number of hardware blocks, which 
 perform
 such operations. They can be used in paralel to the main GPU module to
 offload CPU from processing grapics or video data. One of example use 
 of
 such modules is implementing video overlay, which usually requires 
 color
 space conversion from NV12 (or similar) to RGB32 color space and 
 scaling to
 target window size.

 The proposed API is heavily inspired by atomic KMS approach - it is 
 also
 based on DRM objects and their properties. A new DRM object is 
 introduced:
 picture processor (called pp for convenience). Such objects have a set 
 of
 standard DRM properties, which describes the operation to be performed 
 by
 respective hardware module. In typical case those properties are a 
 source
 fb id and rectangle (x, y, width, height) and destination fb id and
 rectangle. Optionally a rotation property can be also specified if
 supported by the given hardware. To perform an operation on image data,
 userspace provides a set of properties and their values for given 
 fbproc
 object in a similar way as object and properties are provided for
 performing atomic page flip / mode setting.

 The proposed API consists of the 3 new ioctls:
 - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
   processors,
 - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
   processor,
 - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
   property set.

 The proposed API is extensible. Drivers can attach their own, custom
 properties to add support for more advanced picture processing (for 
 example
 blending).

 This proposal aims to replace Exynos DRM IPP (Image Post Processing)
 subsystem. IPP API is over-engineered in general, but not really 
 extensible
 on the other side. It is also buggy, with significant design flaws - 
 the
 biggest issue is the fact that the API covers memory-2-memory picture
 operations together with CRTC writeback and duplicating features, which
 belongs to video plane. Comparing with IPP subsystem, the PP framework 
 is
 smaller (1807 vs 778 lines) and allows driver simplification (Exynos
 rotator driver smaller by over 200 lines).
>>> This seems to be the kind of hardware that is typically supported by 
>>> V4L2.
>>> Stupid question, why DRM ?
>>
>> Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
>>
>> 1. we want to replace existing Exynos IPP subsystem:
>>  - it is used only in some internal/vendor trees, not in open-source
>>  - we want it to have sane and potentially extensible userspace API
>>  - but we don't want to loose its functionality
>>
>> 2. we want to have simple API for performing single image processing
>> operation:
>>  - typically it will be used by compositing window manager, this means 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Inki Dae]

2017년 05월 10일 15:38에 Tomasz Figa 이(가) 쓴 글:
> On Wed, May 10, 2017 at 2:27 PM, Inki Dae  wrote:
>> Hi Tomasz,
>>
>> 2017년 05월 10일 14:38에 Tomasz Figa 이(가) 쓴 글:
>>> Hi Everyone,
>>>
>>> On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:


 2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
> Hi Marek,
>
> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>> Hi Laurent,
>>
>> On 2017-04-20 12:25, Laurent Pinchart wrote:
>>> Hi Marek,
>>>
>>> (CC'ing Sakari Ailus)
>>>
>>> Thank you for the patches.
>>>
>>> On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
 Dear all,

 This is an updated proposal for extending EXYNOS DRM API with generic
 support for hardware modules, which can be used for processing image 
 data
>>> >from the one memory buffer to another. Typical memory-to-memory 
>>> >operations
 are: rotation, scaling, colour space conversion or mix of them. This 
 is a
 follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
 processors", which has been rejected as "not really needed in the DRM
 core":
 http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html

 In this proposal I moved all the code to Exynos DRM driver, so now this
 will be specific only to Exynos DRM. I've also changed the name from
 framebuffer processor (fbproc) to picture processor (pp) to avoid 
 confusion
 with fbdev API.

 Here is a bit more information what picture processors are:

 Embedded SoCs are known to have a number of hardware blocks, which 
 perform
 such operations. They can be used in paralel to the main GPU module to
 offload CPU from processing grapics or video data. One of example use 
 of
 such modules is implementing video overlay, which usually requires 
 color
 space conversion from NV12 (or similar) to RGB32 color space and 
 scaling to
 target window size.

 The proposed API is heavily inspired by atomic KMS approach - it is 
 also
 based on DRM objects and their properties. A new DRM object is 
 introduced:
 picture processor (called pp for convenience). Such objects have a set 
 of
 standard DRM properties, which describes the operation to be performed 
 by
 respective hardware module. In typical case those properties are a 
 source
 fb id and rectangle (x, y, width, height) and destination fb id and
 rectangle. Optionally a rotation property can be also specified if
 supported by the given hardware. To perform an operation on image data,
 userspace provides a set of properties and their values for given 
 fbproc
 object in a similar way as object and properties are provided for
 performing atomic page flip / mode setting.

 The proposed API consists of the 3 new ioctls:
 - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
   processors,
 - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
   processor,
 - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
   property set.

 The proposed API is extensible. Drivers can attach their own, custom
 properties to add support for more advanced picture processing (for 
 example
 blending).

 This proposal aims to replace Exynos DRM IPP (Image Post Processing)
 subsystem. IPP API is over-engineered in general, but not really 
 extensible
 on the other side. It is also buggy, with significant design flaws - 
 the
 biggest issue is the fact that the API covers memory-2-memory picture
 operations together with CRTC writeback and duplicating features, which
 belongs to video plane. Comparing with IPP subsystem, the PP framework 
 is
 smaller (1807 vs 778 lines) and allows driver simplification (Exynos
 rotator driver smaller by over 200 lines).
>>> This seems to be the kind of hardware that is typically supported by 
>>> V4L2.
>>> Stupid question, why DRM ?
>>
>> Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
>>
>> 1. we want to replace existing Exynos IPP subsystem:
>>  - it is used only in some internal/vendor trees, not in open-source
>>  - we want it to have sane and potentially extensible userspace API
>>  - but we don't want to loose its functionality
>>
>> 2. we want to have simple API for performing single image processing
>> operation:
>>  - typically it will be used by compositing window manager, 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Daniel Vetter]

On Wed, May 10, 2017 at 03:27:02PM +0900, Inki Dae wrote:
> Hi Tomasz,
> 
> 2017년 05월 10일 14:38에 Tomasz Figa 이(가) 쓴 글:
> > Hi Everyone,
> > 
> > On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:
> >>
> >>
> >> 2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
> >>> Hi Marek,
> >>>
> >>> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>  Hi Laurent,
> 
>  On 2017-04-20 12:25, Laurent Pinchart wrote:
> > Hi Marek,
> >
> > (CC'ing Sakari Ailus)
> >
> > Thank you for the patches.
> >
> > On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> >> Dear all,
> >>
> >> This is an updated proposal for extending EXYNOS DRM API with generic
> >> support for hardware modules, which can be used for processing image 
> >> data
> > >from the one memory buffer to another. Typical memory-to-memory 
> > >operations
> >> are: rotation, scaling, colour space conversion or mix of them. This 
> >> is a
> >> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
> >> processors", which has been rejected as "not really needed in the DRM
> >> core":
> >> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
> >>
> >> In this proposal I moved all the code to Exynos DRM driver, so now this
> >> will be specific only to Exynos DRM. I've also changed the name from
> >> framebuffer processor (fbproc) to picture processor (pp) to avoid 
> >> confusion
> >> with fbdev API.
> >>
> >> Here is a bit more information what picture processors are:
> >>
> >> Embedded SoCs are known to have a number of hardware blocks, which 
> >> perform
> >> such operations. They can be used in paralel to the main GPU module to
> >> offload CPU from processing grapics or video data. One of example use 
> >> of
> >> such modules is implementing video overlay, which usually requires 
> >> color
> >> space conversion from NV12 (or similar) to RGB32 color space and 
> >> scaling to
> >> target window size.
> >>
> >> The proposed API is heavily inspired by atomic KMS approach - it is 
> >> also
> >> based on DRM objects and their properties. A new DRM object is 
> >> introduced:
> >> picture processor (called pp for convenience). Such objects have a set 
> >> of
> >> standard DRM properties, which describes the operation to be performed 
> >> by
> >> respective hardware module. In typical case those properties are a 
> >> source
> >> fb id and rectangle (x, y, width, height) and destination fb id and
> >> rectangle. Optionally a rotation property can be also specified if
> >> supported by the given hardware. To perform an operation on image data,
> >> userspace provides a set of properties and their values for given 
> >> fbproc
> >> object in a similar way as object and properties are provided for
> >> performing atomic page flip / mode setting.
> >>
> >> The proposed API consists of the 3 new ioctls:
> >> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
> >>   processors,
> >> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
> >>   processor,
> >> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
> >>   property set.
> >>
> >> The proposed API is extensible. Drivers can attach their own, custom
> >> properties to add support for more advanced picture processing (for 
> >> example
> >> blending).
> >>
> >> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> >> subsystem. IPP API is over-engineered in general, but not really 
> >> extensible
> >> on the other side. It is also buggy, with significant design flaws - 
> >> the
> >> biggest issue is the fact that the API covers memory-2-memory picture
> >> operations together with CRTC writeback and duplicating features, which
> >> belongs to video plane. Comparing with IPP subsystem, the PP framework 
> >> is
> >> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> >> rotator driver smaller by over 200 lines).
> > This seems to be the kind of hardware that is typically supported by 
> > V4L2.
> > Stupid question, why DRM ?
> 
>  Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
> 
>  1. we want to replace existing Exynos IPP subsystem:
>   - it is used only in some internal/vendor trees, not in open-source
>   - we want it to have sane and potentially extensible userspace API
>   - but we don't want to loose its functionality
> 
>  2. we want to have simple API for performing single image processing
>  operation:
>   - typically it will be used by compositing window manager, this means 
>  that
> some parameters of the 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Tomasz Figa]

On Wed, May 10, 2017 at 2:27 PM, Inki Dae  wrote:
> Hi Tomasz,
>
> 2017년 05월 10일 14:38에 Tomasz Figa 이(가) 쓴 글:
>> Hi Everyone,
>>
>> On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:
>>>
>>>
>>> 2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
 Hi Marek,

 On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
> Hi Laurent,
>
> On 2017-04-20 12:25, Laurent Pinchart wrote:
>> Hi Marek,
>>
>> (CC'ing Sakari Ailus)
>>
>> Thank you for the patches.
>>
>> On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
>>> Dear all,
>>>
>>> This is an updated proposal for extending EXYNOS DRM API with generic
>>> support for hardware modules, which can be used for processing image 
>>> data
>> >from the one memory buffer to another. Typical memory-to-memory 
>> >operations
>>> are: rotation, scaling, colour space conversion or mix of them. This is 
>>> a
>>> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
>>> processors", which has been rejected as "not really needed in the DRM
>>> core":
>>> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
>>>
>>> In this proposal I moved all the code to Exynos DRM driver, so now this
>>> will be specific only to Exynos DRM. I've also changed the name from
>>> framebuffer processor (fbproc) to picture processor (pp) to avoid 
>>> confusion
>>> with fbdev API.
>>>
>>> Here is a bit more information what picture processors are:
>>>
>>> Embedded SoCs are known to have a number of hardware blocks, which 
>>> perform
>>> such operations. They can be used in paralel to the main GPU module to
>>> offload CPU from processing grapics or video data. One of example use of
>>> such modules is implementing video overlay, which usually requires color
>>> space conversion from NV12 (or similar) to RGB32 color space and 
>>> scaling to
>>> target window size.
>>>
>>> The proposed API is heavily inspired by atomic KMS approach - it is also
>>> based on DRM objects and their properties. A new DRM object is 
>>> introduced:
>>> picture processor (called pp for convenience). Such objects have a set 
>>> of
>>> standard DRM properties, which describes the operation to be performed 
>>> by
>>> respective hardware module. In typical case those properties are a 
>>> source
>>> fb id and rectangle (x, y, width, height) and destination fb id and
>>> rectangle. Optionally a rotation property can be also specified if
>>> supported by the given hardware. To perform an operation on image data,
>>> userspace provides a set of properties and their values for given fbproc
>>> object in a similar way as object and properties are provided for
>>> performing atomic page flip / mode setting.
>>>
>>> The proposed API consists of the 3 new ioctls:
>>> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
>>>   processors,
>>> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
>>>   processor,
>>> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
>>>   property set.
>>>
>>> The proposed API is extensible. Drivers can attach their own, custom
>>> properties to add support for more advanced picture processing (for 
>>> example
>>> blending).
>>>
>>> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
>>> subsystem. IPP API is over-engineered in general, but not really 
>>> extensible
>>> on the other side. It is also buggy, with significant design flaws - the
>>> biggest issue is the fact that the API covers memory-2-memory picture
>>> operations together with CRTC writeback and duplicating features, which
>>> belongs to video plane. Comparing with IPP subsystem, the PP framework 
>>> is
>>> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
>>> rotator driver smaller by over 200 lines).
>> This seems to be the kind of hardware that is typically supported by 
>> V4L2.
>> Stupid question, why DRM ?
>
> Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
>
> 1. we want to replace existing Exynos IPP subsystem:
>  - it is used only in some internal/vendor trees, not in open-source
>  - we want it to have sane and potentially extensible userspace API
>  - but we don't want to loose its functionality
>
> 2. we want to have simple API for performing single image processing
> operation:
>  - typically it will be used by compositing window manager, this means 
> that
>some parameters of the processing might change on each vblank (like
>destination rectangle for example). This api allows such change on each
>operation 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Inki Dae]

Hi Tomasz,

2017년 05월 10일 14:38에 Tomasz Figa 이(가) 쓴 글:
> Hi Everyone,
> 
> On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:
>>
>>
>> 2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
>>> Hi Marek,
>>>
>>> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
 Hi Laurent,

 On 2017-04-20 12:25, Laurent Pinchart wrote:
> Hi Marek,
>
> (CC'ing Sakari Ailus)
>
> Thank you for the patches.
>
> On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
>> Dear all,
>>
>> This is an updated proposal for extending EXYNOS DRM API with generic
>> support for hardware modules, which can be used for processing image data
> >from the one memory buffer to another. Typical memory-to-memory 
> >operations
>> are: rotation, scaling, colour space conversion or mix of them. This is a
>> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
>> processors", which has been rejected as "not really needed in the DRM
>> core":
>> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
>>
>> In this proposal I moved all the code to Exynos DRM driver, so now this
>> will be specific only to Exynos DRM. I've also changed the name from
>> framebuffer processor (fbproc) to picture processor (pp) to avoid 
>> confusion
>> with fbdev API.
>>
>> Here is a bit more information what picture processors are:
>>
>> Embedded SoCs are known to have a number of hardware blocks, which 
>> perform
>> such operations. They can be used in paralel to the main GPU module to
>> offload CPU from processing grapics or video data. One of example use of
>> such modules is implementing video overlay, which usually requires color
>> space conversion from NV12 (or similar) to RGB32 color space and scaling 
>> to
>> target window size.
>>
>> The proposed API is heavily inspired by atomic KMS approach - it is also
>> based on DRM objects and their properties. A new DRM object is 
>> introduced:
>> picture processor (called pp for convenience). Such objects have a set of
>> standard DRM properties, which describes the operation to be performed by
>> respective hardware module. In typical case those properties are a source
>> fb id and rectangle (x, y, width, height) and destination fb id and
>> rectangle. Optionally a rotation property can be also specified if
>> supported by the given hardware. To perform an operation on image data,
>> userspace provides a set of properties and their values for given fbproc
>> object in a similar way as object and properties are provided for
>> performing atomic page flip / mode setting.
>>
>> The proposed API consists of the 3 new ioctls:
>> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
>>   processors,
>> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
>>   processor,
>> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
>>   property set.
>>
>> The proposed API is extensible. Drivers can attach their own, custom
>> properties to add support for more advanced picture processing (for 
>> example
>> blending).
>>
>> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
>> subsystem. IPP API is over-engineered in general, but not really 
>> extensible
>> on the other side. It is also buggy, with significant design flaws - the
>> biggest issue is the fact that the API covers memory-2-memory picture
>> operations together with CRTC writeback and duplicating features, which
>> belongs to video plane. Comparing with IPP subsystem, the PP framework is
>> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
>> rotator driver smaller by over 200 lines).
> This seems to be the kind of hardware that is typically supported by V4L2.
> Stupid question, why DRM ?

 Let me elaborate a bit on the reasons for implementing it in Exynos DRM:

 1. we want to replace existing Exynos IPP subsystem:
  - it is used only in some internal/vendor trees, not in open-source
  - we want it to have sane and potentially extensible userspace API
  - but we don't want to loose its functionality

 2. we want to have simple API for performing single image processing
 operation:
  - typically it will be used by compositing window manager, this means that
some parameters of the processing might change on each vblank (like
destination rectangle for example). This api allows such change on each
operation without any additional cost. V4L2 requires to reinitialize
queues with new configuration on such change, what means that a bunch of
ioctls has to be called.
>>>
>>> What do you mean by re-initialising the queue? Format, buffers or 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Tomasz Figa]

Hi Everyone,

On Wed, May 10, 2017 at 9:24 AM, Inki Dae  wrote:
>
>
> 2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
>> Hi Marek,
>>
>> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>>> Hi Laurent,
>>>
>>> On 2017-04-20 12:25, Laurent Pinchart wrote:
 Hi Marek,

 (CC'ing Sakari Ailus)

 Thank you for the patches.

 On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> Dear all,
>
> This is an updated proposal for extending EXYNOS DRM API with generic
> support for hardware modules, which can be used for processing image data
 >from the one memory buffer to another. Typical memory-to-memory operations
> are: rotation, scaling, colour space conversion or mix of them. This is a
> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
> processors", which has been rejected as "not really needed in the DRM
> core":
> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
>
> In this proposal I moved all the code to Exynos DRM driver, so now this
> will be specific only to Exynos DRM. I've also changed the name from
> framebuffer processor (fbproc) to picture processor (pp) to avoid 
> confusion
> with fbdev API.
>
> Here is a bit more information what picture processors are:
>
> Embedded SoCs are known to have a number of hardware blocks, which perform
> such operations. They can be used in paralel to the main GPU module to
> offload CPU from processing grapics or video data. One of example use of
> such modules is implementing video overlay, which usually requires color
> space conversion from NV12 (or similar) to RGB32 color space and scaling 
> to
> target window size.
>
> The proposed API is heavily inspired by atomic KMS approach - it is also
> based on DRM objects and their properties. A new DRM object is introduced:
> picture processor (called pp for convenience). Such objects have a set of
> standard DRM properties, which describes the operation to be performed by
> respective hardware module. In typical case those properties are a source
> fb id and rectangle (x, y, width, height) and destination fb id and
> rectangle. Optionally a rotation property can be also specified if
> supported by the given hardware. To perform an operation on image data,
> userspace provides a set of properties and their values for given fbproc
> object in a similar way as object and properties are provided for
> performing atomic page flip / mode setting.
>
> The proposed API consists of the 3 new ioctls:
> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
>   processors,
> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
>   processor,
> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
>   property set.
>
> The proposed API is extensible. Drivers can attach their own, custom
> properties to add support for more advanced picture processing (for 
> example
> blending).
>
> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> subsystem. IPP API is over-engineered in general, but not really 
> extensible
> on the other side. It is also buggy, with significant design flaws - the
> biggest issue is the fact that the API covers memory-2-memory picture
> operations together with CRTC writeback and duplicating features, which
> belongs to video plane. Comparing with IPP subsystem, the PP framework is
> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> rotator driver smaller by over 200 lines).
 This seems to be the kind of hardware that is typically supported by V4L2.
 Stupid question, why DRM ?
>>>
>>> Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
>>>
>>> 1. we want to replace existing Exynos IPP subsystem:
>>>  - it is used only in some internal/vendor trees, not in open-source
>>>  - we want it to have sane and potentially extensible userspace API
>>>  - but we don't want to loose its functionality
>>>
>>> 2. we want to have simple API for performing single image processing
>>> operation:
>>>  - typically it will be used by compositing window manager, this means that
>>>some parameters of the processing might change on each vblank (like
>>>destination rectangle for example). This api allows such change on each
>>>operation without any additional cost. V4L2 requires to reinitialize
>>>queues with new configuration on such change, what means that a bunch of
>>>ioctls has to be called.
>>
>> What do you mean by re-initialising the queue? Format, buffers or something
>> else?
>>
>> If you need a larger buffer than what you have already allocated, you'll
>> need to re-allocate, V4L2 or not.
>>
>> We also do lack a way to destroy individual 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Inki Dae]

2017년 04월 26일 07:21에 Sakari Ailus 이(가) 쓴 글:
> Hi Marek,
> 
> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>> Hi Laurent,
>>
>> On 2017-04-20 12:25, Laurent Pinchart wrote:
>>> Hi Marek,
>>>
>>> (CC'ing Sakari Ailus)
>>>
>>> Thank you for the patches.
>>>
>>> On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
 Dear all,

 This is an updated proposal for extending EXYNOS DRM API with generic
 support for hardware modules, which can be used for processing image data
>>> >from the one memory buffer to another. Typical memory-to-memory operations
 are: rotation, scaling, colour space conversion or mix of them. This is a
 follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
 processors", which has been rejected as "not really needed in the DRM
 core":
 http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html

 In this proposal I moved all the code to Exynos DRM driver, so now this
 will be specific only to Exynos DRM. I've also changed the name from
 framebuffer processor (fbproc) to picture processor (pp) to avoid confusion
 with fbdev API.

 Here is a bit more information what picture processors are:

 Embedded SoCs are known to have a number of hardware blocks, which perform
 such operations. They can be used in paralel to the main GPU module to
 offload CPU from processing grapics or video data. One of example use of
 such modules is implementing video overlay, which usually requires color
 space conversion from NV12 (or similar) to RGB32 color space and scaling to
 target window size.

 The proposed API is heavily inspired by atomic KMS approach - it is also
 based on DRM objects and their properties. A new DRM object is introduced:
 picture processor (called pp for convenience). Such objects have a set of
 standard DRM properties, which describes the operation to be performed by
 respective hardware module. In typical case those properties are a source
 fb id and rectangle (x, y, width, height) and destination fb id and
 rectangle. Optionally a rotation property can be also specified if
 supported by the given hardware. To perform an operation on image data,
 userspace provides a set of properties and their values for given fbproc
 object in a similar way as object and properties are provided for
 performing atomic page flip / mode setting.

 The proposed API consists of the 3 new ioctls:
 - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
   processors,
 - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
   processor,
 - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
   property set.

 The proposed API is extensible. Drivers can attach their own, custom
 properties to add support for more advanced picture processing (for example
 blending).

 This proposal aims to replace Exynos DRM IPP (Image Post Processing)
 subsystem. IPP API is over-engineered in general, but not really extensible
 on the other side. It is also buggy, with significant design flaws - the
 biggest issue is the fact that the API covers memory-2-memory picture
 operations together with CRTC writeback and duplicating features, which
 belongs to video plane. Comparing with IPP subsystem, the PP framework is
 smaller (1807 vs 778 lines) and allows driver simplification (Exynos
 rotator driver smaller by over 200 lines).
>>> This seems to be the kind of hardware that is typically supported by V4L2.
>>> Stupid question, why DRM ?
>>
>> Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
>>
>> 1. we want to replace existing Exynos IPP subsystem:
>>  - it is used only in some internal/vendor trees, not in open-source
>>  - we want it to have sane and potentially extensible userspace API
>>  - but we don't want to loose its functionality
>>
>> 2. we want to have simple API for performing single image processing
>> operation:
>>  - typically it will be used by compositing window manager, this means that
>>some parameters of the processing might change on each vblank (like
>>destination rectangle for example). This api allows such change on each
>>operation without any additional cost. V4L2 requires to reinitialize
>>queues with new configuration on such change, what means that a bunch of
>>ioctls has to be called.
> 
> What do you mean by re-initialising the queue? Format, buffers or something
> else?
> 
> If you need a larger buffer than what you have already allocated, you'll
> need to re-allocate, V4L2 or not.
> 
> We also do lack a way to destroy individual buffers in V4L2. It'd be up to
> implementing that and some work in videobuf2.
> 
> Another thing is that V4L2 is very stream oriented. For most devices that's
> fine as a lot of the parameters are not 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Marek Szyprowski]

Hi Tobias and Nicolas,

On 2017-04-26 17:16, Tobias Jakobi wrote:

Nicolas Dufresne wrote:

Le mercredi 26 avril 2017 à 01:21 +0300, Sakari Ailus a écrit :

Hi Marek,

On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:

Hi Laurent,

On 2017-04-20 12:25, Laurent Pinchart wrote:

Hi Marek,

(CC'ing Sakari Ailus)

Thank you for the patches.

On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:

Dear all,

This is an updated proposal for extending EXYNOS DRM API with generic
support for hardware modules, which can be used for processing image data
from the one memory buffer to another. Typical memory-to-memory operations
are: rotation, scaling, colour space conversion or mix of them. This is a
follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
processors", which has been rejected as "not really needed in the DRM
core":
http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html

In this proposal I moved all the code to Exynos DRM driver, so now this
will be specific only to Exynos DRM. I've also changed the name from
framebuffer processor (fbproc) to picture processor (pp) to avoid confusion
with fbdev API.

Here is a bit more information what picture processors are:

Embedded SoCs are known to have a number of hardware blocks, which perform
such operations. They can be used in paralel to the main GPU module to
offload CPU from processing grapics or video data. One of example use of
such modules is implementing video overlay, which usually requires color
space conversion from NV12 (or similar) to RGB32 color space and scaling to
target window size.

The proposed API is heavily inspired by atomic KMS approach - it is also
based on DRM objects and their properties. A new DRM object is introduced:
picture processor (called pp for convenience). Such objects have a set of
standard DRM properties, which describes the operation to be performed by
respective hardware module. In typical case those properties are a source
fb id and rectangle (x, y, width, height) and destination fb id and
rectangle. Optionally a rotation property can be also specified if
supported by the given hardware. To perform an operation on image data,
userspace provides a set of properties and their values for given fbproc
object in a similar way as object and properties are provided for
performing atomic page flip / mode setting.

The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
   processors,
- DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
   processor,
- DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
   property set.

The proposed API is extensible. Drivers can attach their own, custom
properties to add support for more advanced picture processing (for example
blending).

This proposal aims to replace Exynos DRM IPP (Image Post Processing)
subsystem. IPP API is over-engineered in general, but not really extensible
on the other side. It is also buggy, with significant design flaws - the
biggest issue is the fact that the API covers memory-2-memory picture
operations together with CRTC writeback and duplicating features, which
belongs to video plane. Comparing with IPP subsystem, the PP framework is
smaller (1807 vs 778 lines) and allows driver simplification (Exynos
rotator driver smaller by over 200 lines).

Just a side note, we have written code in GStreamer using the Exnynos 4
FIMC IPP driver. I don't know how many, if any, deployment still exist
(Exynos 4 is relatively old now), but there exist userspace for the
FIMC driver. We use this for color transformation (from tiled to
linear) and scaling. The FIMC driver is in fact quite stable in
upstream kernel today. The GScaler V4L2 M2M driver on Exynos 5 is
largely based on it and has received some maintenance to properly work
in GStreamer. unlike this DRM API, you can reuse the same userspace
code across multiple platforms (which we do already). We have also
integrated this driver in Chromium in the past (not upstream though).

I am well aware that the blitter driver has not got much attention
though. But again, V4L2 offers a generic interface to userspace
application. Fixing this driver could enable some work like this one:

https://bugzilla.gnome.org/show_bug.cgi?id=772766

This work in progress feature is a generic hardware accelerated video
mixer. It has been tested with IMX.6 v4l2 m2m blitter driver (which I
believe is in staging right now). Again, unlike the exynos/drm, this
code could be reused between platforms.

In general, the problem with the DRM approach is that it only targets
displays. We often need to use these IP block for stream pre/post
processing outside a "playback" use case.

just a short note that this is not true. You can use all this
functionality e.g. through render nodes, without needing to have a
display attached to your system.


Yes. As an alternative I also plan to provide generic V4L2-style mem2mem

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Marek Szyprowski]

Hi Sakari,

On 2017-04-26 00:21, Sakari Ailus wrote:

Hi Marek,

On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:

Hi Laurent,

On 2017-04-20 12:25, Laurent Pinchart wrote:

Hi Marek,

(CC'ing Sakari Ailus)

Thank you for the patches.

On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:

Dear all,

This is an updated proposal for extending EXYNOS DRM API with generic
support for hardware modules, which can be used for processing image data

>from the one memory buffer to another. Typical memory-to-memory operations

are: rotation, scaling, colour space conversion or mix of them. This is a
follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
processors", which has been rejected as "not really needed in the DRM
core":
http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html

In this proposal I moved all the code to Exynos DRM driver, so now this
will be specific only to Exynos DRM. I've also changed the name from
framebuffer processor (fbproc) to picture processor (pp) to avoid confusion
with fbdev API.

Here is a bit more information what picture processors are:

Embedded SoCs are known to have a number of hardware blocks, which perform
such operations. They can be used in paralel to the main GPU module to
offload CPU from processing grapics or video data. One of example use of
such modules is implementing video overlay, which usually requires color
space conversion from NV12 (or similar) to RGB32 color space and scaling to
target window size.

The proposed API is heavily inspired by atomic KMS approach - it is also
based on DRM objects and their properties. A new DRM object is introduced:
picture processor (called pp for convenience). Such objects have a set of
standard DRM properties, which describes the operation to be performed by
respective hardware module. In typical case those properties are a source
fb id and rectangle (x, y, width, height) and destination fb id and
rectangle. Optionally a rotation property can be also specified if
supported by the given hardware. To perform an operation on image data,
userspace provides a set of properties and their values for given fbproc
object in a similar way as object and properties are provided for
performing atomic page flip / mode setting.

The proposed API consists of the 3 new ioctls:
- DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
   processors,
- DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
   processor,
- DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
   property set.

The proposed API is extensible. Drivers can attach their own, custom
properties to add support for more advanced picture processing (for example
blending).

This proposal aims to replace Exynos DRM IPP (Image Post Processing)
subsystem. IPP API is over-engineered in general, but not really extensible
on the other side. It is also buggy, with significant design flaws - the
biggest issue is the fact that the API covers memory-2-memory picture
operations together with CRTC writeback and duplicating features, which
belongs to video plane. Comparing with IPP subsystem, the PP framework is
smaller (1807 vs 778 lines) and allows driver simplification (Exynos
rotator driver smaller by over 200 lines).

This seems to be the kind of hardware that is typically supported by V4L2.
Stupid question, why DRM ?

Let me elaborate a bit on the reasons for implementing it in Exynos DRM:

1. we want to replace existing Exynos IPP subsystem:
  - it is used only in some internal/vendor trees, not in open-source
  - we want it to have sane and potentially extensible userspace API
  - but we don't want to loose its functionality

2. we want to have simple API for performing single image processing
operation:
  - typically it will be used by compositing window manager, this means that
some parameters of the processing might change on each vblank (like
destination rectangle for example). This api allows such change on each
operation without any additional cost. V4L2 requires to reinitialize
queues with new configuration on such change, what means that a bunch of
ioctls has to be called.

What do you mean by re-initialising the queue? Format, buffers or something
else?


In case of compositor use case, the parameter that is being changed most
frequently is source and/or destination rectangle position and/or size.


If you need a larger buffer than what you have already allocated, you'll
need to re-allocate, V4L2 or not.

We also do lack a way to destroy individual buffers in V4L2. It'd be up to
implementing that and some work in videobuf2.


Well if we would use V4l2, buffers will always come as dmabuf objects. There
is a hard limit of the number of buffers that can be imported to v4l2/vb2
queue to get buffer ids. This also limits easy processing of the buffers
in the compositor, because you would need to reinitialize the v4l2 queues
to get new set of v4l2/vb2 buffer ids.

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Nicolas Dufresne]

Le mercredi 26 avril 2017 à 21:31 +0200, Tobias Jakobi a écrit :
> I'm pretty sure you have misread Marek's description of the patchset.
> The picture processor API should replaced/deprecate the IPP API that is
> currently implemented in the Exynos DRM.
> 
> In particular this affects the following files:
> - drivers/gpu/drm/exynos/exynos_drm_ipp.{c,h}
> - drivers/gpu/drm/exynos/exynos_drm_fimc.{c,h}
> - drivers/gpu/drm/exynos/exynos_drm_gsc.{c,h}
> - drivers/gpu/drm/exynos/exynos_drm_rotator.{c,h}
> 
> I know only two places where the IPP API is actually used. Tizen and my
> experimental mpv backend.

Sorry for the noise then.

regards,
Nicolas

signature.asc
Description: This is a digitally signed message part

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Tobias Jakobi]

Hey,

Nicolas Dufresne wrote:
> Le mercredi 26 avril 2017 à 18:52 +0200, Tobias Jakobi a écrit :
>> Hello again,
>>
>>
>> Nicolas Dufresne wrote:
>>> Le mercredi 26 avril 2017 à 01:21 +0300, Sakari Ailus a écrit :
 Hi Marek,

 On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
> Hi Laurent,
>
> On 2017-04-20 12:25, Laurent Pinchart wrote:
>> Hi Marek,
>>
>> (CC'ing Sakari Ailus)
>>
>> Thank you for the patches.
>>
>> On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
>>> Dear all,
>>>
>>> This is an updated proposal for extending EXYNOS DRM API with generic
>>> support for hardware modules, which can be used for processing image 
>>> data
>>> from the one memory buffer to another. Typical memory-to-memory 
>>> operations
>>> are: rotation, scaling, colour space conversion or mix of them. This is 
>>> a
>>> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
>>> processors", which has been rejected as "not really needed in the DRM
>>> core":
>>> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
>>>
>>> In this proposal I moved all the code to Exynos DRM driver, so now this
>>> will be specific only to Exynos DRM. I've also changed the name from
>>> framebuffer processor (fbproc) to picture processor (pp) to avoid 
>>> confusion
>>> with fbdev API.
>>>
>>> Here is a bit more information what picture processors are:
>>>
>>> Embedded SoCs are known to have a number of hardware blocks, which 
>>> perform
>>> such operations. They can be used in paralel to the main GPU module to
>>> offload CPU from processing grapics or video data. One of example use of
>>> such modules is implementing video overlay, which usually requires color
>>> space conversion from NV12 (or similar) to RGB32 color space and 
>>> scaling to
>>> target window size.
>>>
>>> The proposed API is heavily inspired by atomic KMS approach - it is also
>>> based on DRM objects and their properties. A new DRM object is 
>>> introduced:
>>> picture processor (called pp for convenience). Such objects have a set 
>>> of
>>> standard DRM properties, which describes the operation to be performed 
>>> by
>>> respective hardware module. In typical case those properties are a 
>>> source
>>> fb id and rectangle (x, y, width, height) and destination fb id and
>>> rectangle. Optionally a rotation property can be also specified if
>>> supported by the given hardware. To perform an operation on image data,
>>> userspace provides a set of properties and their values for given fbproc
>>> object in a similar way as object and properties are provided for
>>> performing atomic page flip / mode setting.
>>>
>>> The proposed API consists of the 3 new ioctls:
>>> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
>>>   processors,
>>> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
>>>   processor,
>>> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
>>>   property set.
>>>
>>> The proposed API is extensible. Drivers can attach their own, custom
>>> properties to add support for more advanced picture processing (for 
>>> example
>>> blending).
>>>
>>> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
>>> subsystem. IPP API is over-engineered in general, but not really 
>>> extensible
>>> on the other side. It is also buggy, with significant design flaws - the
>>> biggest issue is the fact that the API covers memory-2-memory picture
>>> operations together with CRTC writeback and duplicating features, which
>>> belongs to video plane. Comparing with IPP subsystem, the PP framework 
>>> is
>>> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
>>> rotator driver smaller by over 200 lines).
>>>
>>> Just a side note, we have written code in GStreamer using the Exnynos 4
>>> FIMC IPP driver. I don't know how many, if any, deployment still exist
>>> (Exynos 4 is relatively old now), but there exist userspace for the
>>> FIMC driver.
>>
>> I was searching for this code, but I didn't find anything. Are you sure
>> you really mean the FIMC IPP in Exynos DRM, and not just the FIMC driver
>> from the V4L2 subsystem?
> 
> Oops, I manage to be unclear. Having two drivers on the same IP isn't
> helping. We wrote code around the FIMC driver on V4L2 side. This
> driver:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/exynos4-is/fimc-m2m.c
> 
> And this code:
> 
> https://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/sys/v4l2/gstv4l2transform.c
> 
> Unless I have miss-read, the proposal here is to deprecate the V4L side
> and improve the DRM 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Nicolas Dufresne]

Le mercredi 26 avril 2017 à 18:52 +0200, Tobias Jakobi a écrit :
> Hello again,
> 
> 
> Nicolas Dufresne wrote:
> > Le mercredi 26 avril 2017 à 01:21 +0300, Sakari Ailus a écrit :
> > > Hi Marek,
> > > 
> > > On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
> > > > Hi Laurent,
> > > > 
> > > > On 2017-04-20 12:25, Laurent Pinchart wrote:
> > > > > Hi Marek,
> > > > > 
> > > > > (CC'ing Sakari Ailus)
> > > > > 
> > > > > Thank you for the patches.
> > > > > 
> > > > > On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> > > > > > Dear all,
> > > > > > 
> > > > > > This is an updated proposal for extending EXYNOS DRM API with 
> > > > > > generic
> > > > > > support for hardware modules, which can be used for processing 
> > > > > > image data
> > > > > > from the one memory buffer to another. Typical memory-to-memory 
> > > > > > operations
> > > > > > are: rotation, scaling, colour space conversion or mix of them. 
> > > > > > This is a
> > > > > > follow-up of my previous proposal "[RFC 0/2] New feature: 
> > > > > > Framebuffer
> > > > > > processors", which has been rejected as "not really needed in the 
> > > > > > DRM
> > > > > > core":
> > > > > > http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
> > > > > > 
> > > > > > In this proposal I moved all the code to Exynos DRM driver, so now 
> > > > > > this
> > > > > > will be specific only to Exynos DRM. I've also changed the name from
> > > > > > framebuffer processor (fbproc) to picture processor (pp) to avoid 
> > > > > > confusion
> > > > > > with fbdev API.
> > > > > > 
> > > > > > Here is a bit more information what picture processors are:
> > > > > > 
> > > > > > Embedded SoCs are known to have a number of hardware blocks, which 
> > > > > > perform
> > > > > > such operations. They can be used in paralel to the main GPU module 
> > > > > > to
> > > > > > offload CPU from processing grapics or video data. One of example 
> > > > > > use of
> > > > > > such modules is implementing video overlay, which usually requires 
> > > > > > color
> > > > > > space conversion from NV12 (or similar) to RGB32 color space and 
> > > > > > scaling to
> > > > > > target window size.
> > > > > > 
> > > > > > The proposed API is heavily inspired by atomic KMS approach - it is 
> > > > > > also
> > > > > > based on DRM objects and their properties. A new DRM object is 
> > > > > > introduced:
> > > > > > picture processor (called pp for convenience). Such objects have a 
> > > > > > set of
> > > > > > standard DRM properties, which describes the operation to be 
> > > > > > performed by
> > > > > > respective hardware module. In typical case those properties are a 
> > > > > > source
> > > > > > fb id and rectangle (x, y, width, height) and destination fb id and
> > > > > > rectangle. Optionally a rotation property can be also specified if
> > > > > > supported by the given hardware. To perform an operation on image 
> > > > > > data,
> > > > > > userspace provides a set of properties and their values for given 
> > > > > > fbproc
> > > > > > object in a similar way as object and properties are provided for
> > > > > > performing atomic page flip / mode setting.
> > > > > > 
> > > > > > The proposed API consists of the 3 new ioctls:
> > > > > > - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available 
> > > > > > picture
> > > > > >   processors,
> > > > > > - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
> > > > > >   processor,
> > > > > > - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by 
> > > > > > given
> > > > > >   property set.
> > > > > > 
> > > > > > The proposed API is extensible. Drivers can attach their own, custom
> > > > > > properties to add support for more advanced picture processing (for 
> > > > > > example
> > > > > > blending).
> > > > > > 
> > > > > > This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> > > > > > subsystem. IPP API is over-engineered in general, but not really 
> > > > > > extensible
> > > > > > on the other side. It is also buggy, with significant design flaws 
> > > > > > - the
> > > > > > biggest issue is the fact that the API covers memory-2-memory 
> > > > > > picture
> > > > > > operations together with CRTC writeback and duplicating features, 
> > > > > > which
> > > > > > belongs to video plane. Comparing with IPP subsystem, the PP 
> > > > > > framework is
> > > > > > smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> > > > > > rotator driver smaller by over 200 lines).
> > 
> > Just a side note, we have written code in GStreamer using the Exnynos 4
> > FIMC IPP driver. I don't know how many, if any, deployment still exist
> > (Exynos 4 is relatively old now), but there exist userspace for the
> > FIMC driver.
> 
> I was searching for this code, but I didn't find anything. Are you sure
> you really mean the FIMC IPP in Exynos DRM, and not just the FIMC driver
> from the V4L2 subsystem?

Oops, 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Tobias Jakobi]

Hello again,


Nicolas Dufresne wrote:
> Le mercredi 26 avril 2017 à 01:21 +0300, Sakari Ailus a écrit :
>> Hi Marek,
>>
>> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>>> Hi Laurent,
>>>
>>> On 2017-04-20 12:25, Laurent Pinchart wrote:
 Hi Marek,

 (CC'ing Sakari Ailus)

 Thank you for the patches.

 On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> Dear all,
>
> This is an updated proposal for extending EXYNOS DRM API with generic
> support for hardware modules, which can be used for processing image data
> from the one memory buffer to another. Typical memory-to-memory operations
> are: rotation, scaling, colour space conversion or mix of them. This is a
> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
> processors", which has been rejected as "not really needed in the DRM
> core":
> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
>
> In this proposal I moved all the code to Exynos DRM driver, so now this
> will be specific only to Exynos DRM. I've also changed the name from
> framebuffer processor (fbproc) to picture processor (pp) to avoid 
> confusion
> with fbdev API.
>
> Here is a bit more information what picture processors are:
>
> Embedded SoCs are known to have a number of hardware blocks, which perform
> such operations. They can be used in paralel to the main GPU module to
> offload CPU from processing grapics or video data. One of example use of
> such modules is implementing video overlay, which usually requires color
> space conversion from NV12 (or similar) to RGB32 color space and scaling 
> to
> target window size.
>
> The proposed API is heavily inspired by atomic KMS approach - it is also
> based on DRM objects and their properties. A new DRM object is introduced:
> picture processor (called pp for convenience). Such objects have a set of
> standard DRM properties, which describes the operation to be performed by
> respective hardware module. In typical case those properties are a source
> fb id and rectangle (x, y, width, height) and destination fb id and
> rectangle. Optionally a rotation property can be also specified if
> supported by the given hardware. To perform an operation on image data,
> userspace provides a set of properties and their values for given fbproc
> object in a similar way as object and properties are provided for
> performing atomic page flip / mode setting.
>
> The proposed API consists of the 3 new ioctls:
> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
>   processors,
> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
>   processor,
> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
>   property set.
>
> The proposed API is extensible. Drivers can attach their own, custom
> properties to add support for more advanced picture processing (for 
> example
> blending).
>
> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> subsystem. IPP API is over-engineered in general, but not really 
> extensible
> on the other side. It is also buggy, with significant design flaws - the
> biggest issue is the fact that the API covers memory-2-memory picture
> operations together with CRTC writeback and duplicating features, which
> belongs to video plane. Comparing with IPP subsystem, the PP framework is
> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> rotator driver smaller by over 200 lines).
> 
> Just a side note, we have written code in GStreamer using the Exnynos 4
> FIMC IPP driver. I don't know how many, if any, deployment still exist
> (Exynos 4 is relatively old now), but there exist userspace for the
> FIMC driver.
I was searching for this code, but I didn't find anything. Are you sure
you really mean the FIMC IPP in Exynos DRM, and not just the FIMC driver
from the V4L2 subsystem?


With best wishes,
Tobias



> We use this for color transformation (from tiled to
> linear) and scaling. The FIMC driver is in fact quite stable in
> upstream kernel today. The GScaler V4L2 M2M driver on Exynos 5 is
> largely based on it and has received some maintenance to properly work
> in GStreamer. unlike this DRM API, you can reuse the same userspace
> code across multiple platforms (which we do already). We have also
> integrated this driver in Chromium in the past (not upstream though).
> 
> I am well aware that the blitter driver has not got much attention
> though. But again, V4L2 offers a generic interface to userspace
> application. Fixing this driver could enable some work like this one:
> 
> https://bugzilla.gnome.org/show_bug.cgi?id=772766
> 
> This work in progress feature is a generic hardware accelerated video
> 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Tobias Jakobi]

Hello everyone,


Nicolas Dufresne wrote:
> Le mercredi 26 avril 2017 à 01:21 +0300, Sakari Ailus a écrit :
>> Hi Marek,
>>
>> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
>>> Hi Laurent,
>>>
>>> On 2017-04-20 12:25, Laurent Pinchart wrote:
 Hi Marek,

 (CC'ing Sakari Ailus)

 Thank you for the patches.

 On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> Dear all,
>
> This is an updated proposal for extending EXYNOS DRM API with generic
> support for hardware modules, which can be used for processing image data
> from the one memory buffer to another. Typical memory-to-memory operations
> are: rotation, scaling, colour space conversion or mix of them. This is a
> follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
> processors", which has been rejected as "not really needed in the DRM
> core":
> http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
>
> In this proposal I moved all the code to Exynos DRM driver, so now this
> will be specific only to Exynos DRM. I've also changed the name from
> framebuffer processor (fbproc) to picture processor (pp) to avoid 
> confusion
> with fbdev API.
>
> Here is a bit more information what picture processors are:
>
> Embedded SoCs are known to have a number of hardware blocks, which perform
> such operations. They can be used in paralel to the main GPU module to
> offload CPU from processing grapics or video data. One of example use of
> such modules is implementing video overlay, which usually requires color
> space conversion from NV12 (or similar) to RGB32 color space and scaling 
> to
> target window size.
>
> The proposed API is heavily inspired by atomic KMS approach - it is also
> based on DRM objects and their properties. A new DRM object is introduced:
> picture processor (called pp for convenience). Such objects have a set of
> standard DRM properties, which describes the operation to be performed by
> respective hardware module. In typical case those properties are a source
> fb id and rectangle (x, y, width, height) and destination fb id and
> rectangle. Optionally a rotation property can be also specified if
> supported by the given hardware. To perform an operation on image data,
> userspace provides a set of properties and their values for given fbproc
> object in a similar way as object and properties are provided for
> performing atomic page flip / mode setting.
>
> The proposed API consists of the 3 new ioctls:
> - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
>   processors,
> - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
>   processor,
> - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
>   property set.
>
> The proposed API is extensible. Drivers can attach their own, custom
> properties to add support for more advanced picture processing (for 
> example
> blending).
>
> This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> subsystem. IPP API is over-engineered in general, but not really 
> extensible
> on the other side. It is also buggy, with significant design flaws - the
> biggest issue is the fact that the API covers memory-2-memory picture
> operations together with CRTC writeback and duplicating features, which
> belongs to video plane. Comparing with IPP subsystem, the PP framework is
> smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> rotator driver smaller by over 200 lines).
> 
> Just a side note, we have written code in GStreamer using the Exnynos 4
> FIMC IPP driver. I don't know how many, if any, deployment still exist
> (Exynos 4 is relatively old now), but there exist userspace for the
> FIMC driver. We use this for color transformation (from tiled to
> linear) and scaling. The FIMC driver is in fact quite stable in
> upstream kernel today. The GScaler V4L2 M2M driver on Exynos 5 is
> largely based on it and has received some maintenance to properly work
> in GStreamer. unlike this DRM API, you can reuse the same userspace
> code across multiple platforms (which we do already). We have also
> integrated this driver in Chromium in the past (not upstream though).
> 
> I am well aware that the blitter driver has not got much attention
> though. But again, V4L2 offers a generic interface to userspace
> application. Fixing this driver could enable some work like this one:
> 
> https://bugzilla.gnome.org/show_bug.cgi?id=772766
> 
> This work in progress feature is a generic hardware accelerated video
> mixer. It has been tested with IMX.6 v4l2 m2m blitter driver (which I
> believe is in staging right now). Again, unlike the exynos/drm, this
> code could be reused between platforms.
> 
> In general, 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Nicolas Dufresne]

Le mercredi 26 avril 2017 à 01:21 +0300, Sakari Ailus a écrit :
> Hi Marek,
> 
> On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
> > Hi Laurent,
> > 
> > On 2017-04-20 12:25, Laurent Pinchart wrote:
> > > Hi Marek,
> > > 
> > > (CC'ing Sakari Ailus)
> > > 
> > > Thank you for the patches.
> > > 
> > > On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> > > > Dear all,
> > > > 
> > > > This is an updated proposal for extending EXYNOS DRM API with generic
> > > > support for hardware modules, which can be used for processing image 
> > > > data
> > > > from the one memory buffer to another. Typical memory-to-memory 
> > > > operations
> > > > are: rotation, scaling, colour space conversion or mix of them. This is 
> > > > a
> > > > follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
> > > > processors", which has been rejected as "not really needed in the DRM
> > > > core":
> > > > http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
> > > > 
> > > > In this proposal I moved all the code to Exynos DRM driver, so now this
> > > > will be specific only to Exynos DRM. I've also changed the name from
> > > > framebuffer processor (fbproc) to picture processor (pp) to avoid 
> > > > confusion
> > > > with fbdev API.
> > > > 
> > > > Here is a bit more information what picture processors are:
> > > > 
> > > > Embedded SoCs are known to have a number of hardware blocks, which 
> > > > perform
> > > > such operations. They can be used in paralel to the main GPU module to
> > > > offload CPU from processing grapics or video data. One of example use of
> > > > such modules is implementing video overlay, which usually requires color
> > > > space conversion from NV12 (or similar) to RGB32 color space and 
> > > > scaling to
> > > > target window size.
> > > > 
> > > > The proposed API is heavily inspired by atomic KMS approach - it is also
> > > > based on DRM objects and their properties. A new DRM object is 
> > > > introduced:
> > > > picture processor (called pp for convenience). Such objects have a set 
> > > > of
> > > > standard DRM properties, which describes the operation to be performed 
> > > > by
> > > > respective hardware module. In typical case those properties are a 
> > > > source
> > > > fb id and rectangle (x, y, width, height) and destination fb id and
> > > > rectangle. Optionally a rotation property can be also specified if
> > > > supported by the given hardware. To perform an operation on image data,
> > > > userspace provides a set of properties and their values for given fbproc
> > > > object in a similar way as object and properties are provided for
> > > > performing atomic page flip / mode setting.
> > > > 
> > > > The proposed API consists of the 3 new ioctls:
> > > > - DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
> > > >   processors,
> > > > - DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
> > > >   processor,
> > > > - DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
> > > >   property set.
> > > > 
> > > > The proposed API is extensible. Drivers can attach their own, custom
> > > > properties to add support for more advanced picture processing (for 
> > > > example
> > > > blending).
> > > > 
> > > > This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> > > > subsystem. IPP API is over-engineered in general, but not really 
> > > > extensible
> > > > on the other side. It is also buggy, with significant design flaws - the
> > > > biggest issue is the fact that the API covers memory-2-memory picture
> > > > operations together with CRTC writeback and duplicating features, which
> > > > belongs to video plane. Comparing with IPP subsystem, the PP framework 
> > > > is
> > > > smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> > > > rotator driver smaller by over 200 lines).

Just a side note, we have written code in GStreamer using the Exnynos 4
FIMC IPP driver. I don't know how many, if any, deployment still exist
(Exynos 4 is relatively old now), but there exist userspace for the
FIMC driver. We use this for color transformation (from tiled to
linear) and scaling. The FIMC driver is in fact quite stable in
upstream kernel today. The GScaler V4L2 M2M driver on Exynos 5 is
largely based on it and has received some maintenance to properly work
in GStreamer. unlike this DRM API, you can reuse the same userspace
code across multiple platforms (which we do already). We have also
integrated this driver in Chromium in the past (not upstream though).

I am well aware that the blitter driver has not got much attention
though. But again, V4L2 offers a generic interface to userspace
application. Fixing this driver could enable some work like this one:

https://bugzilla.gnome.org/show_bug.cgi?id=772766

This work in progress feature is a generic hardware accelerated video
mixer. It has been tested with IMX.6 v4l2 m2m blitter driver 

Re: [RFC 0/4] Exynos DRM: add Picture Processor extension

[Sakari Ailus]

Hi Marek,

On Thu, Apr 20, 2017 at 01:23:09PM +0200, Marek Szyprowski wrote:
> Hi Laurent,
> 
> On 2017-04-20 12:25, Laurent Pinchart wrote:
> >Hi Marek,
> >
> >(CC'ing Sakari Ailus)
> >
> >Thank you for the patches.
> >
> >On Thursday 20 Apr 2017 11:13:36 Marek Szyprowski wrote:
> >>Dear all,
> >>
> >>This is an updated proposal for extending EXYNOS DRM API with generic
> >>support for hardware modules, which can be used for processing image data
> >>from the one memory buffer to another. Typical memory-to-memory operations
> >>are: rotation, scaling, colour space conversion or mix of them. This is a
> >>follow-up of my previous proposal "[RFC 0/2] New feature: Framebuffer
> >>processors", which has been rejected as "not really needed in the DRM
> >>core":
> >>http://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg146286.html
> >>
> >>In this proposal I moved all the code to Exynos DRM driver, so now this
> >>will be specific only to Exynos DRM. I've also changed the name from
> >>framebuffer processor (fbproc) to picture processor (pp) to avoid confusion
> >>with fbdev API.
> >>
> >>Here is a bit more information what picture processors are:
> >>
> >>Embedded SoCs are known to have a number of hardware blocks, which perform
> >>such operations. They can be used in paralel to the main GPU module to
> >>offload CPU from processing grapics or video data. One of example use of
> >>such modules is implementing video overlay, which usually requires color
> >>space conversion from NV12 (or similar) to RGB32 color space and scaling to
> >>target window size.
> >>
> >>The proposed API is heavily inspired by atomic KMS approach - it is also
> >>based on DRM objects and their properties. A new DRM object is introduced:
> >>picture processor (called pp for convenience). Such objects have a set of
> >>standard DRM properties, which describes the operation to be performed by
> >>respective hardware module. In typical case those properties are a source
> >>fb id and rectangle (x, y, width, height) and destination fb id and
> >>rectangle. Optionally a rotation property can be also specified if
> >>supported by the given hardware. To perform an operation on image data,
> >>userspace provides a set of properties and their values for given fbproc
> >>object in a similar way as object and properties are provided for
> >>performing atomic page flip / mode setting.
> >>
> >>The proposed API consists of the 3 new ioctls:
> >>- DRM_IOCTL_EXYNOS_PP_GET_RESOURCES: to enumerate all available picture
> >>   processors,
> >>- DRM_IOCTL_EXYNOS_PP_GET: to query capabilities of given picture
> >>   processor,
> >>- DRM_IOCTL_EXYNOS_PP_COMMIT: to perform operation described by given
> >>   property set.
> >>
> >>The proposed API is extensible. Drivers can attach their own, custom
> >>properties to add support for more advanced picture processing (for example
> >>blending).
> >>
> >>This proposal aims to replace Exynos DRM IPP (Image Post Processing)
> >>subsystem. IPP API is over-engineered in general, but not really extensible
> >>on the other side. It is also buggy, with significant design flaws - the
> >>biggest issue is the fact that the API covers memory-2-memory picture
> >>operations together with CRTC writeback and duplicating features, which
> >>belongs to video plane. Comparing with IPP subsystem, the PP framework is
> >>smaller (1807 vs 778 lines) and allows driver simplification (Exynos
> >>rotator driver smaller by over 200 lines).
> >This seems to be the kind of hardware that is typically supported by V4L2.
> >Stupid question, why DRM ?
> 
> Let me elaborate a bit on the reasons for implementing it in Exynos DRM:
> 
> 1. we want to replace existing Exynos IPP subsystem:
>  - it is used only in some internal/vendor trees, not in open-source
>  - we want it to have sane and potentially extensible userspace API
>  - but we don't want to loose its functionality
> 
> 2. we want to have simple API for performing single image processing
> operation:
>  - typically it will be used by compositing window manager, this means that
>some parameters of the processing might change on each vblank (like
>destination rectangle for example). This api allows such change on each
>operation without any additional cost. V4L2 requires to reinitialize
>queues with new configuration on such change, what means that a bunch of
>ioctls has to be called.

What do you mean by re-initialising the queue? Format, buffers or something
else?

If you need a larger buffer than what you have already allocated, you'll
need to re-allocate, V4L2 or not.

We also do lack a way to destroy individual buffers in V4L2. It'd be up to
implementing that and some work in videobuf2.

Another thing is that V4L2 is very stream oriented. For most devices that's
fine as a lot of the parameters are not changeable during streaming,
especially if the pipeline is handled by multiple drivers. That said, for
devices that process data from memory to memory