RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Mauro and all,

> > Subject: RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> >
> > Hi Mauro,
> >
> > > > > Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> > > > >
> > > > > Hi,
> > > > >
> > > > > Em Fri, 17 Nov 2017 02:58:56 + "Mani, Rajmohan"
> > > > > <rajmohan.m...@intel.com> escreveu:
> > > > >
> > > > > > Here is an update on the IPU3 documentation that we are
> > > > > > currently working
> > > > > on.
> > > > > >
> > > > > > Image processing in IPU3 relies on the following.
> > > > > >
> > > > > > 1) HW configuration to enable ISP and
> > > > > > 2) setting customer specific 3A Tuning / Algorithm Parameters
> > > > > > to achieve
> > > > > desired image quality.
> > > > > >
> > > > > > We intend to provide documentation on ImgU driver programming
> > > > > > interface
> > > > > to help users of this driver to configure and enable ISP HW to
> > > > > meet their needs.  This documentation will include details on
> > > > > complete
> > > > > V4L2 Kernel driver interface and IO-Control parameters, except
> > > > > for the ISP internal algorithm and its parameters (which is
> > > > > Intel proprietary
> > IP).
> > > > >
> > > > > Sakari asked me to take a look on this thread, specifically on
> > > > > this email. I took a look on the other e-mails from this thread
> > > > > that are discussing about this IP block.
> > > > >
> > > > > I understand that Intel wants to keep their internal 3A
> > > > > algorithm protected, just like other vendors protect their own
> > > > > algos. It was never a requirement to open whatever algorithm are
> > > > > used inside a hardware (or firmware). The only requirement is
> > > > > that firmwares should be licensed with redistribution
> > > > > permission, ideally merged at linux-firmware
> > > > git tree.
> > > > >
> > > > > Yet, what I don't understand is why Intel also wants to also
> > > > > protect the interface for such 3A hardware/firmware algorithm.
> > > > > The parameters that are passed from an userspace application to
> > > > > Intel ISP logic doesn't contain the algorithm itself. What's the
> > > > > issue of documenting the meaning of each parameter?
> > > > >
> > > >
> > > > Thanks for looking into this.
> > > >
> > > > To achieve improved image quality using IPU3, 3A (Auto White
> > > > balance, Auto Focus and Auto Exposure) Tuning parameters specific
> > > > to a given camera sensor module, are converted to Intel ISP
> > > > algorithm parameters in user space camera HAL using AIC (Automatic
> > > > ISP
> > Configuration) library.
> > > >
> > > > As a unique design of Intel ISP, it exposes very detailed
> > > > algorithm parameters (~ 1 parameters) to configure ISP's image
> > > > processing algorithm per each image fame in runtime. Typical
> > > > Camera SW developers (including those at
> > > > Intel) are not expected to fully understand and directly set these
> > > > parameters to configure the ISP algorithm blocks. Due to the
> > > > above, a user space AIC library (in binary form) is provided to
> > > > generate ISP Algorithm specific parameters, for a given set of 3A
> > > > tuning parameters. It significantly reduces the efforts of SW
> > > > development in ISP HW
> > > configuration.
> > > >
> > > > On the other hand, the ISP algorithm details could be deduced
> > > > readily through these detailed parameters by other ISP experts
> > > > outside
> > Intel.
> > > > This is the reason that we want to keep these parameter
> > > > definitions as Intel
> > > proprietary IP.
> > > >
> > > > We are fully aware of your concerns on how to enable open source
> > > > developers to use Intel ISP through up-streamed Kernel Driver.
> > > > Internally, we are working on the license for this AIC library
> > > > release now (as Hans said NDA license is not acceptable). We
> > > > believe thi

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Adding Tomasz...

> -Original Message-
> From: Mani, Rajmohan
> Sent: Tuesday, February 20, 2018 5:56 PM
> To: 'Mauro Carvalho Chehab' <mche...@s-opensource.com>
> Cc: Zhi, Yong <yong@intel.com>; 'linux-me...@vger.kernel.org'  me...@vger.kernel.org>; 'sakari.ai...@linux.intel.com'
> <sakari.ai...@linux.intel.com>; Zheng, Jian Xu <jian.xu.zh...@intel.com>;
> Toivonen, Tuukka <tuukka.toivo...@intel.com>; Hu, Jerry W
> <jerry.w...@intel.com>; 'a...@arndb.de' <a...@arndb.de>; 'h...@lst.de'
> <h...@lst.de>; 'robin.mur...@arm.com' <robin.mur...@arm.com>;
> 'iommu@lists.linux-foundation.org' <iommu@lists.linux-foundation.org>
> Subject: RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> Hi Mauro,
> 
> > > > Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> > > >
> > > > Hi,
> > > >
> > > > Em Fri, 17 Nov 2017 02:58:56 + "Mani, Rajmohan"
> > > > <rajmohan.m...@intel.com> escreveu:
> > > >
> > > > > Here is an update on the IPU3 documentation that we are
> > > > > currently working
> > > > on.
> > > > >
> > > > > Image processing in IPU3 relies on the following.
> > > > >
> > > > > 1) HW configuration to enable ISP and
> > > > > 2) setting customer specific 3A Tuning / Algorithm Parameters to
> > > > > achieve
> > > > desired image quality.
> > > > >
> > > > > We intend to provide documentation on ImgU driver programming
> > > > > interface
> > > > to help users of this driver to configure and enable ISP HW to
> > > > meet their needs.  This documentation will include details on
> > > > complete
> > > > V4L2 Kernel driver interface and IO-Control parameters, except for
> > > > the ISP internal algorithm and its parameters (which is Intel 
> > > > proprietary
> IP).
> > > >
> > > > Sakari asked me to take a look on this thread, specifically on
> > > > this email. I took a look on the other e-mails from this thread
> > > > that are discussing about this IP block.
> > > >
> > > > I understand that Intel wants to keep their internal 3A algorithm
> > > > protected, just like other vendors protect their own algos. It was
> > > > never a requirement to open whatever algorithm are used inside a
> > > > hardware (or firmware). The only requirement is that firmwares
> > > > should be licensed with redistribution permission, ideally merged
> > > > at linux-firmware
> > > git tree.
> > > >
> > > > Yet, what I don't understand is why Intel also wants to also
> > > > protect the interface for such 3A hardware/firmware algorithm. The
> > > > parameters that are passed from an userspace application to Intel
> > > > ISP logic doesn't contain the algorithm itself. What's the issue
> > > > of documenting the meaning of each parameter?
> > > >
> > >
> > > Thanks for looking into this.
> > >
> > > To achieve improved image quality using IPU3, 3A (Auto White
> > > balance, Auto Focus and Auto Exposure) Tuning parameters specific to
> > > a given camera sensor module, are converted to Intel ISP algorithm
> > > parameters in user space camera HAL using AIC (Automatic ISP
> Configuration) library.
> > >
> > > As a unique design of Intel ISP, it exposes very detailed algorithm
> > > parameters (~ 1 parameters) to configure ISP's image processing
> > > algorithm per each image fame in runtime. Typical Camera SW
> > > developers (including those at
> > > Intel) are not expected to fully understand and directly set these
> > > parameters to configure the ISP algorithm blocks. Due to the above,
> > > a user space AIC library (in binary form) is provided to generate
> > > ISP Algorithm specific parameters, for a given set of 3A tuning
> > > parameters. It significantly reduces the efforts of SW development
> > > in ISP HW
> > configuration.
> > >
> > > On the other hand, the ISP algorithm details could be deduced
> > > readily through these detailed parameters by other ISP experts outside
> Intel.
> > > This is the reason that we want to keep these parameter definitions
> > > as Intel
> > proprietary IP.
> > >
> > > We are fully aware of your concerns on how to ena

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Mauro,

> > > Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> > >
> > > Hi,
> > >
> > > Em Fri, 17 Nov 2017 02:58:56 +
> > > "Mani, Rajmohan" <rajmohan.m...@intel.com> escreveu:
> > >
> > > > Here is an update on the IPU3 documentation that we are currently
> > > > working
> > > on.
> > > >
> > > > Image processing in IPU3 relies on the following.
> > > >
> > > > 1) HW configuration to enable ISP and
> > > > 2) setting customer specific 3A Tuning / Algorithm Parameters to
> > > > achieve
> > > desired image quality.
> > > >
> > > > We intend to provide documentation on ImgU driver programming
> > > > interface
> > > to help users of this driver to configure and enable ISP HW to meet
> > > their needs.  This documentation will include details on complete
> > > V4L2 Kernel driver interface and IO-Control parameters, except for
> > > the ISP internal algorithm and its parameters (which is Intel proprietary 
> > > IP).
> > >
> > > Sakari asked me to take a look on this thread, specifically on this
> > > email. I took a look on the other e-mails from this thread that are
> > > discussing about this IP block.
> > >
> > > I understand that Intel wants to keep their internal 3A algorithm
> > > protected, just like other vendors protect their own algos. It was
> > > never a requirement to open whatever algorithm are used inside a
> > > hardware (or firmware). The only requirement is that firmwares
> > > should be licensed with redistribution permission, ideally merged at
> > > linux-firmware
> > git tree.
> > >
> > > Yet, what I don't understand is why Intel also wants to also protect
> > > the interface for such 3A hardware/firmware algorithm. The
> > > parameters that are passed from an userspace application to Intel
> > > ISP logic doesn't contain the algorithm itself. What's the issue of
> > > documenting the meaning of each parameter?
> > >
> >
> > Thanks for looking into this.
> >
> > To achieve improved image quality using IPU3, 3A (Auto White balance,
> > Auto Focus and Auto Exposure) Tuning parameters specific to a given
> > camera sensor module, are converted to Intel ISP algorithm parameters
> > in user space camera HAL using AIC (Automatic ISP Configuration) library.
> >
> > As a unique design of Intel ISP, it exposes very detailed algorithm
> > parameters (~ 1 parameters) to configure ISP's image processing
> > algorithm per each image fame in runtime. Typical Camera SW developers
> > (including those at
> > Intel) are not expected to fully understand and directly set these
> > parameters to configure the ISP algorithm blocks. Due to the above, a
> > user space AIC library (in binary form) is provided to generate ISP
> > Algorithm specific parameters, for a given set of 3A tuning
> > parameters. It significantly reduces the efforts of SW development in ISP HW
> configuration.
> >
> > On the other hand, the ISP algorithm details could be deduced readily
> > through these detailed parameters by other ISP experts outside Intel.
> > This is the reason that we want to keep these parameter definitions as Intel
> proprietary IP.
> >
> > We are fully aware of your concerns on how to enable open source
> > developers to use Intel ISP through up-streamed Kernel Driver.
> > Internally, we are working on the license for this AIC library release
> > now (as Hans said NDA license is not acceptable). We believe this will
> > be more efficient way to help open source developers.
> >
> > This AIC library release would be a binary-only release. This AIC
> > library does not use any kernel uAPIs directly. The user space Camera
> > HAL that uses kernel uAPIs is available at
> > https://chromium.googlesource.com/chromiumos/platform/arc-
> > camera/+/master
> >

The AIC library (in binary form) is available here.
https://storage.googleapis.com/chromeos-localmirror/distfiles/intel-3a-libs-bin-2017.09.27.tbz2

Licensing information can be found in ./LICENSE.intel_3a_library file after 
unzipping the tar file.

> 
> Just pinging to know your thoughts on this.
> 
> Thanks
> Raj
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Mauro,

> -Original Message-
> From: Mani, Rajmohan
> Sent: Tuesday, December 26, 2017 2:31 PM
> To: 'Mauro Carvalho Chehab' <mche...@s-opensource.com>
> Cc: Zhi, Yong <yong@intel.com>; linux-me...@vger.kernel.org;
> sakari.ai...@linux.intel.com; Zheng, Jian Xu <jian.xu.zh...@intel.com>;
> Toivonen, Tuukka <tuukka.toivo...@intel.com>; Hu, Jerry W
> <jerry.w...@intel.com>; a...@arndb.de; h...@lst.de;
> robin.mur...@arm.com; iommu@lists.linux-foundation.org
> Subject: RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> Hi Mauro,
> 
> > -Original Message-
> > From: Mauro Carvalho Chehab [mailto:mche...@s-opensource.com]
> > Sent: Wednesday, December 20, 2017 5:58 AM
> > To: Mani, Rajmohan <rajmohan.m...@intel.com>
> > Cc: Zhi, Yong <yong@intel.com>; linux-me...@vger.kernel.org;
> > sakari.ai...@linux.intel.com; Zheng, Jian Xu
> > <jian.xu.zh...@intel.com>; Toivonen, Tuukka
> > <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
> > a...@arndb.de; h...@lst.de; robin.mur...@arm.com;
> > iommu@lists.linux-foundation.org
> > Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> >
> > Hi,
> >
> > Em Fri, 17 Nov 2017 02:58:56 +
> > "Mani, Rajmohan" <rajmohan.m...@intel.com> escreveu:
> >
> > > Here is an update on the IPU3 documentation that we are currently
> > > working
> > on.
> > >
> > > Image processing in IPU3 relies on the following.
> > >
> > > 1) HW configuration to enable ISP and
> > > 2) setting customer specific 3A Tuning / Algorithm Parameters to
> > > achieve
> > desired image quality.
> > >
> > > We intend to provide documentation on ImgU driver programming
> > > interface
> > to help users of this driver to configure and enable ISP HW to meet
> > their needs.  This documentation will include details on complete V4L2
> > Kernel driver interface and IO-Control parameters, except for the ISP
> > internal algorithm and its parameters (which is Intel proprietary IP).
> >
> > Sakari asked me to take a look on this thread, specifically on this
> > email. I took a look on the other e-mails from this thread that are
> > discussing about this IP block.
> >
> > I understand that Intel wants to keep their internal 3A algorithm
> > protected, just like other vendors protect their own algos. It was
> > never a requirement to open whatever algorithm are used inside a
> > hardware (or firmware). The only requirement is that firmwares should
> > be licensed with redistribution permission, ideally merged at linux-firmware
> git tree.
> >
> > Yet, what I don't understand is why Intel also wants to also protect
> > the interface for such 3A hardware/firmware algorithm. The parameters
> > that are passed from an userspace application to Intel ISP logic
> > doesn't contain the algorithm itself. What's the issue of documenting
> > the meaning of each parameter?
> >
> 
> Thanks for looking into this.
> 
> To achieve improved image quality using IPU3, 3A (Auto White balance, Auto
> Focus and Auto Exposure) Tuning parameters specific to a given camera sensor
> module, are converted to Intel ISP algorithm parameters in user space camera
> HAL using AIC (Automatic ISP Configuration) library.
> 
> As a unique design of Intel ISP, it exposes very detailed algorithm parameters
> (~ 1 parameters) to configure ISP's image processing algorithm per each
> image fame in runtime. Typical Camera SW developers (including those at
> Intel) are not expected to fully understand and directly set these parameters 
> to
> configure the ISP algorithm blocks. Due to the above, a user space AIC library
> (in binary form) is provided to generate ISP Algorithm specific parameters, 
> for a
> given set of 3A tuning parameters. It significantly reduces the efforts of SW
> development in ISP HW configuration.
> 
> On the other hand, the ISP algorithm details could be deduced readily through
> these detailed parameters by other ISP experts outside Intel. This is the 
> reason
> that we want to keep these parameter definitions as Intel proprietary IP.
> 
> We are fully aware of your concerns on how to enable open source developers
> to use Intel ISP through up-streamed Kernel Driver. Internally, we are working
> on the license for this AIC library release now (as Hans said NDA license is 
> not
> acceptable). We believe this will be more efficient way to help open source
> developers.
> 
> This AIC library release would be a binary-only release. This AIC library 
> does not
> use any kernel uAPIs directly. The user space Camera HAL that uses kernel
> uAPIs is available at
> https://chromium.googlesource.com/chromiumos/platform/arc-
> camera/+/master
> 

Just pinging to know your thoughts on this.

Thanks
Raj
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Mauro,

> -Original Message-
> From: Mauro Carvalho Chehab [mailto:mche...@s-opensource.com]
> Sent: Wednesday, December 20, 2017 5:58 AM
> To: Mani, Rajmohan <rajmohan.m...@intel.com>
> Cc: Zhi, Yong <yong@intel.com>; linux-me...@vger.kernel.org;
> sakari.ai...@linux.intel.com; Zheng, Jian Xu <jian.xu.zh...@intel.com>;
> Toivonen, Tuukka <tuukka.toivo...@intel.com>; Hu, Jerry W
> <jerry.w...@intel.com>; a...@arndb.de; h...@lst.de;
> robin.mur...@arm.com; iommu@lists.linux-foundation.org
> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> Hi,
> 
> Em Fri, 17 Nov 2017 02:58:56 +
> "Mani, Rajmohan" <rajmohan.m...@intel.com> escreveu:
> 
> > Here is an update on the IPU3 documentation that we are currently working
> on.
> >
> > Image processing in IPU3 relies on the following.
> >
> > 1) HW configuration to enable ISP and
> > 2) setting customer specific 3A Tuning / Algorithm Parameters to achieve
> desired image quality.
> >
> > We intend to provide documentation on ImgU driver programming interface
> to help users of this driver to configure and enable ISP HW to meet their
> needs.  This documentation will include details on complete V4L2 Kernel driver
> interface and IO-Control parameters, except for the ISP internal algorithm and
> its parameters (which is Intel proprietary IP).
> 
> Sakari asked me to take a look on this thread, specifically on this email. I 
> took a
> look on the other e-mails from this thread that are discussing about this IP
> block.
> 
> I understand that Intel wants to keep their internal 3A algorithm protected,
> just like other vendors protect their own algos. It was never a requirement to
> open whatever algorithm are used inside a hardware (or firmware). The only
> requirement is that firmwares should be licensed with redistribution
> permission, ideally merged at linux-firmware git tree.
> 
> Yet, what I don't understand is why Intel also wants to also protect the
> interface for such 3A hardware/firmware algorithm. The parameters that are
> passed from an userspace application to Intel ISP logic doesn't contain the
> algorithm itself. What's the issue of documenting the meaning of each
> parameter?
> 

Thanks for looking into this.

To achieve improved image quality using IPU3, 3A (Auto White balance, Auto Focus
and Auto Exposure) Tuning parameters specific to a given camera sensor module,
are converted to Intel ISP algorithm parameters in user space camera HAL using
AIC (Automatic ISP Configuration) library.

As a unique design of Intel ISP, it exposes very detailed algorithm parameters
(~ 1 parameters) to configure ISP's image processing algorithm per each
image fame in runtime. Typical Camera SW developers (including those at Intel)
are not expected to fully understand and directly set these parameters to
configure the ISP algorithm blocks. Due to the above, a user space AIC library
(in binary form) is provided to generate ISP Algorithm specific parameters, for
a given set of 3A tuning parameters. It significantly reduces the efforts of SW
development in ISP HW configuration.

On the other hand, the ISP algorithm details could be deduced readily through
these detailed parameters by other ISP experts outside Intel. This is the reason
that we want to keep these parameter definitions as Intel proprietary IP.

We are fully aware of your concerns on how to enable open source developers
to use Intel ISP through up-streamed Kernel Driver. Internally, we are working
on the license for this AIC library release now (as Hans said NDA license is not
acceptable). We believe this will be more efficient way to help open source
developers.

This AIC library release would be a binary-only release. This AIC library does
not use any kernel uAPIs directly. The user space Camera HAL that uses kernel
uAPIs is available at 
https://chromium.googlesource.com/chromiumos/platform/arc-camera/+/master

Thanks
Raj
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mauro Carvalho Chehab]

Hi,

Em Fri, 17 Nov 2017 02:58:56 +
"Mani, Rajmohan"  escreveu:

> Here is an update on the IPU3 documentation that we are currently working on.
> 
> Image processing in IPU3 relies on the following.
> 
> 1) HW configuration to enable ISP and
> 2) setting customer specific 3A Tuning / Algorithm Parameters to achieve 
> desired image quality. 
> 
> We intend to provide documentation on ImgU driver programming interface to 
> help users of this driver to configure and enable ISP HW to meet their needs. 
>  This documentation will include details on complete V4L2 Kernel driver 
> interface and IO-Control parameters, except for the ISP internal algorithm 
> and its parameters (which is Intel proprietary IP).

Sakari asked me to take a look on this thread, specifically on this
email. I took a look on the other e-mails from this thread that are
discussing about this IP block.

I understand that Intel wants to keep their internal 3A algorithm
protected, just like other vendors protect their own algos. It was never
a requirement to open whatever algorithm are used inside a hardware
(or firmware). The only requirement is that firmwares should be licensed
with redistribution permission, ideally merged at linux-firmware git
tree.

Yet, what I don't understand is why Intel also wants to also protect
the interface for such 3A hardware/firmware algorithm. The parameters
that are passed from an userspace application to Intel ISP logic 
doesn't contain the algorithm itself. What's the issue of documenting
the meaning of each parameter?


Thanks,
Mauro
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Tomasz, Hans,

> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> On 12/05/2017 04:22 AM, Tomasz Figa wrote:
> > Hi Raj,
> >
> > On Tue, Dec 5, 2017 at 9:13 AM, Mani, Rajmohan
> <rajmohan.m...@intel.com> wrote:
> >> Hi Hans,
> >>
> >> Thanks for your patience and sharing your thoughts on this.
> >>
> >>> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> >>>
> >>> Hi Rajmohan,
> >>>
> >>> On 11/17/2017 03:58 AM, Mani, Rajmohan wrote:
> >>>> Hi Sakari and all,
> >>>>
> >>>>> -Original Message-
> >>>>> From: Zhi, Yong
> >>>>> Sent: Tuesday, October 17, 2017 8:47 PM
> >>>>> To: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com
> >>>>> Cc: Zheng, Jian Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan
> >>>>> <rajmohan.m...@intel.com>; Toivonen, Tuukka
> >>>>> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
> >>>>> a...@arndb.de; h...@lst.de; robin.mur...@arm.com;
> >>>>> iommu@lists.linux- foundation.org; Zhi, Yong <yong@intel.com>
> >>>>> Subject: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> >>>>>
> >>>>> This patchset adds support for the Intel IPU3 (Image Processing
> >>>>> Unit) ImgU which is essentially a modern memory-to-memory ISP. It
> >>>>> implements raw Bayer to YUV image format conversion as well as a
> >>>>> large number of other pixel processing algorithms for improving
> >>>>> the image
> >>> quality.
> >>>>>
> >>>>> Meta data formats are defined for image statistics (3A, i.e.
> >>>>> automatic white balance, exposure and focus, histogram and local
> >>>>> area contrast
> >>>>> enhancement) as well as for the pixel processing algorithm parameters.
> >>>>> The documentation for these formats is currently not included in
> >>>>> the patchset but will be added in a future version of this set.
> >>>>>
> >>>>
> >>>> Here is an update on the IPU3 documentation that we are currently
> >>>> working
> >>> on.
> >>>>
> >>>> Image processing in IPU3 relies on the following.
> >>>>
> >>>> 1) HW configuration to enable ISP and
> >>>> 2) setting customer specific 3A Tuning / Algorithm Parameters to
> >>>> achieve
> >>> desired image quality.
> >>>>
> >>>> We intend to provide documentation on ImgU driver programming
> >>>> interface
> >>> to help users of this driver to configure and enable ISP HW to meet
> >>> their needs.  This documentation will include details on complete
> >>> V4L2 Kernel driver interface and IO-Control parameters, except for
> >>> the ISP internal algorithm and its parameters (which is Intel proprietary 
> >>> IP).
> >>>>
> >>>> We will also provide an user space library in binary form to help
> >>>> users of this
> >>> driver, to convert the public 3A tuning parameters to IPU3 algorithm
> >>> parameters. This tool will be released under NDA to the users of this
> driver.
> >>>
> >>> So I discussed this situation with Sakari in Prague during the ELCE.
> >>> I am not happy with adding a driver to the kernel that needs an NDA
> >>> to be usable. I can't agree to that. It's effectively the same as
> >>> firmware that's only available under an NDA and we wouldn't accept such
> drivers either.
> >>>
> >>
> >> Ack
> >>
> >>> There are a few options:
> >>>
> >>> 1) Document the ISP parameters and that format they are stored in
> >>> allowing for
> >>>open source implementations. While this is the ideal, I suspect that 
> >>> this
> is
> >>>a no-go for Intel.
> >>>
> >>
> >> Ack
> >>
> >>> 2) The driver can be used without using these ISP parameters and still
> achieve
> >>>'OK' quality. I.e., it's usable for basic webcam usage under normal 
> >>> lighting
> >>>conditions. I'm not sure if this is possible at all, though.
> >>>
> >>
> >> This is something that we have tri

Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Hans Verkuil]

On 12/05/2017 04:22 AM, Tomasz Figa wrote:
> Hi Raj,
> 
> On Tue, Dec 5, 2017 at 9:13 AM, Mani, Rajmohan <rajmohan.m...@intel.com> 
> wrote:
>> Hi Hans,
>>
>> Thanks for your patience and sharing your thoughts on this.
>>
>>> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
>>>
>>> Hi Rajmohan,
>>>
>>> On 11/17/2017 03:58 AM, Mani, Rajmohan wrote:
>>>> Hi Sakari and all,
>>>>
>>>>> -Original Message-
>>>>> From: Zhi, Yong
>>>>> Sent: Tuesday, October 17, 2017 8:47 PM
>>>>> To: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com
>>>>> Cc: Zheng, Jian Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan
>>>>> <rajmohan.m...@intel.com>; Toivonen, Tuukka
>>>>> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
>>>>> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
>>>>> foundation.org; Zhi, Yong <yong@intel.com>
>>>>> Subject: [PATCH v4 00/12] Intel IPU3 ImgU patchset
>>>>>
>>>>> This patchset adds support for the Intel IPU3 (Image Processing Unit)
>>>>> ImgU which is essentially a modern memory-to-memory ISP. It
>>>>> implements raw Bayer to YUV image format conversion as well as a
>>>>> large number of other pixel processing algorithms for improving the image
>>> quality.
>>>>>
>>>>> Meta data formats are defined for image statistics (3A, i.e.
>>>>> automatic white balance, exposure and focus, histogram and local area
>>>>> contrast
>>>>> enhancement) as well as for the pixel processing algorithm parameters.
>>>>> The documentation for these formats is currently not included in the
>>>>> patchset but will be added in a future version of this set.
>>>>>
>>>>
>>>> Here is an update on the IPU3 documentation that we are currently working
>>> on.
>>>>
>>>> Image processing in IPU3 relies on the following.
>>>>
>>>> 1) HW configuration to enable ISP and
>>>> 2) setting customer specific 3A Tuning / Algorithm Parameters to achieve
>>> desired image quality.
>>>>
>>>> We intend to provide documentation on ImgU driver programming interface
>>> to help users of this driver to configure and enable ISP HW to meet their
>>> needs.  This documentation will include details on complete V4L2 Kernel 
>>> driver
>>> interface and IO-Control parameters, except for the ISP internal algorithm 
>>> and
>>> its parameters (which is Intel proprietary IP).
>>>>
>>>> We will also provide an user space library in binary form to help users of 
>>>> this
>>> driver, to convert the public 3A tuning parameters to IPU3 algorithm
>>> parameters. This tool will be released under NDA to the users of this 
>>> driver.
>>>
>>> So I discussed this situation with Sakari in Prague during the ELCE. I am 
>>> not
>>> happy with adding a driver to the kernel that needs an NDA to be usable. I
>>> can't agree to that. It's effectively the same as firmware that's only 
>>> available
>>> under an NDA and we wouldn't accept such drivers either.
>>>
>>
>> Ack
>>
>>> There are a few options:
>>>
>>> 1) Document the ISP parameters and that format they are stored in allowing
>>> for
>>>open source implementations. While this is the ideal, I suspect that 
>>> this is
>>>a no-go for Intel.
>>>
>>
>> Ack
>>
>>> 2) The driver can be used without using these ISP parameters and still 
>>> achieve
>>>'OK' quality. I.e., it's usable for basic webcam usage under normal 
>>> lighting
>>>conditions. I'm not sure if this is possible at all, though.
>>>
>>
>> This is something that we have tried and are able to do image capture with
>> the default ISP parameters using ov5670 sensor.
>> Additionally we had to set optimal values for the ov5670 sensor's exposure 
>> and
>> gain settings.
> 
> Does it mean hardcoding some ov5670-specific settings in the ISP
> driver? If not, I guess it might be good enough?

It doesn't look too bad, but it's hard to tell from just a single
frame. I think you can work with Sakari on this, if he also thinks it's
good enough, then I am happy with that.

> 
>>
>> Ple

Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Tomasz Figa via iommu]

Hi Raj,

On Tue, Dec 5, 2017 at 9:13 AM, Mani, Rajmohan <rajmohan.m...@intel.com> wrote:
> Hi Hans,
>
> Thanks for your patience and sharing your thoughts on this.
>
>> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
>>
>> Hi Rajmohan,
>>
>> On 11/17/2017 03:58 AM, Mani, Rajmohan wrote:
>> > Hi Sakari and all,
>> >
>> >> -Original Message-
>> >> From: Zhi, Yong
>> >> Sent: Tuesday, October 17, 2017 8:47 PM
>> >> To: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com
>> >> Cc: Zheng, Jian Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan
>> >> <rajmohan.m...@intel.com>; Toivonen, Tuukka
>> >> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
>> >> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
>> >> foundation.org; Zhi, Yong <yong@intel.com>
>> >> Subject: [PATCH v4 00/12] Intel IPU3 ImgU patchset
>> >>
>> >> This patchset adds support for the Intel IPU3 (Image Processing Unit)
>> >> ImgU which is essentially a modern memory-to-memory ISP. It
>> >> implements raw Bayer to YUV image format conversion as well as a
>> >> large number of other pixel processing algorithms for improving the image
>> quality.
>> >>
>> >> Meta data formats are defined for image statistics (3A, i.e.
>> >> automatic white balance, exposure and focus, histogram and local area
>> >> contrast
>> >> enhancement) as well as for the pixel processing algorithm parameters.
>> >> The documentation for these formats is currently not included in the
>> >> patchset but will be added in a future version of this set.
>> >>
>> >
>> > Here is an update on the IPU3 documentation that we are currently working
>> on.
>> >
>> > Image processing in IPU3 relies on the following.
>> >
>> > 1) HW configuration to enable ISP and
>> > 2) setting customer specific 3A Tuning / Algorithm Parameters to achieve
>> desired image quality.
>> >
>> > We intend to provide documentation on ImgU driver programming interface
>> to help users of this driver to configure and enable ISP HW to meet their
>> needs.  This documentation will include details on complete V4L2 Kernel 
>> driver
>> interface and IO-Control parameters, except for the ISP internal algorithm 
>> and
>> its parameters (which is Intel proprietary IP).
>> >
>> > We will also provide an user space library in binary form to help users of 
>> > this
>> driver, to convert the public 3A tuning parameters to IPU3 algorithm
>> parameters. This tool will be released under NDA to the users of this driver.
>>
>> So I discussed this situation with Sakari in Prague during the ELCE. I am not
>> happy with adding a driver to the kernel that needs an NDA to be usable. I
>> can't agree to that. It's effectively the same as firmware that's only 
>> available
>> under an NDA and we wouldn't accept such drivers either.
>>
>
> Ack
>
>> There are a few options:
>>
>> 1) Document the ISP parameters and that format they are stored in allowing
>> for
>>open source implementations. While this is the ideal, I suspect that this 
>> is
>>a no-go for Intel.
>>
>
> Ack
>
>> 2) The driver can be used without using these ISP parameters and still 
>> achieve
>>'OK' quality. I.e., it's usable for basic webcam usage under normal 
>> lighting
>>conditions. I'm not sure if this is possible at all, though.
>>
>
> This is something that we have tried and are able to do image capture with
> the default ISP parameters using ov5670 sensor.
> Additionally we had to set optimal values for the ov5670 sensor's exposure and
> gain settings.

Does it mean hardcoding some ov5670-specific settings in the ISP
driver? If not, I guess it might be good enough?

>
> Please see if the following image looks to meet the 'OK' quality.
>
> git clone https://github.com/RajmohanMani/ipu3-misc.git
> ipu3-misc/ov5670.jpg is the image file.
>
>> 3) Make the library available without requiring an NDA.
>>
>
> We are also actively exploring this option to see if this can be done.
>
>> 4) Provide a non-NDA library (ideally open source) that achieves at minimum
>>the quality as described in 2: i.e. usable for basic webcam.
>>
>
> I see this is the same as option 3) + open sourcing the library.
> Open sourcing the library does not look to be an option.
> I will reconfirm this.

In my understanding, that could be quite different from option 3). The
open source library would not have to implement all of the
capabilities, just enough to get the "OK" quality and the implemented
part could use some simpler algorithms not covered by IP.

Best regards,
Tomasz
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Hans,

Thanks for your patience and sharing your thoughts on this.

> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> Hi Rajmohan,
> 
> On 11/17/2017 03:58 AM, Mani, Rajmohan wrote:
> > Hi Sakari and all,
> >
> >> -Original Message-
> >> From: Zhi, Yong
> >> Sent: Tuesday, October 17, 2017 8:47 PM
> >> To: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com
> >> Cc: Zheng, Jian Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan
> >> <rajmohan.m...@intel.com>; Toivonen, Tuukka
> >> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
> >> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
> >> foundation.org; Zhi, Yong <yong@intel.com>
> >> Subject: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> >>
> >> This patchset adds support for the Intel IPU3 (Image Processing Unit)
> >> ImgU which is essentially a modern memory-to-memory ISP. It
> >> implements raw Bayer to YUV image format conversion as well as a
> >> large number of other pixel processing algorithms for improving the image
> quality.
> >>
> >> Meta data formats are defined for image statistics (3A, i.e.
> >> automatic white balance, exposure and focus, histogram and local area
> >> contrast
> >> enhancement) as well as for the pixel processing algorithm parameters.
> >> The documentation for these formats is currently not included in the
> >> patchset but will be added in a future version of this set.
> >>
> >
> > Here is an update on the IPU3 documentation that we are currently working
> on.
> >
> > Image processing in IPU3 relies on the following.
> >
> > 1) HW configuration to enable ISP and
> > 2) setting customer specific 3A Tuning / Algorithm Parameters to achieve
> desired image quality.
> >
> > We intend to provide documentation on ImgU driver programming interface
> to help users of this driver to configure and enable ISP HW to meet their
> needs.  This documentation will include details on complete V4L2 Kernel driver
> interface and IO-Control parameters, except for the ISP internal algorithm and
> its parameters (which is Intel proprietary IP).
> >
> > We will also provide an user space library in binary form to help users of 
> > this
> driver, to convert the public 3A tuning parameters to IPU3 algorithm
> parameters. This tool will be released under NDA to the users of this driver.
> 
> So I discussed this situation with Sakari in Prague during the ELCE. I am not
> happy with adding a driver to the kernel that needs an NDA to be usable. I
> can't agree to that. It's effectively the same as firmware that's only 
> available
> under an NDA and we wouldn't accept such drivers either.
> 

Ack

> There are a few options:
> 
> 1) Document the ISP parameters and that format they are stored in allowing
> for
>open source implementations. While this is the ideal, I suspect that this 
> is
>a no-go for Intel.
> 

Ack

> 2) The driver can be used without using these ISP parameters and still achieve
>'OK' quality. I.e., it's usable for basic webcam usage under normal 
> lighting
>conditions. I'm not sure if this is possible at all, though.
> 

This is something that we have tried and are able to do image capture with
the default ISP parameters using ov5670 sensor.
Additionally we had to set optimal values for the ov5670 sensor's exposure and
gain settings.

Please see if the following image looks to meet the 'OK' quality.

git clone https://github.com/RajmohanMani/ipu3-misc.git
ipu3-misc/ov5670.jpg is the image file.

> 3) Make the library available without requiring an NDA.
> 

We are also actively exploring this option to see if this can be done.

> 4) Provide a non-NDA library (ideally open source) that achieves at minimum
>the quality as described in 2: i.e. usable for basic webcam.
> 

I see this is the same as option 3) + open sourcing the library.
Open sourcing the library does not look to be an option.
I will reconfirm this.

> 5) Other solutions I didn't think of?
> 
> I think 4 might be the best option, especially if it is open sourced and just 
> uses
> non-IP 3A algorithms. It could even be added to the v4l-utils git repo.
> 
> A closed source non-NDA library might also work, but you will need to check
> what Mauro thinks about that.

I believe this is option 3) that you are referring here.
Depending on the progress that we make on option 3), we can work on the
next steps.

> My problem is that such libraries tend to be
> abandoned after a few years and then bit-rot sets in. An open-source solution
> is much easier to maintain in the long term.
> 
> Regards,
> 
>   Hans
> 

___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Hans Verkuil]

Hi Rajmohan,

On 11/17/2017 03:58 AM, Mani, Rajmohan wrote:
> Hi Sakari and all,
> 
>> -Original Message-
>> From: Zhi, Yong
>> Sent: Tuesday, October 17, 2017 8:47 PM
>> To: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com
>> Cc: Zheng, Jian Xu ; Mani, Rajmohan
>> ; Toivonen, Tuukka
>> ; Hu, Jerry W ;
>> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
>> foundation.org; Zhi, Yong 
>> Subject: [PATCH v4 00/12] Intel IPU3 ImgU patchset
>>
>> This patchset adds support for the Intel IPU3 (Image Processing Unit) ImgU
>> which is essentially a modern memory-to-memory ISP. It implements raw
>> Bayer to YUV image format conversion as well as a large number of other pixel
>> processing algorithms for improving the image quality.
>>
>> Meta data formats are defined for image statistics (3A, i.e. automatic white
>> balance, exposure and focus, histogram and local area contrast
>> enhancement) as well as for the pixel processing algorithm parameters.
>> The documentation for these formats is currently not included in the patchset
>> but will be added in a future version of this set.
>>
> 
> Here is an update on the IPU3 documentation that we are currently working on.
> 
> Image processing in IPU3 relies on the following.
> 
> 1) HW configuration to enable ISP and
> 2) setting customer specific 3A Tuning / Algorithm Parameters to achieve 
> desired image quality. 
> 
> We intend to provide documentation on ImgU driver programming interface to 
> help users of this driver to configure and enable ISP HW to meet their needs. 
>  This documentation will include details on complete V4L2 Kernel driver 
> interface and IO-Control parameters, except for the ISP internal algorithm 
> and its parameters (which is Intel proprietary IP).
> 
> We will also provide an user space library in binary form to help users of 
> this driver, to convert the public 3A tuning parameters to IPU3 algorithm 
> parameters. This tool will be released under NDA to the users of this driver.

So I discussed this situation with Sakari in Prague during the ELCE. I am not
happy with adding a driver to the kernel that needs an NDA to be usable. I can't
agree to that. It's effectively the same as firmware that's only available under
an NDA and we wouldn't accept such drivers either.

There are a few options:

1) Document the ISP parameters and that format they are stored in allowing for
   open source implementations. While this is the ideal, I suspect that this is
   a no-go for Intel.

2) The driver can be used without using these ISP parameters and still achieve
   'OK' quality. I.e., it's usable for basic webcam usage under normal lighting
   conditions. I'm not sure if this is possible at all, though.

3) Make the library available without requiring an NDA.

4) Provide a non-NDA library (ideally open source) that achieves at minimum
   the quality as described in 2: i.e. usable for basic webcam.

5) Other solutions I didn't think of?

I think 4 might be the best option, especially if it is open sourced and just
uses non-IP 3A algorithms. It could even be added to the v4l-utils git repo.

A closed source non-NDA library might also work, but you will need to check
what Mauro thinks about that. My problem is that such libraries tend to be
abandoned after a few years and then bit-rot sets in. An open-source solution
is much easier to maintain in the long term.

Regards,

Hans

> 
>> The algorithm parameters need to be considered specific to a given frame and
>> typically a large number of these parameters change on frame to frame basis.
>> Additionally, the parameters are highly structured (and not a flat space of
>> independent configuration primitives). They also reflect the data structures
>> used by the firmware and the hardware. On top of that, the algorithms require
>> highly specialized user space to make meaningful use of them. For these
>> reasons it has been chosen video buffers to pass the parameters to the 
>> device.
>>
>> On individual patches:
>>
>> The heart of ImgU is the CSS, or Camera Subsystem, which contains the image
>> processors and HW accelerators.
>>
>> The 3A statistics and other firmware parameter computation related functions
>> are implemented in patch 8.
>>
>> All h/w programming related code can be found in patch 9.
>>
>> To access DDR via ImgU's own memory space, IPU3 is also equipped with its
>> own MMU unit, the driver is implemented in patch 2.
>>
>> Currently, the MMU driver has dependency on two exported symbols
>> (iommu_group_ref_get and iommu_group_get_for_dev))to build as ko.
>>
>> Patch 3 uses above IOMMU driver for DMA mem related functions.
>>
>> Patch 5-10 are basically IPU3 CSS specific implementations:
>>
>> 6 and 7 provide some utility functions and manage IPU3 fw download and
>> install.
>>
>> The firmware which is called ipu3-fw.bin can be 

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Mani, Rajmohan]

Hi Sakari and all,

> -Original Message-
> From: Zhi, Yong
> Sent: Tuesday, October 17, 2017 8:47 PM
> To: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com
> Cc: Zheng, Jian Xu ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ;
> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
> foundation.org; Zhi, Yong 
> Subject: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> This patchset adds support for the Intel IPU3 (Image Processing Unit) ImgU
> which is essentially a modern memory-to-memory ISP. It implements raw
> Bayer to YUV image format conversion as well as a large number of other pixel
> processing algorithms for improving the image quality.
> 
> Meta data formats are defined for image statistics (3A, i.e. automatic white
> balance, exposure and focus, histogram and local area contrast
> enhancement) as well as for the pixel processing algorithm parameters.
> The documentation for these formats is currently not included in the patchset
> but will be added in a future version of this set.
> 

Here is an update on the IPU3 documentation that we are currently working on.

Image processing in IPU3 relies on the following.

1) HW configuration to enable ISP and
2) setting customer specific 3A Tuning / Algorithm Parameters to achieve 
desired image quality. 

We intend to provide documentation on ImgU driver programming interface to help 
users of this driver to configure and enable ISP HW to meet their needs.  This 
documentation will include details on complete V4L2 Kernel driver interface and 
IO-Control parameters, except for the ISP internal algorithm and its parameters 
(which is Intel proprietary IP).

We will also provide an user space library in binary form to help users of this 
driver, to convert the public 3A tuning parameters to IPU3 algorithm 
parameters. This tool will be released under NDA to the users of this driver.

> The algorithm parameters need to be considered specific to a given frame and
> typically a large number of these parameters change on frame to frame basis.
> Additionally, the parameters are highly structured (and not a flat space of
> independent configuration primitives). They also reflect the data structures
> used by the firmware and the hardware. On top of that, the algorithms require
> highly specialized user space to make meaningful use of them. For these
> reasons it has been chosen video buffers to pass the parameters to the device.
> 
> On individual patches:
> 
> The heart of ImgU is the CSS, or Camera Subsystem, which contains the image
> processors and HW accelerators.
> 
> The 3A statistics and other firmware parameter computation related functions
> are implemented in patch 8.
> 
> All h/w programming related code can be found in patch 9.
> 
> To access DDR via ImgU's own memory space, IPU3 is also equipped with its
> own MMU unit, the driver is implemented in patch 2.
> 
> Currently, the MMU driver has dependency on two exported symbols
> (iommu_group_ref_get and iommu_group_get_for_dev))to build as ko.
> 
> Patch 3 uses above IOMMU driver for DMA mem related functions.
> 
> Patch 5-10 are basically IPU3 CSS specific implementations:
> 
> 6 and 7 provide some utility functions and manage IPU3 fw download and
> install.
> 
> The firmware which is called ipu3-fw.bin can be downloaded from:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
> (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f)
> 
> Patch 9 and 10 are of the same file, the latter implements interface functions
> for access fw & hw capabilities defined in patch 8.
> 
> Patch 11 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work:
> 
> 
> 
> 
> Patch 12 uses Kconfig and Makefile created by IPU3 cio2 patch series.
> 
> Link to user space implementation:
> 
>  camera/+/master>
> 
> Device topology:
> 
> ./media-ctl -d /dev/media0 -p
> Media controller API version 4.14.0
> 
> Media device information
> 
> driver  ipu3-imgu
> model   ipu3-imgu
> serial
> bus info:00:05.0
> hw revision 0x0
> driver version  4.14.0
> 
> Device topology
> - entity 1: ipu3-imgu:0 (8 pads, 8 links)
> type V4L2 subdev subtype Unknown flags 0
> device node name /dev/v4l-subdev0
>   pad0: Sink
>   [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown]
>   <- "input":0 [ENABLED,IMMUTABLE]
>   pad1: Sink
>   [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown]
>   <- "parameters":0 []
>   pad2: Source
>   [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown]
>   -> "output":0 []
>   pad3: Source
>   

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Zhi, Yong]

Hi, Sakari,

> -Original Message-
> From: linux-media-ow...@vger.kernel.org [mailto:linux-media-
> ow...@vger.kernel.org] On Behalf Of Sakari Ailus
> Sent: Friday, October 20, 2017 2:10 AM
> To: Zhi, Yong <yong@intel.com>
> Cc: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com; Zheng, Jian
> Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan
> <rajmohan.m...@intel.com>; Toivonen, Tuukka
> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
> foundation.org
> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> Hi Yong,
> 
> On Tue, Oct 17, 2017 at 10:46:48PM -0500, Yong Zhi wrote:
> > This patchset adds support for the Intel IPU3 (Image Processing Unit)
> > ImgU which is essentially a modern memory-to-memory ISP. It implements
> > raw Bayer to YUV image format conversion as well as a large number of
> > other pixel processing algorithms for improving the image quality.
> >
> > Meta data formats are defined for image statistics (3A, i.e. automatic
> > white balance, exposure and focus, histogram and local area contrast
> > enhancement) as well as for the pixel processing algorithm parameters.
> > The documentation for these formats is currently not included in the
> > patchset but will be added in a future version of this set.
> >
> > The algorithm parameters need to be considered specific to a given
> > frame and typically a large number of these parameters change on frame
> > to frame basis. Additionally, the parameters are highly structured
> > (and not a flat space of independent configuration primitives). They
> > also reflect the data structures used by the firmware and the
> > hardware. On top of that, the algorithms require highly specialized
> > user space to make meaningful use of them. For these reasons it has
> > been chosen video buffers to pass
> 
> Do you have a to-do list for this patchset? I think it would be useful to
> maintain one, in case not all the comments have been addressed.
> 

Sure, will add in next update.

> --
> Sakari Ailus
> e-mail: sakari.ai...@iki.fi
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Sakari Ailus]

Hi Yong,

On Tue, Oct 17, 2017 at 10:46:48PM -0500, Yong Zhi wrote:
> This patchset adds support for the Intel IPU3 (Image Processing Unit)
> ImgU which is essentially a modern memory-to-memory ISP. It implements
> raw Bayer to YUV image format conversion as well as a large number of
> other pixel processing algorithms for improving the image quality.
> 
> Meta data formats are defined for image statistics (3A, i.e. automatic
> white balance, exposure and focus, histogram and local area contrast
> enhancement) as well as for the pixel processing algorithm parameters.
> The documentation for these formats is currently not included in the
> patchset but will be added in a future version of this set.
> 
> The algorithm parameters need to be considered specific to a given frame
> and typically a large number of these parameters change on frame to frame
> basis. Additionally, the parameters are highly structured (and not a flat
> space of independent configuration primitives). They also reflect the
> data structures used by the firmware and the hardware. On top of that,
> the algorithms require highly specialized user space to make meaningful
> use of them. For these reasons it has been chosen video buffers to pass

Do you have a to-do list for this patchset? I think it would be useful to
maintain one, in case not all the comments have been addressed.

-- 
Sakari Ailus
e-mail: sakari.ai...@iki.fi
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Zhi, Yong]

Hi, Christoph,

Thanks for the message.

> -Original Message-
> From: Christoph Hellwig [mailto:h...@lst.de]
> Sent: Tuesday, October 17, 2017 11:23 PM
> To: Zhi, Yong <yong@intel.com>
> Cc: linux-me...@vger.kernel.org; sakari.ai...@linux.intel.com; Zheng, Jian
> Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan
> <rajmohan.m...@intel.com>; Toivonen, Tuukka
> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>;
> a...@arndb.de; h...@lst.de; robin.mur...@arm.com; iommu@lists.linux-
> foundation.org
> Subject: Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset
> 
> Please keep everyone on CC for all the patches, othervise they are complete
> unreviable and will be ignored.

Last time, Tomasz instructed me to cc you and other iommu experts only for 
mmu/dmamap patches(3 of 12), should I resend the 12 patches to both Linux-media 
and iommu list? or to Linux-media and cc everyone? Or just send the missing 
patches to iommu list and you folks this time? 

Yong 
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v4 00/12] Intel IPU3 ImgU patchset

[Christoph Hellwig]

Please keep everyone on CC for all the patches, othervise they are
complete unreviable and will be ignored.
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu