[RFC] Standardize YUV support in the fbdev API
Hi Florian,
On Saturday 21 May 2011 00:33:02 Florian Tobias Schandinat wrote:
> On 05/17/2011 10:07 PM, Laurent Pinchart wrote:
> > Hi everybody,
> >
> > I need to implement support for a YUV frame buffer in an fbdev driver. As
> > the fbdev API doesn't support this out of the box, I've spent a couple
> > of days reading fbdev (and KMS) code and thinking about how we could
> > cleanly add YUV support to the API. I'd like to share my findings and
> > thoughts, and hopefully receive some comments back.
>
> Thanks. I think you did already a good job, hope we can get it done this
> time.
Thanks. I'll keep pushing then :-)
> > Given the overlap between the KMS, V4L2 and fbdev APIs, the need to share
> > data and buffers between those subsystems, and the planned use of V4L2
> > FCCs in the KMS overlay API, I believe using V4L2 FCCs to identify fbdev
> > formats would be a wise decision.
>
> I agree.
There seems to be a general agreement on this, so I'll consider this settled.
> > To select a frame buffer YUV format, the fb_var_screeninfo structure will
> > need to be extended with a format field. The fbdev API and ABI must not
> > be broken, which prevents us from changing the current structure layout
> > and replacing the existing format selection mechanism (through the red,
> > green, blue and alpha bitfields) completely.
>
> I agree.
>
> > - Other solutions are possible, such as adding new ioctls. Those
> > solutions are more intrusive, and require larger changes to both
> > userspace and kernelspace code.
>
> I'm against (ab)using the nonstd field (probably the only sane thing we can
> do with it is declare it non-standard which interpretation is completely
> dependent on the specific driver) or requiring previously unused fields to
> have a special value so I'd like to suggest a different method:
>
> I remembered an earlier discussion:
> [ http://marc.info/?l=linux-fbdev=129896686208130=2 ]
>
> On 03/01/2011 08:07 AM, Geert Uytterhoeven wrote:
> > On Tue, Mar 1, 2011 at 04:13, Damian wrote:
> >> On 2011/02/24 15:05, Geert Uytterhoeven wrote:
> >>> For YUV (do you mean YCbCr?), I'm inclined to suggest adding a new
> >>> FB_VISUAL_*
> >>> type instead, which indicates the fb_var_screeninfo.{red,green,blue}
> >>> fields are
> >>> YCbCr instead of RGB.
> >>> Depending on the frame buffer organization, you also need new
> >>> FB_TYPE_*/FB_AUX_*
> >>> types.
> >>
> >> I just wanted to clarify here. Is your comment about these new flags
> >> in specific reference to this patch or to Magnus' "going forward"
> >> comment? It
> >
> > About new flags.
> >
> >> seems like the beginnings of a method to standardize YCbCr support in
> >> fbdev across all platforms.
> >> Also, do I understand correctly that FB_VISUAL_ would specify the
> >> colorspace
> >
> > FB_VISUAL_* specifies how pixel values (which may be tuples) are mapped
> > to colors on the screen, so to me it looks like the sensible way to set
> > up YCbCr.
> >
> >> (RGB, YCbCr), FB_TYPE_* would be a format specifier (i.e. planar,
> >> semiplanar, interleaved, etc)? I'm not really sure what you are
> >> referring to with the FB_AUX_* however.
> >
> > Yep, FB_TYPE_* specifies how pixel values/tuples are laid out in frame
> > buffer memory.
> >
> > FB_AUX_* is only used if a specific value of FB_TYPE_* needs an
> > additional parameter (e.g. the interleave value for interleaved
> > bitplanes).
>
> Adding new standard values for these fb_fix_screeninfo fields would solve
> the issue for framebuffers which only support a single format.
I've never liked changing fixed screen information :-) It would be consistent
with the API though.
> If you have the need to switch
Yes I need that. This requires an API to set the mode through
fb_var_screeninfo, which is why I skipped modifying fb_fix_screeinfo.
A new FB_TYPE_* could be used to report that we use a 4CC-based mode, with the
exact mode reported in one of the fb_fix_screeninfo reserved fields (or the
type_aux field). This would duplicate the information passed through
fb_var_screeninfo though. Do you think it's worth it ?
> I guess it would be a good idea to add a new flag to the vmode bitfield in
> fb_var_screeninfo which looks like a general purpose modifier for the
> videomode. You could than reuse any RGB-specific field you like to pass more
> information.
That looks good to me. The grayscale field could be reused to pass the 4CC.
> Maybe we should also use this chance to declare one of the fix_screeninfo
> reserved fields to be used for capability flags or an API version as we can
> assume that those are 0 (at least in sane drivers).
That's always good, although it's not a hard requirement for the purpose of
YUV support.
--
Regards,
Laurent Pinchart
[RFC] Standardize YUV support in the fbdev API
Hi Laurent,
On 05/23/2011 09:00 PM, Laurent Pinchart wrote:
> On Saturday 21 May 2011 00:33:02 Florian Tobias Schandinat wrote:
>> On 05/17/2011 10:07 PM, Laurent Pinchart wrote:
>>> - Other solutions are possible, such as adding new ioctls. Those
>>> solutions are more intrusive, and require larger changes to both
>>> userspace and kernelspace code.
>>
>> I'm against (ab)using the nonstd field (probably the only sane thing we can
>> do with it is declare it non-standard which interpretation is completely
>> dependent on the specific driver) or requiring previously unused fields to
>> have a special value so I'd like to suggest a different method:
>>
>> I remembered an earlier discussion:
>> [ http://marc.info/?l=linux-fbdev=129896686208130=2 ]
>>
>> On 03/01/2011 08:07 AM, Geert Uytterhoeven wrote:
>> > On Tue, Mar 1, 2011 at 04:13, Damian wrote:
>> >> On 2011/02/24 15:05, Geert Uytterhoeven wrote:
>> >>> For YUV (do you mean YCbCr?), I'm inclined to suggest adding a new
>> >>> FB_VISUAL_*
>> >>> type instead, which indicates the fb_var_screeninfo.{red,green,blue}
>> >>> fields are
>> >>> YCbCr instead of RGB.
>> >>> Depending on the frame buffer organization, you also need new
>> >>> FB_TYPE_*/FB_AUX_*
>> >>> types.
>> >>
>> >> I just wanted to clarify here. Is your comment about these new flags
>> >> in specific reference to this patch or to Magnus' "going forward"
>> >> comment? It
>> >
>> > About new flags.
>> >
>> >> seems like the beginnings of a method to standardize YCbCr support in
>> >> fbdev across all platforms.
>> >> Also, do I understand correctly that FB_VISUAL_ would specify the
>> >> colorspace
>> >
>> > FB_VISUAL_* specifies how pixel values (which may be tuples) are mapped
>> > to colors on the screen, so to me it looks like the sensible way to set
>> > up YCbCr.
>> >
>> >> (RGB, YCbCr), FB_TYPE_* would be a format specifier (i.e. planar,
>> >> semiplanar, interleaved, etc)? I'm not really sure what you are
>> >> referring to with the FB_AUX_* however.
>> >
>> > Yep, FB_TYPE_* specifies how pixel values/tuples are laid out in frame
>> > buffer memory.
>> >
>> > FB_AUX_* is only used if a specific value of FB_TYPE_* needs an
>> > additional parameter (e.g. the interleave value for interleaved
>> > bitplanes).
>>
>> Adding new standard values for these fb_fix_screeninfo fields would solve
>> the issue for framebuffers which only support a single format.
>
> I've never liked changing fixed screen information :-) It would be consistent
> with the API though.
Fixed does only mean that it can't be directly manipulated by applications. The
driver has to modify it anyway on about every mode change (line_length). Yes
perhaps some of these fields would be in var today and certainly others
shouldn't exist at all but I do not blame anyone for not being capable to look
into the future.
>> If you have the need to switch
>
> Yes I need that. This requires an API to set the mode through
> fb_var_screeninfo, which is why I skipped modifying fb_fix_screeinfo.
>
> A new FB_TYPE_* could be used to report that we use a 4CC-based mode, with the
> exact mode reported in one of the fb_fix_screeninfo reserved fields (or the
> type_aux field). This would duplicate the information passed through
> fb_var_screeninfo though. Do you think it's worth it ?
I think it's more like a FB_VISUAL_FOURCC as you want to express how the color
<-> pixel mapping is. The FB_TYPE_* is more about whether pixel are packed or
represented as planes (the 2 format groups mentioned on fourcc.org).
That's certainly something I'd introduce as it would (hopefully) work to
prevent
old applications which don't know your extension manipulating a FOURCC format
thinking that it is RGB.
So I think we should
fix.visual = FB_VISUAL_FOURCC;
fix.type = FB_TYPE_PACKED_PIXELS | FB_TYPE_PLANES; /* (?) */
or maybe add a FB_TYPE_FOURCC and rely only on the information in FOURCC (as
things like UYVY could become confusing as macropixel!=pixel)
>> I guess it would be a good idea to add a new flag to the vmode bitfield in
>> fb_var_screeninfo which looks like a general purpose modifier for the
>> videomode. You could than reuse any RGB-specific field you like to pass more
>> information.
>
> That looks good to me. The grayscale field could be reused to pass the 4CC.
var.grayscale = ;
var.vmode = FB_VMODE_FOURCC;
and if this vmode flag is not set it means traditional mode (based on RGBA).
>> Maybe we should also use this chance to declare one of the fix_screeninfo
>> reserved fields to be used for capability flags or an API version as we can
>> assume that those are 0 (at least in sane drivers).
>
> That's always good, although it's not a hard requirement for the purpose of
> YUV support.
Sure. But it's good to let the application know whether you support the new
extension or whether you just ignore the flag. So I'm voting for a
[RFC] Standardize YUV support in the fbdev API
On Monday, May 23, 2011 13:55:21 Marek Szyprowski wrote: > Hello, > > On Wednesday, May 18, 2011 8:54 AM Hans Verkuil wrote: > > > On Wednesday, May 18, 2011 00:44:26 Felipe Contreras wrote: > > > On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart > > > wrote: > > > > I need to implement support for a YUV frame buffer in an fbdev driver. > > As the > > > > fbdev API doesn't support this out of the box, I've spent a couple of > > days > > > > reading fbdev (and KMS) code and thinking about how we could cleanly > > add YUV > > > > support to the API. I'd like to share my findings and thoughts, and > > hopefully > > > > receive some comments back. > > > > > > > > The terms 'format', 'pixel format', 'frame buffer format' and 'data > > format' > > > > will be used interchangeably in this e-mail. They all refer to the way > > pixels > > > > are stored in memory, including both the representation of a pixel as > > integer > > > > values and the layout of those integer values in memory. > > > > > > This is a great proposal. It was about time! > > > > > > > The third solution has my preference. Comments and feedback will be > > > > appreciated. I will then work on a proof of concept and submit patches. > > > > > > I also would prefer the third solution. I don't think there's much > > > difference from the user-space point of view, and a new ioctl would be > > > cleaner. Also the v4l2 fourcc's should do. > > > > I agree with this. > > > > We might want to take the opportunity to fix this section of the V4L2 Spec: > > > > http://www.xs4all.nl/~hverkuil/spec/media.html#pixfmt-rgb > > > > There are two tables, 2.6 and 2.7. But 2.6 is almost certainly wrong and > > should be removed. > > That's definitely true. I was confused at the beginning when I saw 2 > different tables describing the same pixel formats. > > I suspect many if not all V4L2 drivers are badly broken for > > big-endian systems and report the wrong pixel formats. > > > > Officially the pixel formats reflect the contents of the memory. But > > everything is swapped on a big endian system, so you are supposed to > > report a different pix format. > > I always thought that pix_format describes the layout of video data in > memory on byte level, which is exactly the same on both little- and big- > endian systems. Correct. > You can notice swapped data only if you access memory > by units larger than byte, like 16bit or 32bit integers. BTW - I would > really like to avoid little- and big- endian flame, but your statement > about 'everything is swapped on a big endian system' is completely > wrong. It is rather the characteristic of little-endian system not big > endian one if you display the content of the same memory first using > byte access and then using word/long access. You are correct, I wasn't thinking it through. > > I can't remember seeing any driver do that. Some have built-in swapping, > > though, and turn that on if needed. > > The drivers shouldn't do ANY byte swapping at all. Only tools that > extract pixel data with some 'accelerated' methods (like 32bit integer > casting and bit-level shifting) should be aware of endianess. > > > I really need to run some tests, but I've been telling myself this for > > years now :-( > > I've checked the BTTV board in my PowerMac/G4 and the display was > correct with xawtv. It is just a matter of selecting correct pix format > basing on the information returned by xsever. > > Best regards > Just forget my post (except for the part of cleaning up the tables :-) ). Regards, Hans
[RFC] Standardize YUV support in the fbdev API
Hello, On Wednesday, May 18, 2011 8:54 AM Hans Verkuil wrote: > On Wednesday, May 18, 2011 00:44:26 Felipe Contreras wrote: > > On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart > > wrote: > > > I need to implement support for a YUV frame buffer in an fbdev driver. > As the > > > fbdev API doesn't support this out of the box, I've spent a couple of > days > > > reading fbdev (and KMS) code and thinking about how we could cleanly > add YUV > > > support to the API. I'd like to share my findings and thoughts, and > hopefully > > > receive some comments back. > > > > > > The terms 'format', 'pixel format', 'frame buffer format' and 'data > format' > > > will be used interchangeably in this e-mail. They all refer to the way > pixels > > > are stored in memory, including both the representation of a pixel as > integer > > > values and the layout of those integer values in memory. > > > > This is a great proposal. It was about time! > > > > > The third solution has my preference. Comments and feedback will be > > > appreciated. I will then work on a proof of concept and submit patches. > > > > I also would prefer the third solution. I don't think there's much > > difference from the user-space point of view, and a new ioctl would be > > cleaner. Also the v4l2 fourcc's should do. > > I agree with this. > > We might want to take the opportunity to fix this section of the V4L2 Spec: > > http://www.xs4all.nl/~hverkuil/spec/media.html#pixfmt-rgb > > There are two tables, 2.6 and 2.7. But 2.6 is almost certainly wrong and > should be removed. That's definitely true. I was confused at the beginning when I saw 2 different tables describing the same pixel formats. I suspect many if not all V4L2 drivers are badly broken for > big-endian systems and report the wrong pixel formats. > > Officially the pixel formats reflect the contents of the memory. But > everything is swapped on a big endian system, so you are supposed to > report a different pix format. I always thought that pix_format describes the layout of video data in memory on byte level, which is exactly the same on both little- and big- endian systems. You can notice swapped data only if you access memory by units larger than byte, like 16bit or 32bit integers. BTW - I would really like to avoid little- and big- endian flame, but your statement about 'everything is swapped on a big endian system' is completely wrong. It is rather the characteristic of little-endian system not big endian one if you display the content of the same memory first using byte access and then using word/long access. > I can't remember seeing any driver do that. Some have built-in swapping, > though, and turn that on if needed. The drivers shouldn't do ANY byte swapping at all. Only tools that extract pixel data with some 'accelerated' methods (like 32bit integer casting and bit-level shifting) should be aware of endianess. > I really need to run some tests, but I've been telling myself this for > years now :-( I've checked the BTTV board in my PowerMac/G4 and the display was correct with xawtv. It is just a matter of selecting correct pix format basing on the information returned by xsever. Best regards -- Marek Szyprowski Samsung Poland R Center
Re: [RFC] Standardize YUV support in the fbdev API
On Monday, May 23, 2011 13:55:21 Marek Szyprowski wrote: Hello, On Wednesday, May 18, 2011 8:54 AM Hans Verkuil wrote: On Wednesday, May 18, 2011 00:44:26 Felipe Contreras wrote: On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart laurent.pinch...@ideasonboard.com wrote: I need to implement support for a YUV frame buffer in an fbdev driver. As the fbdev API doesn't support this out of the box, I've spent a couple of days reading fbdev (and KMS) code and thinking about how we could cleanly add YUV support to the API. I'd like to share my findings and thoughts, and hopefully receive some comments back. The terms 'format', 'pixel format', 'frame buffer format' and 'data format' will be used interchangeably in this e-mail. They all refer to the way pixels are stored in memory, including both the representation of a pixel as integer values and the layout of those integer values in memory. This is a great proposal. It was about time! The third solution has my preference. Comments and feedback will be appreciated. I will then work on a proof of concept and submit patches. I also would prefer the third solution. I don't think there's much difference from the user-space point of view, and a new ioctl would be cleaner. Also the v4l2 fourcc's should do. I agree with this. We might want to take the opportunity to fix this section of the V4L2 Spec: http://www.xs4all.nl/~hverkuil/spec/media.html#pixfmt-rgb There are two tables, 2.6 and 2.7. But 2.6 is almost certainly wrong and should be removed. That's definitely true. I was confused at the beginning when I saw 2 different tables describing the same pixel formats. I suspect many if not all V4L2 drivers are badly broken for big-endian systems and report the wrong pixel formats. Officially the pixel formats reflect the contents of the memory. But everything is swapped on a big endian system, so you are supposed to report a different pix format. I always thought that pix_format describes the layout of video data in memory on byte level, which is exactly the same on both little- and big- endian systems. Correct. You can notice swapped data only if you access memory by units larger than byte, like 16bit or 32bit integers. BTW - I would really like to avoid little- and big- endian flame, but your statement about 'everything is swapped on a big endian system' is completely wrong. It is rather the characteristic of little-endian system not big endian one if you display the content of the same memory first using byte access and then using word/long access. You are correct, I wasn't thinking it through. I can't remember seeing any driver do that. Some have built-in swapping, though, and turn that on if needed. The drivers shouldn't do ANY byte swapping at all. Only tools that extract pixel data with some 'accelerated' methods (like 32bit integer casting and bit-level shifting) should be aware of endianess. I really need to run some tests, but I've been telling myself this for years now :-( I've checked the BTTV board in my PowerMac/G4 and the display was correct with xawtv. It is just a matter of selecting correct pix format basing on the information returned by xsever. Best regards Just forget my post (except for the part of cleaning up the tables :-) ). Regards, Hans ___ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
RE: [RFC] Standardize YUV support in the fbdev API
Hello, On Wednesday, May 18, 2011 8:54 AM Hans Verkuil wrote: On Wednesday, May 18, 2011 00:44:26 Felipe Contreras wrote: On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart laurent.pinch...@ideasonboard.com wrote: I need to implement support for a YUV frame buffer in an fbdev driver. As the fbdev API doesn't support this out of the box, I've spent a couple of days reading fbdev (and KMS) code and thinking about how we could cleanly add YUV support to the API. I'd like to share my findings and thoughts, and hopefully receive some comments back. The terms 'format', 'pixel format', 'frame buffer format' and 'data format' will be used interchangeably in this e-mail. They all refer to the way pixels are stored in memory, including both the representation of a pixel as integer values and the layout of those integer values in memory. This is a great proposal. It was about time! The third solution has my preference. Comments and feedback will be appreciated. I will then work on a proof of concept and submit patches. I also would prefer the third solution. I don't think there's much difference from the user-space point of view, and a new ioctl would be cleaner. Also the v4l2 fourcc's should do. I agree with this. We might want to take the opportunity to fix this section of the V4L2 Spec: http://www.xs4all.nl/~hverkuil/spec/media.html#pixfmt-rgb There are two tables, 2.6 and 2.7. But 2.6 is almost certainly wrong and should be removed. That's definitely true. I was confused at the beginning when I saw 2 different tables describing the same pixel formats. I suspect many if not all V4L2 drivers are badly broken for big-endian systems and report the wrong pixel formats. Officially the pixel formats reflect the contents of the memory. But everything is swapped on a big endian system, so you are supposed to report a different pix format. I always thought that pix_format describes the layout of video data in memory on byte level, which is exactly the same on both little- and big- endian systems. You can notice swapped data only if you access memory by units larger than byte, like 16bit or 32bit integers. BTW - I would really like to avoid little- and big- endian flame, but your statement about 'everything is swapped on a big endian system' is completely wrong. It is rather the characteristic of little-endian system not big endian one if you display the content of the same memory first using byte access and then using word/long access. I can't remember seeing any driver do that. Some have built-in swapping, though, and turn that on if needed. The drivers shouldn't do ANY byte swapping at all. Only tools that extract pixel data with some 'accelerated' methods (like 32bit integer casting and bit-level shifting) should be aware of endianess. I really need to run some tests, but I've been telling myself this for years now :-( I've checked the BTTV board in my PowerMac/G4 and the display was correct with xawtv. It is just a matter of selecting correct pix format basing on the information returned by xsever. Best regards -- Marek Szyprowski Samsung Poland RD Center ___ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC] Standardize YUV support in the fbdev API
Hi Florian,
On Saturday 21 May 2011 00:33:02 Florian Tobias Schandinat wrote:
On 05/17/2011 10:07 PM, Laurent Pinchart wrote:
Hi everybody,
I need to implement support for a YUV frame buffer in an fbdev driver. As
the fbdev API doesn't support this out of the box, I've spent a couple
of days reading fbdev (and KMS) code and thinking about how we could
cleanly add YUV support to the API. I'd like to share my findings and
thoughts, and hopefully receive some comments back.
Thanks. I think you did already a good job, hope we can get it done this
time.
Thanks. I'll keep pushing then :-)
Given the overlap between the KMS, V4L2 and fbdev APIs, the need to share
data and buffers between those subsystems, and the planned use of V4L2
FCCs in the KMS overlay API, I believe using V4L2 FCCs to identify fbdev
formats would be a wise decision.
I agree.
There seems to be a general agreement on this, so I'll consider this settled.
To select a frame buffer YUV format, the fb_var_screeninfo structure will
need to be extended with a format field. The fbdev API and ABI must not
be broken, which prevents us from changing the current structure layout
and replacing the existing format selection mechanism (through the red,
green, blue and alpha bitfields) completely.
I agree.
- Other solutions are possible, such as adding new ioctls. Those
solutions are more intrusive, and require larger changes to both
userspace and kernelspace code.
I'm against (ab)using the nonstd field (probably the only sane thing we can
do with it is declare it non-standard which interpretation is completely
dependent on the specific driver) or requiring previously unused fields to
have a special value so I'd like to suggest a different method:
I remembered an earlier discussion:
[ http://marc.info/?l=linux-fbdevm=129896686208130w=2 ]
On 03/01/2011 08:07 AM, Geert Uytterhoeven wrote:
On Tue, Mar 1, 2011 at 04:13, Damiandhobs...@igel.co.jp wrote:
On 2011/02/24 15:05, Geert Uytterhoeven wrote:
For YUV (do you mean YCbCr?), I'm inclined to suggest adding a new
FB_VISUAL_*
type instead, which indicates the fb_var_screeninfo.{red,green,blue}
fields are
YCbCr instead of RGB.
Depending on the frame buffer organization, you also need new
FB_TYPE_*/FB_AUX_*
types.
I just wanted to clarify here. Is your comment about these new flags
in specific reference to this patch or to Magnus' going forward
comment? It
About new flags.
seems like the beginnings of a method to standardize YCbCr support in
fbdev across all platforms.
Also, do I understand correctly that FB_VISUAL_ would specify the
colorspace
FB_VISUAL_* specifies how pixel values (which may be tuples) are mapped
to colors on the screen, so to me it looks like the sensible way to set
up YCbCr.
(RGB, YCbCr), FB_TYPE_* would be a format specifier (i.e. planar,
semiplanar, interleaved, etc)? I'm not really sure what you are
referring to with the FB_AUX_* however.
Yep, FB_TYPE_* specifies how pixel values/tuples are laid out in frame
buffer memory.
FB_AUX_* is only used if a specific value of FB_TYPE_* needs an
additional parameter (e.g. the interleave value for interleaved
bitplanes).
Adding new standard values for these fb_fix_screeninfo fields would solve
the issue for framebuffers which only support a single format.
I've never liked changing fixed screen information :-) It would be consistent
with the API though.
If you have the need to switch
Yes I need that. This requires an API to set the mode through
fb_var_screeninfo, which is why I skipped modifying fb_fix_screeinfo.
A new FB_TYPE_* could be used to report that we use a 4CC-based mode, with the
exact mode reported in one of the fb_fix_screeninfo reserved fields (or the
type_aux field). This would duplicate the information passed through
fb_var_screeninfo though. Do you think it's worth it ?
I guess it would be a good idea to add a new flag to the vmode bitfield in
fb_var_screeninfo which looks like a general purpose modifier for the
videomode. You could than reuse any RGB-specific field you like to pass more
information.
That looks good to me. The grayscale field could be reused to pass the 4CC.
Maybe we should also use this chance to declare one of the fix_screeninfo
reserved fields to be used for capability flags or an API version as we can
assume that those are 0 (at least in sane drivers).
That's always good, although it's not a hard requirement for the purpose of
YUV support.
--
Regards,
Laurent Pinchart
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC] Standardize YUV support in the fbdev API
Hi Laurent,
On 05/23/2011 09:00 PM, Laurent Pinchart wrote:
On Saturday 21 May 2011 00:33:02 Florian Tobias Schandinat wrote:
On 05/17/2011 10:07 PM, Laurent Pinchart wrote:
- Other solutions are possible, such as adding new ioctls. Those
solutions are more intrusive, and require larger changes to both
userspace and kernelspace code.
I'm against (ab)using the nonstd field (probably the only sane thing we can
do with it is declare it non-standard which interpretation is completely
dependent on the specific driver) or requiring previously unused fields to
have a special value so I'd like to suggest a different method:
I remembered an earlier discussion:
[ http://marc.info/?l=linux-fbdevm=129896686208130w=2 ]
On 03/01/2011 08:07 AM, Geert Uytterhoeven wrote:
On Tue, Mar 1, 2011 at 04:13, Damiandhobs...@igel.co.jp wrote:
On 2011/02/24 15:05, Geert Uytterhoeven wrote:
For YUV (do you mean YCbCr?), I'm inclined to suggest adding a new
FB_VISUAL_*
type instead, which indicates the fb_var_screeninfo.{red,green,blue}
fields are
YCbCr instead of RGB.
Depending on the frame buffer organization, you also need new
FB_TYPE_*/FB_AUX_*
types.
I just wanted to clarify here. Is your comment about these new flags
in specific reference to this patch or to Magnus' going forward
comment? It
About new flags.
seems like the beginnings of a method to standardize YCbCr support in
fbdev across all platforms.
Also, do I understand correctly that FB_VISUAL_ would specify the
colorspace
FB_VISUAL_* specifies how pixel values (which may be tuples) are mapped
to colors on the screen, so to me it looks like the sensible way to set
up YCbCr.
(RGB, YCbCr), FB_TYPE_* would be a format specifier (i.e. planar,
semiplanar, interleaved, etc)? I'm not really sure what you are
referring to with the FB_AUX_* however.
Yep, FB_TYPE_* specifies how pixel values/tuples are laid out in frame
buffer memory.
FB_AUX_* is only used if a specific value of FB_TYPE_* needs an
additional parameter (e.g. the interleave value for interleaved
bitplanes).
Adding new standard values for these fb_fix_screeninfo fields would solve
the issue for framebuffers which only support a single format.
I've never liked changing fixed screen information :-) It would be consistent
with the API though.
Fixed does only mean that it can't be directly manipulated by applications. The
driver has to modify it anyway on about every mode change (line_length). Yes
perhaps some of these fields would be in var today and certainly others
shouldn't exist at all but I do not blame anyone for not being capable to look
into the future.
If you have the need to switch
Yes I need that. This requires an API to set the mode through
fb_var_screeninfo, which is why I skipped modifying fb_fix_screeinfo.
A new FB_TYPE_* could be used to report that we use a 4CC-based mode, with the
exact mode reported in one of the fb_fix_screeninfo reserved fields (or the
type_aux field). This would duplicate the information passed through
fb_var_screeninfo though. Do you think it's worth it ?
I think it's more like a FB_VISUAL_FOURCC as you want to express how the color
- pixel mapping is. The FB_TYPE_* is more about whether pixel are packed or
represented as planes (the 2 format groups mentioned on fourcc.org).
That's certainly something I'd introduce as it would (hopefully) work to prevent
old applications which don't know your extension manipulating a FOURCC format
thinking that it is RGB.
So I think we should
fix.visual = FB_VISUAL_FOURCC;
fix.type = FB_TYPE_PACKED_PIXELS | FB_TYPE_PLANES; /* (?) */
or maybe add a FB_TYPE_FOURCC and rely only on the information in FOURCC (as
things like UYVY could become confusing as macropixel!=pixel)
I guess it would be a good idea to add a new flag to the vmode bitfield in
fb_var_screeninfo which looks like a general purpose modifier for the
videomode. You could than reuse any RGB-specific field you like to pass more
information.
That looks good to me. The grayscale field could be reused to pass the 4CC.
var.grayscale = FOURCC_FORMAT;
var.vmode = FB_VMODE_FOURCC;
and if this vmode flag is not set it means traditional mode (based on RGBA).
Maybe we should also use this chance to declare one of the fix_screeninfo
reserved fields to be used for capability flags or an API version as we can
assume that those are 0 (at least in sane drivers).
That's always good, although it's not a hard requirement for the purpose of
YUV support.
Sure. But it's good to let the application know whether you support the new
extension or whether you just ignore the flag. So I'm voting for a
fix.caps = FB_CAP_FOURCC;
Best regards,
Florian Tobias Schandinat
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
[RFC] Standardize YUV support in the fbdev API
Hi Laurent,
On 05/17/2011 10:07 PM, Laurent Pinchart wrote:
> Hi everybody,
>
> I need to implement support for a YUV frame buffer in an fbdev driver. As the
> fbdev API doesn't support this out of the box, I've spent a couple of days
> reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
> support to the API. I'd like to share my findings and thoughts, and hopefully
> receive some comments back.
Thanks. I think you did already a good job, hope we can get it done this time.
> Given the overlap between the KMS, V4L2 and fbdev APIs, the need to share data
> and buffers between those subsystems, and the planned use of V4L2 FCCs in the
> KMS overlay API, I believe using V4L2 FCCs to identify fbdev formats would be
> a wise decision.
I agree.
> To select a frame buffer YUV format, the fb_var_screeninfo structure will need
> to be extended with a format field. The fbdev API and ABI must not be broken,
> which prevents us from changing the current structure layout and replacing the
> existing format selection mechanism (through the red, green, blue and alpha
> bitfields) completely.
I agree.
> - Other solutions are possible, such as adding new ioctls. Those solutions are
> more intrusive, and require larger changes to both userspace and kernelspace
> code.
I'm against (ab)using the nonstd field (probably the only sane thing we can do
with it is declare it non-standard which interpretation is completely dependent
on the specific driver) or requiring previously unused fields to have a special
value so I'd like to suggest a different method:
I remembered an earlier discussion:
[ http://marc.info/?l=linux-fbdev=129896686208130=2 ]
On 03/01/2011 08:07 AM, Geert Uytterhoeven wrote:
> On Tue, Mar 1, 2011 at 04:13, Damian wrote:
>> On 2011/02/24 15:05, Geert Uytterhoeven wrote:
>>> For YUV (do you mean YCbCr?), I'm inclined to suggest adding a new
>>> FB_VISUAL_*
>>> type instead, which indicates the fb_var_screeninfo.{red,green,blue}
>>> fields are
>>> YCbCr instead of RGB.
>>> Depending on the frame buffer organization, you also need new
>>> FB_TYPE_*/FB_AUX_*
>>> types.
>>
>> I just wanted to clarify here. Is your comment about these new flags in
>> specific reference to this patch or to Magnus' "going forward" comment? It
>
> About new flags.
>
>> seems like the beginnings of a method to standardize YCbCr support in fbdev
>> across all platforms.
>> Also, do I understand correctly that FB_VISUAL_ would specify the colorspace
>
> FB_VISUAL_* specifies how pixel values (which may be tuples) are mapped to
> colors on the screen, so to me it looks like the sensible way to set up
> YCbCr.
>
>> (RGB, YCbCr), FB_TYPE_* would be a format specifier (i.e. planar,
>> semiplanar, interleaved, etc)? I'm not really sure what you are referring
>> to with the FB_AUX_* however.
>
> Yep, FB_TYPE_* specifies how pixel values/tuples are laid out in frame buffer
> memory.
>
> FB_AUX_* is only used if a specific value of FB_TYPE_* needs an additional
> parameter (e.g. the interleave value for interleaved bitplanes).
Adding new standard values for these fb_fix_screeninfo fields would solve the
issue for framebuffers which only support a single format. If you have the need
to switch I guess it would be a good idea to add a new flag to the vmode
bitfield in fb_var_screeninfo which looks like a general purpose modifier for
the videomode. You could than reuse any RGB-specific field you like to pass
more
information.
Maybe we should also use this chance to declare one of the fix_screeninfo
reserved fields to be used for capability flags or an API version as we can
assume that those are 0 (at least in sane drivers).
Good luck,
Florian Tobias Schandinat
Re: [RFC] Standardize YUV support in the fbdev API
Hi Laurent,
On 05/17/2011 10:07 PM, Laurent Pinchart wrote:
Hi everybody,
I need to implement support for a YUV frame buffer in an fbdev driver. As the
fbdev API doesn't support this out of the box, I've spent a couple of days
reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
support to the API. I'd like to share my findings and thoughts, and hopefully
receive some comments back.
Thanks. I think you did already a good job, hope we can get it done this time.
Given the overlap between the KMS, V4L2 and fbdev APIs, the need to share data
and buffers between those subsystems, and the planned use of V4L2 FCCs in the
KMS overlay API, I believe using V4L2 FCCs to identify fbdev formats would be
a wise decision.
I agree.
To select a frame buffer YUV format, the fb_var_screeninfo structure will need
to be extended with a format field. The fbdev API and ABI must not be broken,
which prevents us from changing the current structure layout and replacing the
existing format selection mechanism (through the red, green, blue and alpha
bitfields) completely.
I agree.
- Other solutions are possible, such as adding new ioctls. Those solutions are
more intrusive, and require larger changes to both userspace and kernelspace
code.
I'm against (ab)using the nonstd field (probably the only sane thing we can do
with it is declare it non-standard which interpretation is completely dependent
on the specific driver) or requiring previously unused fields to have a special
value so I'd like to suggest a different method:
I remembered an earlier discussion:
[ http://marc.info/?l=linux-fbdevm=129896686208130w=2 ]
On 03/01/2011 08:07 AM, Geert Uytterhoeven wrote:
On Tue, Mar 1, 2011 at 04:13, Damiandhobs...@igel.co.jp wrote:
On 2011/02/24 15:05, Geert Uytterhoeven wrote:
For YUV (do you mean YCbCr?), I'm inclined to suggest adding a new
FB_VISUAL_*
type instead, which indicates the fb_var_screeninfo.{red,green,blue}
fields are
YCbCr instead of RGB.
Depending on the frame buffer organization, you also need new
FB_TYPE_*/FB_AUX_*
types.
I just wanted to clarify here. Is your comment about these new flags in
specific reference to this patch or to Magnus' going forward comment? It
About new flags.
seems like the beginnings of a method to standardize YCbCr support in fbdev
across all platforms.
Also, do I understand correctly that FB_VISUAL_ would specify the colorspace
FB_VISUAL_* specifies how pixel values (which may be tuples) are mapped to
colors on the screen, so to me it looks like the sensible way to set up YCbCr.
(RGB, YCbCr), FB_TYPE_* would be a format specifier (i.e. planar,
semiplanar, interleaved, etc)? I'm not really sure what you are referring
to with the FB_AUX_* however.
Yep, FB_TYPE_* specifies how pixel values/tuples are laid out in frame buffer
memory.
FB_AUX_* is only used if a specific value of FB_TYPE_* needs an additional
parameter (e.g. the interleave value for interleaved bitplanes).
Adding new standard values for these fb_fix_screeninfo fields would solve the
issue for framebuffers which only support a single format. If you have the need
to switch I guess it would be a good idea to add a new flag to the vmode
bitfield in fb_var_screeninfo which looks like a general purpose modifier for
the videomode. You could than reuse any RGB-specific field you like to pass more
information.
Maybe we should also use this chance to declare one of the fix_screeninfo
reserved fields to be used for capability flags or an API version as we can
assume that those are 0 (at least in sane drivers).
Good luck,
Florian Tobias Schandinat
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
[RFC] Standardize YUV support in the fbdev API
On Wednesday, May 18, 2011 00:44:26 Felipe Contreras wrote: > On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart > wrote: > > I need to implement support for a YUV frame buffer in an fbdev driver. As > > the > > fbdev API doesn't support this out of the box, I've spent a couple of days > > reading fbdev (and KMS) code and thinking about how we could cleanly add YUV > > support to the API. I'd like to share my findings and thoughts, and > > hopefully > > receive some comments back. > > > > The terms 'format', 'pixel format', 'frame buffer format' and 'data format' > > will be used interchangeably in this e-mail. They all refer to the way > > pixels > > are stored in memory, including both the representation of a pixel as > > integer > > values and the layout of those integer values in memory. > > This is a great proposal. It was about time! > > > The third solution has my preference. Comments and feedback will be > > appreciated. I will then work on a proof of concept and submit patches. > > I also would prefer the third solution. I don't think there's much > difference from the user-space point of view, and a new ioctl would be > cleaner. Also the v4l2 fourcc's should do. I agree with this. We might want to take the opportunity to fix this section of the V4L2 Spec: http://www.xs4all.nl/~hverkuil/spec/media.html#pixfmt-rgb There are two tables, 2.6 and 2.7. But 2.6 is almost certainly wrong and should be removed. I suspect many if not all V4L2 drivers are badly broken for big-endian systems and report the wrong pixel formats. Officially the pixel formats reflect the contents of the memory. But everything is swapped on a big endian system, so you are supposed to report a different pix format. I can't remember seeing any driver do that. Some have built-in swapping, though, and turn that on if needed. I really need to run some tests, but I've been telling myself this for years now :-( Regards, Hans
[RFC] Standardize YUV support in the fbdev API
Hi everybody,
I need to implement support for a YUV frame buffer in an fbdev driver. As the
fbdev API doesn't support this out of the box, I've spent a couple of days
reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
support to the API. I'd like to share my findings and thoughts, and hopefully
receive some comments back.
The terms 'format', 'pixel format', 'frame buffer format' and 'data format'
will be used interchangeably in this e-mail. They all refer to the way pixels
are stored in memory, including both the representation of a pixel as integer
values and the layout of those integer values in memory.
History and current situation
-
The fbdev API predates YUV frame buffers. In those old days developers only
cared (and probably thought) about RGB frame buffers, and they developed an
API that could express all kind of weird RGB layout in memory (such as R-
GGG- for instance, I can't imagine hardware implementing that).
This resulted in individual bit field description for the red, green, blue and
alpha components:
struct fb_bitfield {
__u32 offset; /* beginning of bitfield*/
__u32 length; /* length of bitfield */
__u32 msb_right; /* != 0 : Most significant bit is */
/* right */
};
Grayscale formats were pretty common, so a grayscale field tells color formats
(grayscale == 0) from grayscale formats (grayscale != 0).
People already realized that hardware developers were crazily inventive (the
word to remember here is crazily), and that non-standard formats would be
needed at some point. The fb_var_screeninfo ended up containing the following
format-related fields.
struct fb_var_screeninfo {
...
__u32 bits_per_pixel; /* guess what */
__u32 grayscale; /* != 0 Graylevels instead of colors */
struct fb_bitfield red;/* bitfield in fb mem if true color, */
struct fb_bitfield green; /* else only length is significant */
struct fb_bitfield blue;
struct fb_bitfield transp; /* transparency */
__u32 nonstd; /* != 0 Non standard pixel format */
...
};
Two bits have been specified for the nonstd field:
#define FB_NONSTD_HAM 1 /* Hold-And-Modify (HAM)*/
#define FB_NONSTD_REV_PIX_IN_B 2 /* order of pixels in each byte is reversed
*/
The FB_NONSTD_HAM bit is used by the video/amifb.c driver only to enable HAM
mode[1]. I very much doubt that any new hardware will implement HAM mode (and
I certainly hope none will).
The FB_NONSTD_REV_PIX_IN_B is used in video/fb_draw.h by the generic bitblit,
fillrect and copy area implementations, not directly by drivers.
A couple of drivers hardcode the nonstd field to 1 for some reason. Those are
video/arcfb.c (1bpp gray display), video/hecubafb.c (1bpp gray display) and
video/metronomefb.c (8bpp gray display).
The following drivers use nonstd for various other (and sometimes weird)
purposes:
video/arkfb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/fsl-diu-fb.c
Set var->nonstd bits into var->sync (why?)
video/pxafb.c
Encode frame buffer xpos and ypos in the nonstd field
video/s3fb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/vga16fb.c
When panning in non-8bpp, non-text mode, decrement xoffset
Do some other weird stuff when not 0
video/i810/i810_main.c
Select direct color mode when set to 1 (truecolor otherwise)
Fast forward a couple of years, hardware provides support for YUV frame
buffers. Several drivers had to provide YUV format selection to applications,
without any standard API to do so. All of them used the nonstd field for that
purpose:
media/video/ivtv/
Enable YUV mode when set to 1
video/pxafb.c
Encode pixel format in the nonstd field
video/sh_mobile_lcdfb.c
If bpp == 12 and nonstd != 0, enable NV12 mode
If bpp == 16 or bpp == 24, ?
video/omap/omapfb_main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422, YUV420 or YUY422)
video/omap2/omapfb/omapfb-main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422 or YUY422)
Those drivers use the nonstd field in different, incompatible ways.
Other related APIs
--
Beside the fbdev API, two other kernel/userspace APIs deal with video-related
modes and formats.
- Kernel Mode Setting (KMS)
The KMS API is similar in purpose to XRandR. Its main purpose is to provide a
kernel API to configure output video modes. As such, it doesn't care about
frame buffer formats, as they are irrelevant at the CRTC output.
In addition to handling
Re: [RFC] Standardize YUV support in the fbdev API
On Wednesday, May 18, 2011 00:44:26 Felipe Contreras wrote: On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart laurent.pinch...@ideasonboard.com wrote: I need to implement support for a YUV frame buffer in an fbdev driver. As the fbdev API doesn't support this out of the box, I've spent a couple of days reading fbdev (and KMS) code and thinking about how we could cleanly add YUV support to the API. I'd like to share my findings and thoughts, and hopefully receive some comments back. The terms 'format', 'pixel format', 'frame buffer format' and 'data format' will be used interchangeably in this e-mail. They all refer to the way pixels are stored in memory, including both the representation of a pixel as integer values and the layout of those integer values in memory. This is a great proposal. It was about time! The third solution has my preference. Comments and feedback will be appreciated. I will then work on a proof of concept and submit patches. I also would prefer the third solution. I don't think there's much difference from the user-space point of view, and a new ioctl would be cleaner. Also the v4l2 fourcc's should do. I agree with this. We might want to take the opportunity to fix this section of the V4L2 Spec: http://www.xs4all.nl/~hverkuil/spec/media.html#pixfmt-rgb There are two tables, 2.6 and 2.7. But 2.6 is almost certainly wrong and should be removed. I suspect many if not all V4L2 drivers are badly broken for big-endian systems and report the wrong pixel formats. Officially the pixel formats reflect the contents of the memory. But everything is swapped on a big endian system, so you are supposed to report a different pix format. I can't remember seeing any driver do that. Some have built-in swapping, though, and turn that on if needed. I really need to run some tests, but I've been telling myself this for years now :-( Regards, Hans ___ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
[RFC] Standardize YUV support in the fbdev API
Oops. Nevermind, you already have looked at what ivtvfb does. -Andy
[RFC] Standardize YUV support in the fbdev API
On Wed, 2011-05-18 at 00:07 +0200, Laurent Pinchart wrote:
> Hi everybody,
>
> I need to implement support for a YUV frame buffer in an fbdev driver. As the
> fbdev API doesn't support this out of the box, I've spent a couple of days
> reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
> support to the API. I'd like to share my findings and thoughts, and hopefully
> receive some comments back.
I haven't looked at anything below, but I'll mention that the ivtv
driver presents an fbdev interface for the YUV output stream of the
CX23415.
It may be worth a look and asking Hans what are the short-comings.
-Andy
> The terms 'format', 'pixel format', 'frame buffer format' and 'data format'
> will be used interchangeably in this e-mail. They all refer to the way pixels
> are stored in memory, including both the representation of a pixel as integer
> values and the layout of those integer values in memory.
>
>
> History and current situation
> -
>
> The fbdev API predates YUV frame buffers. In those old days developers only
> cared (and probably thought) about RGB frame buffers, and they developed an
> API that could express all kind of weird RGB layout in memory (such as R-
> GGG- for instance, I can't imagine hardware implementing that).
> This resulted in individual bit field description for the red, green, blue
> and
> alpha components:
>
> struct fb_bitfield {
> __u32 offset; /* beginning of bitfield*/
> __u32 length; /* length of bitfield */
> __u32 msb_right; /* != 0 : Most significant bit is */
>/* right */
> };
>
> Grayscale formats were pretty common, so a grayscale field tells color
> formats
> (grayscale == 0) from grayscale formats (grayscale != 0).
>
> People already realized that hardware developers were crazily inventive (the
> word to remember here is crazily), and that non-standard formats would be
> needed at some point. The fb_var_screeninfo ended up containing the following
> format-related fields.
>
> struct fb_var_screeninfo {
> ...
> __u32 bits_per_pixel; /* guess what */
> __u32 grayscale; /* != 0 Graylevels instead of colors */
>
> struct fb_bitfield red;/* bitfield in fb mem if true color, */
> struct fb_bitfield green; /* else only length is significant */
> struct fb_bitfield blue;
> struct fb_bitfield transp; /* transparency */
>
> __u32 nonstd; /* != 0 Non standard pixel format */
> ...
> };
>
> Two bits have been specified for the nonstd field:
>
> #define FB_NONSTD_HAM 1 /* Hold-And-Modify (HAM)*/
> #define FB_NONSTD_REV_PIX_IN_B 2 /* order of pixels in each byte is
> reversed
> */
>
> The FB_NONSTD_HAM bit is used by the video/amifb.c driver only to enable HAM
> mode[1]. I very much doubt that any new hardware will implement HAM mode (and
> I certainly hope none will).
>
> The FB_NONSTD_REV_PIX_IN_B is used in video/fb_draw.h by the generic bitblit,
> fillrect and copy area implementations, not directly by drivers.
>
> A couple of drivers hardcode the nonstd field to 1 for some reason. Those are
> video/arcfb.c (1bpp gray display), video/hecubafb.c (1bpp gray display) and
> video/metronomefb.c (8bpp gray display).
>
> The following drivers use nonstd for various other (and sometimes weird)
> purposes:
>
> video/arkfb.c
> Used in 4bpp mode only, to control fb_setcolreg operation
> video/fsl-diu-fb.c
> Set var->nonstd bits into var->sync (why?)
> video/pxafb.c
> Encode frame buffer xpos and ypos in the nonstd field
> video/s3fb.c
> Used in 4bpp mode only, to control fb_setcolreg operation
> video/vga16fb.c
> When panning in non-8bpp, non-text mode, decrement xoffset
> Do some other weird stuff when not 0
> video/i810/i810_main.c
> Select direct color mode when set to 1 (truecolor otherwise)
>
> Fast forward a couple of years, hardware provides support for YUV frame
> buffers. Several drivers had to provide YUV format selection to applications,
> without any standard API to do so. All of them used the nonstd field for that
> purpose:
>
> media/video/ivtv/
> Enable YUV mode when set to 1
> video/pxafb.c
> Encode pixel format in the nonstd field
> video/sh_mobile_lcdfb.c
> If bpp == 12 and nonstd != 0, enable NV12 mode
> If bpp == 16 or bpp == 24, ?
> video/omap/omapfb_main.c
> Select direct color mode when set to 1 (depend on bpp otherwise)
> Used as a pixel format identifier (YUV422, YUV420 or YUY422)
> video/omap2/omapfb/omapfb-main.c
> Select direct color mode when set to 1 (depend on bpp otherwise)
> Used as a pixel format identifier
[RFC] Standardize YUV support in the fbdev API
Hi everybody,
I need to implement support for a YUV frame buffer in an fbdev driver. As the
fbdev API doesn't support this out of the box, I've spent a couple of days
reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
support to the API. I'd like to share my findings and thoughts, and hopefully
receive some comments back.
The terms 'format', 'pixel format', 'frame buffer format' and 'data format'
will be used interchangeably in this e-mail. They all refer to the way pixels
are stored in memory, including both the representation of a pixel as integer
values and the layout of those integer values in memory.
History and current situation
-
The fbdev API predates YUV frame buffers. In those old days developers only
cared (and probably thought) about RGB frame buffers, and they developed an
API that could express all kind of weird RGB layout in memory (such as R-
GGG- for instance, I can't imagine hardware implementing that).
This resulted in individual bit field description for the red, green, blue and
alpha components:
struct fb_bitfield {
__u32 offset; /* beginning of bitfield*/
__u32 length; /* length of bitfield */
__u32 msb_right; /* != 0 : Most significant bit is */
/* right */
};
Grayscale formats were pretty common, so a grayscale field tells color formats
(grayscale == 0) from grayscale formats (grayscale != 0).
People already realized that hardware developers were crazily inventive (the
word to remember here is crazily), and that non-standard formats would be
needed at some point. The fb_var_screeninfo ended up containing the following
format-related fields.
struct fb_var_screeninfo {
...
__u32 bits_per_pixel; /* guess what */
__u32 grayscale; /* != 0 Graylevels instead of colors */
struct fb_bitfield red;/* bitfield in fb mem if true color, */
struct fb_bitfield green; /* else only length is significant */
struct fb_bitfield blue;
struct fb_bitfield transp; /* transparency */
__u32 nonstd; /* != 0 Non standard pixel format */
...
};
Two bits have been specified for the nonstd field:
#define FB_NONSTD_HAM 1 /* Hold-And-Modify (HAM)*/
#define FB_NONSTD_REV_PIX_IN_B 2 /* order of pixels in each byte is reversed
*/
The FB_NONSTD_HAM bit is used by the video/amifb.c driver only to enable HAM
mode[1]. I very much doubt that any new hardware will implement HAM mode (and
I certainly hope none will).
The FB_NONSTD_REV_PIX_IN_B is used in video/fb_draw.h by the generic bitblit,
fillrect and copy area implementations, not directly by drivers.
A couple of drivers hardcode the nonstd field to 1 for some reason. Those are
video/arcfb.c (1bpp gray display), video/hecubafb.c (1bpp gray display) and
video/metronomefb.c (8bpp gray display).
The following drivers use nonstd for various other (and sometimes weird)
purposes:
video/arkfb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/fsl-diu-fb.c
Set var-nonstd bits into var-sync (why?)
video/pxafb.c
Encode frame buffer xpos and ypos in the nonstd field
video/s3fb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/vga16fb.c
When panning in non-8bpp, non-text mode, decrement xoffset
Do some other weird stuff when not 0
video/i810/i810_main.c
Select direct color mode when set to 1 (truecolor otherwise)
Fast forward a couple of years, hardware provides support for YUV frame
buffers. Several drivers had to provide YUV format selection to applications,
without any standard API to do so. All of them used the nonstd field for that
purpose:
media/video/ivtv/
Enable YUV mode when set to 1
video/pxafb.c
Encode pixel format in the nonstd field
video/sh_mobile_lcdfb.c
If bpp == 12 and nonstd != 0, enable NV12 mode
If bpp == 16 or bpp == 24, ?
video/omap/omapfb_main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422, YUV420 or YUY422)
video/omap2/omapfb/omapfb-main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422 or YUY422)
Those drivers use the nonstd field in different, incompatible ways.
Other related APIs
--
Beside the fbdev API, two other kernel/userspace APIs deal with video-related
modes and formats.
- Kernel Mode Setting (KMS)
The KMS API is similar in purpose to XRandR. Its main purpose is to provide a
kernel API to configure output video modes. As such, it doesn't care about
frame buffer formats, as they are irrelevant at the CRTC output.
In addition to handling
Re: [RFC] Standardize YUV support in the fbdev API
On Wed, May 18, 2011 at 1:07 AM, Laurent Pinchart laurent.pinch...@ideasonboard.com wrote: I need to implement support for a YUV frame buffer in an fbdev driver. As the fbdev API doesn't support this out of the box, I've spent a couple of days reading fbdev (and KMS) code and thinking about how we could cleanly add YUV support to the API. I'd like to share my findings and thoughts, and hopefully receive some comments back. The terms 'format', 'pixel format', 'frame buffer format' and 'data format' will be used interchangeably in this e-mail. They all refer to the way pixels are stored in memory, including both the representation of a pixel as integer values and the layout of those integer values in memory. This is a great proposal. It was about time! The third solution has my preference. Comments and feedback will be appreciated. I will then work on a proof of concept and submit patches. I also would prefer the third solution. I don't think there's much difference from the user-space point of view, and a new ioctl would be cleaner. Also the v4l2 fourcc's should do. Cheers. -- Felipe Contreras ___ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [RFC] Standardize YUV support in the fbdev API
On Wed, 2011-05-18 at 00:07 +0200, Laurent Pinchart wrote:
Hi everybody,
I need to implement support for a YUV frame buffer in an fbdev driver. As the
fbdev API doesn't support this out of the box, I've spent a couple of days
reading fbdev (and KMS) code and thinking about how we could cleanly add YUV
support to the API. I'd like to share my findings and thoughts, and hopefully
receive some comments back.
I haven't looked at anything below, but I'll mention that the ivtv
driver presents an fbdev interface for the YUV output stream of the
CX23415.
It may be worth a look and asking Hans what are the short-comings.
-Andy
The terms 'format', 'pixel format', 'frame buffer format' and 'data format'
will be used interchangeably in this e-mail. They all refer to the way pixels
are stored in memory, including both the representation of a pixel as integer
values and the layout of those integer values in memory.
History and current situation
-
The fbdev API predates YUV frame buffers. In those old days developers only
cared (and probably thought) about RGB frame buffers, and they developed an
API that could express all kind of weird RGB layout in memory (such as R-
GGG- for instance, I can't imagine hardware implementing that).
This resulted in individual bit field description for the red, green, blue
and
alpha components:
struct fb_bitfield {
__u32 offset; /* beginning of bitfield*/
__u32 length; /* length of bitfield */
__u32 msb_right; /* != 0 : Most significant bit is */
/* right */
};
Grayscale formats were pretty common, so a grayscale field tells color
formats
(grayscale == 0) from grayscale formats (grayscale != 0).
People already realized that hardware developers were crazily inventive (the
word to remember here is crazily), and that non-standard formats would be
needed at some point. The fb_var_screeninfo ended up containing the following
format-related fields.
struct fb_var_screeninfo {
...
__u32 bits_per_pixel; /* guess what */
__u32 grayscale; /* != 0 Graylevels instead of colors */
struct fb_bitfield red;/* bitfield in fb mem if true color, */
struct fb_bitfield green; /* else only length is significant */
struct fb_bitfield blue;
struct fb_bitfield transp; /* transparency */
__u32 nonstd; /* != 0 Non standard pixel format */
...
};
Two bits have been specified for the nonstd field:
#define FB_NONSTD_HAM 1 /* Hold-And-Modify (HAM)*/
#define FB_NONSTD_REV_PIX_IN_B 2 /* order of pixels in each byte is
reversed
*/
The FB_NONSTD_HAM bit is used by the video/amifb.c driver only to enable HAM
mode[1]. I very much doubt that any new hardware will implement HAM mode (and
I certainly hope none will).
The FB_NONSTD_REV_PIX_IN_B is used in video/fb_draw.h by the generic bitblit,
fillrect and copy area implementations, not directly by drivers.
A couple of drivers hardcode the nonstd field to 1 for some reason. Those are
video/arcfb.c (1bpp gray display), video/hecubafb.c (1bpp gray display) and
video/metronomefb.c (8bpp gray display).
The following drivers use nonstd for various other (and sometimes weird)
purposes:
video/arkfb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/fsl-diu-fb.c
Set var-nonstd bits into var-sync (why?)
video/pxafb.c
Encode frame buffer xpos and ypos in the nonstd field
video/s3fb.c
Used in 4bpp mode only, to control fb_setcolreg operation
video/vga16fb.c
When panning in non-8bpp, non-text mode, decrement xoffset
Do some other weird stuff when not 0
video/i810/i810_main.c
Select direct color mode when set to 1 (truecolor otherwise)
Fast forward a couple of years, hardware provides support for YUV frame
buffers. Several drivers had to provide YUV format selection to applications,
without any standard API to do so. All of them used the nonstd field for that
purpose:
media/video/ivtv/
Enable YUV mode when set to 1
video/pxafb.c
Encode pixel format in the nonstd field
video/sh_mobile_lcdfb.c
If bpp == 12 and nonstd != 0, enable NV12 mode
If bpp == 16 or bpp == 24, ?
video/omap/omapfb_main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422, YUV420 or YUY422)
video/omap2/omapfb/omapfb-main.c
Select direct color mode when set to 1 (depend on bpp otherwise)
Used as a pixel format identifier (YUV422 or YUY422)
Those drivers use the nonstd field in different, incompatible ways.
Other related APIs
Re: [RFC] Standardize YUV support in the fbdev API
Oops. Nevermind, you already have looked at what ivtvfb does. -Andy ___ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
