Re: [FFmpeg-devel] [PATCH, 4/7] lavu/hwcontext_vaapi: add vaapi_format_map support for 0YUV/Y210/Y410
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Mark Thompson
> Sent: Saturday, December 28, 2019 06:41
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH, 4/7] lavu/hwcontext_vaapi: add
> vaapi_format_map support for 0YUV/Y210/Y410
>
> On 04/12/2019 14:44, Linjie Fu wrote:
> > VA_RT_FORMAT describes the desired sampling format for surface.
> >
> > When creating surface, VA_RT_FORMAT will be used firstly to choose
> > the expected fourcc/media_format for the surface. And the fourcc
> > will be revised by the value of VASurfaceAttribPixelFormat.
> >
> > Add vaapi_format_map support for new pixel_format.
> > This is fundamental for both VA-API and QSV.
> >
> > Signed-off-by: Linjie Fu
> > ---
> > libavutil/hwcontext_vaapi.c | 7 +++
> > 1 file changed, 7 insertions(+)
> >
> > diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> > index cf11764..296234b 100644
> > --- a/libavutil/hwcontext_vaapi.c
> > +++ b/libavutil/hwcontext_vaapi.c
> > @@ -116,6 +116,13 @@ static const VAAPIFormatDescriptor
> vaapi_format_map[] = {
> > #endif
> > MAP(UYVY, YUV422, UYVY422, 0),
> > MAP(YUY2, YUV422, YUYV422, 0),
> > +#ifdef VA_FOURCC_Y210
> > +MAP(Y210, YUV422_10, Y210, 0),
> > +#endif
> > +MAP(AYUV, YUV444,0YUV, 0),
> > +#ifdef VA_FOURCC_Y410
> > +MAP(Y410, YUV444_10, Y410, 0),
> > +#endif
> > MAP(411P, YUV411, YUV411P, 0),
> > MAP(422V, YUV422, YUV440P, 0),
> > MAP(444P, YUV444, YUV444P, 0),
>
> (I've now bought an Ice Lake machine so that I can actually investigate what's
> going on here properly.)
Appreciated it.
> The real problem is that the formats advertised by the driver for H.265/VP9
> YUV 4:4:4 (AYUV and Y410) are incorrect. There isn't any alpha support
> anywhere in the hardware (none of the multilayer stuff), so using formats
> with alpha in them is wrong and makes all of the attempts to use it extremely
> confusing.
>
> To fix that we need to change the driver to use formats which actually reflect
> reality. This is mostly straightforward, because the inner code in the driver
> mainly cares about layout rather than what is actually in each channel.
>
> Some patches which kindof work (not particularly well tested):
>
> libva: <http://ixia.jkqxz.net/~mrt/4b8a1bb668ce23702f993f9ae7a8f96c/0001-
> Add-fourcc-values-for-YUV-4-4-4-formats.patch>
> iHD: <http://ixia.jkqxz.net/~mrt/4b8a1bb668ce23702f993f9ae7a8f96c/0001-
> Do-not-advertise-support-for-nonexistent-alpha-chann.patch>
I got some concerns for the modifications:
1. RT FORMAT YUV444 would not be bind with VA_FOURCC_444P any more
@@ -2136,10 +2151,13 @@ DdiMedia_CreateSurfaces2(
expected_fourcc = VA_FOURCC_Y216;
break;
case VA_RT_FORMAT_YUV444:
-expected_fourcc = VA_FOURCC_444P;
+expected_fourcc = VA_FOURCC_XYUV;
break;
VAAPI decode of jpeg requires VA_RT_FORMAT_YUV444 with expected_fourcc =
VA_FOURCC_444P.
(Though it's not used frequently), changing the expected fourcc into XYUV may
cause a change for
media_format from Media_Format_444P to Media_Format_XYUV.
2. The changes in MediaLibvaCaps::QuerySurfaceAttributes which eliminate
AYUV/Y410/Y416 may
affect the pipelines already implemented in MSDK and GStreamer. (If so, there
is a lot to be done/rework)
> I've called the non-alpha formats XYUV and XV30 to match libdrm (see
> <https://cgit.freedesktop.org/mesa/drm/tree/include/drm/drm_fourcc.h#n
> 164>). The 12-bit case also needs some correction - it's currently using
> Y216
> and I think assuming that the user knows they need to ignore the alpha
> channel and the lower bits elsewhere, but we might need to have distinct
> formats to make it work properly outside the driver.
>
> - Mark
>
> PS: It might help to split out the 4:2:2 case (which doesn't have these
> problems because the formats there don't have fake alpha channels or
> unknown depth) to go forward, and look separately at fixing the driver for
> the others.
Thanks, I'll split and get 4:2:2 stuffs done firstly.
- linjie
BTW, what's the plan for submit these modifications in libva/media-driver?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH, 4/7] lavu/hwcontext_vaapi: add vaapi_format_map support for 0YUV/Y210/Y410
On 04/12/2019 14:44, Linjie Fu wrote:
> VA_RT_FORMAT describes the desired sampling format for surface.
>
> When creating surface, VA_RT_FORMAT will be used firstly to choose
> the expected fourcc/media_format for the surface. And the fourcc
> will be revised by the value of VASurfaceAttribPixelFormat.
>
> Add vaapi_format_map support for new pixel_format.
> This is fundamental for both VA-API and QSV.
>
> Signed-off-by: Linjie Fu
> ---
> libavutil/hwcontext_vaapi.c | 7 +++
> 1 file changed, 7 insertions(+)
>
> diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> index cf11764..296234b 100644
> --- a/libavutil/hwcontext_vaapi.c
> +++ b/libavutil/hwcontext_vaapi.c
> @@ -116,6 +116,13 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = {
> #endif
> MAP(UYVY, YUV422, UYVY422, 0),
> MAP(YUY2, YUV422, YUYV422, 0),
> +#ifdef VA_FOURCC_Y210
> +MAP(Y210, YUV422_10, Y210, 0),
> +#endif
> +MAP(AYUV, YUV444,0YUV, 0),
> +#ifdef VA_FOURCC_Y410
> +MAP(Y410, YUV444_10, Y410, 0),
> +#endif
> MAP(411P, YUV411, YUV411P, 0),
> MAP(422V, YUV422, YUV440P, 0),
> MAP(444P, YUV444, YUV444P, 0),
(I've now bought an Ice Lake machine so that I can actually investigate what's
going on here properly.)
The real problem is that the formats advertised by the driver for H.265/VP9 YUV
4:4:4 (AYUV and Y410) are incorrect. There isn't any alpha support anywhere in
the hardware (none of the multilayer stuff), so using formats with alpha in
them is wrong and makes all of the attempts to use it extremely confusing.
To fix that we need to change the driver to use formats which actually reflect
reality. This is mostly straightforward, because the inner code in the driver
mainly cares about layout rather than what is actually in each channel.
Some patches which kindof work (not particularly well tested):
libva:
iHD:
I've called the non-alpha formats XYUV and XV30 to match libdrm (see
).
The 12-bit case also needs some correction - it's currently using Y216 and I
think assuming that the user knows they need to ignore the alpha channel and
the lower bits elsewhere, but we might need to have distinct formats to make it
work properly outside the driver.
- Mark
PS: It might help to split out the 4:2:2 case (which doesn't have these
problems because the formats there don't have fake alpha channels or unknown
depth) to go forward, and look separately at fixing the driver for the others.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH, 4/7] lavu/hwcontext_vaapi: add vaapi_format_map support for 0YUV/Y210/Y410
VA_RT_FORMAT describes the desired sampling format for surface.
When creating surface, VA_RT_FORMAT will be used firstly to choose
the expected fourcc/media_format for the surface. And the fourcc
will be revised by the value of VASurfaceAttribPixelFormat.
Add vaapi_format_map support for new pixel_format.
This is fundamental for both VA-API and QSV.
Signed-off-by: Linjie Fu
---
libavutil/hwcontext_vaapi.c | 7 +++
1 file changed, 7 insertions(+)
diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
index cf11764..296234b 100644
--- a/libavutil/hwcontext_vaapi.c
+++ b/libavutil/hwcontext_vaapi.c
@@ -116,6 +116,13 @@ static const VAAPIFormatDescriptor vaapi_format_map[] = {
#endif
MAP(UYVY, YUV422, UYVY422, 0),
MAP(YUY2, YUV422, YUYV422, 0),
+#ifdef VA_FOURCC_Y210
+MAP(Y210, YUV422_10, Y210, 0),
+#endif
+MAP(AYUV, YUV444,0YUV, 0),
+#ifdef VA_FOURCC_Y410
+MAP(Y410, YUV444_10, Y410, 0),
+#endif
MAP(411P, YUV411, YUV411P, 0),
MAP(422V, YUV422, YUV440P, 0),
MAP(444P, YUV444, YUV444P, 0),
--
2.7.4
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
