Hi,
2014-09-18 11:59 GMT+02:00 Rémi Denis-Courmont <[email protected]>:
> Le 2014-09-18 12:20, Gwenole Beauchesne a écrit :
>>>
>>> Except not. Maybe VA only needs a pointer and if we forego type safety,
>>> it
>>> can easily be made into a void pointer:
>>>
>>> int av_hwaccel_alloc(AVCodecContext *, enum AVPixelFormat, void *);
>>>
>>> Yes but... VDPAU needs a 32-bits integer *and* a pointer, so that does
>>> not
>>> quite work. And I would not assume that other interfaces can conveniently
>>> be
>>> made into a single void pointer either.
>>>
>>> Also, not that I care myself, but I don't see a credible story for
>>> preemption handling if there is a generic function.
>>
>>
>> In this case,
>> - av_hwaccel_device, where you could have an
>> av_hwaccel_{bind,import}_HWACCEL_device()
>> - av_hwaccel_context (internal or public struct), and the void * above
>> is your av_hwaccel_device.
>
>
> Compare with file descriptors: They are all destroyed with close(), but the
> functions and function prototypes to create them varies by type.
>
>> bonus point: you can still have the freedom to delegate hwaccel_device
>> allocation to libavcodec too.
>
>
> You could define a function for the special case that the application wants
> to use hardware decoding in a completely transparent fashion. But that is a
> special case of its own. I still don't see the benefit in not having
> specific functions for what are non-transparent cases anyway.
The transparent usage model doesn't need anything special, but a flag
and possibly the hwaccel_device if libavcodec is not told to allocate
its own.
>> Your hwaccel_device is where you could import a pool of hwaccel
>> surfaces if needed.
>>
>> However, we need to have at least one common baseline usage model.
>
>
> This may be possible for decoding-only acceleration (into CPU memory), but I
> don't think it even makes sense for full acceleration. Surface allocation is
> per-backend, post-processing is per-backend and rendering is per-backend.
Surface allocation is per-backend, and video processing is
per-backend, yes. But that backend implementation can live in
libavcodec in most practical use cases on Linux at least. There is no
real reason to have that process be a per-application burden either...
As to rendering, on certain platforms (Linux), this can be a quite
common thing... called EGL for instance. i.e. you'd just need an
additional way to expose the underlying surface buffer handles.
The bottom line would be:
1. Create an hwaccel device, if needed
2. Create an hwaccel context for libavcodec
3. (CPU case) no further work needed, you have the raw YUV pixels,
(GPU case) you could opt for exposing a native hwaccel buffer handle
that the rendering layer could handle (EGL)
That shouldn't be more complicated than that. Of course, more
elaborated usage models are still allowed, e.g.
- Supply your own surface pool ;
- Let the application developer supplice himself to handle various
combos of { vaapi, vdpau, omxil } x { x11, wayland, egl } ;
- etc.
I honestly think we should start off an application developer point of
view. Do we really want him to know the finest details of per-hwaccel
initialization process, hand cruft various rendering options, etc.? Or
should we try to make usages simple, while still making it possible to
have more elaborated use-cases?
>> Otherwise, client application developers would get confused on the
>> normal operational model.
>
>
> The only case where you can really abstract the backend away from the
> application developer is where you copy decoded content back to the CPU. And
> that is the most useless case, as it is often slower than software decoding.
I don't really want to focus on legacy platforms where what you
mention might hold. :)
In the future, or today on certain HW/OS combos, transferring to
user-backed memory, or exposing suitable buffer handles for 3rdparty &
common rendering APIs, is quite possible at minimal exra cost
(performance-wise, but also developer time-wise). And my opinion is
not biased, because it does also include a handful number of other
vendors.
>> BTW, we should really stop abusing pixel-formats for identifying
>> hwaccels. We could have an hwaccel_id enum just fine, or a string
>> literal. Or did the AVPixelFormat arg above serve other purpose in
>> your proposal?
>
>
> I agree that for certain use cases, we would need a separate
> get_hw_context() callback before get_format(). Indeed, we cannot use pixel
> formats to distinguish hardware-accelerated decoding to CPU memory, from
> normal software decoding.
>
> *But* when the surfaces are not mapped to the CPU, then I think a separate
> pixel format is actually sound. It describes what to expect of the AVFrame
> structure afterall.
I object here. If a pixel format is needed to represent "go away, no
system memory backed AVFrame here", a unique pixfmt ("hwaccel") is all
that is required, assuming we go the hwaccel_id route, and that
information would already be available elsewhere because we used that
to initialize the hwaccel_context in the first place. i.e. there is no
real need to replicate in pixfmt (I'm vaapi, I'm vdpau, I'm dxva2,
etc.), what we would already know through other means.
>> I mean, if this is for the AVFrame system memory
>> buffers, that's readily a known process.
>
>
> Sure.
Regards,
--
Gwenole Beauchesne
Intel Corporation SAS / 2 rue de Paris, 92196 Meudon Cedex, France
Registration Number (RCS): Nanterre B 302 456 199
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
