2015-11-11 8:27 GMT+08:00 Clément Bœsch <u...@pkh.me>: > From: Clément Bœsch <clem...@stupeflix.com> > > --- > > So here is a first prototype of a higher level API following an > asynchronous model to (for now) simply use the current synchronous API. > > I suggest to look at libavutil/async.h (sorry no doxy yet) and > doc/examples/async_demuxing_decoding.c for an example of usage. > > Basically what the current draft proposes for the user is to instanciate > an asynchronous context, in which you register Readers (the user will > generally do a demuxing in the associated callback to get a packet), in > which you register Decoders (basically just a wrapper for an > AVCodecContext) to which you associate a push frame callback. > > You construct something like this: > > AVAsyncContext( > AVAsyncReader( > AVAsyncDecoder(...), > AVAsyncDecoder(...), > AVAsyncDecoder(...), > ... > ), > AVAsyncReader( > AVAsyncDecoder(...), > AVAsyncDecoder(...), > ... > ), > .... > ) > > You will generally have a Reader per file but that can be anything you > want as long as you can pull of packets. > > Now implementation wise, the interesting part is that every level is > asynchronous: > > - The context will spawn a thread for each reader > - Then in each reader, you will have a thread for each decoder > - Then each decoder will have its own queue of packets, and its own > queue of output frames > - Then along with each decoder, you will have Watcher in its dedicated > thread: the watcher is monitoring the frame queue of its decoder, and > will call the user push frame callback anytime a new one appears in > the queue. > > The interesting part about the watcher is that it will allow the user to > transparently do any operation on the frame without blocking the > decoding or the demuxing: basically, blocking in push_frame for the user > will just block any subsequent call to the callback, but the packet > and frame queues of the decoder will continue to fill up, ready to be > pop'ed. > > Note: the size of the packet and frame queue for each decoder is > configurable > > Note2: the packet queue is not set in the reader so that a decoder that > isn't pulling its packets and decoding fast enough will not block the > others. As a result, the Reader just reads one packet and is then > responsible for broadcasting it to the appropriate decoder (by just > adding it to its internal queue). > > Now all of this doesn't require any modification to the current API so > it doesn't really solve the problem of writing asynchronous > decoders/hwaccel. The goal for the next iteration will be to adjust the > VideoToolbox hwaccel to make use of the asynchronous model, and > basically just make it call the user callback by itself. It will be a > good candidate for such thing. > > Another concerning limitation in the current code is the inability to > seek: basically the user can not lock the codec and reader context to > execute a seek, nor flush the queues. And I'd like suggestion of > direction in that regard:
If we want a fat reader, seek should be handled inside reader, and let user call AVAsyncReader.seek_async(...), to request reader handle seek operation at next round before av_read_frame(). If we want a thin reader, just let user push packet into reader's internal packet queue. Anyhow, I don't think it's a good idea to put av_read_frame() and avformat_seek_file() into different thread. > - first, in threadmessage it seems I can't yet flush the fifo properly > (for example by calling av_frame_free() on each entry of the frame > queue): should I extend the API? Nicolas, maybe you have a suggestion? Maybe serial number in frame_queue as ffplay's packet_queue? > - secondly, how should I allow the user to lock/unlock the reader (and > as a result the likely AVFormatContext user side) and decoder? I added > a few prototypes in async.h but I'm pretty sure that won't fit many > cases. Comments very welcome. IMO, we don't need to allow user to lock/unlock reader/decoder, we just provide thread-safe reader/decoder API. > I hope this is going in a direction most developers like. If it's not > the case, I'm of course open to any reworking but please be specific. > > Also, I'm really sorry I'm not sending this to libav-devel even so I > apparently looked forward a colaboration in a previous thread. I was > planning to but unfortunately the threadmessage API is not there yet and > it would have require way too much work for an initial prototype. When > we come up with something that satisfy everyone and this code becomes > something more than just a draft, I will maybe do the necessary. > Unfortunately, porting VideoToolbox accel, threadmessage and probably a > bunch of other ffmpeg specific features (I can thing about > av_dynarray2_add and maybe more) is going to require a bit too much code > adjustment, so I might just share the async.h and redirect them to a > working implementation here.... > > Anyway, please comment. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel