2015-07-30 16:37 GMT+08:00 wm4 <nfx...@googlemail.com>: > On Thu, 30 Jul 2015 16:08:36 +0800 > Zhang Rui <bbcal...@gmail.com> wrote: > >> 2015-07-30 0:01 GMT+08:00 Clément Bœsch <u...@pkh.me>: >> > On Tue, Jul 28, 2015 at 09:01:50PM +0800, Zhang Rui wrote: >> >> 2015-07-28 16:04 GMT+08:00 Clément Bœsch <u...@pkh.me>: >> >> > On Mon, Jul 27, 2015 at 10:10:12PM +0800, Zhang Rui wrote: >> >> >> 2015-07-27 20:14 GMT+08:00 Clément Bœsch <u...@pkh.me>: >> >> >> > On Mon, Jul 27, 2015 at 06:13:30PM +0800, Zhang Rui wrote: >> >> >> >> 2015-07-15 21:51 GMT+08:00 Clément Bœsch <u...@pkh.me>: >> >> >> >> > On Sat, Jun 20, 2015 at 01:33:00PM +0200, Sebastien Zwickert >> >> >> >> > wrote: >> >> >> >> >> Old videtotoolbox patch rebased and updated to target the new >> >> >> >> >> HWAccel API. >> >> >> >> >> As VDA is a wrapper of VideoToolbox framework, the changes base >> >> >> >> >> vda implementation >> >> >> >> >> upon the videotoolbox implementation to factorize common part of >> >> >> >> >> code. >> >> >> >> >> >> >> >> >> >> --- >> >> >> >> >> Changelog | 1 + >> >> >> >> >> MAINTAINERS | 1 + >> >> >> >> >> Makefile | 5 +- >> >> >> >> >> configure | 19 +- >> >> >> >> >> ffmpeg.h | 2 + >> >> >> >> >> ffmpeg_opt.c | 5 +- >> >> >> >> >> ffmpeg_vda.c | 134 -------- >> >> >> >> >> ffmpeg_videotoolbox.c | 147 +++++++++ >> >> >> >> >> libavcodec/Makefile | 12 +- >> >> >> >> >> libavcodec/allcodecs.c | 5 + >> >> >> >> >> libavcodec/h263dec.c | 3 + >> >> >> >> >> libavcodec/h264_slice.c | 4 + >> >> >> >> >> libavcodec/mpeg12dec.c | 3 + >> >> >> >> >> libavcodec/vda.c | 2 +- >> >> >> >> >> libavcodec/vda_h264.c | 154 +--------- >> >> >> >> >> libavcodec/vda_internal.h | 33 -- >> >> >> >> >> libavcodec/vda_vt_internal.h | 57 ++++ >> >> >> >> >> libavcodec/version.h | 2 +- >> >> >> >> >> libavcodec/videotoolbox.c | 705 >> >> >> >> >> +++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> >> libavcodec/videotoolbox.h | 126 ++++++++ >> >> >> >> >> libavutil/pixdesc.c | 4 + >> >> >> >> >> libavutil/pixfmt.h | 2 + >> >> >> >> > >> >> >> >> > I tried on iOS (iPhone 6+ so last one unless I'm mistaken) with a >> >> >> >> > 240fps >> >> >> >> > footage, and it seems the decode calls take more than 5ms, which >> >> >> >> > seems not >> >> >> >> > enough. Do you have any idea what could cause this? (I'm running >> >> >> >> > a timer >> >> >> >> > after/before avcodec_decode_video*()) >> >> >> >> >> >> >> >> 1. kVTDecodeFrame_EnableAsynchronousDecompression can make it >> >> >> >> faster, >> >> >> >> but not suitable for sync-style API (avcodec_decode_video*()) >> >> >> > >> >> >> > not suitable in the sense that it wouldn't work, or it would have >> >> >> > undesirable side effects? >> >> >> >> >> >> It would work, but it would be better and easier if decoder's input >> >> >> and output are decoupled. >> >> > >> >> >> Anyhow, as wm4 said, it's probably not worth the trouble. >> >> >> >> >> > >> >> > Well, if it makes possible for ffmpeg to decode 240fps footage @ 240Hz >> >> > that's kind of worth the trouble IMO. The HW decoder seems to be able to >> >> > decode frames between 700 to 1100µs, which makes it possible to handle >> >> > such material. A factor 5 to 6 with FFmpeg is kind of a problem IMO. >> >> >> >> Would you mind to share your 240fps test sample? >> >> I'd like to take a benchmark in my own player with async VideoToolbox. >> >> https://github.com/Bilibili/ijkplayer >> > >> > Well, you can just capture a video with the slowmo mode in the camera of >> > the iphone (that's where you plan to bench, right?). >> >> Thank you. >> >> This is my test result on iPhone6 (iOS 8.4): >> >> Metadata: >> major_brand : qt >> minor_version : 0 >> compatible_brands: qt >> creation_time : 2015-07-30 06:02:29 >> encoder : 8.4 >> encoder-zho : 8.4 >> date : 2015-07-30T14:02:29+0800 >> date-zho : 2015-07-30T14:02:29+0800 >> location : +31.2423+121.5203+020.306/ >> location-zho : +31.2423+121.5203+020.306/ >> model : iPhone 6 >> model-zho : iPhone 6 >> make : Apple >> make-zho : Apple >> Duration: 00:00:31.12, start: 0.000000, bitrate: 40163 kb/s >> Stream #0:0(und), 1, 1/2400: Video: h264, 1 reference frame (avc1 >> / 0x31637661), yuv420p(tv, bt709, left), 1280x720, 1/4800, 40069 kb/s, >> 239.84 fps, 240 tbr, 2400 tbn, 4800 tbc (default) >> Metadata: >> creation_time : 2015-07-30 06:02:29 >> handler_name : Core Media Data Handler >> encoder : H.264 >> Stream #0:1(und), 87, 1/44100: Audio: aac (LC) (mp4a / >> 0x6134706D), 44100 Hz, mono, fltp, 62 kb/s (default) >> Metadata: >> creation_time : 2015-07-30 06:02:29 >> handler_name : Core Media Data Handler >> >> 1. Frames and time are count in VTB callback. >> 2. Without rendering. >> 3. At most 3 AVPackets are sent to VTB before any decoded frame being >> received in callback. >> >> ----- sync ----- >> 192.886031 fps, 5.184409 ms/frame, 7440 frames >> 202.669572 fps, 4.934140 ms/frame, 7440 frames >> 207.768997 fps, 4.813038 ms/frame, 7440 frames >> 204.986913 fps, 4.878360 ms/frame, 7440 frames >> 200.836820 fps, 4.979167 ms/frame, 7440 frames >> 221.152131 fps, 4.521774 ms/frame, 7440 frames >> >> ----- async ----- >> 451.840155 fps, 2.213172 ms/frame, 7440 frames >> 450.117974 fps, 2.221640 ms/frame, 7440 frames >> 450.663275 fps, 2.218952 ms/frame, 7440 frames >> 453.686200 fps, 2.204167 ms/frame, 7440 frames >> 454.517686 fps, 2.200134 ms/frame, 7440 frames >> 454.739930 fps, 2.199059 ms/frame, 7440 frames > > Could this be fixed by using the async API in lavc and adding more > buffering?
Only easy if libavformat gurantee that an AVPacket contains one and only one frame. I don't know how VTB would act if AVPacket contains multiple frames. And I don't know how to retrieve last delayed frames with avcodec_decode_video*() after EOF. Maybe call it with a zero-len packet? BTW: I think current patch is OK enough. Async-mode could be an experimental option. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
