On Tue, Feb 26, 2013 at 6:18 AM, Brian Matherly <pez4br...@yahoo.com> wrote: >> On Mon, Feb 25, 2013 at 10:26 PM, Dan Dennedy <d...@dennedy.org> wrote: > >>> On Mon, Feb 25, 2013 at 10:20 PM, Dan Dennedy <d...@dennedy.org> >> wrote: >>>> On Mon, Feb 25, 2013 at 8:45 PM, Brian Matherly >> <pez4br...@yahoo.com> wrote: >>>>> Dan, >>>>> >>>>> I added a new module: avsync. It has one producer and one consumer. >> The producer generates a stream with a classic blip/flash pattern. The >> consumer >> attempts to detect the blip/flash and calculate the audio/video sync. The >> consumer will report the caluclated lipsync in milliseconds. A positive >> number >> indicates that audio leads video. A negative number indicates that audio lags >> video. >>>>> >> [,,,] >>>>> melt blipflash -consumer libdv:test.dv >>>>> melt -silent test.dv -consumer blipflash >>>>> avsync = -40.00 >>>>> >>>> >>>> It is interesting to note that >>>> >>>> melt blipflash -consumer libdv:test.dv >>> >>> excuse me, I meant >>> melt -silent libdv:test.dv -consumer blipflash >>> >>> using the libdv producer instead of avformat. >>> >>>> gives 0.00, which lends itself to a possible off-by-one error in the >>>> avformat producer since this is PAL, which is 40ms per frame. >>> >>> I started to look into this, but like the other isolation problem, now >>> I am not sure whether to look at audio or video initially! >> >> I created a blipflash DV with frame numbers: >> melt blipflash -attach dynamictext:#frame# bgcolour=white -consumer >> libdv:test.dv >> >> Then, I generated an image sequence with the libdv producer: >> melt -silent libdvtest.dv -consumer avformat:test-%04d.jpg progressive=1 >> >> It looks fine. Then, using the avformat producer: >> melt -silent test.dv -consumer avformat:test-%04d.jpg progressive=1 >> >> The result shows frame 0 followed by frame 2! >> >> Now, applying an in point: >> melt -silent test.dv in=5 -consumer avformat:test-%04d.jpg progressive=1 >> >> test-0001.jpg shows frame# 6 instead of 5. >> >> Off-by-one in the video handling of the avformat producer. > > Nice! It looks like the module has been useful already. I Hope I didn't just > create a whole bunch of work for you :) >
I digged into that problem last night, found the culprit, and made a quick change to fix it. However, as I suspected might be the case, it created a huge regression for some other cases, namely your mkv and mov tests in the originating post. I am talking ~300ms off. I have put a huge amount of effort into a/v sync and seeking over the years, and I am very reluctant to make changes due to the large amount of testing to balance requirements and get it right. Here is a quick rundown of the requirements: - large variety of formats/codecs - large variety of files (tools and devices used to produce them, each with a variety of versions) - a variety of ffmpeg and now libav versions As if that amount of diversity alone is not enough: - frame accurate and fast seeking where possible, including AVCHD - live (network stream and device) input (no seeking required) - large runs with no memory leaks - image caching to support filters such as YADIF and telecide that need previous and next frames - large numbers of files in a composition without consuming RAM or file handles - a special mode to decode all audio streams in a mux - various combinations of the above - cross-platform - very limited human resources to develop and support All I can say is thank goodness for FFmpeg and libav without which none of this would be possible, but there is still a lot of work for MLT to make effective and comprehensive usage of their API. I do not plan to revisit this task in the near term. With that said... > My plan was to try to script some automated tests to run weekly on the build > server. That way we would know if any avsync regressions occurred. But it > didn't occur to me that we might have to fix up some producers/consumers. My > plan was to use +/-3ms as my pass/fail criteria (hence my previous comment). > But maybe it would make more sense to use one frame duration - at least until > everything gets dialed in. > I would like to develop a plan to test for regression against a baseline so I can try to make a change or improvement and have more automated testing to verify it. Starting with the avformat producer, we need a set of reference inputs, but we do not have them, and it does not make sense to produce them with the MLT avformat consumer. But we can use blipflash producer to generate a raw DV and verify that with dvthumbs.c. Then, we can use the ffmpeg command line tool to generate a variety of encodings. We still do not know if those are correct, but they do provide a best effort baseline against which we can measure deviations to check for regressions, analyze trends, and investigate the hotspots (popular or important formats that show a large offset). > When you run "melt -silent", do you use CTL-C to stop it? Is there any way to > tell melt to exit after x frames? > yes, ctrl+c. If you want to go a certain number of frames set an out point on the producer and termnate_on_pause=1 on the consumer, but the consumer has to be written to exit the consumer thread's loop when that property is set. -- +-DRD-+ ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb _______________________________________________ Mlt-devel mailing list Mlt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mlt-devel
