On Thu, Oct 4, 2012 at 2:31 AM, Ed Rogalsky <[email protected]> wrote:
> Hi Dan,
>
> the last days I worked on the jack seeking and I hope this is the last
> major step before contributing the code to kdenlive.
> There are some problems with the current jack_sync implementation.
>
> I parsed the jack documantion and found that it is a little bit
> unclear. Thats why I studied some jack clients like ardour or
> qtractor and the jackd implementation. Here the results:
>
> Jack Sync theory:
> ==============
>
> The jack_sync callback is designed only for position relocation
> (seeking) and slow sync clients (sync callback registered).
> The idea is to give the clients enough time for seeking. Indeed
> "video clients" are very slow in seeking.
> Starting jack transport is per defintion a "relocation". So
> jack_sync is only called (for registered client) if a relocation is
> requested or the jack transport is started(ing). The jack_sync is
> not the transport transition callback: started => stopped and so
> on. For getting the transport transition info it has to be polled
> e.g. from the jack_process by calling the jack_transport_query
> function (follow_transport_slave principle). There can be fast
> clients with no jack_sync callback registered.
>
> The jack transport is started if all clients return 1 from the
> sync_callback or the registered timeout (here: 5s) elapse. After
> returning 1 the sync_callback is not called for the entire client.
>
> Back to mlt:
>
> Issue 1: _last_pos property
> ======
>
> The _last_pos property is not updated if we have the following timeline:
>
> +---------+----------------------+
> | blank | video.mpg |
> +---------+----------------------+
> 6s e.g. 20s
>
> or
>
> #sh: MLT_CONSUMER=sdl_preview ./melt -jack -blank 125 video-file.mpg
>
> In this case you will notice that the jack transport is started after
> 5s when the timeout elapse and the jackd starts the transport.
> The problem is that in the "blank" time the function
>
> jackrack_get_audio
>
> is not called and we don't get the position update.
This issue is fundamental to the design of MLT, which probably makes
MLT's native support for transport sync non-universal and not suitable
for some applications like Kdenlive unless Kdenlive wants to start
putting silent clips on the timeline instead of blanks. Why? because
MLT does not ask filters to process "null" data, which is what blank
represents. Trying to change that is likely to introduce bugs.
> jack_sync:
>
> else if ( position >= mlt_properties_get_position( properties,
> "_last_pos" ) - 2 ) => based on the "_last_pos" this criteria is not
> matched or incorrect
> {
> mlt_properties_set_int( properties, "_sync_guard", 0
> );
> result = 1;
> }
>
>
> case 1:
>
> not updated "_last_pos" > position => timeout
Getting a non-existent property as a numeric yields a 0, so this case
is highly unlikely.
>
> case 2:
>
> not updated "_last_pos" < position => jack transport starts, but hay
> did we wait for mlt seeking???
This is just firing an event making it dependent upon the app's
behavior, but mlt_producer_seek() is not synchronous (impossible) and
neither does melt wait for the seeked frame to be reported as
displayed by the consumer. It may be possible in Shotcut and Kdenlive
with some work.
>
>
> Issue 2: mlt transport is started ignoring slower sync clients
> ======
>
> jack_sync():
> ....
> else if ( state == JackTransportStarting )
> {
> result = 0;
> if ( !mlt_properties_get_int( properties, "_sync_guard" ) )
> {
> mlt_properties_set_int( properties, "_sync_guard", 1
> );
> mlt_events_fire( properties, "jack-started",
> &position, NULL );
> }
> else if ( position >= mlt_properties_get_position( properties,
> "_last_pos" ) - 2 )
> {
> mlt_properties_set_int( properties, "_sync_guard", 0
> );
> result = 1;
> }
> }
> ...
>
> Considering the fact "_sync_guard == 0" the "jack-started" event is
> fired and the mlt transport starts. This is a problem if we have
> some slower sync clients connected. The jackd holds the jack transport
> in "starting" until all clients return 1 or the timeout elapse.
> In this case the mlt client gets absolutely out of sync.
>
> As mentioned above the mlt transport should be started from the
> jack_process by polling the jack_transport_query (follow_slave
> principle).
> And in the jack_sync only "mlt seeking" should be requested. This is
> the way it is implemented in ardour.
>
> In my private mlt jack branch I introduced the event "jack-starting"
> - giving the appl the time for seeking or starting a very complex and
> slow
> enginge but not the transport :-))
>
> See for inspiration:
> https://github.com/eddrog/mlt/blob/jack/experimental/src/modules/jackrack/filter_jackrack.c
>
> As a work around for "_last_pos" I'm syncing the jack position to
> consumers position. Sorry for the "jack-last-pos-req" event. I tried
> to reach the
> consumers object from the filter. But as result I got just crashes.
Yeah, you cannot really do that. The filter should only fire events,
and let the application access the consumer.
> In the appl I set the "_last_position" property with the conumers
> position value.
>
> My implementation is very experimental but I get pretty good sync
> results (seeking while playing).
>
>
> I know you have a better way to solve the issues but I think my way is
> a good base for inspiration :-)))
>
I think there are some very fundamental issues with the integration of
jack transport sync within a MLT filter for all use cases, and it
should instead be done entirely within Kdenlive.
>
> Issue 3: sound starts to early about 1s
> ======
>
> +---------+----------------------+
> | blank | video.mpg |
> +---------+----------------------+
> 6s e.g. 20s
>
> consumer=sdl_preview
>
> We have again our problematic timeline. In this case the sound starts
> to play ca. 1s to early. I tried to figure out why but hay.... :-))
>
>
>
>
> Issue 4: sound plays after stopping (ca. 1s)
> ======
>
> I have still the problem that after stopping you hear the sound
> playing ca. 1s. It becomes very crazy if you have a transport loop
> e.g. in ardour and
> a sound track in kdenlive. After looping some times and stopping it
> plays and plays and plays (ca. 5s or so). For me this issue is low
> prio I think.
>
>
>
>
> Ok now I'm done - puh.
>
> regards
>
> eddrog
--
+-DRD-+
------------------------------------------------------------------------------
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
_______________________________________________
Mlt-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mlt-devel
