2012/11/4 Dan Dennedy <ddenn...@gmail.com>:
> On Thu, Oct 4, 2012 at 2:31 AM, Ed Rogalsky <ed.rogal...@gmail.com> wrote:
>> Hi Dan,
>>
>> the last days I worked on the jack seeking and I hope this is the last
>> major step before contributing the code to kdenlive.
>> There are some problems with the current jack_sync implementation.
>>
>> I parsed the jack documantion and found that it is a little bit
>> unclear. Thats why I studied some jack clients like ardour or
>> qtractor and the jackd implementation. Here the results:
>>
>> Jack Sync theory:
>> ==============
>>
>> The jack_sync callback is designed only for position relocation
>> (seeking) and slow sync clients (sync callback registered).
>> The idea is to give the clients enough time for seeking. Indeed
>> "video clients" are very slow in seeking.
>> Starting jack transport is per defintion a "relocation". So
>> jack_sync is only called (for registered client) if a relocation is
>> requested or the jack transport is started(ing). The jack_sync is
>> not the transport transition callback: started => stopped and so
>> on. For getting the transport transition info it has to be polled
>> e.g. from the jack_process by calling the jack_transport_query
>> function (follow_transport_slave principle). There can be fast
>> clients with no jack_sync callback registered.
>>
>> The jack transport is started if all clients return 1 from the
>> sync_callback or the registered timeout (here: 5s) elapse. After
>> returning 1 the sync_callback is not called for the entire client.
>>
>> Back to mlt:
>>
>> Issue 1: _last_pos property
>> ======
>>
>> The _last_pos property is not updated if we have the following timeline:
>>
>> +---------+----------------------+
>> | blank | video.mpg |
>> +---------+----------------------+
>> 6s e.g. 20s
>>
>> or
>>
>> #sh: MLT_CONSUMER=sdl_preview ./melt -jack -blank 125 video-file.mpg
>>
>> In this case you will notice that the jack transport is started after
>> 5s when the timeout elapse and the jackd starts the transport.
>> The problem is that in the "blank" time the function
>>
>> jackrack_get_audio
>>
>> is not called and we don't get the position update.
>
> This issue is fundamental to the design of MLT, which probably makes
> MLT's native support for transport sync non-universal and not suitable
> for some applications like Kdenlive unless Kdenlive wants to start
> putting silent clips on the timeline instead of blanks. Why? because
> MLT does not ask filters to process "null" data, which is what blank
> represents. Trying to change that is likely to introduce bugs.
>
>> jack_sync:
>>
>> else if ( position >= mlt_properties_get_position(
>> properties,
>> "_last_pos" ) - 2 ) => based on the "_last_pos" this criteria is not
>> matched or incorrect
>> {
>> mlt_properties_set_int( properties, "_sync_guard", 0
>> );
>> result = 1;
>> }
>>
>>
>> case 1:
>>
>> not updated "_last_pos" > position => timeout
>
> Getting a non-existent property as a numeric yields a 0, so this case
> is highly unlikely.
Oh I see the problem. What you mean is only true for the first time if
the property is
read and it is not set to a value yet => correct you get 0. But
consider the following situation:
In our timeline (see above):
1) move playhead to timecode 7s (after blank). The _last_pos property
is updated to a value != 0 (timecode = 7s).
2) move playhead back to say 2s. The "_last_pos" property (pos == 7s)
is not updated due to the blank problem.
Try to start jack playback => jackd timeout.
>>
>> case 2:
>>
>> not updated "_last_pos" < position => jack transport starts, but hay
>> did we wait for mlt seeking???
>
> This is just firing an event making it dependent upon the app's
> behavior, but mlt_producer_seek() is not synchronous (impossible) and
> neither does melt wait for the seeked frame to be reported as
> displayed by the consumer. It may be possible in Shotcut and Kdenlive
> with some work.
>
>>
>>
>> Issue 2: mlt transport is started ignoring slower sync clients
>> ======
>>
>> jack_sync():
>> ....
>> else if ( state == JackTransportStarting )
>> {
>> result = 0;
>> if ( !mlt_properties_get_int( properties, "_sync_guard" ) )
>> {
>> mlt_properties_set_int( properties, "_sync_guard", 1
>> );
>> mlt_events_fire( properties, "jack-started",
>> &position, NULL );
>> }
>> else if ( position >= mlt_properties_get_position(
>> properties,
>> "_last_pos" ) - 2 )
>> {
>> mlt_properties_set_int( properties, "_sync_guard", 0
>> );
>> result = 1;
>> }
>> }
>> ...
>>
>> Considering the fact "_sync_guard == 0" the "jack-started" event is
>> fired and the mlt transport starts. This is a problem if we have
>> some slower sync clients connected. The jackd holds the jack transport
>> in "starting" until all clients return 1 or the timeout elapse.
>> In this case the mlt client gets absolutely out of sync.
>>
>> As mentioned above the mlt transport should be started from the
>> jack_process by polling the jack_transport_query (follow_slave
>> principle).
>> And in the jack_sync only "mlt seeking" should be requested. This is
>> the way it is implemented in ardour.
>>
>> In my private mlt jack branch I introduced the event "jack-starting"
>> - giving the appl the time for seeking or starting a very complex and
>> slow
>> enginge but not the transport :-))
>>
>> See for inspiration:
>> https://github.com/eddrog/mlt/blob/jack/experimental/src/modules/jackrack/filter_jackrack.c
>>
>> As a work around for "_last_pos" I'm syncing the jack position to
>> consumers position. Sorry for the "jack-last-pos-req" event. I tried
>> to reach the
>> consumers object from the filter. But as result I got just crashes.
>
> Yeah, you cannot really do that. The filter should only fire events,
> and let the application access the consumer.
>
>> In the appl I set the "_last_position" property with the conumers
>> position value.
>>
>> My implementation is very experimental but I get pretty good sync
>> results (seeking while playing).
>>
>>
>> I know you have a better way to solve the issues but I think my way is
>> a good base for inspiration :-)))
>>
>
> I think there are some very fundamental issues with the integration of
> jack transport sync within a MLT filter for all use cases, and it
> should instead be done entirely within Kdenlive.
How should we proceed here? What is your plan / idea:
1) remove jack_sync callback from filter_jackrack and do it in app
or
2) provide some additional events from filter <=> app (similar way I did in the
experimental branch)
>>
>> Issue 3: sound starts to early about 1s
>> ======
>>
>> +---------+----------------------+
>> | blank | video.mpg |
>> +---------+----------------------+
>> 6s e.g. 20s
>>
>> consumer=sdl_preview
>>
>> We have again our problematic timeline. In this case the sound starts
>> to play ca. 1s to early. I tried to figure out why but hay.... :-))
>>
>>
>>
>>
>> Issue 4: sound plays after stopping (ca. 1s)
>> ======
>>
>> I have still the problem that after stopping you hear the sound
>> playing ca. 1s. It becomes very crazy if you have a transport loop
>> e.g. in ardour and
>> a sound track in kdenlive. After looping some times and stopping it
>> plays and plays and plays (ca. 5s or so). For me this issue is low
>> prio I think.
>>
>>
>>
>>
>> Ok now I'm done - puh.
>>
>> regards
>>
>> eddrog
>
>
>
> --
> +-DRD-+
------------------------------------------------------------------------------
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
_______________________________________________
Kdenlive-devel mailing list
Kdenlive-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kdenlive-devel
