Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens and others,

Any thoughts on this? I would like to start working on it.

On 30/09/16 21:32, Felipe Ferreri Tonello wrote:
> Hi Clemens,
> 
> On 30/09/16 19:46, Clemens Ladisch wrote:
>> Felipe Ferreri Tonello wrote:
>>> This event we are discussing is time based.
>>> The only difference is that the event time is not its delivered time,
>>> but some time in the past.
>>>
>>> I just want to make ALSA Sequencer support this idea, which is new and a
>>> requirement for MIDI over BLE to work properly.
>>
>> I see no such requirement in the BLE-MIDI specification, which says:
>> | To maintain precise inter-event timing, this protocol uses 13-bit
>> | millisecond-resolution timestamps to express the render time and event
>> | spacing of MIDI messages.
>> and:
>> | Correlation between the receiver’s clock and the received timestamps
>> | must be performed to ensure accurate rendering of MIDI messages, and
>> | is not addressed in this document.
>>
>> In the context of the ALSA sequencer, "rendering" means delivery.
> 
> Yes, the correlation exists on the current implementation I am working
> on[1].
> 
> The problem is the latency on receiving BLE packets. The timestamp the
> device sends and the "receiver's clock" (ALSA Seq) will very quickly get
> out-of-sync, causing this timestamp to be invalid. Thus we need somehow
> to set the event timestamp to any arbitrary value, in practice it will
> be the BLE-MIDI timestamps (normalized to the central timestamp, of course).
> 
> [1] https://github.com/ftonello/bluez/blob/midi/profiles/midi/midi.c#L138
> 

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens,

On 30/09/16 19:46, Clemens Ladisch wrote:
> Felipe Ferreri Tonello wrote:
>> This event we are discussing is time based.
>> The only difference is that the event time is not its delivered time,
>> but some time in the past.
>>
>> I just want to make ALSA Sequencer support this idea, which is new and a
>> requirement for MIDI over BLE to work properly.
> 
> I see no such requirement in the BLE-MIDI specification, which says:
> | To maintain precise inter-event timing, this protocol uses 13-bit
> | millisecond-resolution timestamps to express the render time and event
> | spacing of MIDI messages.
> and:
> | Correlation between the receiver’s clock and the received timestamps
> | must be performed to ensure accurate rendering of MIDI messages, and
> | is not addressed in this document.
> 
> In the context of the ALSA sequencer, "rendering" means delivery.

Yes, the correlation exists on the current implementation I am working
on[1].

The problem is the latency on receiving BLE packets. The timestamp the
device sends and the "receiver's clock" (ALSA Seq) will very quickly get
out-of-sync, causing this timestamp to be invalid. Thus we need somehow
to set the event timestamp to any arbitrary value, in practice it will
be the BLE-MIDI timestamps (normalized to the central timestamp, of course).

[1] https://github.com/ftonello/bluez/blob/midi/profiles/midi/midi.c#L138

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Clemens Ladisch]

Felipe Ferreri Tonello wrote:
> This event we are discussing is time based.
> The only difference is that the event time is not its delivered time,
> but some time in the past.
>
> I just want to make ALSA Sequencer support this idea, which is new and a
> requirement for MIDI over BLE to work properly.

I see no such requirement in the BLE-MIDI specification, which says:
| To maintain precise inter-event timing, this protocol uses 13-bit
| millisecond-resolution timestamps to express the render time and event
| spacing of MIDI messages.
and:
| Correlation between the receiver’s clock and the received timestamps
| must be performed to ensure accurate rendering of MIDI messages, and
| is not addressed in this document.

In the context of the ALSA sequencer, "rendering" means delivery.


Regards,
Clemens
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Paul,

On 30/09/16 14:48, Paul Davis wrote:
> 
> 
> On Fri, Sep 30, 2016 at 4:35 AM, Felipe Ferreri Tonello
> > wrote:
> 
> 
> >
> > The time of an event is the time at which it is actually delivered.
> > If you want to be compatible with most other applications, you have
> > to deliver the events at the desired time.
> 
> Ok. This is *not* an option. We need to overcome this somehow.
> 
> 
> Don't use the ALSA sequencer. The whole term "sequencer" is a reference
> to organization/scheduling based on time. If you're not going to comply
> with that basic concept of what the sequencer is/was about, use raw MIDI
> ports instead.

But this is the whole point. This event we are discussing is time based.
The only difference is that the event time is not its delivered time,
but some time in the past.

I just want to make ALSA Sequencer support this idea, which is new and a
requirement for MIDI over BLE to work properly.

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Paul Davis]

On Fri, Sep 30, 2016 at 4:35 AM, Felipe Ferreri Tonello <
e...@felipetonello.com> wrote:

>
> >
> > The time of an event is the time at which it is actually delivered.
> > If you want to be compatible with most other applications, you have
> > to deliver the events at the desired time.
>
> Ok. This is *not* an option. We need to overcome this somehow.
>

Don't use the ALSA sequencer. The whole term "sequencer" is a reference to
organization/scheduling based on time. If you're not going to comply with
that basic concept of what the sequencer is/was about, use raw MIDI ports
instead.
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens,

On 20/09/16 18:11, Clemens Ladisch wrote:
> Felipe Ferreri Tonello wrote:
>> On 20/09/16 15:26, Clemens Ladisch wrote:
>>> In other words: any application that does care about timestamps will
>>> never see your timestamps.
>>
>> Yes. And that's why my first question.
>>
>> How can we implement this timestamp feature in ALSA with the current
>> implementation?
> 
> Not at all.
> 
>> If it is not possible, how feasible is for us to add this feature?
> 
> The ALSA sequencer API is an interface between multiple drivers and
> applications.  Adding a feature to the interface will not have any
> measurable effect because no other application will use it.

No other application will use it because it has not been implemented
yet. It doesn't mean that applications who cares about MIDI over BLE
will not use it. It is a very small patch anyway.

> 
> The time of an event is the time at which it is actually delivered.
> If you want to be compatible with most other applications, you have
> to deliver the events at the desired time.

Ok. This is *not* an option. We need to overcome this somehow.

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Clemens Ladisch]

Felipe Ferreri Tonello wrote:
> On 20/09/16 15:26, Clemens Ladisch wrote:
>> In other words: any application that does care about timestamps will
>> never see your timestamps.
>
> Yes. And that's why my first question.
>
> How can we implement this timestamp feature in ALSA with the current
> implementation?

Not at all.

> If it is not possible, how feasible is for us to add this feature?

The ALSA sequencer API is an interface between multiple drivers and
applications.  Adding a feature to the interface will not have any
measurable effect because no other application will use it.

The time of an event is the time at which it is actually delivered.
If you want to be compatible with most other applications, you have
to deliver the events at the desired time.


Regards,
Clemens
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens,

On 20/09/16 15:26, Clemens Ladisch wrote:
> Felipe Ferreri Tonello wrote:
>> On 19/09/16 13:27, Clemens Ladisch wrote:
>>> And applications that care about the time of received events
>>> tell the sequencer to overwrite the timestamp with the actual delivery
>>> time anyway.
>>
>> Applications only need to care about the timestamp field for that event,
>> it doesn't matter who set it (the ALSA Seq scheduler or other client, in
>> this case the BLE driver).
> 
> When the receiving port has timestamping mode enabled, the sequencer
> will overwrite any timestamp that the event had.
> 
> When the receiving port does not have timestamping mode enabled, the
> application will not read the timestamp, because there is no guarantee
> that it is set.
> 
> In other words: any application that does care about timestamps will
> never see your timestamps.

Yes. And that's why my first question.

How can we implement this timestamp feature in ALSA with the current
implementation? If it is not possible, how feasible is for us to add
this feature? Any suggestions?

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Clemens Ladisch]

Felipe Ferreri Tonello wrote:
> On 19/09/16 13:27, Clemens Ladisch wrote:
>> And applications that care about the time of received events
>> tell the sequencer to overwrite the timestamp with the actual delivery
>> time anyway.
>
> Applications only need to care about the timestamp field for that event,
> it doesn't matter who set it (the ALSA Seq scheduler or other client, in
> this case the BLE driver).

When the receiving port has timestamping mode enabled, the sequencer
will overwrite any timestamp that the event had.

When the receiving port does not have timestamping mode enabled, the
application will not read the timestamp, because there is no guarantee
that it is set.

In other words: any application that does care about timestamps will
never see your timestamps.


Regards,
Clemens
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens,

On 19/09/16 13:27, Clemens Ladisch wrote:
> Felipe Ferreri Tonello wrote:
>>  * We want to deliver these events *ASAP* to the application -
>> scheduling adds latency, a lot;
>>  * Timestamps are in the past relative to the central.
>>
>> But I still need the timestamp information. Why? The spec doesn't
>> explain, but it makes sense to believe it is used to have a predictable
>> latency, so if the central device wants to layout these MIDI message,
>> they little or no jitter in between.
> 
> MIDI is a real-time protocol, and the sequencer assumes that events are
> delivered in real time.
> 
> If you want to minimize jitter, you have to impose a certain fixed
> latency, and schedule all events that arrived too early.
> 
> If you want to minimize latency, you have to deliver the events
> immediately.
> 
>> Thus, to be MIDI compliant
> 
> Compliant with what?  The MIDI specification does not say anything about
> timestamps associated with events.

MIDI-BLE[1] (MIDI over Bluetooth Low Energy) _does_ specify timestamp
associated with a MIDI message, like I already mentioned.

> 
>> we need to set this timestamp some how on the event. And I think the
>> simplest way, is to use snd_seq_real_time_t on ev.time.time.
> 
> But nobody will read it. 

Applications which cares about the event timestamp already reads it.

But even then, we can patch and add this functionality that is needed by
MIDI-BLE. Unless we find a better way of doing so.

> And without a queue, the timestamp does not
> make sense. 

Applications who cares about timestamp have their own queue implementation.

> And applications that care about the time of received events
> tell the sequencer to overwrite the timestamp with the actual delivery
> time anyway.

Applications only need to care about the timestamp field for that event,
it doesn't matter who set it (the ALSA Seq scheduler or other client, in
this case the BLE driver).

[1] https://www.midi.org/specifications/item/bluetooth-le-midi

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Clemens Ladisch]

Felipe Ferreri Tonello wrote:
>  * We want to deliver these events *ASAP* to the application -
> scheduling adds latency, a lot;
>  * Timestamps are in the past relative to the central.
>
> But I still need the timestamp information. Why? The spec doesn't
> explain, but it makes sense to believe it is used to have a predictable
> latency, so if the central device wants to layout these MIDI message,
> they little or no jitter in between.

MIDI is a real-time protocol, and the sequencer assumes that events are
delivered in real time.

If you want to minimize jitter, you have to impose a certain fixed
latency, and schedule all events that arrived too early.

If you want to minimize latency, you have to deliver the events
immediately.

> Thus, to be MIDI compliant

Compliant with what?  The MIDI specification does not say anything about
timestamps associated with events.

> we need to set this timestamp some how on the event. And I think the
> simplest way, is to use snd_seq_real_time_t on ev.time.time.

But nobody will read it.  And without a queue, the timestamp does not
make sense.  And applications that care about the time of received events
tell the sequencer to overwrite the timestamp with the actual delivery
time anyway.


Regards,
Clemens
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens,

On 17/09/16 08:42, Clemens Ladisch wrote:
> Felipe Ferreri Tonello wrote:
>> On 16/09/16 18:41, Clemens Ladisch wrote:
>>> Felipe Ferreri Tonello wrote:
 I have a question. I would like to send sequencer events without
 scheduling but with a timestamp information associated with. Is that
 possible?
>>>
>>> You could set the timestamp field of the event, but why bother when
>>> nobody is ever going to read it?
>>
>> Thant's what I am doing[1] but I would like to know if there is a proper
>> method of doing so.
> 
> You can either schedule an event to be delivered in the future, or send
> it to be delivered immediately.
> 
> In the latter case, setting the timestamp does not make sense.

It _does_ on MIDI-BLE. See below.

> 
>> This is *necessary* for MIDI-BLE at least, where the packet provides
>> timestamp information.
> 
> If an event received over bluetooth is not to be delivered immediately,
> you have to schedule it.

The timestamp on MIDI-BLE is part of the BLE packet and represents the
time on the peripheral device (the one sending the MIDI message). Which
means I can't schedule the event because they are in the past when
compared to the central device[1].

It is also acceptable that one BLE packet has multiple MIDI messages
with different timestamps (always increasing). This happens in an
attempt to lower the latency.

To sum up, this is why I can't schedule these events:
 * We want to deliver these events *ASAP* to the application -
scheduling adds latency, a lot;
 * Timestamps are in the past relative to the central.

But I still need the timestamp information. Why? The spec doesn't
explain, but it makes sense to believe it is used to have a predictable
latency, so if the central device wants to layout these MIDI message,
they little or no jitter in between.

Thus, to be MIDI compliant, we need to set this timestamp some how on
the event. And I think the simplest way, is to use snd_seq_real_time_t
on ev.time.time.

Any thoughts?

[1] Central device represents the one establishing the connection to a
Bluetooth GAP peripheral, similar to a host on USB.

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Clemens Ladisch]

Felipe Ferreri Tonello wrote:
> On 16/09/16 18:41, Clemens Ladisch wrote:
>> Felipe Ferreri Tonello wrote:
>>> I have a question. I would like to send sequencer events without
>>> scheduling but with a timestamp information associated with. Is that
>>> possible?
>>
>> You could set the timestamp field of the event, but why bother when
>> nobody is ever going to read it?
>
> Thant's what I am doing[1] but I would like to know if there is a proper
> method of doing so.

You can either schedule an event to be delivered in the future, or send
it to be delivered immediately.

In the latter case, setting the timestamp does not make sense.

> This is *necessary* for MIDI-BLE at least, where the packet provides
> timestamp information.

If an event received over bluetooth is not to be delivered immediately,
you have to schedule it.


Regards,
Clemens
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hi Clemens,

On 16/09/16 18:41, Clemens Ladisch wrote:
> Felipe Ferreri Tonello wrote:
>> I have a question. I would like to send sequencer events without
>> scheduling but with a timestamp information associated with. Is that
>> possible?
> 
> You could set the timestamp field of the event, but why bother when
> nobody is ever going to read it?

Thant's what I am doing[1] but I would like to know if there is a proper
method of doing so.

I suggest us to use this method as a standard way of doing it. This is
*necessary* for MIDI-BLE at least, where the packet provides timestamp
information.

If that's the case, we can patch some main libraries to use it if it is
available, and update ALSA Sequencer documentation.

[1] https://github.com/ftonello/bluez/blob/midi/profiles/midi/midi.c

-- 
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

Re: [LAD] ALSA Sequencer timestamp on event without scheduling

[Clemens Ladisch]

Felipe Ferreri Tonello wrote:
> I have a question. I would like to send sequencer events without
> scheduling but with a timestamp information associated with. Is that
> possible?

You could set the timestamp field of the event, but why bother when
nobody is ever going to read it?


Regards,
Clemens
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev

[LAD] ALSA Sequencer timestamp on event without scheduling

[Felipe Ferreri Tonello]

Hello all,

I have a question. I would like to send sequencer events without
scheduling but with a timestamp information associated with. Is that
possible?

Thanks,
Felipe


0x92698E6A.asc
Description: application/pgp-keys
___
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev