Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens and others, Any thoughts on this? I would like to start working on it. On 30/09/16 21:32, Felipe Ferreri Tonello wrote: > Hi Clemens, > > On 30/09/16 19:46, Clemens Ladisch wrote: >> Felipe Ferreri Tonello wrote: >>> This event we are discussing is time based. >>> The only difference is that the event time is not its delivered time, >>> but some time in the past. >>> >>> I just want to make ALSA Sequencer support this idea, which is new and a >>> requirement for MIDI over BLE to work properly. >> >> I see no such requirement in the BLE-MIDI specification, which says: >> | To maintain precise inter-event timing, this protocol uses 13-bit >> | millisecond-resolution timestamps to express the render time and event >> | spacing of MIDI messages. >> and: >> | Correlation between the receiver’s clock and the received timestamps >> | must be performed to ensure accurate rendering of MIDI messages, and >> | is not addressed in this document. >> >> In the context of the ALSA sequencer, "rendering" means delivery. > > Yes, the correlation exists on the current implementation I am working > on[1]. > > The problem is the latency on receiving BLE packets. The timestamp the > device sends and the "receiver's clock" (ALSA Seq) will very quickly get > out-of-sync, causing this timestamp to be invalid. Thus we need somehow > to set the event timestamp to any arbitrary value, in practice it will > be the BLE-MIDI timestamps (normalized to the central timestamp, of course). > > [1] https://github.com/ftonello/bluez/blob/midi/profiles/midi/midi.c#L138 > -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens, On 30/09/16 19:46, Clemens Ladisch wrote: > Felipe Ferreri Tonello wrote: >> This event we are discussing is time based. >> The only difference is that the event time is not its delivered time, >> but some time in the past. >> >> I just want to make ALSA Sequencer support this idea, which is new and a >> requirement for MIDI over BLE to work properly. > > I see no such requirement in the BLE-MIDI specification, which says: > | To maintain precise inter-event timing, this protocol uses 13-bit > | millisecond-resolution timestamps to express the render time and event > | spacing of MIDI messages. > and: > | Correlation between the receiver’s clock and the received timestamps > | must be performed to ensure accurate rendering of MIDI messages, and > | is not addressed in this document. > > In the context of the ALSA sequencer, "rendering" means delivery. Yes, the correlation exists on the current implementation I am working on[1]. The problem is the latency on receiving BLE packets. The timestamp the device sends and the "receiver's clock" (ALSA Seq) will very quickly get out-of-sync, causing this timestamp to be invalid. Thus we need somehow to set the event timestamp to any arbitrary value, in practice it will be the BLE-MIDI timestamps (normalized to the central timestamp, of course). [1] https://github.com/ftonello/bluez/blob/midi/profiles/midi/midi.c#L138 -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Felipe Ferreri Tonello wrote: > This event we are discussing is time based. > The only difference is that the event time is not its delivered time, > but some time in the past. > > I just want to make ALSA Sequencer support this idea, which is new and a > requirement for MIDI over BLE to work properly. I see no such requirement in the BLE-MIDI specification, which says: | To maintain precise inter-event timing, this protocol uses 13-bit | millisecond-resolution timestamps to express the render time and event | spacing of MIDI messages. and: | Correlation between the receiver’s clock and the received timestamps | must be performed to ensure accurate rendering of MIDI messages, and | is not addressed in this document. In the context of the ALSA sequencer, "rendering" means delivery. Regards, Clemens ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Paul, On 30/09/16 14:48, Paul Davis wrote: > > > On Fri, Sep 30, 2016 at 4:35 AM, Felipe Ferreri Tonello > mailto:e...@felipetonello.com>> wrote: > > > > > > The time of an event is the time at which it is actually delivered. > > If you want to be compatible with most other applications, you have > > to deliver the events at the desired time. > > Ok. This is *not* an option. We need to overcome this somehow. > > > Don't use the ALSA sequencer. The whole term "sequencer" is a reference > to organization/scheduling based on time. If you're not going to comply > with that basic concept of what the sequencer is/was about, use raw MIDI > ports instead. But this is the whole point. This event we are discussing is time based. The only difference is that the event time is not its delivered time, but some time in the past. I just want to make ALSA Sequencer support this idea, which is new and a requirement for MIDI over BLE to work properly. -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
On Fri, Sep 30, 2016 at 4:35 AM, Felipe Ferreri Tonello < e...@felipetonello.com> wrote: > > > > > The time of an event is the time at which it is actually delivered. > > If you want to be compatible with most other applications, you have > > to deliver the events at the desired time. > > Ok. This is *not* an option. We need to overcome this somehow. > Don't use the ALSA sequencer. The whole term "sequencer" is a reference to organization/scheduling based on time. If you're not going to comply with that basic concept of what the sequencer is/was about, use raw MIDI ports instead. ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens, On 20/09/16 18:11, Clemens Ladisch wrote: > Felipe Ferreri Tonello wrote: >> On 20/09/16 15:26, Clemens Ladisch wrote: >>> In other words: any application that does care about timestamps will >>> never see your timestamps. >> >> Yes. And that's why my first question. >> >> How can we implement this timestamp feature in ALSA with the current >> implementation? > > Not at all. > >> If it is not possible, how feasible is for us to add this feature? > > The ALSA sequencer API is an interface between multiple drivers and > applications. Adding a feature to the interface will not have any > measurable effect because no other application will use it. No other application will use it because it has not been implemented yet. It doesn't mean that applications who cares about MIDI over BLE will not use it. It is a very small patch anyway. > > The time of an event is the time at which it is actually delivered. > If you want to be compatible with most other applications, you have > to deliver the events at the desired time. Ok. This is *not* an option. We need to overcome this somehow. -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Felipe Ferreri Tonello wrote: > On 20/09/16 15:26, Clemens Ladisch wrote: >> In other words: any application that does care about timestamps will >> never see your timestamps. > > Yes. And that's why my first question. > > How can we implement this timestamp feature in ALSA with the current > implementation? Not at all. > If it is not possible, how feasible is for us to add this feature? The ALSA sequencer API is an interface between multiple drivers and applications. Adding a feature to the interface will not have any measurable effect because no other application will use it. The time of an event is the time at which it is actually delivered. If you want to be compatible with most other applications, you have to deliver the events at the desired time. Regards, Clemens ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens, On 20/09/16 15:26, Clemens Ladisch wrote: > Felipe Ferreri Tonello wrote: >> On 19/09/16 13:27, Clemens Ladisch wrote: >>> And applications that care about the time of received events >>> tell the sequencer to overwrite the timestamp with the actual delivery >>> time anyway. >> >> Applications only need to care about the timestamp field for that event, >> it doesn't matter who set it (the ALSA Seq scheduler or other client, in >> this case the BLE driver). > > When the receiving port has timestamping mode enabled, the sequencer > will overwrite any timestamp that the event had. > > When the receiving port does not have timestamping mode enabled, the > application will not read the timestamp, because there is no guarantee > that it is set. > > In other words: any application that does care about timestamps will > never see your timestamps. Yes. And that's why my first question. How can we implement this timestamp feature in ALSA with the current implementation? If it is not possible, how feasible is for us to add this feature? Any suggestions? -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Felipe Ferreri Tonello wrote: > On 19/09/16 13:27, Clemens Ladisch wrote: >> And applications that care about the time of received events >> tell the sequencer to overwrite the timestamp with the actual delivery >> time anyway. > > Applications only need to care about the timestamp field for that event, > it doesn't matter who set it (the ALSA Seq scheduler or other client, in > this case the BLE driver). When the receiving port has timestamping mode enabled, the sequencer will overwrite any timestamp that the event had. When the receiving port does not have timestamping mode enabled, the application will not read the timestamp, because there is no guarantee that it is set. In other words: any application that does care about timestamps will never see your timestamps. Regards, Clemens ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens, On 19/09/16 13:27, Clemens Ladisch wrote: > Felipe Ferreri Tonello wrote: >> * We want to deliver these events *ASAP* to the application - >> scheduling adds latency, a lot; >> * Timestamps are in the past relative to the central. >> >> But I still need the timestamp information. Why? The spec doesn't >> explain, but it makes sense to believe it is used to have a predictable >> latency, so if the central device wants to layout these MIDI message, >> they little or no jitter in between. > > MIDI is a real-time protocol, and the sequencer assumes that events are > delivered in real time. > > If you want to minimize jitter, you have to impose a certain fixed > latency, and schedule all events that arrived too early. > > If you want to minimize latency, you have to deliver the events > immediately. > >> Thus, to be MIDI compliant > > Compliant with what? The MIDI specification does not say anything about > timestamps associated with events. MIDI-BLE[1] (MIDI over Bluetooth Low Energy) _does_ specify timestamp associated with a MIDI message, like I already mentioned. > >> we need to set this timestamp some how on the event. And I think the >> simplest way, is to use snd_seq_real_time_t on ev.time.time. > > But nobody will read it. Applications which cares about the event timestamp already reads it. But even then, we can patch and add this functionality that is needed by MIDI-BLE. Unless we find a better way of doing so. > And without a queue, the timestamp does not > make sense. Applications who cares about timestamp have their own queue implementation. > And applications that care about the time of received events > tell the sequencer to overwrite the timestamp with the actual delivery > time anyway. Applications only need to care about the timestamp field for that event, it doesn't matter who set it (the ALSA Seq scheduler or other client, in this case the BLE driver). [1] https://www.midi.org/specifications/item/bluetooth-le-midi -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Felipe Ferreri Tonello wrote: > * We want to deliver these events *ASAP* to the application - > scheduling adds latency, a lot; > * Timestamps are in the past relative to the central. > > But I still need the timestamp information. Why? The spec doesn't > explain, but it makes sense to believe it is used to have a predictable > latency, so if the central device wants to layout these MIDI message, > they little or no jitter in between. MIDI is a real-time protocol, and the sequencer assumes that events are delivered in real time. If you want to minimize jitter, you have to impose a certain fixed latency, and schedule all events that arrived too early. If you want to minimize latency, you have to deliver the events immediately. > Thus, to be MIDI compliant Compliant with what? The MIDI specification does not say anything about timestamps associated with events. > we need to set this timestamp some how on the event. And I think the > simplest way, is to use snd_seq_real_time_t on ev.time.time. But nobody will read it. And without a queue, the timestamp does not make sense. And applications that care about the time of received events tell the sequencer to overwrite the timestamp with the actual delivery time anyway. Regards, Clemens ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens, On 17/09/16 08:42, Clemens Ladisch wrote: > Felipe Ferreri Tonello wrote: >> On 16/09/16 18:41, Clemens Ladisch wrote: >>> Felipe Ferreri Tonello wrote: I have a question. I would like to send sequencer events without scheduling but with a timestamp information associated with. Is that possible? >>> >>> You could set the timestamp field of the event, but why bother when >>> nobody is ever going to read it? >> >> Thant's what I am doing[1] but I would like to know if there is a proper >> method of doing so. > > You can either schedule an event to be delivered in the future, or send > it to be delivered immediately. > > In the latter case, setting the timestamp does not make sense. It _does_ on MIDI-BLE. See below. > >> This is *necessary* for MIDI-BLE at least, where the packet provides >> timestamp information. > > If an event received over bluetooth is not to be delivered immediately, > you have to schedule it. The timestamp on MIDI-BLE is part of the BLE packet and represents the time on the peripheral device (the one sending the MIDI message). Which means I can't schedule the event because they are in the past when compared to the central device[1]. It is also acceptable that one BLE packet has multiple MIDI messages with different timestamps (always increasing). This happens in an attempt to lower the latency. To sum up, this is why I can't schedule these events: * We want to deliver these events *ASAP* to the application - scheduling adds latency, a lot; * Timestamps are in the past relative to the central. But I still need the timestamp information. Why? The spec doesn't explain, but it makes sense to believe it is used to have a predictable latency, so if the central device wants to layout these MIDI message, they little or no jitter in between. Thus, to be MIDI compliant, we need to set this timestamp some how on the event. And I think the simplest way, is to use snd_seq_real_time_t on ev.time.time. Any thoughts? [1] Central device represents the one establishing the connection to a Bluetooth GAP peripheral, similar to a host on USB. -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Felipe Ferreri Tonello wrote: > On 16/09/16 18:41, Clemens Ladisch wrote: >> Felipe Ferreri Tonello wrote: >>> I have a question. I would like to send sequencer events without >>> scheduling but with a timestamp information associated with. Is that >>> possible? >> >> You could set the timestamp field of the event, but why bother when >> nobody is ever going to read it? > > Thant's what I am doing[1] but I would like to know if there is a proper > method of doing so. You can either schedule an event to be delivered in the future, or send it to be delivered immediately. In the latter case, setting the timestamp does not make sense. > This is *necessary* for MIDI-BLE at least, where the packet provides > timestamp information. If an event received over bluetooth is not to be delivered immediately, you have to schedule it. Regards, Clemens ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Hi Clemens, On 16/09/16 18:41, Clemens Ladisch wrote: > Felipe Ferreri Tonello wrote: >> I have a question. I would like to send sequencer events without >> scheduling but with a timestamp information associated with. Is that >> possible? > > You could set the timestamp field of the event, but why bother when > nobody is ever going to read it? Thant's what I am doing[1] but I would like to know if there is a proper method of doing so. I suggest us to use this method as a standard way of doing it. This is *necessary* for MIDI-BLE at least, where the packet provides timestamp information. If that's the case, we can patch some main libraries to use it if it is available, and update ALSA Sequencer documentation. [1] https://github.com/ftonello/bluez/blob/midi/profiles/midi/midi.c -- Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
Re: [LAD] ALSA Sequencer timestamp on event without scheduling
Felipe Ferreri Tonello wrote: > I have a question. I would like to send sequencer events without > scheduling but with a timestamp information associated with. Is that > possible? You could set the timestamp field of the event, but why bother when nobody is ever going to read it? Regards, Clemens ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev
[LAD] ALSA Sequencer timestamp on event without scheduling
Hello all, I have a question. I would like to send sequencer events without scheduling but with a timestamp information associated with. Is that possible? Thanks, Felipe 0x92698E6A.asc Description: application/pgp-keys ___ Linux-audio-dev mailing list Linux-audio-dev@lists.linuxaudio.org http://lists.linuxaudio.org/listinfo/linux-audio-dev