> Terminology is a problem. I generally use "latency" to mean the maximum > time between input and output, i.e. when a midi message arrives or a > timer interrupt occurs, how long can it take for the event to be > delivered to the application, processed, and delivered to the output > device? (As I type, I see this defn is not quite right, but it's the > right idea.)
I see, so that kind of latency only occurs on midi thru for me, since that's the only "real time response" part. > > The thing that puzzles me, is why would someone *not* do it this way? > > Since I'm new to audio programming stuff I'm perfectly willing to > > believe I've missed something everyone else understands here... > > > I think what you're saying is that you are polling and busy waiting. The > main argument against this is that you're using a lot of CPU power to > determine that there is nothing to do. The simplest alternative is to Not really... the only thing that's polling is the *read* side, not the write side. If you imagine a write_midi command that accepts [(Time, Msg)] on stdin, then it's similar to "generate_midi I write_midi", which shouldn't involve any polling since reading on stdin blocks until there's data. The interesting thing that I've realized since is that it's handy to be able to have the [(Time, Msg)] come in any order, while portmidi and the OS want it sorted. If the input list is unsorted, you get back into having latency again, because you have to decide when to stop waiting to see if a newer event will arrive, e.g. (calling it "padding" to avoid the l-word): write_thread msgs = forever sleep for (newest_msg - (now-padding)) or until a newer msg arrives if (new - newest_msg <= padding), send it Now someone sending a msg that should go out in less than "padding" time will be disappointed, because "write_thread" already told the OS about a later msg. Even if I compute the 'msgs' stream completely in advance, I still have to merge 'thru' data with it. > If you don't like polling at all, then the standard practice is to > sleep waiting for an input event or timeout based on the time of the > next scheduled event. That way, you only wake up when there is work > to do. This has the advantage that you wake up immediately to handle > events, but usually at the cost of more overhead. > > > In the case of audio, I would say "play at this ... > > Audio is essentially an uninterrupted sequence that is clocked in/out by > the sample clock in the A-toD/D-to-A system. The clock is generally > independent of the OS system clock and the two are not calibrated with > each other (e.g. according to the OS, you will never have exactly 44100 > samples per second or any other round number). Therefore, you don't send > samples with timestamps, you just start the stream and keep computing, > usually in some sort of callback/demand scheme. If you want to play a > short sequence of samples, you generally mix them into a stream that is > already started to avoid glitches on the output and to avoid the > overhead of setting up the stream and taking it down. Once things are > running, there is some latency associated with the number of samples > buffered. On output, this number decreases continuously at the sample > rate and increases by some integer number every time you write a block > of samples. Often, the exact number cannot be determined. I see, hence the need to sync the midi stream with the audio stream clock rather than the other way around. I was thinking about scheduling the beginning times of audio chunks and then letting them play along with the midi, but your response makes it clear that this won't work too well on longer chunks of audio, and I should always use the audio clock for timing. I guess this is what the time_proc for the open functions is for. Then I can do something like write a little scheduler that attaches a sample stream to an audio clock position, and a routine that watches the audio clock and starts copying samples into the buffer when it's time (scheduled_time * sample_rate - buffer_size). Then I tell portmidi to play midi events at (scheduled_time * sample_rate) and give it the audio clock, and trust that it will do the midi equivalent of "timestamp - buffer_size"... except maybe it can't do that since it doesn't know time_proc is synced to the sample rate and how many samples the constant (?) output delay is equivalent to. Does this sound right? It still doesn't seem to jibe with the "latency" parameter of Pm_Write... > > since portmidi doesn't look particularly > > reentrant-safe. > > It's not: as a new user, could you tell me where this important > information would be noticed and should be written? If it applies to all functions, then at the top of portmidi.h somewhere before the individual function documentation would be nice. > > The thing about polling, is if I set it to 1ms, that basically adds > > 1ms to my best case latency (actually, worse, it inserts a 0-1ms > > jitter... > > This is not necessarily true. Your system latency (the time the OS might > wait before running a thread that is ready to run) is probably at least > 1ms, so whether you sleep for 1ms or 1ns or 0s, you have no guarantee > that the next instruction will be executed sooner than 1ms from now. By > inserting a little latency and using timestamps, you can reduce jitter > independently of whether you sleep or not. We're now back on the reading side, right? And since events come in (hopefully accurately) timestamped, this only applies to sending data through as quickly as possible. In a thru situation, timestamps are unused. poll to read a msg (how much latency it adds): 1 sleep for n ms (random between 0-n, depending on if the msg arrived at the beginning or end of the sleep) 2 unknown amount of time before OS actually wakes me up (random time, but small) 3 pull msg off the input queue, schedule immediate responses to be sent "right now" (constant and very small, if I'm just looking up tables) 4 time for OS midi driver to process and send the "right now" msg (probably random, but small) So total time from the msg arriving at midi in to the response going out is 1+2+3+4. It seems to me like a blocking implementation would be the same, except it would replace step 1 with "OS figures out that my process can unblock". Both are a random amount of time, which is why I thought it would hinge on whether it takes longer than 1ms to figure out you can unblock. And since in thru I'm always sending events out as soon as I get them, I don't see how inserting latency will help matters, or how you could use timestamps here. > In my opinion, one shouldn't worry much about a couple of ms of jitter > in MIDI data. Rather than schedule things in the future with timestamps, > I prefer to just pretend that I'm running infinitely fast and actually > poll every ms or two. I don't feel this detracts from performance, but > maybe if I were writing a sequencer (or PortMidi!), I'd put in some > extra effort to support timestamps and finer time control. You're talking about a realtime receive and retransmit kind of situation, right? I don't think 1-4ms is audible anyway, so I'm not overly concerned. But running a polling loop at 1ms seems like it would unnecessarily eat cpu and add to latency. In a playback of recorded data, like a sequencer, there's no jitter to worry about since the start time of each event is already determined, and its just the driver's job to get as close to that as possible. Wouldn't adding another quick polling loop to the writing side just I feel like we're still talking past each other here... maybe getting the "activities" confused. Sorry it's taking me so long to "get it"! The way I see it, we have three activities: playback recorded data: schedule (Time, Message), record new data: record (Time, Message), and respond ASAP to incoming data, e.g. for thru: take (Time, Message) to scheduling (0, Message). #1 shouldn't require fast response, it just needs to occasionally pull things off the low level input queue so it doesn't fill up. #2 shouldn't require fast response either, it just occasionally refills the low level output queue to avoid dumping too much data on it at once. #3 requires fast response, and also needs to merge into the output queue, so it causes a problem with #2 since the low level output queue wants events with increasing timestamps. So #3 actually has to be down at the scheduling level, *below* #2, to merge properly. So this seems to indicate that I should have #2 and #3 use separate streams, which the OS then merges. Either that, or I have to bypass output timestamps entirely and write my own scheduler, which is essentially reimplementing what the OS already implements. So that suggests to me that portmidi should provide a platform independent way to use the OS-level stream merging, or use a simulated internal one. Otherwise everyone winds up writing their own scheduler and bypassing the existing one... right? I think I'll look at existing sequencers to see what they do for that. > > Interesting... but as a cross platform library, isn't it portmidi's > > job to know what the underlying driver does? If the OS driver does > > buffering and merging and tests show that it does it "right", then > > it's easy, we just use the OS's implementation. If not, the library > > emulates that by doing the buffering itself. ... > I agree. PortMidi does not help you to send real-time messages while > transmitting sysex data, certainly not in any system-independent way. I > wonder how many, if any, sequencers do this "right"? Although it's an > interesting and challenging problem, I wonder if there is any demand for > this. I.e., is anyone depending on a drum machine synchronizing to MIDI > clocks while dumping patches to a synthesizer? Well, I'm guessing most people use sysex like I do: one big burst when loading the song and that's it, and probably don't use midi sync at all. And I shouldn't argue for a feature I wouldn't use myself, it was just from the "a library should be complete" angle. If I want to sync with another program I'll probably try to use jack or something anyway. So I think I'll ignore this for now, and only think about it if it actually becomes an issue. > We'll soon be adding pyPortMidi to the PortMidi release, and I'd be > happy to add haskell bindings. The advantage is that if we put them in > one place, it's possible to say "SVN version 83 was compiled and tested > with haskell." The disadvantage is that people may expect every change > to be tested across Common Lisp, C, haskell, python, etc, and in fact > that's not practical without a lot more effort. Maybe the readme can say "the maintainer says he last used it with this version of portmidi against this version of this compiler, and here's his email address". _______________________________________________ media_api mailing list media_api@create.ucsb.edu http://lists.create.ucsb.edu/mailman/listinfo/media_api
