On Sat, 2003-06-14 at 04:45, Frank van de Pol wrote: > On Sat, Jun 14, 2003 at 02:12:57AM +1000, Allan Klinbail wrote: > > This is all quite interesting.. > > Thanks for your reply Allan, I've put some of my controversial opinions in > it, please don't feel offended!
No offence taken at all.. this was a very good and informative read... > > > > > >From working with hardware synths I'm used to listening to everything in > > real time.... over and over and over again.. no big deal... often this > > is where new ideas are formed and also problems in the mix discovered.. > > > > Personally I'm just excited that we have real-time soft synths. > > Me too, unlike some other people I have a 'traditional' setup with mostly hardware > synths, effects, mixing desks. Some portions are done by the soft synths, > but I do see that more and more of the fine hardware is being virtualised in > the software (at the expense of especially the user interface / user > interaction).. Agreed, and even some of the good hardware sytns are becoming software dependent. i.e. With my AN-200 not all of the parameters can be accessed through the hardware interface.. I have to boot into that other OS to run the software to create patches with all the flexibility that the device offers. > > > > > To my knowledge MIDI in itself is a real-time spec except for sysex... > > which is used more often for management rather than real-time control.. > > (although it can be). However the notes below do indicate a dedication > > to real-time.... although some concerns do arise > > > > "Note that for the softsynth to get advantage of this the application > > > > should enqueue the events (a bit) ahead of time" > > > > ON outset it would seem that someone who is using an external MIDI > > controller device (fader box e.t.c..) or even an internal Midi slider > > app may suffer as us humans work in real time.. our real time messages > > would then be queuing after the above events...(am I making sense? let > > me try another phrase) essentially a real-time tweak of a knob (say > > routed to a filter, envelope or LFO) may not register with the system > > as the sequencer app only received these messages in real time and not > > "slightly before"..... This would make recording performances extremely > > difficult (and in techno,electronica et.al synth tweaking is the > > performance in itself)... Imagine trying to pre-empt yourself by a > > matter of milliseconds.. Playing on traditional instruments it's often > > hard to even do it in time. I'm not convinced trying to do "better > > than real-time" is actually such a musical concept. I Would be happy to > > see a bounce feature that works in better than real-time.. but not if it > > is going to sacrifice the performance of MusE or any other app in > > real-time. > > Perhaps I failed to make my point clear, or even that I forgot to tell the > whole context... > > Some of my points of view: > > - you want the latency (time between user action, eg. tweaking a real-time > control or pressing a key) as low as possible. But don't get trapped in > all the marketing stories people want to play > > My experience is that even with without low latency it is possible to play > an instrument, though is more difficult and takes more practice to get the > timing right. A latency of >20 ms makes a piano feel kind of sluggish; a > latency <10ms gives your the feeling of instant sound. The curch organ is > a nice example of an instrument that is difficult to play because of > extremely high latency, but good musicians do manage... Yep I had a go at that.. and I found it really frustrating.. Then when I changed school the new one had a Hammond B2 (about 1.5 times the size of a the legendary B-3) but htey wouldn't let me play it cause I wasn't a church organ player... (But I was attending the school of Deep Purple, and Black Sabbath at the time.. so god did I want to play on that) > > A nice example of constant delay I keep on using is the propagation > velocity of sound waves. At 20 deg, the speed of sound is approximate 330 > m/s. Which means that moving the speakers 1 meter further away will cost > you 3ms more latency... > > "Recording engineers" who claim that they can't make a decent mix if the > latency is above 10ms might be right with that statement, but I'll be glad > to put a bet for a crate of beer on it that they can't do a decent mix > with no latency either ;-) > > - Though the human brain can compensate for latency (constant delay), it > can't for jitter (stochastic variable delay). If there is a lot of jitter, > it just won't sound right, the groove just isn't there. > > Every device has latency, every keyboard has latency (time between hitting > the key, and event or sound generated), every MIDI synth has latency (time > between NOTE ON event coming in and sound generated). The amount of latency > depends on the brand/model of the device (eg. my AKAI S3K sampler has lower > latency than my Kawai K1 synth etc.). This is true > > This brings op a rather interesting 3rd point: > > - In the end, all that counts is that the parts of the composition arrive at > the listner's ears at the right time. This implies that you would need to > know about the latencies in your system, and compensate for it: > > If an event is know ahead of time (sequencer playback, but also > pre-recorded audio), use that knowledge to ease the system and gain better > timing accuracy. This way you can start the longish things before the fast > things, with as goal that they are completed at the same time. > > Now you do know your latencies, but an event comes in to be played 'right > now' (eg. a live keyboard player. You have the option to either try to > play is best effort, with a certain (hopefully small) latency, or delay > everything to mask the individual instrument's latency. > > In the end the only thing that matters is that it sounds OK and that when > the recording is reproduced it sound the same... (already not a trivial > question) > > > When comparing a MIDI with a block based digital audio system (eg. Jack) we > see an interesting difference: MIDI is event driven, while the audio portion > is in fact sampled, with fixed frames per second (numer of frames per second > depends on sample rate and block size). > > Due to the block size processing this inevitely introduces some latency, but > what's worse, when viewed from real-time perspective, this latency also has > a jitter of up to 1 frame size (which might be quite large!). > > when a softsynth, running such a frame based audio engine is driven by > real-time events like MIDI controls, it has basically 2 options: > > 1. play the event as soon as possible, ie. next frame, but this introduces a > large amount of jitter. > > 2. determine somehow the timestamp within the frame the event needs to be > played, and delay it till that time arrises. > > For live performance one would typically opt for the first option, unless > the sound systems has fairly hight latency (which translates in jitter to > the event system!!!!). For other cases the 2nd option would be preferd, > since the data is either known ahead of time, or one can introduce a small > constant delay. > > When sending the data from one app to another, this might even make the > whole timing worse, unless the last one in the chain still has access to the > original timing data, and has the ability to compensate. > > The moral of my story is that timing hints won't save your ass, but might > help you if you either know the event ahead of time (eg. recorded midi > sequence), or can introduce a constant delay. A sequencer system does not > have a crystal globe to predict the future. Being aware of the artifacts of > mixing real-time and frame based systems might help us builing the right > tools for great musical compositions... > > > ps: I'm still a very lousy keyboard player, and am heading for some serious > beer now. Cheers! > mmmmm beeer (no beer never really helped my keyboard playing either.. but it improved how I perceived I played) > > > > > > "Since > > > > the audio sample rate can also be used as a clock master for the alsa > > > > sequencer this would be a good option to ensure synchronisation." > > > > So long as sample-rate is converted into minutes and seconds etc.. to > > allow for adjusting BPM.. but I'm sure this consideration has been taken > > on-board > > > > > > > > > > > On Tue, 2003-06-10 at 23:51, Robert Jonsson wrote: > > > tisdagen den 10 juni 2003 13.21 skrev Frank van de Pol: > > > > On Tue, Jun 10, 2003 at 08:30:39AM +0200, Robert Jonsson wrote: > > > > > Hi, > > > > > > > > > > > In fact the bounce feature in MusE is "realtime". It means that you > > > > > > have to wait the real duration of the track to be rendered. > > > > > > In a non "realtime" mode the track is rendered as fast as computer can. > > > > > > > > > > AFAICT the realtimeness of the bounce feature is like that because of > > > > > design constraints. Okay, bouncing wavetracks should be possible in > > > > > non-realtime, but not when using softsynths. > > > > > > > > > > This is because all softsynths use alsa-sequencer as the input interface. > > > > > And if I'm not missing anything, this interface is strictly realtime > > > > > based. (perhaps it can be tweaked by timestamping every note and sending > > > > > them in batches? it seems very hard though.) > > > > > > > > You are right, with the current alsa-sequencer the softsynths are driven by > > > > realtime events. Though an application can enqueue the events to the > > > > priority queues with delivery timestamp, the scheduling is handled > > > > internally by the alsa sequencer. This causes some problems (especially for > > > > sample accurate synchronisation with JACK or LADSPA synth plugins (XAP?)), > > > > but also for network transparency and support for MIDI interfaces which > > > > accepts timing hints (Steinberg LTB or Emagic AMT ... if specs of the > > > > protocol were available :-( ). > > > > > > > > During the LAD meeting at Karlsruhe we discussed this and sketched a > > > > alsa-sequencer roadmap that focusses on transition of the alsa-sequencer > > > > from kernel to userspace and better integration with softsynths / JACK. > > > > A few things from this are very much related to your track bouncing / > > > > off-line rendering thing: > > > > > > > > - Provide facility to delegate scheduling to the client. The implementation > > > > would be to deliver the events directly (without queuing) with the > > > > timestamp attached to the registered client port. This would allow the > > > > client to get the events before the deadline (time at which the event > > > > should be played) and use that additional time to put the events at the > > > > right sample position. > > > > > > > > Note that for the softsynth to get advantage of this the application > > > > should enqueue the events (a bit) ahead of time and pass the timestamp. > > > > Some of the current applications (including MusE) use the alsa-sequencer > > > > only as event router and drive it real-time. > > > > > > > > Since the softsynth/plugin has no notion of the acutal time (only the > > > > media time and sample position), rendering at arbitrary speed should be > > > > possible: bounce faster than realtime or even slower than realtime for > > > > those complex patches. > > > > > > > > - JACK is real-time, and bound to the sample rate of the soundcard. Since > > > > the audio sample rate can also be used as a clock master for the alsa > > > > sequencer this would be a good option to ensure synchronisation. The > > > > transport of JACK and alsa sequencer can be tied together (either one of > > > > the two acting as master, a run-time configurable option) to provide > > > > uniform transport and media time amongst the applications that hook into > > > > the JACK and/or alsa sequencer framework. > > > > > > > > For the offline rendering no nice scheme has been worked out yet; I guess > > > > it would be something along the lines where the application that owns the > > > > sequencer queue has full control on the transport, moving media time at the > > > > speed the frames are actually rendered, and the app(s) generating the > > > > events keeping at least one sample frame ahead of time. > > > > > > > > Frank. > > > > > > Okay, I didn't know that this had been up on the table, how far has this work > > > progressed, was it just the Karlsruhe meeting or has more thinking occured? > > > (fyi I'm CC:ing LAD, it might be a more appropriate place for this > > > discussion..). > > > > > > Regards, > > > Robert -- Son of Zev <[EMAIL PROTECTED]>
