On Thursday 06 February 2003 12.58, Laurent de Soras [Ohm Force] wrote: > Late reply, I was quite busy and english writing takes me > time. I commented various people quotes here. However I > haven't read recent posts yet, I'm a bit overhelmed by > the reply avalanche. :)
Well, we're crazy about plugin APIs around here! ;-) [...] > > I'm questioning whether > > having a simpler query based system may be easier. I don't like > > the idea that you have to instantiate to query, though. > > Many plug-in standards require instanciation before fetching > properties. PTAF must be able to make good wrappers for existing > plug-ins, industry requires it. That's one point, although I think you could have the plugin factory of a wrapper do the instantiation "secretly". A wrapper *is* a plugin host anyway, so this shouldn't be a major issue. Either way, for various other reasons, I'm beginning to think that there are other reasons to use lightweight plugin instances for metadata requests. We have already mentioned plugins that can "mutate", and generic wrappers (ie wrappers that have a control for selecting the alien plugin to wrap) belong in that category. > > Yeah, I've thought about that too. Rather hairy stuff. The only > > easy way to avoid it is to strictly define what the host should > > tell the plugin before asking anything, and then just leave it at > > that. > > We can also add somewhere a watchdog limiting the recursion. Yeah, but it doesn't really solve the problem, does it? If you get endless recusion (or rather ping-pong, if we're talking about events; not much better though), it's because there's something wrong with the host, the plugin or both. There has to be a way to guarantee that dependencies don't become circular... Can we have some actual examples to break down? I've never written a host that changes it's parameters based on plugin properties, so I've never seen the problem in real life. > > [...the "sequencer" thread...] > > As suggested we can call it "host", or "main" thread. In my opinion > generic forms of hosts have several "virtual" threads : > > - Audio: handles audio processing and related sequencing > - Main/Background: handles generic operations, state transitions, What state transitions? > as well as background disk streaming, data preparing for audio > thread, etc. > - GUI thread (at least on Win and Mac OSs), can theoretically > be merged with the main thread, but it's not recommended (Mac > programmers have the bad habit to poll the mouse within a > blocking loop when user is holds a click). Well, they didn't have much choise in the old days... Win16 had the same problem. Provided we're talking about Mac OS X, it certainly is nothing but a bad habit, though. Anyway, I know what you mean by these threads - but I still don't see it as relevant to a plugin API. A GUI-less audio server may well have only one thread, and either way, plugins should not be required to be thread safe, unless they explicitly ask for it (trouble, that is ;-) themselves. [...] > > Well, you might still be able to comment on an idea: Splitting up > > the GUI in two parts; one that runs in the same process as the > > DSP plugin, and one that runs in another process, possibly on > > another machine. > > Yeah, I had this in mind when i was talking about lauching the > GUI from the plug-in. Yeah... Though I think all this should be left to the host, regardless. Just tell the host where your GUI is. If it's a "local half" plugin (that in turn refers to an external GUI), it could be in the same binary, as it's still just a plugin with a minimum of external dependencies. (Whether or not it wants to run in the audio thread is another matter. It can still use the same API - and the same rules WRT GUI toolkits still apply; don't use them.) > > You would need a feature that allows control outputs to be marked > > as "active" or "passive". This allows hosts to control how things > > are handled. > > > > If I see a knob that is turning via some old automation, I > > should be able to grab it in the UI and take over. It is not > > clear to me, however, whether the host should remember that and > > leave control with the highest prio controller, or just make it > > temporary. This needs more thought. It may not be the domain of > > the API at all, but the host. > > But the token system is just about the same concept, isn't it ? Not really. The difference is that my approach doesn't require GUIs and hosts to use a special API just for talking to controls. Also, tokens don't mix with the direct connection system we're using. (XAP plugins generally don't send events through the host, but rather directly to the input queues of other plugins.) > For a given parameter, there is at most one active client at a > time, others are passive and receive only change notifications. > Arbitration is done by host, who is allowed to deprive a client > from his token to give it to a client requesting it. This doesn't work if some clients are out-of process. Control outputs with active/passive modes do, since you never have to inform client whether or not their data is used. > For example it makes sense to give less priority to the > automation system than to user via GUI, so the latter can > steal its token, just by requesting it. When user has released > the knob, it give back the token to the host making it > available for any use (can transmits it implicitly to the > automation system). Sure, but it just seems like a more complicated way of doing it, especially when threading and out-of-process issues are taken in account. > > I actually think having to try to *call* a function to find out > > whether it's supported or not is rather nasty... > > Good point. > > Anyway I would like to minimize the number of functions. Good idea, of course. And besides, this even further reduces the need for "advanced" methods of maintaining supported and unsupported calls. > Functions are handy when the calling conventions > of both parts are compatible. If the language you use > doesn't support this convention natively, all functions > have to be wrapped, which is a real pain. Really? Either way, we also have to consider the importance of wrapping the API. How many write VST hosts and plugins in other languages than C++? (I know that some do. FruityLoops is written in Delphi, IIRC.) We also have to consider the balance between the work required to implement a wrapper/language binding and the impact simplifying that has on the native API, in terms of performance and ease of use. > More, there are still issues with existing plug-ins, reporting > wrongly if they support some opcode or not (talk to this issue > to Plogue people :). Bugs are always bugs. A simple testing host in the SDK would eliminate all valid excuses for this particular bug. (And it could of course be included in the host SDK as well, to warn every user about broken plugins.) Either way, I don't see a real problem here. Just don't call a NULL pointer, ok? :-) > > I think this separation is a mistake. Of course, we can't have an > > API for complex monolith synths scale perfectly to modular synth > > units, but I don't see why one should explicitly prevent some > > scaling into that range. Not a major design goal, though; just > > something to keep in mind. > > The separation between coarse/fine grained plug-ins was just > a design goal for the API. Fine grained plug-ins would have > more issues, like the ability to do fast sample-per-sample > processing (for feedback loops). Yeah, but you can't really get around that. You can still implement a host that does small block (not the Chevy V8! :-) sub nets for feedback loops, but it's going to be rather expensive. OTOH, that's *always* expensive... My point though; expensive is something you could live with. It might even matter to you. *Impossible* is a different story entirely. [...] > > How about defining buffer alignment as "what works good for > > whatever extensions that are available on this hardware"...? > > Good, but it requires the host to be updated when new hardware > is released. Or just make it a setting in "advanced options"...? (If you're not very interested in users upgrading your software, that is! ;-) > We can add a host property indicating the current > alignment so the plug-in can check it before continuing. Very good idea. There *are* platforms where a plugin could crash the host if they ignore this. > > you'll get a lot of arguments about that on this list - linux > > people tend to have a 486 or pentium stashed somewhere. :) > > Professional musicians and studios have recent hardware, at > least it's what survey results show us. They don't really have much of a choice, do they? > Recent audio software > produced by the industry also tend to be hungry, requiring > fast hardware - I don't say it's good, it's just a fact, and > industry is the main target for PTAF. It's a fact, and it's not as much a result of inefficient APIs as it is a result of people wanting "cooler" DSP effects. I don't have a problem with this, but I *do* have, and always will have, a problem with wasting resources for no good reason. Thus, I tend to look for optimal solutions at all times. If it's something I'll have to live with for years, I look harder - just in case it turns out that it actually matters. > If you plan to make a standard, it's better to build it for > having use over years. After all, MIDI is still here and will > probably last 20 years again. OK things are a bit different > with pure software protocols, but arguing little performance > boost against obvious programming safety is pointless for me > (sadly programmer's brain doesn't follows the Moore law). Very good points, and of course, I'm not proposing that we should make life hard for plugin authors just to save a few cycles. It's just that the wasted cycles is not the *only* problem I have with the dispatcher approach, so if there's a better solution (that also happens to be a bit faster), why not use that? > > Agreed - but they also attach value to efficient software. > > Yes, but given my experience with customer support, unsafty > displeasure is one or two orders of magnitude above efficency > pleasure. Very good point! If it *crashes*, who cares if it's slightly faster? > > That's one way... I thought a lot about that for MAIA, but I'm > > not sure I like it. Using events becomes somewhat pointless when > > you change the rules and use another callback. Function calls are > > cleaner and easier for everyone for this kind of stuff. > > The advantage of events is that you can have several of them at > once, giving the plug-in the possibility to optimise operations. Yes. As a matter of fact, I have *exactly* this problem with the FX plugins in Audiality. Initializing the parameters of the xdelay or reverb effects means they have to recalculate all taps almost once per parameter. *heh* They're about the last controls to use function calls, and it's a real problem. I could handle it by implementing the changes in the process() call, though, just like with events. It's just that the changes are not RT safe, so they have to send the work off to another thread. Dealing with that *and* the function call API, well... I'll just leave it until I've fixed the plugin API. > > We've just spoken of calling the process() function with a > > duration of 0 samples. Events are then processed immediately, > > and no second API is needed. > > Dunno if this factorisation is really good. Different goals, > different states for call, it should be a different opcode/func. There's a *big* hidden issue with thinking about different interfaces yor different states. States are related to what context a plugin runs in, and using different APIs in different states doesn't really help. You can't perform non RT operations on a running plugin just because it has a separate interface for it! Either the plugin must be thread safe, or the host has to take the plugin out of the RT thread first. > > Anyway, I'm nervous about the (supposedly real time) connection > > event, as it's not obvious that any plugin can easilly handly > > connections in a real time safe manner. > > Don't mix up audio configuration (which is not real-time) > and connections. In most of cases I see, (de)connections > is reflected in the plug-in by changing a flag or a variable, > making the processing a bit different. For example a plug-in > has a 2-in/2-out configuration. When only one input or output > channel is connected, it gives the ability to make the > processing mono, or to (de)activate vu-meters, etc. You're totally right. I wasn't thinking straight. [...] > > This is where it gets hairy. We prefer to think of scale tuning > > as something that is generally done by specialized plugins, or by > > whatever sends the events (host/sequencer) *before* the synths. > > That way, every synth doesn't have to implement scale support to > > be usable. > > That's why Base Pitch is float. But because MIDI is still there, > existing plug-ins will be happy to round (after scaling if 1/oct) > Base Pitch to get the note number. Yeah, I realize that, but there's still a difference between explicit 1.0/note and 1.0/octave with the scale assumed to be 12tET. If the scale *isn't* 12tET, rounding Base Pitch won't work as expected. > > As it is with VST, most plugins are essentially useless to people > > that don't use 12tET, unless multiple channels + pitch bend is a > > sufficient work-around. > > VST has also a finetuning delivered with each Note On. Yes, I remember now that you say it. However, it has very limited range (cents; sint8_t), and the MIDI pitch field still doubles as the "note ID", so this is in no way a substitute for continous pitch, or even sufficient for handling scales that are substantially different from 12tET. [...] > > The only part I don't like is assuming that Base Pitch isn't > > linear pitch, but effectively note pith, mapped in unknown ways. > > However many (hard/soft) synth can be tuned this way. It is a > convenient way to play exotic scales using traditional keyboard, > piano roll or other editors / performance devices. What's wrong with using a generic scale converter plugin *before* any synth you want to do this with? That's one of the major benefits of using continous, linear pitch, IMO. I don't think synths should have internal scale converters, unless they have *very* good reasons. (And I can't think of any.) > > Right, but MIDI is integers, and the range defines the > > resolution. With floats, why have 2.0 if you can have 1.0...? > > To make 1.0 the middle, default position. Yeah, but what is a "default" velocity in real life...? It's something that was invented for the MIDI protocol, to support controllers without velocity sensitivity. As even the cheapest "toy" MIDI controllers are velocity sensitive these days, it is essentially irrelevant, even to MIDI devices. [...] > > Anyway, what I'm suggesting is basically that this event should > > use the same format as the running musical time "counter" that > > any tempo or beat sync plugin would maintain. It seems easier to > > just count ticks, and then convert to beats, bars or whatever > > when you need those units. > > But how do you know on which bar the plug-in is ? Somewhat like in VST; I ask for a time info struct. > Tempo and > time signature can change in the course of the song and it > is not obvious to deduce the song position from just an > absolute time reference or a number of ticks or beats. That's why tempo and beat sync plugins in VST have to ask for time info for every block. The only difference with XAP is that if you have input controls for musical time, you'll get timestamped events whenever interesting changes occur, so if you just listen, you can have a sample accurate view of the musical timeline at all times. (XAP beats VST in this regard, since VST can only give you time info for the first frame in each block. XAP plugins can maintain sample accurate beat sync even if there are several *meter* changes within a single block. All VST can tell you is the tick time for points in audio time, so you could miss meter changes.) > > Yes. And bars and beats is just *one* valid unit for musical > > time. There's also SMPTE, hours/minutes/seconds, HDR audio time > > and other stuff. Why bars and beats of all these, rather than > > just some linear, single value representation? > > Could be seconds or ticks, but the latter make more sense as it's > > locked to the song even if the tempo is changed. > > For reasons mentioned above, both time in second an musical > representation are needed. I'm not denying that. > SMPTE etc can be deduce from the > absolute time in seconds or samples. Yep, but that's true for musical time as well - although musical time *is* indeed of much more interest to many more plugins, so special-casing it in the API is probably motivated. (Which is why XAP has a set of controls specifically for musical time.) > > Accuracy is *not* infinite. In fact, you'll have fractions with > > more decimals that float or double can handle even in lots of > > hard quantized music, and the tick subdivisions used by most > > sequencers won't produce "clean" float values for much more than > > a few values. > > But is that so important ? Float or int, we manipulate numbers > with a finite precision. Sure, but multiplication of two integers always gives you an exact result. (Yes, even if it doesn't fit! It's just that high mid and level languages can't make use of the upper half of the result - which I find really rather annoying... *heh*) > And if someone is happy with tick > scales multiple of 3, someone else will complain about not > supporting multiples of 19, etc. Yeah, that's why sequencers use these "weird" numbers... They don't cover *everything* of course, but they're high enough that rounding times to the nearest tick is ok - and then you can keep calculations in the sequencer exact. > For the drifting question, it is neglictible with double, and > can be easily resync'd if required. Ohm Force's plug-ins use > double as positions and steps for the LFO phases and it works > fine without resync during hours, on any fraction of beat. > > If you need exact subdivisions, you can still convert the > floating point value into the nearest integer tick. Well, I can't disagree, as I used to argue for your position, and have yet to see actual proof of the rounding errors/drift being a problem. Accurate musical time calculations belong in the sequencer domain, as we can't support them anyway, unless possibly if we switch to musical time timestamps. > > The host would of course know the range of each control, so the > > only *real* issue is that natural control values mean more work > > for hosts. > > No it just would store and manipulate normalized values. > Natural values would be needed only for parameter connection > between plug-ins, if user chooses to connect natural values. There will be conversion work to do *somewhere* no matter what. However, I don't like the idea of making all controls have hard ranges. It restricts the users in ways not necessarily intended by plugin authors, since there's no way to set controls to values outside the range the plugin author decided to use, even if there's no technical reason why it couldn't be done. You're basically throwing away interesting possibilities. > > What's wrong with text and/or raw data controls? > > They often require a variable amount of memory, making > them difficult to manage in the real-time thread. Yes, I'm perfectly aware of that... *heh* > However there is a need for this kind of stuff for sure. > I thought more about encapsulating them into specific > events. Yeah, that's the approach I have in mind. Chained fixed size blocks or something... It's not too much fun working with, but we *are* dealing with real time here; not word processing. People wanting to write real time software will simply have to accept that it's a different world, since there's no way we can changes the laws of nature. > > 1) none - host can autogenerate from hints > > 2) layout - plugin provides XML or something suggesting it's UI, > > host draws it > > 3) graphics+layout - plugin provides XML or something as well as > > graphics - host is responsible for animating according to plugin > > spec 4) total - plugin provides binary UI code (possibly bytecode > > or lib calls) > > I agree with 1) and 4). For my experience, when a plug-in > has 4), it has also 1), but it's not *always* the case > because developers don't feel necessary to implement the > hints correctly (dev time matters). Adding more UI > description is likely not to be supported by either host > or plug-in, making them useless. Some of it can be made optional, of course. In fact, even range and default values could be made optional. Just define the default range for an unmarked control as [0, 1], and the default value as 0. Whether the range should be soft or hard, I'm not sure, but soft seems more useful. > Good solution would > be using portable UI libs, as someone suggested here, > but everyone knows it's the Holy Grail. Yeah... *heh* Even the nicest toolkit/scripting solution will only work for *most* plugins; not the ones that want new, cool visual effects. Though SDL is as portable as it gets, and what you can't do with that can't be done, basically. I can assure you that an API like that *will* work for any plugin GUI, one way or another. How? All it provides is basic blitting and pixel level access to the screen and other surfaces. This is stuff that any platform with graphics can support, and most GUI toolkits also have some form of low level canvas or other component that can do it. I've even implemented the SDL 2D API on top of OpenGL (glSDL), for lightning fast rendering where h/w acceleration is available. (Pretty much the only way to have fully accelerated SDL on X at this point.) If we want to spend time on anything along the lines of a portable GUI toolkit, SDL's graphics functionality is the *first* thing we should implement, or it's just a waste of time. Then we can start thinking about building something like VSTGUI and/or other stuff on top of that. > > The spec should not dictate ABI, either. The ABI is an articat > > of the platform. If my platform uses 32 registers to pass C > > function arguments, it is already binary imcompatible with your > > PC :) > > The only way to guarantee real portability through platforms > and programming languages is to describe the API at the byte > level. This include calling convention. IMHO We can assume > that every target system has a stack, and can pass parameters > on it. Good point. I think we can count on standard C calling conventions to work and be supported by all languages on every platform. No big deal if it isn't as fast as the default conventions - the heavy traffic goes through the event system anyway, and there are no per-event function calls. Of course, a language that cannot interface with the event struct would be a problem. Requirements: Pointers, 32 bit integers, 32 bit floats and 64 bit floats. (We *could* make all integers 64 bit on 64 bit CPUs, but it seems insane and pointless to me. 64 bit machines don't have infinite memory bandwidth...) [...] > If you want to generate only ASCII in your program, just write > only ASCII, it will be supported by UTF-8. However if you have > to read strings, it's better to be aware that it's UTF-8, > because incoming strings may contain multi-byte characters. ...and what that means to your average plugin is basically "Don't start splitting strings arbitrarily!" Would be interesting to know which ASCII values are valid inside multibyte charatcers, BTW. Is there a risk you'll see false slashes, colons and things like that in paths, if you don't parse the UTF-8 properly? (There isn't IIRC, but I'll have to read up on this.) [...] //David Olofson - Programmer, Composer, Open Source Advocate .- The Return of Audiality! --------------------------------. | Free/Open Source Audio Engine for use in Games or Studio. | | RT and off-line synth. Scripting. Sample accurate timing. | `---------------------------> http://olofson.net/audiality -' --- http://olofson.net --- http://www.reologica.se ---
