Hi Max, you are welcome. You can start with the David Yeh phd thesis (someone posted the link here in this thread), which is a good summary of the state of the art. Then you will likely go thru the references to find the master papers dated late 1990s by Borin-DePoli (my professors btw). Just wanted to tell you that implicit FD is intriguing and nowadays almost everywhere in pro audio. When I was at the university I simulated in _realtime_ the Van Der Pol and the Chua Felderhoff circuits, leading to chaotic behaviors. I dont think you can get those behaviors with explcit methods, because they become pretty quick unstable.
Have fun! M. -----Messaggio originale----- Da: music-dsp-boun...@music.columbia.edu [mailto:music-dsp-boun...@music.columbia.edu] Per conto di Max Little Inviato: sabato 16 novembre 2013 16:06 A: A discussion list for music-related DSP Oggetto: Re: [music-dsp] R: R: R: R: Implicit integration is an important term, ZDF is not Hi Marco Yes, you're right, you can simulate at the lower component level which ought then to be monotonic. It is a trap for the unwary though: if you try to use implicit FD at the 'functional' level you can get non-monotonicity even if there is monotonicity at the component level, or so it would seem from our discussion here. I think using explicit FD is hard enough as it is, my expectation would be that implicit FD is much more difficult to use in practice - considering that we haven't even begun to look at all the other issues such as those that arise in numerical methods for root finding, which don't arise in explicit FD. But that's another topic of course! Anyway, thanks for bringing up this interesting theory about uniqueness, if you have any academic references I'd be grateful! Cheers Max On 16 Nov 2013 12:55, "Marco Lo Monaco" <marco.lomon...@teletu.it> wrote: > Hi Max, > yes you can do that, of course it is quite typical. But a multiplier > itself is made of a lot of "monotone" components that arranged in that > way create a quadratic law. I was referring to basic discrete > circuits. > BTW the Van der Pole was made by him, with tetrodes in a fancy > feedback > topology: I never investigated on that analog model (only studied the > general form). I guess that monotonicity is likely lost when building > in oscillators. > > In my experience choosing the branch is always done on physical > considerations about signals. I will investigate further when it will > be the time to face those problem again. > Btw, in the Wikipedia example you choose the right positive branch of > the sqrt because you have constraints on the signal y belonging to > [0,a]. What happen if you wouldnt have such a constraint? > > M. > -----Messaggio originale----- > Da: music-dsp-boun...@music.columbia.edu > [mailto:music-dsp-boun...@music.columbia.edu] Per conto di Max Little > Inviato: sabato 16 novembre 2013 12:49 > A: A discussion list for music-related DSP > Oggetto: Re: [music-dsp] R: R: R: Implicit integration is an important > term, ZDF is not > > Hi Marco > > Thanks - yes, OK with monotonicity that makes sense. So, I picked out > the squaring example and the van der Pol equations to try to prompt > whether this was required or not. For the rather trivial square > example, can't you just rig up an analogue multiplier with standard to > square the input? I picked up my old copy of Horowitz and Hill and on > page 140, they describe analogue multipliers 'exploiting the > g_m-versus-I_C characteristics of bipolar transistors, using matched > arrays to circumvent problems of offset and bias shifts'. Sounds like > this can be built using standard components. So, I remain to be > convinced that implicit FD, coupled with this uniqueness theorem, will > always be a good choice for FD simulations of all analogue audio > circuitry, regardless. I think you still have to pay careful attention > to what the circuit actually implements. > > In any case, as you say, you can always choose one branch or the other > if monotonicity doesn't hold, but I'm not sure if you can always make > the 'right' choice of branch in every case - there may be situations > where you have to 'hop branches' so to speak. I think it gets quite difficult then. > > Cheers > Max > > On 16 November 2013 10:40, Marco Lo Monaco <marco.lomon...@teletu.it> > wrote: > > Yes Max, it has been for 10 years that I have been intrigued by this > > approach :) > > > > AFAIK there is no electronic component among the ones I mentioned > > that can realize a Fnl(x) = x^2 nonlinearity (meaning no bipole in "nature" > > can do that). > > To tell the truth, I forgot to say that mononicity is a required > > thing to make things work for sure :)))) (almost all nonlinear > > electronic devices are monotone in some sense, a diode is an > > example! On the contrary a tunnel diode or a neon bulb have > > nonlinearities that are not monotone and can be more tricky to > > handle, but they are > rarely(never?) > used in audio). > > Also the K matrix systems has always practical values that lead to > > Dini condition to be satisfied easily. > > > > In the ODE you suggested the implicit function G(p, y) = (K*y+p)^2 - > > y = 0 is only locally explicitable when: > > > > G'(y) = 2*K*(p + K*y) - 1 != 0 > > > > Setting up the system: > > G'(y) = 0, > > G(p,y) = 0 > > > > And solving for p0,y0 leads > > > > (p0, y0) = [1/4/K, 1/4/K^2] > > > > thus that point is the only one point where the implicit function > > Dini's theorem fails to be applied. > > > > For every point other than p0,y0, there can be more than one > > solution (branches of implicit function) and an appropriate choice > > must be made (exactly the same thing that happens when choosing the > > sign of the sqr root in the Wikipedia example). Nonetheless the > > solution is guaranteed and unique. Note that this kind of > > non-monotone functions are typically involved with deterministic > > chaos (Chua RLC and Van der Pole are two famous examples, so very > > likely period doubling and bifurcation will happen when a driving force is applied). > > > > So, to answer your question I would say that the solutions are > > always unique by appropriate choice of the branch of implicit function G. > > Nonetheless such a quadratic nonlinearity AFAIK is not met in any > > practical electronic nonlinear device (at least in my experience). > > > > Hope to have helped again. > > > > Ciao > > > > Marco > > > > -----Messaggio originale----- > > Da: music-dsp-boun...@music.columbia.edu > > [mailto:music-dsp-boun...@music.columbia.edu] Per conto di Max > > Little > > Inviato: giovedì 14 novembre 2013 21:34 > > A: A discussion list for music-related DSP > > Oggetto: Re: [music-dsp] R: R: Implicit integration is an important > > term, ZDF is not > > > > Hi Marco > > > > Thanks - sounds intriguing although I don't really follow your > > argument. So, to simplify, let us consider this ODE: > > > > dy/dt=-y^2 > > > > If you can build a circuit using your component examples, then you > > would have non-uniqueness with the backward Euler method. Or, are > > you effectively saying that you can't possibly build such a circuit > > out of the components you mention? > > > > Max > > > > On 14 November 2013 20:20, Marco Lo Monaco > > <marco.lomon...@teletu.it> > wrote: > >> Hi Max, > >> you need to convert your nonlinear system into a state space model > >> with some addiction for dealing non-linearity, reaching a set of 6 > >> matrixes instead of the 4 usual ones (see Yeh's PHD thesis for a > >> good > > starting point). > >> Once you use an implicit integration scheme (like bilinear) you > >> will always reach (after all the calculations) to a non linear > >> system of eqs in the form of y - Fnl(Ky + p) = 0 where p is an > >> historical > >> contribute(=memory) of the system known at time n. You will have to > >> solve in some way (typically Newton > >> Raphson) that nonlinear system. > >> Given the type of Fnl in normal audio circuits (diodes, tubes, > >> transistor and opamp) the uniqueness of the solution of that eq is > >> guaranteed with the conditions I told you, and it is a straight > >> consequence of Dini's implicit function > > (http://en.wikipedia.org/wiki/Implicit_function_theorem) theorem. > >> So if you let > >> y - Fnl(Ky + p) = G(p,y) > >> Since you want to solve the implicit function G(p, y) = 0, the > >> conditions to do this locally (around some y0) is dG/dy != 0. Given > >> the form of G in the multivariate case (MIMO) you will reach the > >> condition I explained in my last post. > >> Note that differentiability of Fnl is _not_ requested. What is > >> requested is the differentiability of dG/dy, which if met for all y > >> implies global explicability of G, thus only one solution is > >> possible and unique. This conditions is less restrictive than you > >> could expect, in fact, for instance it works also for simplified > >> PWL (non > >> differentiable) saturator characteristic function like opamps. > >> I've never found in my 10 yrs experience switching capacitors audio > >> circuits to be modeled and with prominent nonlinearity (I only > >> dealt with bucket brigade analog delay lines but its a simplified > >> story for them). I think nonetheless it is possible to model > >> switching circuits by exchanging the state variables among > >> different nonlinear state space representations (samplerate is then > >> a limit). I dont know if it is realistic or worth to do that, but > >> as a matter of principle it probably > > should be at least possible. > >> > >> Hope this helped > >> > >> Marco > >> > >> > >> > >> -----Messaggio originale----- > >> Da: music-dsp-boun...@music.columbia.edu > >> [mailto:music-dsp-boun...@music.columbia.edu] Per conto di Max > >> Little > >> Inviato: giovedì 14 novembre 2013 19:07 > >> A: A discussion list for music-related DSP > >> Oggetto: Re: [music-dsp] R: Implicit integration is an important > >> term, ZDF is not > >> > >> Hi Marco > >> > >> I don't know, in this way you're ruling out rather simple > >> non-invertible nonlinearities such as the humble quadratic which > >> can occur > > in 'working' > >> nonlinear circuits: > >> http://en.wikipedia.org/wiki/Van_der_Pol_equation > >> > >> The rather trivial example here shows the problem quite clearly > >> (see backward Euler method): > >> http://en.wikipedia.org/wiki/Explicit_and_implicit_methods > >> > >> Also this analysis only works if you have differentiability and > >> there are some rather ubiquitous, practical nonlinear circuits > >> (switched capacitors for example) where this doesn't hold. > >> > >> Max > >> > >> On 14 November 2013 14:52, Marco Lo Monaco > >> <marco.lomon...@teletu.it> > > wrote: > >>> Hi Max, > >>> the uniquess is granted by the Dini's theorem which is satisfied > >>> always in common practical analog schematic (unless you are > >>> dealing with some exotheric Chua's multiple DC operating points). > >>> > >>> That condition is met if > >>> > >>> det(Jnl*K-I)!=0 > >>> > >>> where Jnl is the Jacobian of the MIMO non linearity and K is the K > >>> matrix (see DePoli et alias). > >>> It's when the implicit method becomes "explicit" only locally (or > >>> globally) and you can break the uncomputabilty of the graph, > >>> defeating the instantanoues delay in z-domain. > >>> In all the practical case the nonlinear implicit function is > >>> guaranteed to be globally "explicitable". That also makes sense > >>> since you are modeling a physical system that is actually working > >>> and must have uniqueness of solution. > >>> > >>> Marco > >>> > >>> -----Messaggio originale----- > >>> Da: music-dsp-boun...@music.columbia.edu > >>> [mailto:music-dsp-boun...@music.columbia.edu] Per conto di Max > >>> Little > >>> Inviato: giovedì 14 novembre 2013 15:14 > >>> A: A discussion list for music-related DSP > >>> Oggetto: Re: [music-dsp] Implicit integration is an important > >>> term, ZDF is not > >>> > >>> Thanks Ross. > >>> > >>> Good point about the practical utility of implicit FD and > >>> increasing computational power. There's also all the issues about > >>> uniqueness of implicit FDs arising from nonlinear IVPs, and then > >>> there's stability, convergence, weather the resulting method is > >>> essentially non-oscillatory etc. I suppose there are additional > >>> issues to do with frequency response which may be what matters most in audio DSP. > >>> > >>> Max > >>> > >>> > >>> On 14 November 2013 14:06, Ross Bencina > >>> <rossb-li...@audiomulch.com> > >> wrote: > >>>> On 14/11/2013 11:41 PM, Max Little wrote: > >>>>> > >>>>> I may have misread, but the discussion seems to suggest that > >>>>> this discipline is just discovering implicit finite > >>>>> differencing! Is that really the case? If so, that would be odd, > >>>>> because implicit methods have been around for a very long time in numerical analysis. > >>>> > >>>> > >>>> Hi Max, > >>>> > >>>> I think you would be extrapolating too far to say that a few > >>>> people tossing around ideas on a mailing list are representative > >>>> of the trends of an entire discipline. On this mailing list I > >>>> would struggle to guess which "this discipline" you are refering to. > >>>> Suffice to say that a lot of the people discussing things in this > >>>> thread are developers > >>> not research scientists. > >>>> > >>>> Some practitioners are just "discovering" new practicable > >>>> applications of implicit finite differencing in the last 10 years > >>>> or so. One good reason for this is that in the past these > >>>> techniques were completely irrelevant because they were too > >>>> expensive to apply in real time at the required scale (100+ > >>>> synthesizer voices, 100+ DAW channels). It also seems that the > >>>> market has changed such that people will pay for a monophonic > >>>> synth that burns a whole > >>>> i7 CPU core. > >>>> > >>>> Cheers, > >>>> > >>>> Ross. > >>>> > >>>> -- > >>>> dupswapdrop -- the music-dsp mailing list and website: > >>>> subscription info, FAQ, source code archive, list archive, book > >>>> reviews, dsp links http://music.columbia.edu/cmc/music-dsp > >>>> http://music.columbia.edu/mailman/listinfo/music-dsp > >>> > >>> > >>> > >>> -- > >>> Max Little (www.maxlittle.net) > >>> Wellcome Trust/MIT Fellow and Assistant Professor, Aston > >>> University TED Fellow (fellows.ted.com/profiles/max-little) > >>> Visiting Assistant Professor, MIT > >>> Room MB318A, Aston University > >>> Aston Triangle, Birmingham, B4 7ET, UK UK +44 7710 609564/+44 121 > >>> 204 > >>> 5327 Skype dr.max.little > >>> -- > >>> dupswapdrop -- the music-dsp mailing list and website: > >>> subscription info, FAQ, source code archive, list archive, book > >>> reviews, dsp links http://music.columbia.edu/cmc/music-dsp > >>> http://music.columbia.edu/mailman/listinfo/music-dsp > >>> > >>> -- > >>> dupswapdrop -- the music-dsp mailing list and website: > >>> subscription info, FAQ, source code archive, list archive, book > >>> reviews, dsp links http://music.columbia.edu/cmc/music-dsp > >>> http://music.columbia.edu/mailman/listinfo/music-dsp > >> > >> > >> > >> -- > >> Max Little (www.maxlittle.net) > >> Wellcome Trust/MIT Fellow and Assistant Professor, Aston University > >> TED Fellow (fellows.ted.com/profiles/max-little) > >> Visiting Assistant Professor, MIT > >> Room MB318A, Aston University > >> Aston Triangle, Birmingham, B4 7ET, UK UK +44 7710 609564/+44 121 > >> 204 > >> 5327 Skype dr.max.little > >> -- > >> dupswapdrop -- the music-dsp mailing list and website: > >> subscription info, FAQ, source code archive, list archive, book > >> reviews, dsp links http://music.columbia.edu/cmc/music-dsp > >> http://music.columbia.edu/mailman/listinfo/music-dsp > >> > >> -- > >> dupswapdrop -- the music-dsp mailing list and website: > >> subscription info, FAQ, source code archive, list archive, book > >> reviews, dsp links http://music.columbia.edu/cmc/music-dsp > >> http://music.columbia.edu/mailman/listinfo/music-dsp > > > > > > > > -- > > Max Little (www.maxlittle.net) > > Wellcome Trust/MIT Fellow and Assistant Professor, Aston University > > TED Fellow (fellows.ted.com/profiles/max-little) > > Visiting Assistant Professor, MIT > > Room MB318A, Aston University > > Aston Triangle, Birmingham, B4 7ET, UK UK +44 7710 609564/+44 121 > > 204 > > 5327 Skype dr.max.little > > -- > > dupswapdrop -- the music-dsp mailing list and website: > > subscription info, FAQ, source code archive, list archive, book > > reviews, dsp links http://music.columbia.edu/cmc/music-dsp > > http://music.columbia.edu/mailman/listinfo/music-dsp > > > > -- > > dupswapdrop -- the music-dsp mailing list and website: > > subscription info, FAQ, source code archive, list archive, book > > reviews, dsp links http://music.columbia.edu/cmc/music-dsp > > http://music.columbia.edu/mailman/listinfo/music-dsp > > > > -- > Max Little (www.maxlittle.net) > Wellcome Trust/MIT Fellow and Assistant Professor, Aston University > TED Fellow (fellows.ted.com/profiles/max-little) > Visiting Assistant Professor, MIT > Room MB318A, Aston University > Aston Triangle, Birmingham, B4 7ET, UK UK +44 7710 609564/+44 121 204 > 5327 Skype dr.max.little > -- > dupswapdrop -- the music-dsp mailing list and website: > subscription info, FAQ, source code archive, list archive, book > reviews, dsp links http://music.columbia.edu/cmc/music-dsp > http://music.columbia.edu/mailman/listinfo/music-dsp > > -- > dupswapdrop -- the music-dsp mailing list and website: > subscription info, FAQ, source code archive, list archive, book > reviews, dsp links http://music.columbia.edu/cmc/music-dsp > http://music.columbia.edu/mailman/listinfo/music-dsp > -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
