Re: [music-dsp] Computational complexity of common DSP algorithms
On 2020-03-19, Dario Sanfilippo wrote: Thanks for your email, all good points. From the top of your head, could you please point me to a reference for the measurement of calculations needed in direct-form filters? The best computational complexity in direct form convolution is guaranteed to be zero-delay convolution in the Gardner form, so O(N log M) where N is the number of samples processed and M is the support of the eventual weight/measure of the nonzero portion of the LTI impulse response. (There are methods which lead to longer responses at lower cost, but they all trade in something for something. At full innovation density/critical sampling, you cannot do any better.) This is because this sort of processing can be reduced to a binary sorting problem, which is guaranteed to asymptotically take as much time. (It is a research problem whether non-binary sorting algorithms could perchance be generalized to this setting as well; no such generalizations exist as of now.) https://www.google.com/url?sa=t=j==s=web=1=rja=8=2ahUKEwi74uPM56foAhWnxcQBHeG1BU0QFjAAegQIAxAB=http%3A%2F%2Fwww.cs.ust.hk%2Fmjg_lib%2Fbibs%2FDPSu%2FDPSu.Files%2FGa95.PDF=AOvVaw1lFWuE1IrzjRZDOD-VAP05 Gardner's algorithm is then best in more than one way. It doesn't just yield the best asymptotic performance, but exactly zero delay as well. And the guy didn't even stop there. He also broke the traditional overlap-add-FFT-convolution algorithm in running parts, so that it 1) becomes an exponential cascade of overlapping processes, whose running time sums to constant one, instead of peaking at any stage, even as a whole, and 2) it optimally reuses previous half-results in order to derive the next one. (That has a lot to do with both the OLA structure, and at the same time with how FFT is a realization of a full Fourier rotation in a function space, by coordinate-wise, parallel rotations, in a logarithmic/divide-and-conquer cascade.) I've in fact thought that I should apply the algorithm to certain coding theoretic problems, just now. Because, since it's guaranteed to be zero-delay, you can apply it willy-nilly in decision feedback decoding, on the coding side. And since it's guaranteed to be optimal on the LTI side of things, and it's a fully neutral, general, and provably efficient LTI-DSP primitive, why not take advantage of it...? ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3751464, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] the dominance transformation
As Gerzon describes it, the dominance transformation is a Lorentz boost in a certain direction, or a four-dimensional hyperbolic rotation, between the first four ambisonic channels WXYZ. As far as I know, that sort of thing only works neatly in plain 4D Minkowsky space, with the possible extensions becoming approximate at higher finite orders of surface spherical harmonical decomposition. Since I'm no relativistic quantum physicist, I'm not too sure about how to even begin to calculate with something like a relativistic atomic orbital above the first two S and P stateS -- corresponding to a rigid boost of a higher order decomposition, which we'd probably be after if we wanted to formalize dominance for higher orders, and which is also beyond a strictly local analysis in both the acoustic and EM fields. What I *do* know though, is that in the 1D setting of pulse radar time series, especially in connection with chirped pulses, hyperbolic phase rotations used to be approximated piecewise using physical allpass filters in various constant coefficients. Also, it's possible to piece back together a perfect LTI response from a heterodyne system which implements each band as a simpler non-LTI, SSB-modemed system. Thus, long story short, is there a theory out there of heterodyning and bandshifting akin to the SSB-radio one for spherical harmonics? It certainly couldn't work as nicely as it does with normal frequencies, since there e.g. just isn't enough room for the information if you downshift from second order to first; conversely there *is* going down linearly in normal frequency from (x,2x) to (0,x). But since the spherical harmonic progression too is neatly *quadratic* instead of linear, *maybe* there is somewhere a theory which lets us *quadratically* transpose spatial harmonics, and so modulate willy nilly? Anybody privy to anything like that? Because if there is such a theory, formally transposing the dominance operator one minimum level up could clear the way to actually characterising and even implementing higher order dominance for good. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Auto-tune sounds like vocoder
On 2019-01-15, David Reaves wrote: I’m wondering about why the ever-prevalent auto-tune effect in much of today's (cough!) music (cough!) seems, to my ears, to have such a vocoder-y sound to it. Are the two effects related? If you want to do autotuning of not just one (over)tone but many, you will have to do something approaching a constant bandwidth-time product. Something approaching 1/f versus t stretch in your prosessing. When you try that out, the only invariant out there which works out fully is the minimal Fourier coupling between frequency and impulse response; that same thing can be extended into a coupling between all harmonic series in music, once we impose a time-bandwidth symmetry on our stuff. After that, we only need to analyze our caugh-caugh -stuff, continuous or discrete, as one edemplar of the Lorenz group of time-frequency dilations, modulo an imperfect comb filter in frequency. What necessarily results is a more or less spectrally spread out 1/f -decaying, more or less periodic in period instead of frequency, inverse-harmonic decomposition. That is the only way retuning *can* work even with two sinusoidal signals at a time, eventually the mathematical rigidity of the overall problem will make us do something like a spectrally nice remodulation, with few enough aharmonical intermodulation products. Since we have to estimate, statistically, all of our incoming "voices", we will have to also take care of their cross terms after remodulation. Those terms will have infinite degree, because of the 1/f term in both time and frequency, after remodulation. This makes the optimum solution hard to find, even under the 1/f, 1/t symmetry we already know the continuous statistics of the problem dictate. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theory "best" explanation
On 2017-08-25, Nigel Redmon wrote: http://www.earlevel.com/main/tag/sampling-theory-series/?order=asc Personally I'd make it much simpler at the top. Just tell them sampling is what it is: taking an instantaneous value of a signal at regular intervals. Then tell them that is all it takes to reconstruct the waveform under the assumption of bandlimitation -- a high-falutin term for "doesn't change too fast between your samples". Even a simpleton can grasp that idea. Then if somebody wants to go into the nitty-gritty of it, start talking about shift-invariant spaces, eigenfunctions, harmonical analysis, and the rest of the cool stuff. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] basic in-store speaker evaluation
On 2017-07-04, Andy Farnell wrote: Something in the vein of "just put in a test DVD-A, and let your Android app run"? DVD? Maybe in 2000. Not knowing the playback capabilities at the store you'd be better to put the test files online, in a spotify or soundcloud channel. Granted...if you could just drop something like eight channel FLAC's in and have them work out of the box on any and all of your setups. I too thoroughly hate the standardization that DVD-A never was. But you'll have to agree there's one thing which speaks for it: choosing the proper rates and channel counts, every little bit of variance has been certified and test out of the system. It *does* work out of the box, unlike so many newer file based systems. The room and listening position will be variables, needs factoring out clientside, so quite a bit of DSP on the mobile. Not perhaps on the mobile; let the mobile just work as the pickup, and stream the result back to something more workstation-like. Obviously you couldn't help your phone's pickup being uneven, that You need to know the phone, the app must do a client detect and look up a database because there are large variances between devices. If you're only doing comparisons, you don't need that. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] basic in-store speaker evaluation
Is there an extant software out there which lets me do comparisons between various speaker sets? Something in the vein of "just put in a test DVD-A, and let your Android app run"? I mean, something like that ought to be doable per se. You'd just record your normal test signals such as an MLS noise of known characteristic, a 1kHz sine wave, a set of linear chirps of growing magnitude, and a bunch of synch signals, in some reasonable combination. So that you could at least in theory do synchronous detection of whatever you hear from your test DVD-A, simply by listening to it via your phone. Obviously you couldn't help your phone's pickup being uneven, that way. But if you put the phone in the far field, pretty much the only thing you'd be missing would be its linear characteristic. That would distort any one measurement, but not any linear comparison. Also it needn't distort any nonlinear measurement such as THD, per se. Is there an app for that already? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] ± 45° Hilbert transformer using pair of IIR APFs
On 2017-02-07, Theo Verelst wrote: Like with many transforms, I can't help but practically think that it's hard to make a tradeoff between the meaning of the results, such as [...] Here there's an rather simple optimization criterion: a constant 45 degree phase offset, or perhaps a pair of filters whose relative offset is very near to 45 degrees. This stuff isn't really used for sound synthesis per se, but as a low level primitive, to get high level interesting algorithms. I like the idea of an IIR filter for frequency analysis, or a bank of them with dyadic structure, Would they have the channels in order, or with half of them frequency inverted? What could they be used for? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] ± 45° Hilbert transformer using pair of IIR APFs
On 2017-02-06, robert bristow-johnson wrote: [...] and analytic signal a(t) = x(t) + j y(t) = g(t) cos(w t) + j g(t) sin(w t) = g(t) e^(j w t) the analytic envelope is |a(t)| = sqrt( x(t)^2 + y(t)^2 ) = g(t) so that works great for a single sinusoid. Indeed. Originally the formula was derived for a single sinusoid. At the outset it's very much like you derive group delay from phase delay, given a single sinusoid. However, the formula generalizes just as well. What you have there is a single complex sinusoid, with a gain function g(t) (ostensibly an exponentially decaying one). Leave that out, and you have a single complex sinusoid. Which you can then integrate over frequency to yield your typical Fourier transform, in the s-plane, or sum over the unit circle in the z-plane to yield the discrete version. In either case, you have the complex version of Parseval's theorem backing you up. Now that we went to the Fourier domain, suddenly even those complex sums/integrals work out just fine, as (approximable) sums of complex exponentials (over a discrete basis, by the sampling theorem and such). Suddenly what you get for an arbitrary sum of exponentials, is something whose backwards transform's norm is by Parseval very much the same as you had before for a single frequency. That is why Hilbert-transforms are used to get "instantaneous envelopes" even with wideband signals. Basically the pointwise modulus of a Fourier transform of an analytical signal does what you did above for a single frequency, a sinusoid, while the whole of the Fourier transform does it for all frequencies at the same time, and the finally by Parseval even the sum/integral actually, first locally but then by extension globally, guarantees that the modulus actually follows the whole signal envelope as well. What fucks you up, then, is that a proper Hilbert transform which lets you access the analytical continuation of a real signal, is not only an acausal operator, but also one which converges rather slowly in (future) time. It does so in o(1/t) time/shift, just as the ideal sinc(x) interpolation function does; which is rather slow as it goes, especially in asymptote. That then implies that if you try to do "instantaneous envelopes" using Hilbert transforms and the analytical signals they imply, you might seem to get "something for free". A better and more reactive estimator of envelopes. The theory is sound as well, because of what I said above about Parseval's theorem. It's just that there's *still* no free lunch: if you want to approximate an Hilbert transform pair for real, for any given accuracy you'll have to expend o(1/t) delay in order to derive an o(1^-at) accuracy class Hilbert transformer (or a pair).) But once you toss in additional frequency components into x(t), then that creates high-frequency components into |a(t)|^2 that don't belong in any definition of envelope, no? If you look at it that way, yes. But isn't that the very reason Parseval's theorem is so great? It actually deals away with this kind of stuff, and returns the whole deal towards first order analysis over a function space. what other meaning of "envelope" are you working with, Eric? Finally, there are other meanings. The psychoacoustical ones in particular. Which I'm not too well versed with. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] ± 45° Hilbert transformer using pair of IIR APFs
On 2017-02-06, Eric Brombaugh wrote: well, with a single sinusoid, there should be no intermodulation product so the analytic envelope should be exactly correct. but consider: [...] I might be way off base here, but... As Olli said, both the poles and the zeroes sorta "like to be" on the real line. To me that is kind of a canary. Typically when something works on the real line, it tends to work better when distributed over the unit circle. Often even reflected over the unit point. Olli, I *know* this is again highly intuitionistic. But could you still, as the kind of guy who actually follows through with his math, check this out? I am reasonably, intuitionistically sure, that you started your optimization from a local basin. I'm pretty sure there is at least one other basin even given your optimization criteria, given by alternating zeroes and poles over the left half of the unit circle over the z-plane, just as you said they worked alternately over the real line. Or perchance over some elliptical locus. Please, try that original setup as well. Then tell us what your optimization machinery did with it. I'm reasonably sure it wouldn't converge to what you originally had, even if it converged to *something* good. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Using an actual database for storage of DSP settings
On 2016-12-28, Theo Verelst wrote: Did anyone here do any previous work on such subject ? I mean I don't expect some to come up and say : "sure ! here's a tool that automatically does that for all Ladspa's" or something, but maybe people have ideas... I can't say I've done anything specialized towards sound, but in what career I did once have, I've modelled my fair share of data within the relational framework. If you need help in that regard, I'm available. Enthusiastic, even. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Anyone think about using FPGA for Midi/audio sync ?
On 2016-08-13, Theo Verelst wrote: For a class of applications where at least you would want sample accurate control messages, [...] That's not about music-dsp, but dsp simple. There's a reason why all synthesis architectures out there make distinction between modulation and audio rate events. The first are supposed to be humanly understandable and deliverable even in real time environments. The latter are part of the inner workings of your synthesis algorithm. [...] buffering for efficient pipeline and cache use dictates some form of delay control, which IMHO should be such that from Midi event to note coming out of the DAC, there is always a accurately fixed delay. Not that many of us perfectionistas wouldn't have been thinking about this problem from the start... So I though it might be a good idea to time stamp Midi messages with an Fpga (I use a Xilinx Zynq), and built in some form of timing scheduler in the FPGA to help the kernel. That's plain overkill. All that you need is a well-synchronized realtime clock and a fast consensus algorithm. You can get the first over any extant Ethernet technology in controlled congestion state by using PTP ( https://en.wikipedia.org/wiki/Precision_Time_Protocol ). By rounding your events to the nearest microsecond or so, including time stamps, delaying your events a bit, and going with something like http://www.cse.buffalo.edu/~demirbas/publications/augmentedTime.pdf , you can approach perfection in latency and in fact attain it in local synchronization of the end result, quite without resorting to expensive hardware. Relatively cheap microcontrollers could keep up with that sort of thing any day of the week, without the total cost per node creeping past half that of a Pi. I'm not talking about a hardware Linux "select()" call as kernel accellerator or single sample delay Fpga DSP, or upgrading to dozens of Fpga pins at a hundred times Midi clock rate doing clock edge-accurate timing, but an intersting beginning point for the next generation of accurate DSP software and interfaces. "Accurate DSP software and interfaces." What you're talking about is form beyond function. If you want to do some super-sensitive remotely gated high energy shit in the CERN vein, go ahead. This is what you need. But that doesn't have much to do with MIDI signals or audio, anymore. Certainly not music. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] BW limited peak computation?
On 2016-07-26, Stefan Stenzel wrote: the acid test is when the pre-upsampled data is alternating signs on a large amplitude with *one* sample missing. like: ... -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, -A, +A, ... you might get an unnaturally large peak (many times bigger than |A|) with that. Should be something like the sum of all 2A/(pi*(t+0.5)) for t an integer going from zero to infinity. Not sure if that converges. It doesn't. Intuitively speaking, after some heady limit arguments, a pure ...,+1, -1, +1,-1,... train decodes as a sinusoid at the Nyquist frequency. It does so because of phase cancellation between the various, in-phase but at least double in period sinusoids which go into the Nyquist-Shannon reconstruction. What this particular sequence does is, it sums two one-sided alternating sequences together, too. But it does it so that it leaves out a single sample at the origin, and by doing so, pushes those otherwise nicely cancelling, marginal cosines from the various sinc(x) functions into phase between two sampling times. Obviously what you get then is not convergence, but very fast divergence. All within the band-limited interpolation theorem, which in the limit only applies in general strictly below the Nyquist frequency. (This stuff for once doesn't converge even in the weak, distribution sense.) Now, what I wonder is, could you still somehow pinpoint the temporal location of an extremum between sampling instants, by baseband logic? Because I don't think there can be more than one between any two adjacent sampling times. And if you can pinpoint its location, then I believe you could derive an iterative algorithm relying on higher and higher, recursively computed interpolation formulae which could drive the uncertainty in its location as low as necessary. On the fly. Irrespective of what the real *amplitude* of the thing really is. If you could do that, I think you could then put upper bounds on both the L^1 and L^2 norms of the deviation above full range, between the sampling instants, in closed form. And once you'd done that, I think you could pretty much tame any sonically/musically relevant problem happening "between the samples", as well: just take note of where the peak seems to be for phase stuff, and then take note of how rapidly your interpolands grow, for an approximation of what you should do wrt amplitude. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] how to derive spectrum of random sample-and-hold noise?
On 2015-11-04, robert bristow-johnson wrote: it is the correct way to characterize the spectra of random signals. the spectra (PSD) is the Fourier Transform of autocorrelation and is scaled as magnitude-squared. The normal way to derive the spectrum of S/H-noise goes a bit around these kinds of considerations. It takes as given that we have a certain sampling frequency, which is the same as the S/H frequency. Under that assumption, sample-and-hold takes any value, and holds it constant for a sampling period. You can model that by a convolution with a rectangular function which takes the value one for one sampling period, and which is zero everywhere else. Then the rest of the modelling has to do with normal aliasing analysis. That's at least how they did it before the era of delta-sigma converters. with the assumption of ergodicity, [...] (Semi-)stationarity, I'd say. Ergodicity is a weaker condition, true, but it doesn't then really capture how your usual L^2 correlative measures truly work. i have a sneaky suspicion that this Markov process is gonna be something like pink noise. Something like that, yes, except that you have to factor in aliasing. r[n] = uniform_random(0, 1) if (r[n] <= P) x[n] = uniform_random(-1, 1); else x[n] = x[n-1]; If P==1, that give uniform white noise. If P==0, it yields a constant. If P==.5, half of the time it holds the previous value. In a continuous time Markov process you'd get something like pink noise, yes. But in a discrete time process you have to factor in aliasing. It goes pretty bad, pretty fast. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] how to derive spectrum of random sample-and-hold noise?
On 2015-11-05, robert bristow-johnson wrote: I think I was slightly off when I said that the units of psd are power per unit frequency -- since the whole signal has infinite power, no, i don't think so. Me neither. Power is already by definition energy per unit time. Even if an infinitely long signal often (not always) has infinite energy, *any* practical signal you would be dealing with has finite power over its entire length. And in fact physical signals can't have even infinite length or energy, so even if you go crazy and conceptually integrate everything from the beginning of the time to the End Times, it's reasonable to model any realistic signal as being globally, not just locally, square integrable (the square integral from -inf to +inf being the total energy). the units really need to be power per unit frequency per unit time, which (confusingly) is the same thing as power. the signal has infinite energy because it goes on (with power greater than some positive lower-bound) forever. but it's not infinite power unless it's something like "true" white noise (which has infinite bandwidth). Precisely. So what probably trips some up here is "power per unit time". That's nonsensical. What we need is power per unit frequency, i.e. energy per unit time per unit frequency. what comes out of a random number generator (a good one) is white only up to Nyquist. not infinite-bandwidth white noise. And, as a matter of fact, if you go to the kind of distributional stuff we talked about a while back, you can even deal with real white noise to a degree. That's because you can do local integration in the Fourier domain, where white noise has unity norm. The same argument then explains why a signal which has infinite length and infinite energy in the time domain is absolutely no problem for the kind of analysis we're talking about here: STFT analysis already makes your stuff local in time, so as long as the signal is of finite power, you'll get sensible local results, even if the signal is globally speaking of infinite energy (say like an ideal sinusoid). The only real kink is that when you localise your analysis, you're bringing in an extra degree of freedom: what precisely do the length and shape of your windowing/apodization function do to the results of the analysis. In spectral analysis work, that then mostly revolves around energy concentration within an FFT band, and specral spillover to the adjacent ones. Sometimes statistical estimation theory, and what phase does over successive windows. so the integral, from -Nyquist to +Nyquist of the PSD must equal the variance, as derived from the p.d.f. and that value also has to be the zeroth-lag value of the autocorrelation. Yes, and that by definition. If you have to deal with DC, then you have to separate autocorrelation from autocovariance. Also in that case the DC part spills over asymmetrically after windowing, because essentially it will be AM modulated by the window, and will alias upwards across the zero frequency point. This could be another reason why some special scaling is needed as compared to a finite-length FFT. really, the only scaling would be that comparing the Fourier integral (with truncated and finite limts) to a Riemann summation (which could be expressed as the DFT). As I understand it, scaling is mostly necessary because of numerical concerns. I mean... When you do longer STFT's, the implicit filter represented by each bin grows narrower and more selective. In other words, more and more resonant. If it then so happens that you hit a sinusoid right in the middle of the passband, a growing analysis window leads to an unlimited amount of power gathered on that coefficient. After all, the continuous time Fourier transform of a sinusoid is a Dirac distribution, and with a growing analysis window you'll approach that -- the series doesn't converge in the normal but only the weak sense, so that your STFT bin blows up. So there's a tradeoff between headroom and noise floor, here. Though, I could be talking about a different scaling problem than you folks. I did jump into the fray pretty late. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] about acoustical energy
BTW, now that we're talking about power and energy in the musical, acoustical sense, take a look at how much acoustical energy most speakers radiate. Even the biggest ones. A well tuned line array typically has .001 to .02 power conversion efficiency from electricity to acoustics. Most of its power also radiates away from its intended target. As a result, what you hear in a concert is mere milliwatts of acoustic power. Of those, some land on your ears; there a mere .05mW over the tympani membrane is deafening. Just sayin, us DSP folks typically don't know where the physical limits lie. Because we mostly deal with just numbers, we don't instantly see where the physical restraints on amplitude are, nor the rest of the psychoacoustics. As such, I'd encourage people to do what e.g. RBJ just did: find the absolute limits first, and then calibrate/scale their work based on that. Because that way our DSP-sport ain't just about abject numbers anymore; it has something to do with the real world as well. Its numerics, its physical acoustical counterparts, and whatnot. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] 20k
about in musical DSP, though. Isn't the definition of music something we acutely here, and find pleasing? If so, maybe we should actually speak more about how we hear and feel, in respect to our algorithms? And not so much about how to make an acoustical simulation just right? ;) (And yes, sorry again, I have a tendency to get carried of a bit. No harm, no foul, right...) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] [ot] 20k
On 2015-08-30, robert bristow-johnson wrote: And you know, you can do kernels hundreds of millions of samples long that way in real time, on what machine? does it breathe fire? Basically, yes. The example I've seen came from radar hardware, once again. It was liquid cooled custom ECL logic running at some 10GHz plus, on some exotic high mobility substrate. Doing this and only this, with the coefficients being recycled from a thousand fold banked DRAM fast enough to absorb the refresh cycle into the read one. The butterflies were etched straight into silicon, and they dissipated something like tens of kilowatts of continuous power for processing alone while doing Their Thing. The reason why you'd go to that extreme is that you a) want to track multiple signals at the same time, b) you want to identify them by their resonance too, c) there's no convenient Fourier-kinda way to do that wholesale because every target has its own Doppler shift wrt your antenna, and d) for LPI purposes you have to send out a cryptographically secure direct sequence spread spectrum signal which is a *mile* wide in frequency. As such, the computational requirements quickly become "military grade". So, yes, in the truest sense, the thing did breathe fire: tens of kilowatts is as many times as much as your nastiest gas fired stove puts out. a million samples long (10 or 20 second reverb time), i can sorta see. but hundreds of millions of (dense) samples long, even with fast convolution, seems to me to be outa reach of a modern laptop or desktop PC to do in real time. No shit! :D -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] music-dsp Digest, Vol 2, Issue 47
. :) I'd always thought that Cool Edit/ Audition used an efficient convolution routine to do a lot of it's stuff, and that it's 'FFT filter' used inverse FT to make an FIR. They do, just because of what I said above. Is it possible that a processor that specialises in FFT simply has an efficient method of doing a whole load of interrelated multiplies, and hence would be good for convolution? Yes, and that is precisely why there is dedicated silicon out there for doing FFT's (and related transforms like the DCT). For instance, in image processing, most modern video codecs rely on approximations of 2D Fourier transforms, implemented either as straight discrete cosine transforms of sometimes as approximations to the modified, lapped, MDCT one. Those implement the detail reduction which let modern video codecs do their perceptual quantization thingy. Every time you watch a video on your iPhone, you're nowadays relying on coded-into-silicon approximate 2D Fourier transforms, because doing it in silico saves so much power/battery. For that reason there's hardly an appliance out there with an audio or video output which doesn't have some Fourier minded machinery imprinted on its very hardware. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] [ot] 20k
On 2015-08-30, gwenhwyfaer wrote: Cool, someone else who's as loath to join the 21st century as I am... This kind of makes you wonder about when you'd actually try to brute force tens of thousands of sample long FIR's, no? One example I know of is from brute force cryptanalysis of nonlinearly coupled linear feedback shift register based stream ciphers. There the frequent nonlinear rekeying makes Fourier methods useless, so what you do is you have a ton of bidirectional systolic arrays on an ASIC with explicit rekey lines around the main array, essentially doing rapidly, on-the-fly rekeyed modulo two cross-correlations of such lengths. Another related field is low probability of intercept radar pulse compression, where the Doppler effect makes you want to change your sampling frequency on the fly. Not sure how they precisely do it, but I've seen some (heavily redacted) papers on such architectures. Obviously, that stuff is pretty specialised and totally irrelevant to audio DSP. But every now and then, sure, even something as insane as this really *is* done. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] 20k
On 2015-08-30, Scott Gravenhorst wrote: This amounted to using a microphone to sample the effects of an impulse (starter's pistol or some such) on some audio environment like a church or concert hall, or even a rock face in a forest. The recording was then used as a kernel and convolved with an input signal (such as music). Typically you'd implement that stuff via Fourier methods. It's just so much more efficient that way, and given what you told, I at least don't see any reason to implement in the direct, brute force way. And you know, you can do kernels hundreds of millions of samples long that way in real time, as opposed to perhaps thousands or tens of thousands in the direct form. So if you can at *any* cost avoid the direct calculation, you would. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Compensate for interpolation high frequency signal loss
On 2015-08-19, Ethan Duni wrote: and it doesn't require a table of coefficients, like doing higher-order Lagrange or Hermite would. Robert I think this is where you lost me. Wasn't the premise that memory was cheap, so we can store a big prototype FIR for high quality 512x oversampling? In my (admittedly limited) experience these sorts of tradeoffs come when you need to resample generally, so not just downwards from the original sample rate but upwards as well, and you're doing it all on a dedicated DSP chip. In that case, when your interpolator approaches and goes beyond the Nyquist frequency of the original sample, you need longer and longer approximations of the sinc(x) response, with wonkier and wonkier recursion formulas for online calculation of the coefficients of the interpolating polynomial. Simply because of aliasing suppression, and because you'd like to calculate the coefficients on the fly to save on memory bandwidth. However, if you suitably resample both in the output sampling frequency and in the incoming one, you're left with some margin as far as the interpolator goes, and it's always working downwards, so that it doesn't actually have to do aliasing suppression. An arbitrary low order polynomial is easier to calculate on the fly, then. The crucial part on dedicated DSP chips is that they can generate radix-2 FFT coefficients basically for free, with no table lookup and severely accelerated inline computation as well. That means that you can implement both the input and the output side anti-aliasing/anti-imaging filters via polyphase Fourier methods, for much less effort than the intervening arbitrary interpolation step would ever require. When you do that right, the code is still rather complex since it needs to dynamically mipmap the input sample for larger shifts in frequency, but when done right, you can also get essentially perfect and perfectly flexible results from a signal chain with perhaps 3x the computational load of a baseband third order interpolation polynomial, absent hardware acceleration. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Compensate for interpolation high frequency signal loss
On 2015-08-18, Tom Duffy wrote: In order to reconstruct that sinusoid, you'll need a filter with an infinitely steep transition band. You've demonstrated that SR/2 aliases to 0Hz, i.e. DC. That digital stream of samples is not reconstructable. The conjugate sine to +1, -1, +1, -1, ... is 0, 0, 0, 0... Just phase shift the original sine at the Nyquist frequence. That'll show you that that precise signal cannot be reconstructed without resorting to complex continuation of the signal, on the Fourier plane. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Compensate for interpolation high frequency signal loss
On 2015-08-17, robert bristow-johnson wrote: As I noted in the first reply to this thread, while it’s temping to look at the sinc^2 rolloff of a linear interpolator, for example, and think that compensation would be to boost the highs to undo the rolloff, that won’t work in the general case. Even in Olli Niemitalo’s most excellent paper on interpolation methods (http://yehar.com/blog/wp-content/uploads/2009/08/deip.pdf), he seems to suggest doing this with pre-emphasis, which seems to be a mistake, unless I misunderstood his intentions. Actually it's not that simple. Substandard interpolation methods do lead to high frequency rolloff, which can be corrected to a degree with a complementary filter. But the trouble is, at the same time they lead to aliasing and even nonlinear artifacts, whose high frequency content will be amplified by the compensatory filter as well. As such, that approach is basically sound...but at the same time only within a very narrowly parametrized envelope. to me, it really depends on if you're doing a slowly-varying precision delay in which the pre-emphasis might also be slowly varying. In slowly varying delay it ought to work no matter what. but if the application is sample-rate conversion or similar (like pitch shifting) where the fractional delay is varying all over the place, i think a fixed compensation for sinc^2 might be a good idea. i don't see how it would hurt. especially for the over-sampled case. It doesn't necessarily hurt, but here it isn't guaranteed to do any good either. And it's close to doing something bad instead. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Compensate for interpolation high frequency signal loss
On 2015-08-16, Sham Beam wrote: Is it possible to use a filter to compensate for high frequency signal loss due to interpolation? For example linear or hermite interpolation. Are there any papers that detail what such a filter might look like? Look at Vesa Välimäki's work, and his students'. They did fractional delay delay lines, which had just this problem in the high end. Also, Julius O. Smith's work with waveguides bumped into this very same thing, because they're implemented as (fractional) delay lines as well. Beyond that, most reverb designers could tell you about this sort of thing, only they tend to keep their secret sauce *most* secret. ;) The usual thing you do is to go for higher order interpolation, with the interpolating polynomial being designed for flatter performance over the utility band than the linear spline. It's already very much better at 3rd order, and if you do something like 4th to 5th order with 2x oversampling, it's essentially perfect. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Non-linearity or filtering
steps just touch each other. In the additive framework some of the noise power would be rounded out, so that we'd get noise modulation, but in the subtractive one the power is now just equally divided between adjacent quantizer bins, with one compensating for the other and none of the noise being lost. That's an immensely beautiful result because it means that subtractive doesn't actually add anything to the signal beyond what the quantizer already does, it just randomizes the lattice. 1RPDF is the minimum you have to do if you you want full independence, but you can equally well use 2TPDF, with the additive results holding for the intermediate signal in case you don't have the dither stream available for subtraction. You can imagine why I've been playing with this stuff of late. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] [ot] about entropy encoding
On 2015-08-09, robert bristow-johnson wrote: even so, Shannon information theory is sorta static. it does not deal with the kind of redundancy of a repeated symbol or string. In fact it does so fully, really? like run-length encoding? Shannon's original statement of his theory was actually in terms of Markov chains. Those handle serial correlations of any finite order just fine, so also stuff much more complicated than RLE. (But not exactly the same: RLE can be made to go infite order when the length indicator is encoded using a universal code.) The definitive analysis of RLE like schemes is of course Lempel and Ziv's '79 compression, which Zip and like compressors are based on. one thing that makes this encoding thing difficult is deciding how to communicate to the receiver the data that the signal is close to periodic and what the period is. and when that changes, how to communicate something different. it's the issue of how to define your codebook and whether the receiver knows about it in advance or not. Yes. But in general when you squint a bit and tilt your head, no matter what you do, essentially you're just implicitly declaring the statistics of that one big unified message space Shannon's theory hinges on. That is, what you a priori presume to know about the signals we'll be coding. you could have a variety of different codebooks established in advance, and then send a single short word to tell the receiver which codebook to use. maybe for non-white signals, the LPC coefs for that neighborhood of audio. That's then just equivalent to specifying the space as a discriminated union, instead of the more usual product of successive symbols. ;) In fact LPC and every other DSP transformation we use in codecs are well within Shannon's framework. i didn't know that. it appears to me to be different, almost orthogonal to the Shannon thing. It does, but only because practical implementations can't work with the bigger than universe codebooks and Markov chains Shannon's mathematical analysis admits. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] [ot] about entropy encoding
On 2015-08-10, Peter S wrote: And in *all* other cases except that this very single corner case, surprisal is nonzero, hence entropy is nonzero (whatever the probabilities). Suppose I gave you a signal derived from a cryptographic pseudorandom number generator with a period in the order of 2^256? Its output would by design pass even the most stringent statistical tests you could throw at it. What would your entropy estimator say about it given the output signal? Because its entropy rate would again formally be precisely zero. Since you know how to derive its output, it is irrelevant what comes out. Determinism even of such complicated fashion still equals zero surprisal, over the whole signal. That is another nice corner case which goes to show you cannot really measure entropy unless you know the true, underlying probability distribution of the source a priori. (And algorithmic entropy is always nonzero, even when the Shannon entropy is zero. To understand that, you need to understand algorithmic entropy.) Oh I do know Kolmogorov's theory. Now can you prove the initial packet of code is always within a logarithmically bounded constant of any other? I can't, because it takes extra conditions to prove that. Which precise extra assumptions are needed there? You do not need to specify the probability space 'fully' to know that when you have a probabilistic ensemble of at least two different 'words' (as you call them) from the symbol space with probability different from one, then the Shannon entropy of a message (average information content per message in bits) will be nonzero, without exception. Of course you need to: both of the words could have probability one, so that their occurrence pairwise would also have probability one for certain, and so the joint surprisal could be a flat zero. If you don't understand it, read Shannon's paper. If you cannot apply logic to a simple mathematical formula, that is not my problem. I just did, to yet another edge case. And mind you, I've been doing this since something like age 17-19a in data compression theory. I even feature in the oldskool comp.compression FAQ wrt Burrows-Wheeler Transformation, as the first guy who explained its compression half to the wider net audience. Remind me, when were you born again? ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Non-linearity or filtering
noise that we didn't hear anyways. (Technically we removed about 3/4 of the noise, specifically the noise in the supersonic bands.) Do notice that we're in the business of producing audio systems and software. Not all of them are meant for pure human consumption. For example, in audio forensics work, you'd like your signal chain to be pure enough to pick up line hum at -60dB or even -80dB below the already low noise floor, given enough integration time. Of course now it's not about music-dsp anymore, even close. But if you want your ADC/DAC to be any good at that sort of thing, they have to reach well beyond human capacity. And since such professions necessarily use the same hardware, definitely the same audio formats, and sometimes even shared software, you shouldn't be too swift at dismissing beyond-human audio processing either. Beyond that, Peter, I actually agree with you. 24 bits is beyond what we can dissolve. In fact I'd go with the Acoustic Renaissance for Audio paper which argues that a properly executed, noise shaped, 55kHz @~ 18bpS ought to be enough. https://www.meridian-audio.com/meridian-uploads/ara/araconta.htm#precision . But it then has to be proper as hell, and many practicalities come on the way. Even that paper I mentioned -- one of the most high minded I've *ever* seen in the field -- actually backs down to something besides the ideal. Because it's not just audio theory, it's about practicable solutions as well. I think you ought to be thinking about the same for a change. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] A little frivolous diversion on the effect of using a delay
On 2015-07-20, Tom Duffy wrote: Using separate reverbs on each instrument in a DAW recording gives a richer mix that just a single reverb on the master channel. What it gives you is higher decorrelation across channels. And our ears are used to that, because as soon as you move a sound source even one metre in an enclosed, reverberant space, the precise reverberation pattern changes drastically. We perceptually expect a lot of decorrelation from the decaying part of a reverberant sound...though at the same time less from the early, distinguisable slap echoes. (Or, let's say, we expect a different kind of decorrelation; in the short time frame interaural decorrelation because of delay, and in the longer frame essential whiteness overall.) Tom, look at how DirAC processes its arrivals. Starting from Ville Pulkki's research at then TKK Acoustics Lab, and now continuing at Aalto. It's entirely predicated on this sort of thing in its reverb leg. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] [ot] about entropy encoding
and such. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 ___ music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-30, robert bristow-johnson wrote: but wavetable synthesis *is* a framework that can do that for any periodic (or quasiperiodic) signal. How do you derive the hard bandlimited wavetable for an exponential, rising segment? In closed form, so that your wavetable doesn't already contain aliasing? Then derive it in linear time as well, so as to be compatible with FM and like modulations. Robert, if you can derive that wavetable for any frequency in closed form, I'm pretty sure Vadim would already like you. A lot. there is interpolation or cross-fading between phase-aligned wavetables to worry about, but there is no overlapping of bandlimited grains or wavelets or BLI's (or whatever you wanna call them) to worry about. We all know about that already. Now we're talking about derivations of those wavetables. Cleanly so. So then isn't that the end of the discussion, in *musical* DSP? It sounds like crap, move along, nothing to hear here. or, paraphrasing Duke Ellington, if it sounds like crap, it IS crap. Oh, *yes*. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
the Hilbert transform, and we know how to bandlimit them as well. Might they help? We have seen exactly this for the sine (in my paper), where the sufficient conditions for the BLEP convergence is that the sine frequency is below Nyquist. Notice that the Taylor series for exp is essentially the same as for the sine, therefore they have the same rolloff speed. So, the exponent is bandlimited (in the sense that the BLEP sum converges) under the same conditions as the sine (the frequency must be below Nyquist). If so, it works. But since I haven't done the math, I'm not too certain you can get the sine case to work either without using quadrature terms in addition. And in fact, intuitively speaking, if you had to use those as well, that'd finally tie the knot with my skepticism towards your going with the complex, holomorphic version of Fourier theory, and what you were talking about with Paley-Wiener. I mean, when you introduce a hard phase shift to a sine, you don't just modulate the waveform AM-wise. You introduce a phase discontinuity. On the left side of it the sine has one phase, and on the right side it has another, from -inf to +inf. To me it seems rather obvious, intuitively speaking at least, that the discontinuity doesn't have just a symmetric part, but an asymmetrical one as well. Especially when it gets bandlimited, the way you interpolate the waveform ain't gonna have just Diracs there, but Hilberts as well, and both of all orders. Those can all be derived from derivatives of a Heaviside step, but their infinitesimal or bandlimited versions don't just go up the the frequency band like you'd think. The derivative of a step is the Dirac, the derivative of that is the Hilbert, the derivative of that is in its amplitude behavior just what you'd think once again, but in angle it's the opposite of the Dirac, and at fourth order derivative it returns back to being in phase with the Dirac. That's because those operators are the differential ones which classify by their eigenfunctions the shift-invariant subspaces of functions (and by extension distributions as well) of the function space we started with; originally L^2 of course. Because the Fourier operator is an isometry and whatnot, here, those operators map bijectively onto their generators/characters in the Fourier space; that is, quadrature phase shifts plus amplitude shifts map onto complex numbers. And then the four-fold symmetry of the Fourier transform itself maps back into the four-fold phase symmetry of the (actually Lie ;) ) calculus of the differential Dirac, Hilbert, whatnot, operators, we started with on the distributional side. So to return to the discussion... Have you actually looked at how the phase side of the picture functions? In addition to and in separation with the amplitude/modulus side? Because it's rather different, and might help explain a couple of things in addition to what we've talked about inb4. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-29, Emanuel Landeholm wrote: But all waveforms can be antialiased by brick wall filtering, ie. sine cardinal interpolation. The point is that you can't represent the continuous time waveforms in the usual sampled form, and then apply a sinc filter. Which you need to do in order to synthesize them via a normal D/A converter. Instead you need to perform the convolution with the sinc in implicit form, which lands at a nice, regular, equidistantly sampled form. What we're after here is one of those guaranteed to be bandlimited implicit forms, only somewhat more general than what the conventional BLIT/BLEP framework allows. Certainly not all frameworks can do that, and not for all signals. Closed form discrete summation formulae don't exist for every waveform. At least at the outset, the BLIT/BLEP-framework seemingly cannot handle sine-on-sine FM. Vadim's paper is the first I've seen which handles even arbitrary sync on a sine. And even if what we've been talking about above does go as far as I (following Vadim) suggested, exponential segments are still out of the picture for now. Then, when you fail, you get aliasing, which sounds really bad and behaves nastily under change of parameters. So then isn't that the end of the discussion, in *musical* DSP? It sounds like crap, move along, nothing to hear here. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-22, Vadim Zavalishin wrote: After some googling I rediscovered (I think I already found out it one year ago and forgot) the Paley-Wiener-Schwartz theorem for tempered distributions, which is closely related to what I was aiming at. It'll you land right back at the extended sampling theorem I told about, above. So why fret about the complex extensions? They won't help you with bandlimited stuff at all. Although Hölmander's tightening of the theorem in 1976 might help you understand those BLEPs of yours. Because it quantifies singular supports, i.e. every delta train necessary for the analysis of BLITs and their integrals, starting from continuous time. It's just that you don't need any of that machinery in order to deal with that mode of synthesis, and you can easily see from the distributional theory that you can't do any better. Why make it that complicated? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2-- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-19, robert bristow-johnson wrote: i thought that, because of my misuse of the Dirac delta (from a mathematician's POV, but not from an EE's POV), i didn't think that the model of sampling as multiplication by a stream of delta functions was a living organism in the first place. i thought, from the mathematician's POV, we had to get around this by using the Poisson summation formula [...] In the framework of tempered distributions, all of that follows as well. You can actually do with Dirac deltas what you'd like to do, and what seems natural. Pretty much the only thing you can't do is freely multiply two distributions together, unless they're not just distributions, but functions as well. Convolve if one of the distributions has compact support, or you land within the conventional L_2 theory, or something like that... But otherwise, you can do the funkiest shit. Nota bene, this is not EE stuff per se. This is heady math stuff, used to formalize what you EEs wanted to do all along. It's the kind of collaboration where us math freaks provide the rubber...and then you EE folks can finally fuck your sister in peace and certainty. ;) (Sorry, can't help it, been looking at a lot of stand up comedy of late...) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-19, Ethan Duni wrote: I guess what we lose is the model of sampling as multiplication by a stream of delta functions, but that is more of a pedagogical convenience than a basic requirement to begin with. In fact even that survives fully. In the local integration framework that the tempered distributions carry with them, you can convolve a polynomial by a delta function or any finite derivative of it, and you can also apply a Dirac comb to it so as to sample it. But what does the convergence of the Shannon-Whittaker formula look like in the case of stuff like polynomials? Precisely the same as it does in the case of any other function. You just have to take the convergence in the weak* sense, and then do some extra legwork to return that into a function, from the functional domain. What it returns to is precisely the unique polynomial (or whatnot) you're after. The reconstruction formula, using sinc functions, is exact in that circuitous sense. In the usual setting we get nice results about uniform local convergence, but that requires the asymptotic behavior of the signal being sampled to behave nicely. In a case where it's blowing up at polynomial rate, it seems intuitively that there could be quite strong dependencies on samples far removed in time from any particular region. So the concern would be that it works fine for the ideal sinc interpolator, but could fall apart quite badly for realizable approximations to that. All that is taken care of by the fact that the reconstruction is defined as a transposition of a functional wrt the Schwartz space to begin with. All the mechanics are local because of that. The asymptotics don't matter after that, and the Shannon-Whittaker formula is suddenly defined locally, so that growth rates upto polynomial don't matter at all. Of course some funky global, dual shit happens then: you actually need all of the samples from -inf to +inf in order to define any polynomial, and no finitely supported in time subset will suffice. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-19, Ethan Duni wrote: We theoretically need all samples from -inf to +inf in the regular sampling theorem as well, [...] Not exactly. If you take the typical sampling formula, with equidistant samples, you need them all. But in theory pretty much any numerable number of samples from any compact interval will do. I'm not 100% certain, but with polynomials in the distributional setting, I think you'll actually need -inf to +inf in some sense (equidistant sampling being sufficient but probably not quite necessary), despite the bandlimitation which usually makes the function rigid enough to be analytic, whole, and so resamplable+continuabe from pretty much whatever you have at hand. This happens basically because the sinc function dies off linearly [...] Linearly? It dies off as 1/x. And that's part of the magic. You see: -1/x dominates any decaying exponential, being in a sense their limit -exp(x) dominates any monomial, being in a sense their limit -log(x) dominates any root, being in a sense their limit -there's a fourth one, plus some integral equalities, here This stuff basically delimits in real terms the Schwartz space used to construct tempered distributions. It also delimits the L_p spaces. The fact that the 1/x growth rate is the limit of decaying exponentials and that we go through the weak* topology of the dual space is somehow the reason why we can pass to the 1/x limit of the Shannon-Whittaker interpolation formula, both in the simpler L_2 theory and in the more general distributinal framework. And it's somehow clearly the reason why you can't have but polynomial growth in (tempered) distributions. I don't understand this stuff fully myself, yet, but it's evidently there. So the limiting growth rate of the sinc function cannot be an accident. I think it comes from the dominating real convergence rate of any polynomially bounded tempered distribution, when approximated via milder distributions in the weak* topology. [...] and we are dealing with signals with at most constant-ish asymptotic behavior - so the contribution of a given sample to a given reconstruction region is guaranteed to die off as you get farther away from the region in question. Not quite so. The proper way to say it is when probed locally by nice enough test functions, the reconstruction works the same. That's a bitch because some functions within the space of tempered distributions can be plenty weird. The main counter example I've found is f(x)=sin(x*e^x). That's bounded and continuous, so it induces a well behaved tempered distribution. Then we know that every derivative of a tempered distribution is also a tempered distribution. g(x)=f'(x)=cos(x*e^x)*D(x*e^x)=e^x*cos(x*e^x). That doesn't look polynomially growing at *all*, yet it's part of the space. (The reason is its fast oscillation while it grows.) So for any finite delay, we can get a finite error bound on the reconstruction. But in the case of a polynomial it seems to me that the reconstruction in a given region (around t=0 say) could depend very strongly on samples way off at t = +- 1,000,000,000, since the polynomial is eventually going to be growing faster than the sinc is shrinking. That's the problem: the local integration theory we use with the distributions doesn't work with your usual error metrics or notions of convergence. This sort of argument is meaningless there. What you need to do is bring in the whole set of test functions, in order to construct a nice functional, and then show it can be induced by a function which doesn't integrate in the normal sense against any L_2 function, say. So I'm not seeing how we can get any error bounds for causal, finite-delay approximations to the ideal reconstruction filter in the polynomial case. You'll have to go via the functional transposition operator. We also need the property that the reconstruction can be approximated with realizable filters in a useful way. The sinc convolution is just fine even in this setting. It's just that we just happened to prove its workability in a slightly more general setting. And yes, that blows my mind, too. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-19, Ethan Duni wrote: Not exactly. If you take the typical sampling formula, with equidistant samples, you need them all. Yeah, that's what we're discussing isn't it? Are we? You can approximate any L_2 bandlimited function arbitrarily closely with a finite number of samples. I don't think you can even approach a polynomial in the distributional sense absent the whole infinite set. But in theory pretty much any numerable number of samples from any compact interval will do. Sure, but that's not going to help us with figuring out what comes out of an audio DAC. Yes. And I'm sorry if I sound of like a know-it-all or show-off here. I really am anything *but*. Just interested in this stuff. :) Linearly? It dies off as 1/x. Yeah that's what I mean. Kind of informal, but die off was meant to imply this is what is in the denominator. Check. But 1/x is still pretty special in the denominator. Not quite so. The proper way to say it is when probed locally by nice enough test functions, the reconstruction works the same. I'm not sure we're on the same page here - the statement you were replying to was referring to the classical L2 sampling theorem stuff. If so, again sorry. I have tried to work as much in the distributional setting as I can. The sinc convolution is just fine even in this setting. ??? The sinc convolution is not implementable in any setting. It actually is in the distributional setting. When you go via the weak* topology of the relevant functional space, the functions they induce back implement pure sinc interpolation. The limit is exact. But yeah, in *reality* nothing of the sort can exist. You just have to approximate. It's just that there's nothing new there for any of us, I think. Delta-sigma, yadda-yadda, it's what them chips do all the time for us. Right? ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] [ot] math vs. EE
On 2015-06-19, robert bristow-johnson wrote: we EEs are fucking our sisters when we say that there *is* a function that is zero almost everywhere, yet has an integral of 1. (but when we take the rubber off, we find out that it's a distribution, not a function in the normal sense that one might recognize in anatomy class.) Us wannabe-math-freaks play with things like the by-now classical Cantor function. Continuous, monotonically increasing from 0 to 1, almost everywhere differentiable, with zero slope there. https://en.wikipedia.org/wiki/Cantor_function ...and then we *like* that shit. Cool sister bedamned. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-12, Ethan Duni wrote: Thanks for expanding on that, this is quite interesting stuff. However, if I'm following this correctly, it seems to me that the problem of multiplication of distributions means that the whole basic set-up of the sampling theorem needs to be reworked to make sense in this context. Now that I read up on it... Actually no. Every tempered distribution has a Fourier transform, and if that's compactly supported, the original distribution can be reconstructed via the usual Shannon-Whittaker sinc interpolation formula. That also goes for polynomials and sine modulated polynomials in the continuous domain. Whatever that means in general. No? Yes. While the formalism apparently goes through, I don't have the slightest idea of how to interpret that wrt the usual L^2 theory. I can sort of get that the polynomial-to-series-of-delta-derivatives duality works as it should, and via the Schwartz Representation Theorem captures the asymptotic growth of tempered distributions. But how you'd utilize that in DSP or with its oversampling problems is thus far beyond me. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-09, Ethan Duni wrote: The Fourier transform does not exist for functions that blow up to +- infinity like that. To do frequency domain analysis of those kinds of signals, you need to use the Laplace and/or Z transforms. Actually in the distributional setting polynomials do have Fourier transforms. Naked exponentials, no, but those are evil to begin with. The reason that works is the duality between the Schwartz space and that of tempered distributions themselves. The test functions are required to be rapidly decreasing which means that integrals between them and any function of at most polynomial growth converge, and so polynomials induce perfectly well behaved distributions. In essence the regularization which the Laplace transform gets from its exponential term and varying area of convergence is taken care of by the structure of the Schwartz space, and the whole machinery implements not a global theory of integration, but a local one. I don't know how useful the resulting Fourier transforms would be to the original poster, though: their structure is weird to say the least. Under the Fourier transform polynomials map to linear combinations of the derivatives of various orders of the delta distribution, and their spectrum has as its support the single point x=0. The same goes the other way: derivatives map to monomials of corresponding order. In a vague sense that functional structure at a certain frequency corresponds to the asymptotic behavior of the distribution, while the tamer function-like part corresponds to the shift-invariant structure. The fact that the constant maps to a delta and the successive higher derivatives to monomials of equally higher order sort of correspond to the fact that in order to approximate something with such fiendishly local structure as a delta (corresponding in convolution to taking the value) and its derivatives (convolution with which is the derivative of corresponding order) calls for polynomially increasing amounts of high frequency energy. That is, something you can only handle in the distributional setting, with its functionals and only a local sense of integration. Trying to interpret something like that the way we do in conventional L_2 theory sounds likely to lead to pain. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] [ot] other than sampling theorem, Theo
On 2015-06-11, Theo Verelst wrote: [...] I don't recommend any of the guys I've read from here to presume they'll make it high up the mathematical pecking order by assuming all kinds of previous century generalities, while being even more imprecise about Hilbert Space related math than already at the undergrad EE sampling theory and standard signal analysis and differential equations solving. Could you be any more condescending? For your information, at least distribution spaces do not admit an inner product, much less a complete topology induced by it. Hell, in general they aren't even metrizable. And as for last century mathematics, yes, well anything prior to 2000 technically is just that. But let me point out that math doesn't exactly age, and that 21st century math tends to be beyond most PhD's in its rigor, generality and methods. An EE could well stop at 1800 AD and make do. What we're talking about here is a theory which even now isn't really complete, what with Lojasiewicz and Hörmander doing their seminal work only in '58-'59, and many of the problems of distributional division continuing to generate papers to this date. That being the stuff that lets you operate on rational system functions in this setting, ferfucksake... There are three main operations involved in the relevant sampling theory at hand, the digitization, where the bandwidth limitation should be effective, [...] Yes. [...] the digital processing, which whether you like it or not is very far from perfect (really, it is often, no matter how you insist in fantasies on owning perfect mathematical filter in the sense of the parallel with idealized analog filters), [...] But the problems can be regularized/mollified out of practical significance using 1800s maths. [...] and the reconstruction, which in the absence of processing can be guaranteed to yield back the input signal when it's properly bandwidth limited. Yes, and with deep delta-sigma converters we know how to achieve that as well, over any useful audio bandwidth. Exactly enough for the resulting error to *always* fall under thermionic noise, in practical circuits. [...] without going through the proper motions of understand the academic EE basics, it's a free world, at least in the West, so fine. In fact we do our homework, often without it being in any way connected to a lucrative endeavour. When we don't get it, we ask for help on fora such as these. Which are then supposed to be easy to enter, *precisely* because there's strength in community. I don't believe it is us who are climbing some imaginary ladder of power and prestige. We've been talking math, pure and simple. What you're doing here is pooping on a perfectly vibrant party of others. Please don't do that. I repeat the main error at hand here: it's important to have bandwidth limited synthesis forms, but it is equally important to *quantify* [...] We know, and we have. [...] THEN you could try to get EEs/musicians opinions about inverting and partially preventing the errors in common DAC reconstruction filtering. There is no error, as shown by tens of double-blind empirical tests over the years. If that's your persuasion. Start with the basics, and goof of into some strange faith in miraculous mathematics to solve complexities that are inherent in the problem. Oh, and you just happen to hold the magic key? Math bedamned? Seriously, man. Even worse idea is to mash such idea up with the signal generators and filters, without concern for sample shifting, filtering errors, generator waveform reconstruction issues, and so on. That's not going to be my dream virtual anything. just saying. That's sheer gobbledygook, and as an EE you ought to know so. [...] but it may also kill information that is subsequently lost for later processing, and it will impose a character that probably is boring. Show me the information theoretical argument to that effect. I can follow that kind as well, as I'd surmise quite a number of people here can. But I don't need to worry about accusing the teacher of humiliating me, because I'd be fine good at it. I am reasonably sure it is you who is humiliating himself. But of course I'm open to being proved wrong. Why not do the test? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2-- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Sampling theorem extension
On 2015-06-11, vadim.zavalishin wrote: Not really, if the windowing is done right. The DC offsets have more to do with the following integration step. I'm not sure which integration step you are referring to. The typical framework starts with BLITs, implemented as interpolated wavetable lookup, and then goes via a discrete time summation to derive BLEPs. Right? So the main problem tends to be with the summation, because it's a (borderline) unstable operation. We often build in leakiness there in order to counter the effects of limited numerical precision, leading to long term average cancellation of DC. If we just did everything with bandlimited impulses, the DC error could be controlled exactly -- after all, the sinc interpolation (was it Whittaker?) formula really is interpolating, so that any effects it has on the DC are at most local even after windowing. No systematic DC offset ought to develop. As for the correct window length, this is what is bothering me a little. Depends on how generic the correct window length condition can be. There is no such thing as a correct window length. It's a matter minimizing the interpolation artifacts. Usually by pushing their maximum amplitude under some upper bound over the Fourier domain. The only DC error arising from that is that which the window function itself causes, and it then can be compensated by just adding it back. That's easy, because window functions have compact support. You just have to mind the effect. Ok, so the sampling theorem is not applicable to the exponential function even if we use tempered distributions for the Fourier transform maths. Correct. Now, I don't know whether there is a framework out there which can handle plain exponentials, a well as tempered distributions handle at most polynomial growth. I suspect not, because that would call for the test functions to be faster decaying than any exponential, and such functions are measure theoretically tricky at best. I suspect what you'd at best arrive is would seem very much like the L_p theory or the Laplace transform: various exponential growth rates being quantified by various upper limits of regularization, and so not one single theory where the Fourier transform exists for all your functions at the same time, and the whole thing restricting to the nice L_2 isometry where both functions belong to that space. But as I said, I'm not sure. That sort of stuff goes above my head already. So we don't know, if exp is bandlimited or not. This brings us back to my idea to try to extend the definition of bandlimitedness, by replacing the usage of Fourier transform by the usage of a sequence of windowed sinc convolutions. The trouble is that once you go with such a local description, you start to introduce elements of shift-variance. That sort of thing automatically breaks down most of the nice structure we have with more conventional Fourier transforms. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Did anybody here think about signal integrity
On 2015-06-08, Ethan Duni wrote: But both of these are overkill for day-to-day engineering practice. Especially so, because you can treat the distributional framework as a black box. It has a certain number of rules. If you follow the rules, you'll land with a nice calculus which fully encompasses all of the useful stuff like Dirac impulses. So, as an engineer, you should probably just learn the rules and utilize the stuff; the math folks did the heavy lifting ages ago, and assured you're safe, so why tinker with the internals? 'Cause the stuff *just* *works* *beautifully*. Take it as manna from heaven. In fact in DSP work I'd even say ditch the continuous time wonkiness as a whole. If you do everything you do by the bandlimited recipe, that works as well. Once you know what a sinc(x) pulse looks like, you can freely substitute it within the bandlimit for the Dirac one; the strict mathematical homology from the distributional framework to the theory within a bandlimit *is* there, in the background, so that you don't have to worry. Your math won't blow up. The *only* time you actually have to worry about the continuous time spectrum of a signal are those individual special functions, like FM, usually nonlinear. But even those have standard texts which deal with their approximation theory within the bandlimited framework. If you aim to get things done, as I think an engineers usually does, why the hell bother with more? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Did anybody here think about signal integrity
people have some amazingly broad
theorems based instead of equidistant sampling, to sampling under what
is commonly called bounded rate of innovation.
Though even that theory can't handle whatever. The basic trouble is that
as soon as the support of the Fourier spectrum of your signal set is
something other than compact, the information rate per time goes to
infinity, and no amount of samples suffice in any basis. You can get
around that in some cases, but then you bump into the problem that as
soon as you admit shift invariance, all of the shift invariant subspaces
are actually classified by the Fourier theorem -- mathematically
speaking Fourier theory ain't about frequencies at all, but shifts -- so
that time-invariance will have to go -- not nice at all for audio work.
Let f(t) be a signal such that the statement A(f) holds, let F[n] be
its naively sampled counterpart and and let {D_n} be a sequence of
DACs such that the statement B({D_n}) holds. Then the sequence D_n(F)
converges to f.
There is nor there can be such a general theory. In fact the idea that
you can just naïvely sample pretty much anything is *highly* suspect: as
soon as you bring in any form of integration, measure theory tells you
you can't get shift invariant measures which reflect a pointwise
definition of the underlying functions being integrated. And of course
even defining energy conservation, so that we can distiguish one
physical signal from an amplified version of it, will already call for a
full L_2 theory on your function space.
So, naïve sampling is pretty much out from the start. At the very least
you need to define your functions modulo addition with functions of zero
square norm (i.e. those which vanish almost everywhere).
I made some initial conjectures in this regard in the mentioned
thread, where A had to do with the rolloff speed of the function's
derivatives and D_n was just a sequence of time-windowed sincs with
increasing window size.
Won't work. Not even close. If you have no upper limit to the order of
continuity, I can plug in one of the vast class of continuous, nowhere
differentiable functions, and integrate down to something which fulfils
your condition yet will fail any normal sampling theory under the
assumption of shift-invariance. If you don't have such a limit, then
you're essentially already back to the theory of Schwartz spaces, which
are used in the construction of the topology of the space of tempered
distributions I mentioned above.
--
Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front
+358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2--
dupswapdrop -- the music-dsp mailing list and website:
subscription info, FAQ, source code archive, list archive, book reviews, dsp
links
http://music.columbia.edu/cmc/music-dsp
http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Calculating e^(i*phase) * (H + H')
On 2015-04-06, Larry Trammell (aka RidgeRat) wrote: MF, are you *completely sure* that the H'(w) is intended to be a derivative of H (with respect to what variable)? The H prime notation tends to be used a lot to mean complex conjugate rather than derivative. In the Fourier domain I'd guess H and H' are a Hilbert pair. I.e. summed together they yield a spectrum which will back transform into a real impulse response. Typically those would be zero below the real axis in the complex plane for H and zero above the real axis for H'. In this case they would show conjugate symmetry, but some people striving for generality do feel the need to give the more general form: complex analysis does allow you to additively partition your stuff in the complex transform domain pretty much any which way. That sort of interpretation seems more consistent with the statement that only one of the two spectral motifs must be synthesized. That would make sense when one 'motif' is the complex conjugate of the other -- synthesize one, and you know them both. Quite so. I don't know what that paper is referring to when it says the inverse FFT algorithm only uses the positive frequency half spectrum. What it probably means is that the writer is thinking in terms of full, complex transforms, while the transform code se uses utilizes reals. It might in fact not be DFT/FFT at all, but something like DCT/DST. In that framework it's natural to think of the spectrum derived via the transform as the symmetric real half of a full, complex DFT. Because that's what it really is. In order to go back to reals upon inverse transform, the theory really does then require the complementary half to be summed in. If so, just forget about H'. Its contribution will be implicitly summed in by the fact that you only used a real transform, to and fro. (I'm nowhere sure though.) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] FFT and harmonic distortion (short note)
On 2014-12-08, Ethan Duni wrote: dunno what you're getting at, Theo. both graphics appear fully as expected to me. Yeah I'm at a loss as well. Isn't this stuff well-explained on Wikipedia, and in every book that covers spectral analysis? Well, not *too* well. The difference between spectral spillover and aliasing against nonlinearity, actually is a bit novel for most, still. I think the main reason is that you can't really easily discern the separate effects from a common spectrogram by eye. Even the linear, fully inversible effects often just seem to royally fuck up the spectrum you see, and in the case of spectrally sparse, periodic waveforms, seemingly without any warning or consistency as the period drifts in and out resonance with your analysis filter. Doubly so in the presence of noise/background and varying transform length/analysis bandwidth in the algorithm you used to derive your spectrogram, because it really is rather unintuitive what a progressively more matched filter does to your utility signal, against the floor. After that even a proper theoretical background doesn't shield you from the prima facie reaction: what the fuck is this shit, now, something must have gone wrong, where is it. But yeah, you ought to know about this stuff already. Theo at the very least. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] English as a second language - measuring voice similarity
On 2014-07-23, Theo Verelst wrote: Sometimes, I think the English just make up bogus ways to pronounce things to screw with the rest of us. ... Right. Not just the dialect, but in the case of dialectics it may be right even to consider the time of the secondary language English speaker being tested, and whether it's the Queen's English, or east coast US English or West coast. Yet what else do we have to work with? I'm a second language English speaker myself, and I see no realistic alternative in the STEM field. As such, I've felt from early teenage that I should strive for mastery of the English language, even if it's quite the monster and nowhere near the ideal Lingua Franca. That's the inevitable, tragical logic of of a common language. It doesn't have to be good. The fact that it's common and helps mutual understanding, can make even the Abomination that is English something you actually have to learn. (C. Popper: Logik der Forschung, every scientific theory is true only until it is falsified). Actually Karl, with a hard K. My favourite epistemologer, even above the coherentists. So a farmer in Pennsylvania, or a person from the suburbs of Oxford might have similar ideas about language, but should it be that they get judged on the basis of how well they can mimic a certain language style, or on the content of which they have to say ? Sometimes, in fact yes. Most of the time, of course not. It depends. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Statistics on the (amplitude) FFT of White Noise
On 2014-10-31, Theo Verelst wrote: Now, how probable is it that we get all equal frequency amounts as the output of the this FFT transform (without regarding phase), taking for instance 256 or 4096 bins, and 16 bits accuracy ?! Or, how long would we have to average the bin values to end up equal (and what sort of entropy would that entail)? I don't know what the eventual answer would be, because I'm not a career statistician. But I do know where to look for the answer: https://en.wikipedia.org/wiki/Directional_statistics . That is because when you utilize the FFT (or the Fourier series), you're implicitly using a version of Fourier theory specialised for cyclical signals. Their statistics are always most naturally expressed in the kind of cyclical framework that directional statistics for historical reasons pointed out, and then solved. So, learn to go from the Gaussian distribution to the von Mises one, and then learn to deal with the fact that when you live on a circle, concepts like phase and its continuity become a bit harder to deal with. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Statistics on the (amplitude) FFT of White Noise
On 2014-10-31, Bjorn Roche wrote: I am not sure if the PDFs are preserved across transforms from one orthonormal basis to another, and the answer to your question would depend on that They most certainly are not. As two concrete examples of bases which lead to different induced PDFs upon transformation, just take the DCT and Walsh bases. Not only do the common cases translate differently under those bases, the whole ensemble does, too. Because the Walsh-Hadamard basis doesn't respect time/shift translation symmetry, like the more common Fourier bases by definition do. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Statistics on the (amplitude) FFT of White Noise
On 2014-10-31, Ethan Duni wrote: Transforms between orthogonal bases are basically rotations. I.e., they are linear operators that produce each component of the output as a linear combination of input components. Generally, then, the Central Limit Theorem tells us that the output distributions will tend to be Gaussian, regardless of the input distributions, provided the input components are suitably uncorrelated. At the same time, given the typical non-normal and highly correlated audio signals we have, only transforms approximating the optimal Karhunen-Loeve transform, such as the DCT/FFT/whatever sinusoidal basis family, can be expected to translate that into something nice. If you instead do e.g. an exact partition of unity against a wavelet basis, you can and should expect statistical structure beyond normality to develop. Which is precisely why such bases have been tried out for both audio and video coding, in the first place. There are some details specific to complex numbers/distributions in the case of FFT, and exceptions for transforms that don't actually do much mixing of input components, [...] The most basic Walsh-Hadamard Transform does precisely as much mixing as a DCT/DST/FFT of similar length. You can even prove that by reducing each to a similar dyadic bytterfly network, and then proving each is optimal in their own regard. Then just comparing coefficients, which are +/-1 (90 degrees) in the WHT case, and direction cosines (45 degrees, over which axis precisely in turn?) in the case of the Fourier rotation. They both mix pretty well, and that fact has actually been used to advantage in certain cryptosystems already. At the same time the LTI framework we are best acquainted with on the audio side -- neither deterministically nor in the probabilistic framework -- doesn't really translate too gracefully over to any transform which isn't a Fourier one. And there's also a reason for that: the Fourier framework is provably the only shift-invariant one, bringing on a nice framework, yet at the same time neither what we put in from the mic nor take in via our ears has *any* of those nice qualities for real. Thus, for instance, the need for all of those funky vector quantization schemes in mobile phone work. If you look at them closely, they are a means of dealing with a residual which just does *not* have the overall statistics we started this discussion with, nor would they have even modulo some hypothetical prior LPC pre-encoding stage which was fully LTI from sample to sample. All of those various vector quantization schemes, even in your iPhone, are basically a nonprincipled attempt at modelling via computing power and memory (brute force) that which we couldn't reduce into simplicity via mathematical-statistical analysis, in non-LTI-source-model speech signals and/or background separation. [...] but the general intuition is that linear transforms will preserve Gaussian distributions, and will cause non-Gaussian distributions to become more Gaussian. That much is obviously true, and bears reminding folks as well. For reasonably large FFT sizes and uncorrelated input data, you can generally assume that the FFT coefficients are (complex) Guassians, regardless of what the input data distribution was. The magnitude coefficients, yes. But how about the phase factors? I mean, in multidimensional work such as this, you can't ever assume anything looks the same from all of the possible bases at the same time, especially most of them at the same time, and then if you just chose a minority, nice basis, you should be able to found that choice on something. So why is it that you seem to found your analysis on a Fourier basis? E.g. Gabor derived complete bases do have some theory behind them as well, as do overcomplete time-frequency bases. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Statistics on the (amplitude) FFT of White Noise
On 2014-10-31, Theo Verelst wrote: if I have a sampled noise signal of some kind, regardless how I got it, it isn't going to make a difference for the main characteristics of the proposed measurement which transformation I use, [...] Oh, but it is. Suppose you derived your so called noise sequence from a pseudorandom number generator. Then suppose your transform had a pretransform which inverted your generator. So that it always gave the sequence (1...) for any input sequence given out by your generator. Now everything you'd ever give your transform would be an impulse, and that isn't too interesting or useful. This sort of fuck-all has already been deal with within information and coding theory. But in order to really understand how it should be done, you can't just handwave and say for the main characteristics You have to be able to define those main characteristics in hard math, and to formally quantify which proportion of the set of all possible signals really *do* have those main characteristics, precisely. In the rarest end play, you'd be called upon to prove it in full, exact, excruciating, first order predicate logic as well. Nuke it from the orbit. It's the only way to be sure. Quoth an ex of mine. I just want to [...] That's the very point: this is *not* an I simply can kind of question. To us it sounds kinda like, what are you, stupid, I just want to know how to bed Angelina Jolie tomorrow nite; it can't be too difficult, can it? But of course it can be, and it is. This is nontrivial math, and you can't really go around it without committing what amounts to mathematical and/or technological rape. So... correlate What precisely do you mean by correlate? the measured Measured how? frequency Define frequency? distribution Against which statistic and/or probability space? to normal statistical considerations I know tens of different normal context in which to interpret this. Start by defining yours. (come on, some math persons here should be able to derive the variance per bin neatly, or something). In math you're supposed to prove the possibility, or it doesn't exist. Then you're separately supposed to prove the existence, so as not to appear trivial and laughable. If your proof is nonconstructive, you might be laughed at till dead, and only then might your result be generally accepted. So to be clear: I didn't invite daring mathematical exposes, no matter how fun some of you may think that is (usually it's hard work, no a lot of discussions). Does this seem like fun to you? It's not fun. It's all about duty. I meant to make an example about why FFT and certain statistical normalities can be connected, so possibly DSP for musical or speech signals could benefit. We knew that already. So, nice, you're learning. Do proceed. Just don't lecture us how we should be doing this or that. :) [...] and I don't need theoretical guidance in the reasonable examples I use, they will come together at undergrad level just fine! Oh, but you do, and we'd fuck up your thesis instructor as well. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-14, Max Little wrote: Still, I might find myself finding a use for Hartley's 'entropy', maybe someday. I don't discount any maths really, I don't have any prejudices. Then just do Shannon's definition over a space with equidistributed probability. Define it as so, and you have the precise same fundamental framework. Seriously, there is no difference. It's just that I and many others find that framework unduly limiting. Especially for any analysis involving noise. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Release pyo 0.7.2 (Python dsp library)
On 2014-10-17, Olivier Bélanger wrote: pyo 0.7.2 is now available to download on pyo's web site : Wow, and kudos! - SmoothDelay : a delay line that does not produce clicks or pitch shifting when the delay time is changing. How do you implement the latter part, if you don't mind my asking? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2-- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Release pyo 0.7.2 (Python dsp library)
On 2014-10-17, Olivier Bélanger wrote: I use two playing heads with a user definable crossfade time. The delay time is re-initialized only when a head starts reading the delay line (in the silence). I'd like to amplify rbj's point: if you have a continuous feed of the kind of hypercompressed pop music we all like in both YouTube and games, as the input to the delay... How do you deal with a ten second delay commanded down to a one second delay, if that's done within two seconds in total? How do you keep it from glitching or pitch shifting during those two seconds? (Okay, I know, and rbj knows, these kinds of questions are basically about a crucifixion. About putting someone to the rack and making them squirm. We know that's not what you should really do to a person who's evidently taking their audio-dsp seriously. Enough to release code, even. But we still can't help but latch onto the stuff which sounds fantastic. That stuff just asks for an amount of Inquisition, thumbscrews, and in certain cases even a proper pyre. ;) For any newcomer around here, or any other scientific-technical-mathematical list...that ain't about hate. It's about basic due diligence. Even saying I dunno in return is well enough. Saying tell me more is even better. And if you then tell us how you did it better than we thought was possible...well then now *you* are the Master. Don't underestimate the old master, though... ;) ) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-14, Max Little wrote: Hartley entropy doesn't avoid any use of probability, it simply introduces the assumption that all probabilities are uniform which greatly simplifies all of the calculations. How so? It's defined as the log cardinality of the sample space. It is independent of the actual distribution of the random variable. Because as you can see even from Wikipedia, it coincides with Shannon's definition in the case of an equidistributed source. Thus, Shannon's definition let's you analyze that case, too. Only, Hartley's case doesn't let you analyze but a *very* small part of what the Shannonesque case does. Since Shannon's is the more general and more generally taught case as well, you'd do better to learn it first. The Hartley one is most likely only mentioned by Kolmogorov as a stepping stone to the full-on min-entropy stuff (only fully developed in the recent years, and in very odd niches far apart from compression and signaling), and his own computationally minded work (which could still be fully characterized within Shannon's framework, in theory, but in practice goes into the many separate models territory I talked about, for computational *complexity* reasons; cf. Greg Chaitin too, if you want to really fuck mind up in descriptive complexity ;) ). -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-14, Max Little wrote: Hmm .. don't shoot the messenger! I merely said, it's interesting that you don't actually have to specify the distribution of a random variable to compute the Hartley entropy. No idea if that's useful. Math always has this precise tradeoff: more general but less informative, versus more specific and conducive to a deeper theory. So, true, Hartley's conception is more general and doesn't need as much input fodder. Yet as Ethan said, the theory then assumes too little to be able to answer most of the interesting questions. You can't analyze anything noisy under that one. Shannon's theory on the other hand does incorporate probabilistic reasoning from the very start, and does it in the most general way possible. So it leads to deeper and more useful results, and surprisingly, doesn't require much more machinery than you'd have to have starting with Hartley's definition. Moreover, it really does fully contain Hartley's definition as one special case. That is then why we hail Shannon as the progenitor of information theory. He got the formalism and the viewpoint right, plus did the math, under *very* lax assumptions. So that his viewpoint and framework has guided pretty much everything since then. That's what math proper is about, finding fruitful ways of formalizing hard problems in ways which make them seemingly easy to deal with; Shannon did it better, since his required less legwork even for the general case, without messing up even Hartley's special one. (And of course nobody ever forgot Hartley, either: the one, most fundamental theorem in continuous time analog coding theory is called the Shannon-Hartley theorem. It's just that while Hartley was a formidable, early information theorist, Shannon got his basic framework *just* that little bit better within the discourse. :) ) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-14, Max Little wrote: Maths is really just patterns, lots of them are interesting to me, regardless of whether there is any other extrinsic 'meaning' to those patterns. In that vein, it might even be the most humanistic of sciences. Moreso even than poetry: https://en.wikipedia.org/wiki/Where_Mathematics_Comes_From . And then right back to business, in that humanistic vein; we're wont to stray off topic, which in math means you said something other than first order predicate logic. So let's return. If you look at the real audio signals out there, which statistic would you expect them to follow under the Shannonian framework? A flat one? Or alternatively, what precise good would it do to your analysis, or your code, if you went with the equidistributed, earlier, Hartley framework? Would it help you make mp5 terser than mpeg-2 layer 3, or especially AAC? If so, howso? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: To demonstrate this through an example - could you point out to _where_ the most amount of information is located in the following message? 00010010110011010 There is absolutely no way of knowing that unless you have an underlying probability model of the source. Which is why we already told you that you should read the basics. Let me give you an example. Suppose the message is just one bit, having the values one or zero. In this instance the random variable in question happened to be realized as: 0 What can you tell about it? You can't separate it in any meaningful fashion (unless you do something truly ghastly, mathematically speaking). There is no structure there, no changes, nothing. And yet information theory speaks about that single bit message as well, in a nontrivial way. In this case it happens that the self-information/surprisal of the one bit message was one megabit. How could that be? Well, because I chose to make the source such that the probability of it giving out a zero is 1/(2^-100). It pretty much always gives out a one, so the self-information of a one is negligibly small. But that one time, over the age of many universes, when it *does* give out a zero, the self-information of that is hugely out of proportion. If you then generalize this kind of thinking to your string of numbers, you can't say anything about any of them before you make some assumptions about the underlying probability distribution. Each successive one and zero could mean just about anything, or nothing at all. For example nobody said that the underlying source wouldn't have probability 1 of giving out exactly 00010010110011010 each time, and probability 0 of giving out anything else. In that case the surprisal of that example of yours is exactly zero as well; it contains no information at all because it'd always go down the same way. So, learn your basics, then state your assumptions clearly, and perhaps then we can talk clearly about the *much* harder topic that is perceptual information theory. :) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Rohit Agarwal wrote: You need to show an exclusive 1:1 mapping between your source symbol space and your encoded symbol space. Then you can determine output bitrate based on the probabilities of your source symbols and the lengths of your encoded symbols. One way to do this is with a Huffman Code. Which is optimal down to rounding to 1/2^n probabilities as well. If you want to go below that, then arithmetic coding, block coding, dictionary coding or some variant of them is needed. And of course none of those are mutually exclusive... But I think we're going ahead of things here already. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: Those flipped noise bits add entropy to the message, precisely. No, they do not, if they just follow the same statistics as the original message. Which my algorithm detects, correctly, since your noise is an entropy source. If your algorithm can detect any and all noise bits, then it can also detect any and all bit flips which do not come from a noise source, but a deterministic signaler who wants to utilize your scheme in order to pass on extra information. So, it'd seem that your scheme is able to breach the Shannon bound -- you can try it, with me and most other coding theoretically minded people on this list, but you won't succeed. We'll always come up with a counter-example, even after we already *told* you to read the basics which already tell you why that won't work out. (In fact the exercises in the text books are meant to drill in that counter-example, already, so that you get the real tradeoff.) Your earlier algorithm just segments bitstrings. It doesn't tell you how to assemble those segments back into a code which can be understood unambiguously by any receiver. Unlike the original message could. So what you there did was that you broke a perfectly good message, and never bothered to put it back together. Obviously you can claim any feat that way. As I just claimed above the feat of packing a megabit of information into a single bit; that's trivial. Now put your message back together as well. And once you're done, also do it under noise (a much nastier proposition, I can tell you). -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: Well, if you prefer, you can call my algorithm 'randomness estimator' or 'noise estimator' instead. Personally I prefer to call it 'entropy estimator', because the more random a message is, the more information (=entropy) it contains. Define random. I fail to see why you guys don't realize this trivial correlation. Because we already see how difficult and finicky this stuff can be. For example they're still, after decades of hardest of hardest mathematical analysis, unsure about what randomness really is and how it can be defined. Seriously, just take a look at how mathematicians deal with the idea that certain expansions of certain transcendent real numbers might be normal. Heady mathematicians can't bring themselves to say they're talking about randomness, there, but acquiesce to talking about certain forms of statistical uniformity, one of which is called normality. And that stuff has gone around for decades, with no end in sight. As for entropy estimators, that's as good as being able to extract entropy. Entropy extraction was then already mentioned by someone. That is a *truly* nasty and unintuitive piece of statistics and circuit theory. If you want to understand even a part of it, suddenly you have to know everything from descriptive complexity theory to bounded circuits, topological graph invariants, and whatnot. I too once thought that I had a hang of it, purely by intuition, but fuck no; the live researchers at cryptography -list taught me well better in just a couple of messages. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: Again, it seems my message was lost in translation somewhere... Your message is lost in those fifty self-reflective, little posts of yours. Which is precisely why you were already told to dial it back a bit. I'd also urge you to take up that basic information theory textbook I already linked for you, shut up for a little while, read it, do all of the homework, and then come back. It's not just that you annoy folks a bit. It's also that you'll never learn otherwise. It's for your own good, boy. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: Define random. As I told, my randomness estimation metric is: number of binary state transitions. It is a very good indicator of randomness, feel free test on real-world data or pseudorandom number generators. So by your metric a fully deterministic, binary source which always changes state has the maximum entropy? 010101010101010101010101... ...is always maximally informative to the receiver, even if that was the only message possible? With probability 1? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: 010101010101010101010101... How do you know that that signal is 'fully deterministic', and not a result of coin flips? Because I assumed it to be. Let's say I sent it to you and just made it into a deterministic generator. Without telling you how it was generated. Where is the information, from *your* viewpoint? Maybe I'm one of those cryptographers you're afraid of ;) That remains to be seen. With 1/100 probability. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: Because I assumed it to be. Let's say I sent it to you and just made it into a deterministic generator. Without telling you how it was generated. Where is the information, from *your* viewpoint? In what context? You sent me a long stream of '0101010101...' So? Without context, there is not much I can say about it. Now you're finally getting it. It's about that context at least as much as it is about the signal sent. Everything that is relevant about the context can then also be quantified in one form of probabilistic distribution of the source or another. That's the whole point of what I and Ethan tried to say. Not to mention Claude Shannon. You sent me a sine wave tuned to Nyquist. That is all I can say. No I did not. I sent you a stream of all zeroes. It's just that every time I send anything to anybody, I xor-obfuscate my bitstreams with a maximally fast flipping binary sequence, starting with zero. It has nothing to do with Nyquist either, because I wasn't even working within a sampling framework, but within GF(2)^n, which doesn't possess a workable sampling theory to begin with. Orthodox information theory, coding theory or even the theory of compression makes no distinction there. What I just said is fully cognizable and well defined, quite without any reference to sampling. Also, nobody cares, because the underlying probability distribution is isomorphic to what I started out with, so that mutatis mutandis no more or less information was passed; your context is the same. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: When you're trying to approximate entropy of some arbitrary signal, there is no such context. Of course there is. Each and every one of the classical, dynamically updated probability models in text compression has one, too. The best ones even have papers behind them. With deep statistical analysis to spell out the assumptions in full. Cf. e.g. those behind context tree weighting, then going backwards the PPM* family of higher order Markov models, and finally going back to Lempel and Ziv's LZ77 and LZ78 algorithms, which actually originated as formal proofs of asymptotically optimal compression performance for *any* message (originating from an arbitrarily high order ergodic, stationary, quasi-Markov source, or what was it now...). No I did not. I sent you a stream of all zeroes. It's just that every time I send anything to anybody, I xor-obfuscate my bitstreams with a maximally fast flipping binary sequence, starting with zero. Why not use ROT13 instead? I heard it's better. Very much my point: Shannon's definition of information is fully immune to ROT13. Yours is not. That is also why Shannon talks about information, and has been celebrated as a mathematician who quantified information once and for all. Not you. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] entropy
On 2014-10-12, Peter S wrote: Rather, please go and read some cryptography papers about entropy estimation. Then come back, and we can talk further. PLONK. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] #music-dsp chatroom invitation
On 2014-10-09, Ethan Duni wrote: Again: if I make _very_ loud (= lot of signal energy) naive, non-bandlimited square wave, that _still_ has only 1 bit of entropy. No matter how much I turn up the volume, I do not gain any additional entropy. Still 1 bit. No, you are clearly misunderstanding the basics of how entropy relates to signals In a sense Peter is right. If our underlying model is that of either a signal or no signal, of course it's just one bit per sampling time. It's just that with a model like that, there is no way to represent square waves of different amplitudes, so that it doesn't make any sense to talk about loud and quiet ones, nor sums of square waves. After that the whole entropy of in the signal is one bit for whether it's there, and some arbitrary amount of bits for frequency and phase, which in this case are a rather complicated function of the truncated, one bit samples. So, actually, when you talk about entropy, you ought to define the model it's calculated against, and that leads to the iffy topic of Kolmogorov complexity. In audio work that model then ought to represent everything known about the human hearing, in essence reproducing all of psychoacoustics and thus the ideal audio codec, not to mention the entire probability distribution of the space of signals we're about to be coding; a truly monstrous undertaking. You need way more than 1 bit to represent any square wave - you have to specify the frequency, amplitude and phase offset. Exactly. Plus of course if you allow mixtures as well, then you're into the same territory that Fourier analysis covers, except that your transform is much less well behaved. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] #music-dsp chatroom invitation
On 2014-10-09, Ethan Duni wrote: Look carefully - I'm not speaking about creating _another_ sine wave with 10x volume. No. I'm saying that I amplify the _original_ sine wave by 10x Those kinds of philosophical distinctions do not have any bearing on entropy. Again, in a certain sense, they do. But you have to be careful about what you're saying. Here what is forgotten is that entropy is a property of entire probability distributions, not individual realizations of the associated random variable. In Peter's case there are two encodings of a single one bit stream in a wider PCM word, related by a simple left shift of the associated value. If you think about the distributions, both are purely binary (per sample), and so have precisely one bit of entropy per sample. The crucial part is that that they are drawn from two separate distributions which just happen to have the exact same entropy. You cannot think about either of the signals in the context of the other distribution, because its probability there is identically equal to zero. In the framework you're using, and which is the standard one for PCM signals, you assume an underlying Gaussian distribution for the noise floor and ask how probably a given signal would arise by chance from that, or perhaps, given a perfectly uniform distribution without noise, how likely a certain signal is to occur then. Now in the first case the surprisal of a higher amplitude signal is higher as well, and in the latter one each signal has a continuous surprisal per sample, now directly tied to the bit width. Crucially, in both cases you need to know the entire underlying probability distribution before you can calculate entropy. For entire signals that then not only consists of the distribution of individual samples, but any higher order serial correlations as well, and for psychoacoutic work, you also need to define identical sounding signals as the same, rapidly taking you pretty far from the naive way of counting entropy to first order independent identically distributed samples. Which of course brings us back to your very point: Peter really should understand the basics of information theory before applying it. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] #music-dsp chatroom invitation
On 2014-10-09, Peter S wrote: Could you offer me a reading that you think would clarify the concepts that you think I applied in an improper way? Any standard textbook will do the job. Google's first recommendation is as good as any: Cover Thomas, Elements of Information theory. It's cheap enough, and covers the basics of rate distortion theory as well, which is nice since we really want to end up understanding lossy coding. They devote some space to the difference between entropy, self-information and the lot, too, which is at the base of most of the (formal) misunderstandings in this thread. Though you should know that as usual, I'm not the one that should be giving out readings. I mean I went about information theory the hard way, too: I just delved into text compression and then error correcting codes and then the classical papers, without even having any background in mathematical probability at that point. I can tell you, that is an uphill battle, and the way you end up learning about the stuff doesn't become exactly de jure. Hence my talk about e.g. models -- that comes from text compression, where the underlying probability distribution and so the self-information too is invariably modelled by an actual realization as computable probability model. The same for classical source coding: most of the time you don't even realize how far from the ideal Shannon bound you're working, simply because of computational and algebraic limitations. Well, okay, not perhaps in the foundational papers, but in pretty much everything that is worth writing about and interesting enough to be read. That skews your thinking and especially your vocabulary rather a bit, and makes you less than an ideal pedagogue. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] #music-dsp chatroom invitation
On 2014-10-07, Peter S wrote: I only pointed out that both are 'filterbanks', without stating anything about where exactly the center frequencies are located, nor assuming that their centers are located at the same frequencies. (and the number of bands is also different, obviously.) Talking about filterbanks implicitly says that all of the filters are somehow structurally the same, and linear. Neither is necessarily true of what the cochlea does. For one, there's the weird overlapping between them, and the fact that really the basilar membrane is one continuous waveguide sampled by acoustocilia in a rather odd pattern. We also know the cilia themselves are rather complicated and nonlinear pieces of machinery, with a one-way, rectifying response even to the lowest order of approximation. And of course there's also the descending auditory pathway which makes the organ of Corti as a whole piece of a closed loop control system whose overall function is *still* not exhaustively understood. That's pretty far from how we typically understand filterbanks, so one shouldn't take the analogy too far. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Filtering out unwanted square wave (Radio: DCS/DPL signal)
On 2014-07-30, Bjorn Roche wrote: Thanks for all the info so far. I should have been more careful when I said DCS is a square wave. It's probably more accurately described as an NRZ code. Nevertheless, these suggestions are very useful. I'd actually argue for decoding the signal and starting with the idea that you cancel what you know to be there. That's because typical time or frequency domain methods we use on the music side can never fully remove such a signal from your utility band; they're meant to do psychoacoustically meaningful and as such more or less linear operations, which suppression of a binary modulation scheme definitely is *not*. At the same time, the vast majority of digital signals contain ample redundancy, aimed at making them easy to detect and decode. So, a more or less naïve decode-reencode-subtract loop typically does rather well (if it didn't, you wouldn't be able receive the signal, so that nobody would bother sending it for your nuisance in the first place). What typically messes you up here is the syncronization, especially over terrestrial radio channes, so that you don't *really* want to just subtract anything. But even there you can do a maximum-likelihood center-of-step sampled detection step, and then a round local minimization (preferably againt L^1 norm, but L^2 is obviously more efficient since it can be implemented as an FFT based convoluion) over each transition. The math easily yields a matched filter, which upon sampling gives you a noise tolrant estimate of what should be subracted. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2-- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Instant frequency recognition
On 2014-07-17, Ethan Duni wrote: The thing about this approach is that it requires very strong prior knowledge of the signal structure - to the point of saying quite a lot about how it behaves over all time - in order to work. Well, theoretically, all you have to know is that the signal is bandlimited. When that is the case, it's also analytic, which means that an arbitrarily short piece of it (the analog signal) will be enough to reconstruct all of it as a simple power series. But of course that's the sort of thing you don't actually get to apply in practice. I.e., if you have a signal that you know is a sine wave with given amplitude and phase, you can work out its frequency from a very short length of time. But that's only because you have very strong prior knowledge that relates the behavior of the signal in any short time period to the behavior of the signal over all time. Yes. I guess my point is that I'm struggling to think of an application where such strong prior knowledge exists, and where we'd still need to estimate frequencies from data. A typical example would be FM demodulation in software, or e.g. the detector part of a phased locked loop. There you know that you'll be dealing with a signal that is short term sinusoidal at full amplitude, and want to derive an instantaneous differential frequency estimate with as little delay as possible. That's the kind of problem you can solve optimally in closed form under a wide variety of conditions, e.g. using just three successive samples as input, and those kinds of solutions are actually in use. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Simulating Valve Amps
On 2014-06-19, Ross Bencina wrote: There is a segment of the market that values accurate models--at any computational cost. Then, can you do that at low latency, so that your model is also playable? That's of course the next frontier. And no, there's no shortcut there: those giga-computations not only take memory bandwidth but the kind of low latency which just isn't there. That's why we're seeing this kind of technology in offline plugins, but the musicians still play analog instruments. Which of course have zero latency. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Simulating Valve Amps
On 2014-06-19, Rohit Agarwal wrote: I'm surprised by that statement quite honestly. At a tempo of 200 bpm, this latency would be roughly 10% of the beat interval which seems to me quite small. Then you obviously don't know techno. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Simulating Valve Amps
On 2014-06-17, robert bristow-johnson wrote: And the cabinets are a huge part of the sound. *that*, and the loudspeakers themselves, is the hardest part, no? it's all three: 1. salient (so you can't ignore it), 2. non-linear, and 3. non-memoryless. From what (very little!) I know of hardcore analog simulations, I'd say that is part of a more general and much nastier problem. That's the interaction one: whereas digital signal graphs have a definite direction of signal flow, there's no such thing on the analog side no matter what you do. Sure, you often try to go from low to high impendance and make use of tons of other tricks in order to make your circuit approach a directional signal flow, but lo and behold, most of the nicest sounding circuits actually measurably feed back all the way from the speaker-air-interface to the input jack, affecting anything and everything on the way. The Moog ladder is perhaps the most notorious circuit here, because ladder filters are distributed, equilibrium systems from the start. No true buffering/isolation to be seen, there. But to a lesser degree, at least in order to achieve a proper tube simulacrum, you do always have to consider the full electromechanical equilibrium (and the not-so-equilibrium, e.g. with transients) even within the enclosure-cone-magnet-wires-transformer-terminal-powertubes-powersupply whole. That's then a total bitch to do effectively yet still accurately, in code where the simple operations just don't feed back to the operands, like they used to. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] a weird but salient, LTI-relevant question
On 2014-05-08, Theo Verelst wrote: About the noise, I'll say it only one time: IS YOUR NOISE SOURCE BANDWIDTH LIMITED, [...] This sort of thing is precisely why I left it for you and everybody else to produce. By yourselves. So that only yourself is then to blaim. It really doesn't matter too much as long as you loop the noise well enough. No splicing artifacts which would make the noise easily discernible from looped noise of the same class. Even somewhat lowpassed noise wouldn't be *too* bad -- but then in looping you can't ever lowpass it except using a cyclic convolution, or the splice point can be heard. So, Theo, no analog processing here. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] a weird but salient, LTI-relevant question
On 2014-05-08, robert bristow-johnson wrote: where the RNG for the dither is derived from the LSBs of the last N quantized words? The parity of the last few words as whole. That's because parity/xor has been proven to be an optimal randomness extractor in this sort of thing, and even if it doesn't remain so in the loop I just introduced, it's pretty much the only one which provably disregards both static leading and following zeroes. So that you can embed PCM words of any width, without having to tell how they were embedded, and still achieve in-even-hardware-low-cost synch. Don't say I never thought this out in full. ;) i think that if you cannot hear a different with a butt splice, you won't hear it with a cross fade. Definitely not true. Most means of interpolating over a splice repeat do not really do it over. They work fine within a running sample stream, but once they loop, they do something wrong. Especially at the very first nonrepeating run, and at the end of the last repeat. Or even if the interpolation algorithm is good enough, it's very difficult to make anybody else believe that it was so. Especially any naive listener who knows precious all of DSP and couldn't audit the code. All of the true, golden ears out there are like that. So... ;) if that were the case, Fletcher-Munson curves (or Robinson-Dadson, or pick your researcher) would have equal spacing for all frequencies. the fact that they get squished at the very low and very high frequencies is ostensibly not linear behavior. Of course. But then, this kind of a test shows that that human hearing has even a further, LTI-kinda-looking aspect, which doesn't have to do with those curves either. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] a weird but salient, LTI-relevant question
On 2014-05-08, Olli Niemitalo wrote: Sampo's test should be carried out multiple times to gather statistics, and because repetition will aid in reinforcement of the memory, also the number of repetitions should be controlled or recorded. How about tap to the rhythm of it? Or, more to the point, you should always repeat the test using a different noise stream. You shouldn't be able to learn any statistical deviation from one test to another. The only learning and pattern recognition in play should take place from cycle to cycle, and possibly even so that you're limited from hearing more than two cycles of sequence. (Though it's pretty much impossible to implement that without the cutoff giving you a hint of what the repetition length was.) Interestingly, nobody's taken the test as of yet. Even if it ain't in the least bit a contest, and I already said to begin with that the result might be rather interesting for any and all. Feature-stripped noise should work better in some applications than truly random noise. Perhaps multi-band compression could be used to level it out. If you do anything of the sort, you by definition introduce structure into the signal. After that it ain't noise anymore. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Correctable signal processing (to arrive at wire connection)
On 2014-05-08, Theo Verelst wrote: So give or take a few LSB errors, are digital filters like filters in the analog domain? Yes. So if we have N digital poles, can we create N digital zeros at the same frequencies, convolve those two filters and arrive at a digital wire ? Of course there may be some delay here... Yes. Practical ? Yes. Already done, as your EE eminence well knows. Well, this week I was playing with my Lexicon AD convertors and a good microphone setup, driving my large monitoring system with my latest high quality ground-seperated 384 kHz DA convertor in a real-time situation, and wanted to compensate the small (few dBs here and there) the frequency sensitivity unevenness of the microphone I used, and applied some jack/jack-rack/ladspa Linux filters for that. Worked great. But did it actually constitute a digital wire? -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
[music-dsp] a weird but salient, LTI-relevant question
This is going to sound pretty weird, I'm sure, but could as many people on-list perform the following experiment on themselves and their close ones, as possible? Then report back (privately, so as not to ruin the surprise for everybody else?) Take a long (at least 30 seconds and possibly more) sequence of truly random (AWGN) noise, either from a very long period PRNG or from a primary randomness source. Then starting with very long periods of over 10 seconds, loop the noise, curtailing the period of repetition. Dropping it, say, 200ms at a time at first, and in the end perhaps something like 10ms at a time. When does your ear, perceptually speaking, start to say that the noise repeats? Precisely? I'd be interested in hearing what people on-list have to say about this one. Especially the ones who are curious enough to find the precise limit in milliseconds, and even subject their loved ones to the test. Because, I mean, at least for me this was a total mindfuck, and if you analyze it e.g. via the usual LTI theory of human hearing, the results do not make any sense at all. I think, but I'm not too sure. Whence the question. ;) -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Combining ADCs/DACs to increase bit depth?
On 2014-04-25, Tom Duffy wrote: Combining ADCs/DACs to increase bit depth? I was wondeing if anyone has done this or if it is even possible. Yes, and it's a patent mine-field. A particular case in point, MASH converters. Also certain delta-sigma derived architectures with multibit and/or mixed analog/digital stages within the outest loop. The most interesting case to my mind would be taking a pair of imperfect high rate multibit A/D and D/A converters, connecting them into a delta-sigma loop, and trying to perfect each using the other. To my knowledge theory for doing that sort of thing is nigh-nonexistent, as are the patents. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Does a neutral wire exist
On 2014-04-09, Theo Verelst wrote: Not in the electronics sense, like gold plated, electrically shielded, with 1/3 of the speed of light and such, but in the digital sense! I'd argue that in the analogue sense the best transmission line you can buy is pretty much it, while in the digital world cp /dev/a /dev/b does even better. Of course this is easy to implement, but think about it: how often does a program/module/machine offer the option to record and playback or to transfer simply the information fed to it, with no resampling, no adding of blanks, no slight processing where you don't anticipate it (this happens in audio editing programs), no volume change, etc. ? Rather often. It's even an idiom since times immemorial: the bypass switch. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Audacity Unconference 2014 (AU14)
On 2014-04-03, Martyn Shaw wrote: Audacity have announced their 'Audacity Unconference 2014 (AU14)'. Not able to attend, but might I ask a question? When do they intend to make arbitrary cun'n'paste into arbitrily multichannel streams unconditionally O(1), with a very low constan term too in internal memory? Plus at least make all of their edit code the undo one, upper bounded by the optimal buffer tree algorithm, even if they have to spill to disk? Seriously, mon, Audacity would be pretty good, except that it's data heavy and as a result heavy as Moon crashing down. Just as slow, deleterious and over all editable as well. Kind of like a million giraffes very close to each other You can't claim it wouldn't be cool, but... -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Nyquista?Shannon sampling theorem
On 2014-03-28, Charles Z Henry wrote: Probably everybody knows that you lose something when you mic a bass drum and send the output to a vented box subwoofer. It lags a little bit behind and the tone gets smeared out in time by the resonance. A successful loudspeaker like this would be able to reproduce impulses (drum hits, canons, etc...) with greater clarity. That's an excellent point, and the reasoning is similar to why I for one as a techno freak have always preferred closed speaker designs to reflex ones. A well constructed speaker behaves approximately as a minimum phase system wrt the listener, so the gentler the amplitude variation, the gentler the envelope smear caused by group delay variation as well. That's a particular problem at the low end, and you really, *really* can hear the difference even between a critically tuned reflex and a closed box. If you want to get a hold of what it sounds like and don't want to do real filter design, try an allpass with severe phase nonlinearity at the bass frequencies (a comb allpass section with near unity gain and long delay will do). It's by definition not minimum phase so you can't do anything quantitative done, but it'll get the job one as far as phase goes. A sharp acoustic kick or practically any one of the snappier standard ways of synthesizing tech bass drums will do. They turn from kick to a distinctive slurp, and lose a good amount of their subjective power. Once you get the taste of it, you'll learn to absolutely hate the effect. Of course that'll then try your sanity, since nowadays pretty much every home theatre comes with a subwoofer that is sluggish in that way, not to mention what your typical poorly damped room modes contribute besides. With orchestral music it's not such a problem because the aesthetics favour lively halls and steady state content, but anything electronic or percussive, with lots of sharp transients and a wide bass extension simply turns to mush. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Dither video and articles
noise), the system quasi-periodically self-synchronizes, and after that you can see whether the dither signal is there by doing an approximate, straight correlation with the generated dither stream. All of that means the system becomes pretty robust in the face of cut and paste, formats which can't flag its use via metadata (say, CDs), and even varying bitwidths in the channel, such as seen in libsndfile (the extractor is just a parity over the sample so it's immune to trailing zeroes, and the registration is sample accurate, so that if you lose sync in the middle of the stream, it will be reacquired uniquely at the next sync point, which occur at a set set probability per sample, based on what the extractor lifts from the stream; also there are a couple of minor twists so that the thing doesn't rekey on silent channels etc.). Right now the main problems to be solved are efficiency in software, state size of the RNG in hardware, and avoiding any oscillation from the rekey mechanism through the RNG loop in realistic conditions, and finalizing the correlator code in a form which doesn't use multiplication; might be I have to ditch the bigger xor-shift generator for something leaner and perhaps nonlinear. Comments, anybody? in fact, i think that in a very real manner, Stan Lipshitz and John Vanderkooy and maybe their grad student, Robert Wannamaker, did no less than *save* the red-book CD format in the late 80s, early 90s. and they did it without touching the actual format. same 44.1 kHz, same 2-channels, same 16-bit fixed-point PCM words. they did it with optimizing the quantization to 16 bits and they did that with (1) dithering the quantization and (2) noise-shaping the quantization. OTOH that interpretation is in my opinion an overstatement. Sure, principled dithering in converters and what not solves the low amplitude harshness problem wholesale, but in practice most CD's produced even before additive TPDF dither became the norm were sufficiently, perceptually speaking, autodithered by the broadband signal or the external noise floor. the idea is to get the very best 16-bit words you can outa audio that has been recorded, synthesized, processed, and mixed to a much higher precision. i'm still sorta agnostic about float v. fixed except that i had shown that for the standard IEEE 32-bit floating format (which has 8 exponent bits), that you do better with 32-bit fixed as long as the headroom you need is less than 40 dB. if all you need is 12 dB headroom (and why would anyone need more than that?) you will have 28 dB better S/N ratio with 32-bit fixed-point. Personally I like 32-bit floats. They have 24 bits of mantissa, which would already be sufficient for transparent fixed point. They're already perfect, with the exponent giving the dynamic headroom for resonant filters and whatnot. Now if only you could easily use them *as* fixed point numbers *too*... I mean things like truncated IIR's call for precise control of the exact rounding error, which you can't really achieve with floats, in a dependable manner and using clean code. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Dither video and articles
On 2014-03-28, Emanuel Landeholm wrote: I agree with the points you raise and I'd like to add that you can also trade bandwidth for bits. Totally, and you don't even need to go as far as to apply noise shaping. High sampling rates and linear filtering already raises that question. Okay, in audio DSP you'd typically want to do the real, hardcore, noise shaping trick, at least in release formats with insufficient bits like CD, but e.g. in RF work you immediately bump into these kinds of considerations. One of the nicest examples is something we bumped into a little while ago already, after something Theo said. That's because, as soon as you start doing frequency analyses in the presence of noise, high bandwidth counter-intuitively means that the same precise noise RMS in your signals is spread over a wider bandwidth, so that cutting it out with a filter is per se already an instance of that tradeoff. That then also means that you can't read spectra at all without invoking the concept of resolution bandwidth. FFT's are slightly easier compared to the analog sweeped ones because thought of as filter banks they're critically sampled by definition, but even they lead to nasty surprises for the uninitiated, because the length of the transform leads the individual bins to be narrower. When that's so, in a longer block the noise is distributed over more bins, but steady state sinusoids -- with their infinitely thin Dirac spectra -- stay within a single bin and hence stick out like a sore thumb. Thus, with a long enough transform, something that in the time domain looks like nothing but noise, in the frequency one suddenly has a spike so high that scaling it to range makes the noise floor round off to invisibility. Analog spectra are then even nastier, because they're fundamentally overcompete representations where you have two separate things to worry about: the sweep rate which sets the convolutional spreading of peaks due to amplitude modulation, and the resolution bandwidth, which sets integration time, and so both temporal responsiveness and the noise supression of the matched filter. Funnily enough, eventhough I've been interested in the theoretical aspects of DSP for some two decades now, all such woes of matched filtering and the like are relatively new to me. That leads me to suspect those aspects aren't stressed enough in modern treatments of the subject, and that I might not be the only diginative who has gap in their understanding regarding continuous spectra, matched filtering, statistical detection in the presence of noise, reading the relevant diagrams, and so on. And in fact, while I'm rather critical of Theo's and other audiophile minded folks' claims over things like ultrahigh fidelity formats, I must say their understanding of traditional analog EE and background in tinkering with it probably make them better armed to deal with this side of the field than I'll ever become. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Nyquist-Shannon sampling theorem
deltas, combs, beds of nails, all of them are perfectly fine as long as you remember that they're functionals, not functions. So for instance, it's all well and good to e.g. multiply them absent any hint of (test) functions, as long as their singular supports are disjoint. That sort of thing BTW is why the thing about tempered distributions is not just handwaving. They actually have structure and properties you need to know if you want to get continuous time Fourier analysis in full. And in particular if you want to be proficient in solving ODE's in the Laplace domain. That shit don't fly in all its generality and beauty unless you're at ease with the full calculus of naked distributions, so that you can encode arbitrary ODE's as distributional convolution kernels, and so on. i am simply treating Dirac impulses just like we do for the nascent delta functions of very tiny, but non-zero width. That's also fully kosher once you grasp the abstract machinery. The formal argument for why you're allowed to do that is that test functions are dense in the space of distributions. That means that each and every distribution can be arbitrarily well approximated in the weak topology by a Cauchy sense convergent sequence of C-infinity functions. Thus all that you're actually doing with those nascents of yours is leaving the final passage to the limit implicit. That's BTW one common way of seeing why distributions really are generalized functions and not some arbitrarily exotic set of structures like the full dual of R. As far as mathematical objects go, they're actually pretty tame, domesticated and all-round benign, even if making the idea exact calls for annoying amounts of machinery, in the form of Schwartz spaces and whatnot. In that regard the Wikipedia article in fact gets it mostly right, using another characterization starting with continuous functions (obviously coming from the theory of classical mixed probability distributions, just as the name of the construct does too). the Dirac delta is, strictly speaking, not really a function, as the mathematicians would put it. strictly speaking, if you integrate a function that is zero almost everywhere, the integral is zero, but we lazy-ass electrical engineers say that the Dirac delta is a function that is zero everywhere except the single point when t=0 and we say the integral is 1. Here some background in probability helps a lot. There you're already familiar with the fact that you can represent probability distributions in two forms: the probability density function, and the cumulative distribution function, related to each other by derivation and (here, normalized so that you don't even have the integratino constant in there) antiderivation. The whole original reason for retaining both those representations is that once you start mixing discrete and continuous stuff within the same framework, using functions alone makes that equivalence of representations break down. You can't derive a cumulative probability function which is discontinuous, eventhough the discontinuity lets you systematically represent and operate on discrete concentrations of probability mass. Once you get how that works, and what it historically lead to, distributions in the general feel extremely natural: they're just the minimum closed system of function like objects which preserves the PDF-CDF equivalence, even under iterated differentiation, summation, and even limited forms of multiplication (that actually gets you into things like Colombeau algebras, which are a notch beyond in the machinery department). Plus of course all of this is pretty much the same thing, just from a different angle, that is handled by measure theory. Once you grasp that, everything just clicks into place and suddenly there's absolutely nothing magic or inconvenient about Deltas or even more exotic distributions like the dipole (the negative of the derivative of the Delta). They just work and make your life *much* easier than you ever had it with plain old functions. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Nyquist–Shannon sampling theorem
On 2014-03-27, Doug Houghton wrote: I understand the basics, my question is in the constraints that might be imposed on the signal or functon as referenced by the theory. The basic theory presupposes that the signal is square integrable and bandlimited. That's pretty much it. If you want to make it hard and nasty, you can go well beyond that, but for the most part it suffices that you can ordinarily Riemann integrate the square of your signal, you get a finite total power figure that way, and then once you integrate the product of your signal with any fixed sinusoid, over some fixed frequency cutoff every such product integrates identically to zero. When you have that, you can derive the sampling theorem. It says that taking an equidistant train of instantaneous values of your signal at any rate at or above twice the bandlimit mentioned above is enough fully reconstruct the original waveform. Point for point, exactly, no ifs, ors or buts. So the only real limitation is the upper bandlimit. Is it understood to be repeating? No it doesn't have to be. Yes, there are four separate forms of Fourier analysis which are commonly used, and which have their own analogues of the sampling theorem. Or perhaps rather the sampling theorem itself is a reflection of the same Fourier stuff which all of those forms of analysis rely upon. Two of the forms are periodic in time, which is why you might be thrown off here. But the basic form under which the Shannon-Nyquist sampling theorem is proved is not one of them; it covers all of real square integrable functions, R to R. I'm thinking the math must consider it this way, or rather the difference is abstracted since the signal is assumed to be band limited, which means infinit, which means you can create any random signal by inject the required freuencies at the reuired amplitides and phase from start to finish, even a 20k 2ms blip in the middle of endless silence. If you inject something with time structure, the Fourier transform will decompose it as an integral of a continuum of separate frequencies. This is part of the deeper structure of Fourier analysis, and what in the quantum physics circles is called the uncertainty principle. What we in the DSP circles think of as the tradeoff between time and frequency structure, and operationalize via the idea that time domain convolution becomes a multiplication in the frequency one, and vice versa, is thought of as in the physics circles as the duality between any two conjugate variables, lower-bounded by uncertainty principle. What they call a physical law, us math freaks always called just a basic eigenproperty of any linear operator on R, lower bounded by the eigenfunctions of the class of linear, shift-invariant operators, those being the exponential class, containing complex sinusoids and in the proper limit Gaussians, impulses, infinite sinusoids, and all of their shifted linear combinations. That is a deep, and rich, and beautiful theory in harmonical analysis if you choose to go that way. At its most beautiful it exhibits itself in the class of tempered distributions, which you most easily get via metric completion of the intersection of real functions in square and absolute value norm, and then going to the natural topological dual of the function space which results. If you go there, you can suddenly do things like derivatives of a delta function, and bandlimitation on top of it, in your head. And whatnot. But for the most part you don't want to go there, because there's no return and no end to what follows, and it nary helps you with anything practicable. The better way for a digital bloke is to just assume a bandlimit, and to see what comes out of that. Apply the sampling theorem as it was proven, and then get acquainted with discrete time math as fully as one can. For there lies salvation, and the app which pays your bills. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Nyquist-Shannon sampling theorem
have to go into the theory of tempered distributions. Imagine you had a signal with one sinusoid that slowly fades in and out for the duration of the signal. There is no such thing as a fading sinusoid. Either it is a sinusoid, or it is not. If you fade it, you're multiplying it by an envelope function, often called a window. Then its spectrum, expressed as a sum/integral of sinusoids, is convolved with the spectrum of the envelope function. Sure, the slowly varying envelope mostly has DC and just an eensy-teensy bit of rapidly falling off frequencies around it. But the support of its Fourier transform still aint discrete. It has a nonzero measure, which via convolution makes the support of the spectrum of the sinusoid go from a discrete point to a finite disc (and more often than not the entire real line). You just qualitatively, not just quantitatively, fucked up your sinusoid, making it not-a-sinusoid anymore. Imagine that the the envelope of this sinusoid is the first half of a sinusoid. The envelope can be described as a sinusoid whose period is twice the signal duration. If you were to simply take these two stationary sinusoids (the envelope and the audible tone) and multiply them you end up with a spectrum that contains their sum and difference tones. In that way it can already be thought of as a tone that (slowly) pops in and out, but which is represented as a sum of two stationary sinusoids. Yes. You can do that. But it doesn't tell us anything new beyond the basic sum and difference trigonometric formulae we all know. If you wanted to have the tone come in and out more quickly you could add the first harmonic of a square wave (or several) to the envelope. How would you handle a transcendental period? The Fourier transform, as well as the sampling theorem, handles that directly, without treating it as a special case. For each additional harmonic you add to the envelope you get an additional two sinusoids in spectrum of the whole signal. You can keep adding harmonics up to the Nyquist frequency. This means that your frequencies can pop in and out very quickly, but only as fast as your sampling rate allows. How do you handle phase? Okay, the Fourier series can handle those as well, and in more than one form. So can the DFT. But how would you, in this framework? BTW, in case I sound like a know-it-all, I'm anything but. For example, it's only been a month or so since I learnt the true difference between a matched filter and an inverse filter for real, and I'm still not quite sure whether I really understand the meaning of resolution bandwidth as it pertains to frequency analysis under conditions of non-AWGN noise and/or more deterministic and/or correlated interference. The latter one is especially bad, because it hits right smack in the middle of the Fourier theory I just utilized above, to talk about the envelopes, the spectral domain supports, and whatnot lying behind it. There seriously are no shortcuts with this shit. You really have to mind your assumptions, take proper care, and surprisingly often just grant that you're for now at your wit's end. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2-- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Nyquista?Shannon sampling theorem
On 2014-03-27, gwenhwyfaer wrote: In music the distinction isn't terribly important, because the lower limit of the bandwidth is about 20Hz; other applications may find it more useful. Except for 1812 Overture. That sinks rather near DC at substantial amplitude, given the live cannon in the percussive section. Of course some bright fella then also went ahead and invented a speaker coupled right downto static pressure, presumably just to piss the rest of us off: https://en.wikipedia.org/wiki/Rotary_woofer . I should also go kill myself just about now. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Dither video and articles
On 2014-03-26, Nigel Redmon wrote: Maybe this would be interesting to some list members? A basic and intuitive explanation of audio dither: https://www.youtube.com/watch?v=zWpWIQw7HWU Since it's been quiet and dither was mentioned... Is anybody interested in the development of subtractive dither? I have a broad idea in my mind, and a little bit of code (for once!) as well. Unfortunately nothing too easily adaptable though... Willing to copy and explain all of it, though. :) The video will be followed by a second part, in the coming weeks, that covers details like when, and when not to use dither and noise shaping. I’ll be putting up some additional test files in an article on ear level.com in the next day or so. In any case, thank you kindly. Dithering and noise shaping, both in theory and in practice is *still* something far too few people grasp for real. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2-- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Inquiry: new systems for Live DSP
On 2014-03-14, Olli Niemitalo wrote: Not sure what's going on at the lowest bass frequencies, as the peak there could be a measurement artifact. I'm guessing that's just the minimum phaseness of the kernel intruding. Remember, for a minimum phase filter the phase response equals the log Hilbert transform of the magnitude one. The Hilbert transform is then essentially a derivative plus a linear phase low pass to equalize the magnitude. From that perspective, rapid rolloff at band edge will give precisely the kinds of peaks in the group delay you're seeing in the picture. Or you could take the Hilbert transform pair for the characteristic function of an interval [a,b], which is 1/pi log |(x-a)/(x-b)|; that has a pair of singularities at the endpoints. I'm betting if you plot the group delay on a log frequency scale (or better yet constant ERB), you'll see that they distributed the group delay variation so that it doesn't lead to significant envelope dispersion over any given critical band. Plus of course if you did a full roundtrip through two such converters, any deviation from the ideal brickwall response near the band edges would be squared, making the plot somewhat more difficult to interpret. Why you'd be seeing a lower band edge is beyond me, however. It isn't as though you can't drive delta-sigma converters right down to DC. Maybe there's a capacitive coupling there somewhere, or something... Ardatech calls such a thing a lowest group delay filter and TI calls it a low latency filter. They are useful when running a real-time input--output effect like Guitar Rig, or for software monitoring of inputs, but will hinder some more analytical uses of a sound card. Peter Craven once speculated in the JAES (I think) that such filters might actually lead to more accurate reconstruction of spatial and envelopment cues because they don't exhibit preringing. Another way to go about the same thing is to use higher sampling rates and something like a Bessel filter instead of a sinc, leading to no ringing at all but a much gentler magnitude rolloff. The call's out on such things, nobody really knows if there's any point to them. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] The Uncertainties in Frequency Recognition
On 2014-03-12, Emanuel Landeholm wrote: right at the edge of my dragon territory, Here be dragons? Lol. Precisely that. Estimation is a field that is seriously involved and at some times, at least to me, rather magical looking. Perhaps the most well known example of that is Stein's phenomenon: for joint estimation of three or more independent quantities corrupted by an AWGN process, least mean squares is actually inadmissible since it is dominated by things like the James-Stein estimator. That example alone is deeply disturbing because it shows that while combining multiple independent AWGN processes does lead to multivariate Gaussians, in high enough dimension the multivariate, combined error norm differs in an essential manner from the univariate and bivariate cases. Suddenly in the statistical framework the quadratic nature of the norm enters essentially, and lets you get lower variance estimators by going joint, nonlinear and -- perhaps worst of all -- biased. Estimation theory is filled with surprises like that, and at least I haven't been able to gather any general understanding of where it most likely breaks my intuition. Hence, there be dragons. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] The Uncertainties in Frequency Recognition
On 2014-03-12, Emanuel Landeholm wrote: The intepolation filter only needs to be infinitely long if you need infinite precision. In practice, any dsp filter is windowed to a finite length. There is a relationship between the length of the window and the uncertainty of the frequency. Yes. And by the way, I seem to remember the best joint estimator of frequency, amplitude and phase of a presumedly single sine wave, even in stationary AWGN noise, isn't even linear. Instead it works more like the formulae you use to fit a circle into three points at high amplitudes, and goes down towards expectation maximizing, longer, polynomial kernels from there. Statistical estimation theory is right at the edge of my dragon territory, so I might well be completely mistaken here too, but I actually wouldn't be surprised if the best estimator cannot even be given in closed form in general. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Negative PCA coefficients
On 2014-03-03, Linda Seltzer wrote: Today I ran a principal component analysis in Matlab and the resulting matrix contained negative coefficients. The rows are the components and the columns are the variables. I would understand positive components as weights, with the smallest weights meaning those variables do not contribute greatly to the component. If you think of your dataset as a cloud of points in high dimensional space, PCA essentially finds a rotation which will turn the cloud so that its broadest dimension coincides with the first axis, then the broadest of the remaining orthogonal directions with the second axis, and so on. What that does is a mean square best fit of the data first by a line indexed by the first dimension, then a hyperplane indexed by the first two dimensions, and so on, into a multilinear function of as many dimensions as there are explanatory variables. That's how the PCA extracts information and reduces dimensionality: it tries to explain as much of the variance in the data by reducing it into a progressively more fine grained linear model. If you drop the later factors, the earlier ones are still as good a linear estimate as can be using so many coordinates. And since the model is alwayslinear, the usual statistical thingies like Gaussian distributions, means and variances translate gracefully from the domain of the fit into the range and back again. That makes PCA pretty easy and efficient to work with. (Essentially it's just an eigen decomposition, and a perfect fit for anything quadratic, including Gaussians and the Euclidean metric.) What the minus signs mean is that the best rotation had to flip some of the axes. As somebody said, that't be pretty much the same thing as a negative correlation, except now it's formulated in a multidimensional framework. That happens in cases like the one where you have age as an independent variable and traffic accidents as a dependent one. If you did a straight regression between only the two, you'd get a negative correlation because older people participate in fewer accidents. The same happens in a multidimensional dataset with more variables on either side, too, and the analogy is actually perfect when the only thing affecting the accident rate is the age; in that case the correlation coefficient will be precisely the coefficient between the two variables in PCA, and that correlation will end up being described as the first, most explanatory axis in the result. In general each independent variable will end up affecting each dependent one in various proportions, and so while the coefficients between them still have the same interpretation they do in the context of straight linear correlations, including the sign, their precise magnitudes have to be interpreted in the context of the other ones. That is, if some other variable is more explanatory, it'll gather a higher loading and take away from the coefficient of this one, and might even flip the sign. The latter case could happen e.g. when you go from regressing age with accidents to PCA of accidents against both age and socioeconomic status: suddenly it could surface that SES explains accidents better because wealthy people have more intelligent cars, but then taking that into consideration, *of* the people having better cars, the older ones rely too much on their safety features, making them relatively conditionally more accident prone. (Such a reversion is a kind of reverse example of the well-known statistical fallacy called Simpson's Paradox.) I hope this helps at least a bit. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Iterative decomposition of an arbitrary frequency response by biquad IIR
On 2014-03-05, Ross Bencina wrote: Pretty sure that the oft-cited Knud Bank Christensen paper does LMS fit of a biquad over an arbitrary sampled frequency response. If not, then Serra and the rest of the FOF folks did implementations where formants were implemented with fitted biquads. Pretty sure that literature has to contain the relevant algorithms if used with just a single resonance. Or then just go with second order linear prediction from the impulse response, and transform back to biquad coefficient space. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] Mastering correction by FFT-based filtering followed by 1 octave or 1/10 octave equalizer
On 2014-02-28, Richard Dobson wrote: Apart from obvious true/false things such as digital clipping, it seems to me that very little in musical audio can, in any scientific deterministice way, be called either correct or incorrect. I'd argue even for clipping in some cases. Judiciously used it's just another waveshaper, with nice, crunchy qualities. Yet even the modern fad for death by compression, in the loudness wars is ostensibly a correct commercial decision. Not to mention, it's impossible to correct something like that. The trouble is, nowadays compression is not just a mostly-transparent afterthought used to fit the dynamics into a limited channel and applied in an open loop manner. It's part of the aesthetics of the music, which feeds back so that the music itself is made to be compressed. The effect is the same as with Gregorian chant: if you somehow managed to deconvolve out the whole massive reverb, you wouldn't be left with a more correct version, but something which was never intended to be played except as the excitation to the instrument that is the space. As recording purists we may not like it, but can't really argue about correctness or otherwise. Clearly you have every right to season or otherwise transform your music library in whatever way you choose, for your own listening pleasure. People have done that with hifi tone controls (and even those silly EQ controls) since amplifiers were invented. You're going to be adding a lot of unknowns and distortion sources to the mix, however, if you start with blind processing of whatever comes out of a modern studio. I can just barely agree with the idea of automatic mastering to some kind of a tonal template, but going the other way around I think mostly just leads to unpleasant surprises and a lot of crud which is difficult to avoid without going back at least to the final pre-mastering mix, or perhaps as far as the original composition. I also believe the examples Theo's given earlier lend some perceptual support to my point. Despite lots of clever processing, the end result at least to me just seems hazier than the original. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] R: Best way to do sine hard sync?
On 2014-02-26, robert bristow-johnson wrote: On 2/26/14 4:03 AM, Marco Lo Monaco wrote: yup that was the BLIT stuff, i think, so a sawtooth is the integral if this BandLimited Impulse Train (with a little DC added). Ahaha, funny! Did you set sarcasm mode = on? :))) i guess i hadn't. a bandlimited sawtooth is not the integral of a BLIT (with a touch of DC)?? When you integrate a single impulse, you get a step. When you integrate a series of two impulses, one positive and one negative, you get a rectangular pulse. Integrating a series of such pairs, suitably constructed, buys you all rectangular waveforms in bandlimited form. Triangles are integrated squares with a variable amount of DC, sawtooths are the limiting case of those where you've made one of the slopes of the triangle infinitely fast, that is the underlying rectangular waveform has very uneven duty cycle while the area under the negative and positive halfcycles is still equal (that's one way to approach a delta/an impulse). In all cases you'll probably be using a leaky integrator, so getting rid of the DC, because otherwise you risk blowing up the integrator state. -- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
