Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
ok I now I tried a crude and quick multiresolution FFT analysis at log 2 basis and it seems to work. However my partial and transient tracking does not work naymore on the higher bands since there is too much modulation from the windows now, but it seems that in general this is the direction to go, with some refinements- Am 04.11.2018 um 14:55 schrieb gm: Maybe you could make the analysis with a filterbank, and do the resynthesis with FFT? Years ago I made such a synth based on "analog" Fourier Transforms, (the signal is modulated and rotated down to 0 Frequency and that frequencies around DC are lowpass filtered depending on the bandwitdh yoe need) it it did the inverse transform by modulated sines, but sounded rather reverberant though since the analysis has a rather long IR on the lowest bands. I used 72 Bands based on a Bark scale. Maybe I should try something like that again... However it's absolutely not clear to me how to go from 72 (or what ever) broadband bands to FFT frequencies and bands? Maybe you could just use the amplitude spectrum from the bank and the frequencies from an FFT. Any objections? The frequencies would be wrong before transients, but the benefit is that you get a good spectral envelope and you can have a really sharp transient detection. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
Maybe you could make the analysis with a filterbank, and do the resynthesis with FFT? Years ago I made such a synth based on "analog" Fourier Transforms, (the signal is modulated and rotated down to 0 Frequency and that frequencies around DC are lowpass filtered depending on the bandwitdh yoe need) it it did the inverse transform by modulated sines, but sounded rather reverberant though since the analysis has a rather long IR on the lowest bands. I used 72 Bands based on a Bark scale. Maybe I should try something like that again... However it's absolutely not clear to me how to go from 72 (or what ever) broadband bands to FFT frequencies and bands? Maybe you could just use the amplitude spectrum from the bank and the frequencies from an FFT. Any objections? The frequencies would be wrong before transients, but the benefit is that you get a good spectral envelope and you can have a really sharp transient detection. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
Am 04.11.2018 um 03:03 schrieb Theo Verelst: It might help to understand why in this case you'd chose for the computation according to a IFFT scheme for synthesis. Is it for complimentary processing steps, efficiency, because you have data that fits the practical method in terms of granularity, theoretical interest, example software, some sort of juxtaposition of alternatives, or maybe a well known engineering example where this appears logical ? Originall it was a UI study and proof of concept engine for a granular cloud synth, then I replaced the grain engine with FFT because I had some misconceptions: - I thought you could get the granular cloud sound by randomizing the FFT, but it sounds very different - I thought you would get a better time resolution by using a smaller hop size in the analysis but that is not the case. You would need a smaller window, but then the frequency analysis is bad. - Also I thought you could use a larger FFT for ananlysis and compress them into a smaller ones for resynthesis and benefit from both, but it's the other way it seems, you get the disadvantages from both, at least it didnt work so well here and sounded like the smaller FFT exept that transients are blurred more. - Finally I thought you could reconstruct the waveshape on transients by using th original phase, but it doesn't seem to make a big audible difference and my transient detection doesn't work so well cause the transients are already spread in time by the analysis so this reonstructed waveshape doesn't come in at he right moment. Still I think that somehow it could be possible with the FFT, using a time domain pitch shift, but doing it in that order iFFT -> time domain shift seems to be more complicated compared to the other way : td shift -> FFT, which was my first implementation I think for the other way you always need a variing number of FFTs per time? Also I think the resolution issue will be the same. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
It's a complicated subject when you fill in all the boundary conditions properly, isn't it? Lots of frequency considerations look a bit alike but aren't mathematically equivalent. The human (and animal I suppose) hearing is very sensitive to a lot of these issues in all kinds of convoluted combinations, so finding a heuristic for some problem is easily going to be food for yet another theory getting disproven. Some theories (like Fourier) can only be ultimately, mathematically used to close to infinite accuracy success when applied properly, this is where usually the problem lies. It might help to understand why in this case you'd chose for the computation according to a IFFT scheme for synthesis. Is it for complimentary processing steps, efficiency, because you have data that fits the practical method in terms of granularity, theoretical interest, example software, some sort of juxtaposition of alternatives, or maybe a well known engineering example where this appears logical ? T.V. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
as I wrote before, it depends on the phase angle whether or not the amplitudes add up when you add the numbers with opposite phase, and equal amplitude, the amplitudes cancel, with same phase they add up, with pi radians it is sqrt(2), ect, but I made a quick hack test and it doesn't sound as good as the full spectrum, something in quality is missing I am still investigating and thinking about this so I am not sure what I am missing, I mean there exists a halfsized spectrum that would represent the signal perfectly (or rather, two of them would) maybed it has to do with the fact that you need two successive spectra to represent he same information but I dont really see the effect of that other than it has a better time resolution Am 03.11.2018 um 10:48 schrieb Ross Bencina: [resending, I think I accidentally replied off-list] On 1/11/2018 5:00 AM, gm wrote: > My question rephrased: > Lets assume a spectrum of size N, can you create a meaningfull spectrum of size N/2 > by simply adding every other bin together? > > Neglecting the artefacts of the forward transform, lets say an artificial spectrum > (or a spectrum after peak picking that discards the region around the peaks) > > Lets say two sinusoids in two adjacent bins, will summing them into a single bin of a half sized spectrum > make sense and represent them adequately? > In my limited understanding, yes, but I am not sure, and would like to know why not > if that is not the case. You can analyze this by looking at the definition of the short-time discrete Fourier transform. (Or the corresponding C code for a DFT). Each spectral bin is the sum of samples in the windowed signal multiplied by a complex exponential. Off the top of my head, assuming a rectangular window, I think you'll find that dropping every second bin in the length N spectrum gives you the equivalent of the bin-wise sum of two length N/2 DFTs computed with hop size N/2. Summing adjacent bins would do something different. You could work it out by taking the definition of the DFT and doing some algebra. I think you'd get a spectrum with double the amplitude, frequency shifted by half the bin-spacing. (i.e. the average of the two bin's center frequencies). Ross. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
[resending, I think I accidentally replied off-list] On 1/11/2018 5:00 AM, gm wrote: > My question rephrased: > Lets assume a spectrum of size N, can you create a meaningfull spectrum of size N/2 > by simply adding every other bin together? > > Neglecting the artefacts of the forward transform, lets say an artificial spectrum > (or a spectrum after peak picking that discards the region around the peaks) > > Lets say two sinusoids in two adjacent bins, will summing them into a single bin of a half sized spectrum > make sense and represent them adequately? > In my limited understanding, yes, but I am not sure, and would like to know why not > if that is not the case. You can analyze this by looking at the definition of the short-time discrete Fourier transform. (Or the corresponding C code for a DFT). Each spectral bin is the sum of samples in the windowed signal multiplied by a complex exponential. Off the top of my head, assuming a rectangular window, I think you'll find that dropping every second bin in the length N spectrum gives you the equivalent of the bin-wise sum of two length N/2 DFTs computed with hop size N/2. Summing adjacent bins would do something different. You could work it out by taking the definition of the DFT and doing some algebra. I think you'd get a spectrum with double the amplitude, frequency shifted by half the bin-spacing. (i.e. the average of the two bin's center frequencies). Ross. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
personally, i believe this issue is resolved with the choice of the window size (or "frame length", not to be conflated with "frame hop") that goes into the FFT. for a sufficiently large window, the two sinusoids will appear as two separate peaks in the FFT result. for a sufficiently small window, it will be a single sinusoid, but the amplitude will vary in adjacent windows. that's my spin on the issue. r b-j Original Message ------------ Subject: Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis? From: "gm" Date: Sat, November 3, 2018 2:12 am To: music-dsp@music.columbia.edu -- > > And I think you can model them simply by adding their phasors/bins/numbers... > > for opposite angles they will cancel, for the same angle they will be > amplified > > so the model is correct at the center of the window, but it models just > an instance in time and spreads this instance > > in this way of thinking you basically consider the dft a sine bank > oscillator > > > Am 03.11.2018 um 05:59 schrieb Ross Bencina: >> On 3/11/2018 3:41 AM, Ethan Fenn wrote: >>> No length of FFT will distinguish between a mixture of these sine >>> waves and a single amplitude-modulated one, because they're >>> mathematically identitical! Specifically: >>> >>> sin(440t) + sin(441t) = 2*cos(0.5t)*sin(440.5t) >>> >>> So the question isn't whether an algorithm can distinguish between >>> them but rather which one of these two interpretations it should >>> pick. And I would say in most audio applications the best answer is >>> that it should pick the same interpretation that the human hearing >>> system would. In this example it's clearly the right-hand side. In >>> the case of a large separation (e.g. 440Hz and 550Hz, a major third) >>> it's clearly the left-hand side. And somewhere in between I guess it >>> must be a toss-up. >> >> I guess you could model both simultaneously, with some kind of >> probability weighting. >> >> Ross. > > � � � -- r b-j� � � � � � � � � � � � �r...@audioimagination.com "Imagination is more important than knowledge." � � � � ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
An I think you can model them simply by adding their phasors/bins/numbers... for opposite angles they will cancel, for the same angle they will be amplified so the model is correct at the center of the window, but it models just an instance in time and spreads this instance in this way of thinking you basically consider the dft a sine bank oscillator Am 03.11.2018 um 05:59 schrieb Ross Bencina: On 3/11/2018 3:41 AM, Ethan Fenn wrote: No length of FFT will distinguish between a mixture of these sine waves and a single amplitude-modulated one, because they're mathematically identitical! Specifically: sin(440t) + sin(441t) = 2*cos(0.5t)*sin(440.5t) So the question isn't whether an algorithm can distinguish between them but rather which one of these two interpretations it should pick. And I would say in most audio applications the best answer is that it should pick the same interpretation that the human hearing system would. In this example it's clearly the right-hand side. In the case of a large separation (e.g. 440Hz and 550Hz, a major third) it's clearly the left-hand side. And somewhere in between I guess it must be a toss-up. I guess you could model both simultaneously, with some kind of probability weighting. Ross. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
On 3/11/2018 3:41 AM, Ethan Fenn wrote: No length of FFT will distinguish between a mixture of these sine waves and a single amplitude-modulated one, because they're mathematically identitical! Specifically: sin(440t) + sin(441t) = 2*cos(0.5t)*sin(440.5t) So the question isn't whether an algorithm can distinguish between them but rather which one of these two interpretations it should pick. And I would say in most audio applications the best answer is that it should pick the same interpretation that the human hearing system would. In this example it's clearly the right-hand side. In the case of a large separation (e.g. 440Hz and 550Hz, a major third) it's clearly the left-hand side. And somewhere in between I guess it must be a toss-up. I guess you could model both simultaneously, with some kind of probability weighting. Ross. ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
> > In any case, most signals are not sums of stationary sinusoids. And since > signals are typically buried in noise, or superimposed on top of each > other, so the problem is not well posed. For two very simple examples: > consider two stable sine waves at 440Hz and 441Hz -- you will need a very > long FFT to distinguish this from a single amplitude-modulated sine wave? > or consider a sine wave plus white noise -- the accuracy of frequency and > phase recovery will depend on how much input you have to work with. This is an interesting thing to think about in any sort of spectral modeling. No length of FFT will distinguish between a mixture of these sine waves and a single amplitude-modulated one, because they're mathematically identitical! Specifically: sin(440t) + sin(441t) = 2*cos(0.5t)*sin(440.5t) So the question isn't whether an algorithm can distinguish between them but rather which one of these two interpretations it should pick. And I would say in most audio applications the best answer is that it should pick the same interpretation that the human hearing system would. In this example it's clearly the right-hand side. In the case of a large separation (e.g. 440Hz and 550Hz, a major third) it's clearly the left-hand side. And somewhere in between I guess it must be a toss-up. -Ethan On Wed, Oct 31, 2018 at 2:00 PM, gm wrote: > Thanks for your time > > My question rephrased: > Lets assume a spectrum of size N, can you create a meaningfull spectrum of > size N/2 > by simply adding every other bin together? > > Neglecting the artefacts of the forward transform, lets say an artificial > spectrum > (or a spectrum after peak picking that discards the region around the > peaks) > > Lets say two sinusoids in two adjacent bins, will summing them into a > single bin of a half sized spectrum > make sense and represent them adequately? > In my limited understanding, yes, but I am not sure, and would like to > know why not > if that is not the case. > > > > > > I'm not sure what "compress" means in this context, nor am I sure what >> "fall together" means. But here's some points to note: >> >> A steady state sine wave in the time domain will be transformed by a >> short-time fourier transform into a spectral peak, convolved (in the >> frequency domain) by the spectrum of the analysis envelope. If you know >> that all of your inputs are sine waves, then you can perform "spectral peak >> picking" (AKA MQ analysis) and reduce your signal to a list of sine waves >> and their frequencies and phases -- this is the sinusoidal component of >> Serra's SMS (explained in the pdf linked above). >> >> Note that since a sinusoid ends up placing non-zero values in every FFT >> bin, you'd need to account for that in your spectral estimation, which >> basic MQ does not -- hence it does not perfectly estimate the sinusoids. >> >> In any case, most signals are not sums of stationary sinusoids. And since >> signals are typically buried in noise, or superimposed on top of each >> other, so the problem is not well posed. For two very simple examples: >> consider two stable sine waves at 440Hz and 441Hz -- you will need a very >> long FFT to distinguish this from a single amplitude-modulated sine wave? >> or consider a sine wave plus white noise -- the accuracy of frequency and >> phase recovery will depend on how much input you have to work with. >> >> I think by "compression" you mean "represent sparsely" (i.e. with some >> reduced representation.) The spectral modeling approach is to "model" the >> signal by assuming it has some particular structure (e.g. sinusoids+noise, >> or sinusoids+transients+noise) and then work out how to extract this >> structure from the signal (or to reassemble it for synthesis). >> >> An alternative (more mathematical) approach is to simply assume that the >> signal is sparse in some (unknown) domain. It turns out that if your signal >> is sparse, you can apply a constrained random dimensionality reduction to >> the signal and not lose any information. This is the field of compressed >> sensing. Note that in this case, you haven't recovered any structure. >> >> Ross >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > ___ > dupswapdrop: music-dsp mailing list > music-dsp@music.columbia.edu > https://lists.columbia.edu/mailman/listinfo/music-dsp > > ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
Thanks for your time My question rephrased: Lets assume a spectrum of size N, can you create a meaningfull spectrum of size N/2 by simply adding every other bin together? Neglecting the artefacts of the forward transform, lets say an artificial spectrum (or a spectrum after peak picking that discards the region around the peaks) Lets say two sinusoids in two adjacent bins, will summing them into a single bin of a half sized spectrum make sense and represent them adequately? In my limited understanding, yes, but I am not sure, and would like to know why not if that is not the case. I'm not sure what "compress" means in this context, nor am I sure what "fall together" means. But here's some points to note: A steady state sine wave in the time domain will be transformed by a short-time fourier transform into a spectral peak, convolved (in the frequency domain) by the spectrum of the analysis envelope. If you know that all of your inputs are sine waves, then you can perform "spectral peak picking" (AKA MQ analysis) and reduce your signal to a list of sine waves and their frequencies and phases -- this is the sinusoidal component of Serra's SMS (explained in the pdf linked above). Note that since a sinusoid ends up placing non-zero values in every FFT bin, you'd need to account for that in your spectral estimation, which basic MQ does not -- hence it does not perfectly estimate the sinusoids. In any case, most signals are not sums of stationary sinusoids. And since signals are typically buried in noise, or superimposed on top of each other, so the problem is not well posed. For two very simple examples: consider two stable sine waves at 440Hz and 441Hz -- you will need a very long FFT to distinguish this from a single amplitude-modulated sine wave? or consider a sine wave plus white noise -- the accuracy of frequency and phase recovery will depend on how much input you have to work with. I think by "compression" you mean "represent sparsely" (i.e. with some reduced representation.) The spectral modeling approach is to "model" the signal by assuming it has some particular structure (e.g. sinusoids+noise, or sinusoids+transients+noise) and then work out how to extract this structure from the signal (or to reassemble it for synthesis). An alternative (more mathematical) approach is to simply assume that the signal is sparse in some (unknown) domain. It turns out that if your signal is sparse, you can apply a constrained random dimensionality reduction to the signal and not lose any information. This is the field of compressed sensing. Note that in this case, you haven't recovered any structure. Ross ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
Hi, Sorry, late to the party and unable to read the backlog, but: The "FFT^-1" technique that Robert mentions is from a paper by Rodet and Depalle that I can't find right now. It's widely cited in the literature as "FFT^-1" That paper only deals with steady-state sinusoids however. It won't accurately deal with transients or glides. There has been more recent work on spectral-domain synthesis and I'm fairly sure that some techniques have found their way into some quite famous commercial products. Bonada, J.; Loscos, A.; Cano, P.; Serra, X.; Kenmochi, H. (2001). "Spectral Approach to the Modeling of the Singing Voice". In Proc. of the 111th AES Convention. > My goal is to resynthesize arbitary noises. In that case you need to think about how an FFT represents "arbitrary noises". One approach is to split the signal into sinusoids + noise (a.k.a. spectral modeling synthesis). https://en.wikipedia.org/wiki/Spectral_modeling_synthesis It is worth reviewing Xavier Serra's PhD thesis for the basics (what was already established in the late 1980s.) http://mtg.upf.edu/content/serra-PhD-thesis Here's the PDF: https://repositori.upf.edu/bitstream/handle/10230/34072/Serra_PhDthesis.pdf?sequence=1&isAllowed=y There was a bunch of in the early 90's on real-time additive synthesis at CNMAT, e.g. https://quod.lib.umich.edu/i/icmc/bbp2372.1995.091/1/--bring-your-own-control-to-additive-synthesis?page=root;size=150;view=text Of course there is a ton of more recent work. You could do worse than looking at the papers of Xavier Serra and Jordi Bonada: http://mtg.upf.edu/research/publications On 31/10/2018 1:35 PM, gm wrote: But back to my question, I am serious, could you compress a spectrum by just adding the bins that fall together? I'm not sure what "compress" means in this context, nor am I sure what "fall together" means. But here's some points to note: A steady state sine wave in the time domain will be transformed by a short-time fourier transform into a spectral peak, convolved (in the frequency domain) by the spectrum of the analysis envelope. If you know that all of your inputs are sine waves, then you can perform "spectral peak picking" (AKA MQ analysis) and reduce your signal to a list of sine waves and their frequencies and phases -- this is the sinusoidal component of Serra's SMS (explained in the pdf linked above). Note that since a sinusoid ends up placing non-zero values in every FFT bin, you'd need to account for that in your spectral estimation, which basic MQ does not -- hence it does not perfectly estimate the sinusoids. In any case, most signals are not sums of stationary sinusoids. And since signals are typically buried in noise, or superimposed on top of each other, so the problem is not well posed. For two very simple examples: consider two stable sine waves at 440Hz and 441Hz -- you will need a very long FFT to distinguish this from a single amplitude-modulated sine wave? or consider a sine wave plus white noise -- the accuracy of frequency and phase recovery will depend on how much input you have to work with. I think by "compression" you mean "represent sparsely" (i.e. with some reduced representation.) The spectral modeling approach is to "model" the signal by assuming it has some particular structure (e.g. sinusoids+noise, or sinusoids+transients+noise) and then work out how to extract this structure from the signal (or to reassemble it for synthesis). An alternative (more mathematical) approach is to simply assume that the signal is sparse in some (unknown) domain. It turns out that if your signal is sparse, you can apply a constrained random dimensionality reduction to the signal and not lose any information. This is the field of compressed sensing. Note that in this case, you haven't recovered any structure. Ross ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
Thanks for your answer, it's much apreciated. My goal is to resynthesize arbitary noises. If you do that with wavetables you end up with a pitch of 20 Hz, hence the FFT. My product idea was rubbish though and your post confirms that. For your interest, I recently invented "noisetable synthesis" - take any sound, randomzie the phases or circularly convolve with a long chunk of white noise, you get a long seemless loop of arbitrary spectra. Good for synthetic cymbals and the like. Take any sound and stretch it into an endless loop. Nice for ambient music, too. But back to my question, I am serious, could you compress a spectrum by just adding the bins that fall together? In my understanding, yes, but I am totally not sure. My "understanding" might be totally wrong. Am 31.10.2018 um 03:14 schrieb robert bristow-johnson: Original Message Subject: [music-dsp] two fundamental questions Re: FFT for realtime synthesis? From: "gm" Date: Tue, October 30, 2018 8:17 pm To: music-dsp@music.columbia.edu -- > > > Am 30.10.2018 um 16:30 schrieb gm: >> -Compress the peaks (without the surrounding regions) and noise into >> smaller spectra. >> (but how? - can you simply add those that fall into the same bins?) > snip... >> I am curious about the spectrum compression part, would this work and >> if not why not? > > > That's a very fundamental DSP question and I am surprised no one wants to > answer. > Probably either no one did read the post or it was too stupid. > > My layman's answer is YES OF COURSE, WHAT ARE YOU THINKING but what do I > know, I am all self taught > and even taught myself how to read and write (not kidding) so I am not sure. mr. g, i am not exactly "self-taught". i learned my essential math and electrical engineering in a university setting. but most of these audio techniques are things i have learned since skool and are mostly self-taught. perhaps a quarter century ago i saw a paper about doing something like real-time additive synthesis by use of the inverse FFT, and while i don't remember the details, i remember thinking to myself that it would be very difficult doing synthesis with an iFFT for several reasons. if the FFT is too small (like N=2048) the frequency resolution for each bin is crappy for notes below, say, middle C. if you're not just plopping a "lobe" of template data down onto the iFFT buffer, then you have to find some way to interpolate to move it's position a fraction of a bin. if your FFT is too large, you have a long time to wait between MIDI NoteOn going in and sound coming out. but a big iFFT is what you need for decent frequency resolution at the lower pitches. if you're synthesizing notes that have harmonic partials, then a periodic or quasi-periodic synthesis technique in the time-domain is likely your simplest bet. i am a partisan of Wavetable Synthesis because, with the one restriction that the partials are nearly or perfectly harmonic, wavetable synthesis is general. nearly any existing acoustic instrument can be convincingly synthesized with wavetable synthesis. as long as you have a good pitch detector, it's not hard to analyze a given note and break it into wavetables. but you need to know the period (to a sub-sample precision) and you need to be able to interpolate convincingly between samples. BOTH in the analysis phase and in the synthesis phase. wavetable synthesis works well for notes with an attack (what you might use a sampling and looping "synthesizer" to do), additive synthesis (as long as the partials are nearly harmonic), classic waveforms (like saw, square, PWM, sync-saw, sync-square) and you can cross-fade (or "interpolate") between wavetables to get the ability to change the sound. you need a good tracking pitch detector in the analysis phase. but, for fun, there are other synthesis techniques from the olden days. Classic Subtractive synthesis, Karplus-Strong, Physical Modeling, Non-linear Waveshaping, Geometric Summation formula, IIR impulse response, comb-filter impulse response, "FOF" (something like overlapping wavelets or grains). but they're all time domain. -- r b-j r...@audioimagination.com "Imagination is more important than knowledge." ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
Re: [music-dsp] two fundamental questions Re: FFT for realtime synthesis?
Original Message Subject: [music-dsp] two fundamental questions Re: FFT for realtime synthesis? From: "gm" Date: Tue, October 30, 2018 8:17 pm To: music-dsp@music.columbia.edu -- > > > Am 30.10.2018 um 16:30 schrieb gm: >> -Compress the peaks (without the surrounding regions) and noise into >> smaller spectra. >> (but how? - can you simply add those that fall into the same bins?) > snip... >> I am curious about the spectrum compression part, would this work and >> if not why not? > > > That's a very fundamental DSP question and I am surprised no one wants to > answer. > Probably either no one did read the post or it was too stupid. > > My layman's answer is YES OF COURSE, WHAT ARE YOU THINKING but what do I > know, I am all self taught > and even taught myself how to read and write (not kidding) so I am not sure. mr. g, i am not exactly "self-taught".� i learned my essential math and electrical engineering in a university setting.� but most of these audio techniques are things i have learned since skool and are mostly self-taught. perhaps a quarter century ago i saw a paper about doing something like real-time additive synthesis by use of the inverse FFT, and while i don't remember the details, i remember thinking to myself that it would be very difficult doing synthesis with an iFFT for several reasons.� if the FFT is too small (like N=2048) the frequency resolution for each bin is crappy for notes below, say, middle C.� �if you're not just plopping a "lobe" of template data down onto the iFFT buffer, then you have to find some way to interpolate to move it's position a fraction of a bin.� if your FFT is too large, you have a long time to wait between MIDI NoteOn going in and sound coming out.� but a big iFFT is what you need for decent frequency resolution at the lower pitches. if you're synthesizing notes that have harmonic partials, then a periodic or quasi-periodic synthesis technique in the time-domain is likely your simplest bet.� i am a partisan of Wavetable Synthesis because, with the one restriction that the partials are nearly or perfectly harmonic, wavetable synthesis is general.� nearly any existing acoustic instrument can be convincingly synthesized with wavetable synthesis.� as long as you have a good pitch detector, it's not hard to analyze a given note and break it into wavetables.� but you need to know the period (to a sub-sample precision) and you need to be able to interpolate convincingly between samples.� BOTH in the analysis phase and in the synthesis phase.� wavetable synthesis works well for notes with an attack (what you might use a sampling and looping "synthesizer" to do), additive synthesis (as long as the partials are nearly harmonic), classic waveforms (like saw, square, PWM, sync-saw, sync-square) and you can cross-fade (or "interpolate") between wavetables to get the ability to change the sound.� you need a good tracking pitch detector in the analysis phase. but, for fun, there are other synthesis techniques from the olden days.� Classic Subtractive synthesis, Karplus-Strong, Physical Modeling, Non-linear Waveshaping, Geometric Summation formula, IIR impulse response, comb-filter impulse response, "FOF" (something like overlapping wavelets or grains).� but they're all time domain. � -- r b-j� � � � � � � � � � � � �r...@audioimagination.com "Imagination is more important than knowledge." � � � � ___ dupswapdrop: music-dsp mailing list music-dsp@music.columbia.edu https://lists.columbia.edu/mailman/listinfo/music-dsp
