Hi Raphaël,

you might consider additional spectral descriptors such as MFCC/BFCC, cepstrum, spectral moments like centroid, variance, skewness. For visualization purposes where you want to map perceived sound to visuals a more perceptually oriented scale is also useful, so use barkhausen or mel scale instead of plain FFT. Also psychoacoustic quantities such as sharpness, roughness, tonalness, spectral rolloff, spread, or even time domain quantities like zero crossing rate (e.g. puredata Dissonance Model Toolbox or timbreID package). Have a look also at the stuff from the MTG Barcelona (http://essentia.upf.edu/documentation/) they have mood/key detection and all kinds of things. Transient detection is also important for mapping rhytmical events as opposed to 'steady state' spectral characteristics.

I am very much into this topic of mapping live musical parameters to (generative, real-time rendered) visuals, so I'm very interested in what others have to add to this list.

Best regards,

On 07/10/2018 12:57 PM, Raphaël Ilias wrote:
Hello dear dsp-freaks!

Since a few years I'm very interested in all forms of traditional/classic sound transcription. I often find interesting tools, but currently I am interested to re-code them (with Pure Data and Processing mostly, or Python), or hack them for future artistic use.

So not to reinvent the wheel, I want first to make a quick "state-of-the-art" about classical techniques of "making sound visual", and learn of to implement them.

As examples of the topics i'm interested in :
- Spectral representation with Fast Fourier Transform (linear/logarithmic frequency display & interpolation, time downsampling, colorscaling, visual filtering / thresholding)
- Spectral representation by bands with Q-constant filters bank
- Waveform draw/plotting (especially issue of linear/logarithmic amplitude, upsampling and interpolation, and time/downsampling techniques (decimation, peak, rms)
- Dynamics representation : RMS/peak histograms... variance or...
- and also further analysis tools : musical scaling/filtering, note detection, correlation of different sounds
- partials representation (sinus tracking)

Of course I have tested a few tools, like Spear, Adobe Audition... and a few other command line tools I don't remember the names (but most of them were buggy or Windows-based :§ )

So I wonder if you have other suggestions of software (preferably open-source, hackable or freeware), pseudo-code or academic articles on techniques (I have already partially read : Curtis Roads's L'Audionumérique - Musique et Informatique, Miller Puckette's Theory and techniques of electronic music book, Andy Farnell's Designing Sound and a bit of The Scientist and Engineer's DSP Guide by Steven W. Smith) - I read French and English.

I know this is a very large question !
Anyway I'll welcome any small suggestion from people more advised than Google Search !

thanks by advance,

Raphaël Ilias
*phae.fr <http://phae.fr>*
      17 places des Halles - Ingrandes
      49123 Ingrandes Le Fresne s/ Loire
phae.il...@gmail.com <mailto:phae.il...@gmail.com> / +33 (0) 6 04 45 79 78

dupswapdrop: music-dsp mailing list

dupswapdrop: music-dsp mailing list

Reply via email to