Hi Raphaël,
you might consider additional spectral descriptors such as MFCC/BFCC,
cepstrum, spectral moments like centroid, variance, skewness. For
visualization purposes where you want to map perceived sound to visuals
a more perceptually oriented scale is also useful, so use barkhausen or
mel scale instead of plain FFT. Also psychoacoustic quantities such as
sharpness, roughness, tonalness, spectral rolloff, spread, or even time
domain quantities like zero crossing rate (e.g. puredata Dissonance
Model Toolbox or timbreID package). Have a look also at the stuff from
the MTG Barcelona (http://essentia.upf.edu/documentation/) they have
mood/key detection and all kinds of things. Transient detection is also
important for mapping rhytmical events as opposed to 'steady state'
spectral characteristics.
I am very much into this topic of mapping live musical parameters to
(generative, real-time rendered) visuals, so I'm very interested in what
others have to add to this list.
Best regards,
Patric
On 07/10/2018 12:57 PM, Raphaël Ilias wrote:
Hello dear dsp-freaks!
Since a few years I'm very interested in all forms of
traditional/classic sound transcription.
I often find interesting tools, but currently I am interested to re-code
them (with Pure Data and Processing mostly, or Python), or hack them for
future artistic use.
So not to reinvent the wheel, I want first to make a quick
"state-of-the-art" about classical techniques of "making sound visual",
and learn of to implement them.
As examples of the topics i'm interested in :
- Spectral representation with Fast Fourier Transform
(linear/logarithmic frequency display & interpolation, time
downsampling, colorscaling, visual filtering / thresholding)
- Spectral representation by bands with Q-constant filters bank
- Waveform draw/plotting (especially issue of linear/logarithmic
amplitude, upsampling and interpolation, and time/downsampling
techniques (decimation, peak, rms)
- Dynamics representation : RMS/peak histograms... variance or...
- and also further analysis tools : musical scaling/filtering, note
detection, correlation of different sounds
- partials representation (sinus tracking)
Of course I have tested a few tools, like Spear, Adobe Audition... and a
few other command line tools I don't remember the names (but most of
them were buggy or Windows-based :§ )
So I wonder if you have other suggestions of software (preferably
open-source, hackable or freeware), pseudo-code or academic articles on
techniques (I have already partially read : Curtis Roads's
L'Audionumérique - Musique et Informatique, Miller Puckette's Theory and
techniques of electronic music book, Andy Farnell's Designing Sound and
a bit of The Scientist and Engineer's DSP Guide by Steven W. Smith) - I
read French and English.
I know this is a very large question !
Anyway I'll welcome any small suggestion from people more advised than
Google Search !
thanks by advance,
Raphaël Ilias
___________________________________
*phae.fr <http://phae.fr>*
17 places des Halles - Ingrandes
49123 Ingrandes Le Fresne s/ Loire
phae.il...@gmail.com <mailto:phae.il...@gmail.com> / +33 (0) 6 04 45 79 78
___________
_______________________________________________
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp
_______________________________________________
dupswapdrop: music-dsp mailing list
music-dsp@music.columbia.edu
https://lists.columbia.edu/mailman/listinfo/music-dsp
