Hi Antonio,
On 05/04/15 15:51, Antonio Roberts wrote:
I've been studying the work of Neil Harbisson for awhile and I'm
looking to try and replicate in part the functionality of his eyeborg
using Pure Data. I have already built a simple patch to convert pixels
to sound but now I want to expand on it. Here's my initial research:
http://www.hellocatfood.com/sonification-studiespixel-waves/
"I predict the development of an image to audio sequencer in the near
future."
I made a painting program in gridflow years ago, had some very basic
sonification - gridflow also had a "how to play a car" example
converting a photo by scanlines into audio with FFT.
https://archive.org/details/ClaudiusMaximus_-_Emulsion
https://archive.org/details/ClaudiusMaximus_-_DohPaintII_Session_3
https://archive.org/details/ClaudiusMaximus_-_DohPaintII_Session_2
Colours don't directly relate to sound and so Harbisson and others
must use a scale to assign colours to sounds. Some initial research
brought these up:
http://blogs.plos.org/neuroanthropology/files/2014/03/Harbisson-The-Sound-of-Colors-TED.jpg
http://www.bbc.co.uk/news/magazine-16681630
http://www.lunarplanner.com/Harmonics/planetary-harmonics.html
http://www.flutopedia.com/sound_color.htm
The last one, in the section "Converting RGB Colors to Color
Frequencies", makes a good point - monochromatic wavelength to RGB works
fine, but RGB to wavelength is in general impossible, as colours are
spectral power distributions over continuous wavelengths. Once it's
flattened to RGB, too much information is lost to be able to recreate
the original spectrum (aka metamerism, where 2 different spectrum give
same colour sensation).
Recently I copy/pasted some colour-related stuff from Wikipedia into a
booklet (second link is layed out for printing, first one is better for
screen reading, both have the same content):
http://mathr.co.uk/misc/2015-04-04_colour.pdf
http://mathr.co.uk/misc/2015-04-04_colour_booklet.pdf
The approaches you linked seem to convert wavelengths to colour in a
straightforward way, but for converting from colour to sound I think a
different approach would be better (and indeed needed) - something more
akin to the Munsell perceptual colour system:
https://en.wikipedia.org/wiki/Munsell_color_system
a grossly simplified version would be HSV, which I used in the attached
- a bank of vcf~ fed by noise~, mapping hue to scale note, value to
loudness and pitch, and saturation to filter Q - so that white would
give high noise, grey gives a lower pitch noise, black gives silence,
fully-saturated dark green gives a low note, bright green gives a high
pitch at the same scale note. the filters glitch when the hue jumps
from 0 red (orange side) to 1 red (purple side), could be fixed with
some cleverness probably.
Can anyone think of a way to translate this into pd? In the end I
would like to be able to display a block of colour on screen and have
that generate a specific note.
Any help is appreciated.
Antonio
Claude
--
http://mathr.co.uk
#N canvas 1 51 843 347 10;
#X obj 19 151 gemwin;
#X msg 18 95 create \, 1;
#X msg 25 122 0 \, destroy;
#X obj 17 175 gemhead;
#X obj 17 197 color;
#X obj 17 219 square 3;
#X msg 17 22 \; pd dsp 1;
#X msg 17 57 \; pd dsp 0;
#X obj 136 25 vsl 15 128 0 1 0 0 empty empty R 0 -9 0 10 -262144 -1
-1 2200 1;
#X obj 156 25 vsl 15 128 0 1 0 0 empty empty G 0 -9 0 10 -262144 -1
-1 2000 1;
#X obj 176 25 vsl 15 128 0 1 0 0 empty empty B 0 -9 0 10 -262144 -1
-1 1900 1;
#X obj 132 199 pack f f f;
#X obj 141 176 t b f;
#X obj 185 176 t b f;
#X obj 132 224 rgb2hsv;
#X obj 320 6 unpack f f f;
#X obj 207 25 vsl 15 128 0 1 0 0 empty empty H 0 -9 0 10 -262144 -1
-1 706 1;
#X obj 227 25 vsl 15 128 0 1 0 0 empty empty S 0 -9 0 10 -262144 -1
-1 1732 1;
#X obj 247 25 vsl 15 128 0 1 0 0 empty empty V 0 -9 0 10 -262144 -1
-1 2200 1;
#X obj 320 39 * 12;
#X obj 320 63 mtof;
#X obj 435 57 + 24;
#X obj 278 127 noise~;
#X obj 435 79 mtof;
#X obj 338 169 vcf~;
#X obj 378 169 vcf~;
#X obj 418 169 vcf~;
#X obj 458 169 vcf~;
#X obj 498 169 vcf~;
#X obj 538 169 vcf~;
#X obj 578 169 vcf~;
#X obj 618 169 vcf~;
#X obj 658 169 vcf~;
#X obj 698 169 vcf~;
#X obj 738 169 vcf~;
#X obj 778 169 vcf~;
#X obj 350 140 * 2;
#X obj 278 153 *~ 0;
#X obj 390 140 * 2;
#X obj 430 140 * 2;
#X obj 470 140 * 2;
#X obj 510 140 * 2;
#X obj 550 140 * 2;
#X obj 590 140 * 2;
#X obj 630 140 * 2;
#X obj 670 140 * 2;
#X obj 710 140 * 2;
#X obj 750 140 * 2;
#X obj 790 140 * 2;
#X obj 268 258 dac~;
#X obj 368 95 * 0.707;
#X obj 368 71 pow;
#X obj 369 27 t b f;
#X obj 284 81 sqrt;
#X obj 338 191 /~ 1;
#X obj 378 191 /~ 1;
#X obj 418 191 /~ 1;
#X obj 458 191 /~ 1;
#X obj 498 191 /~ 1;
#X obj 538 191 /~ 1;
#X obj 578 191 /~ 1;
#X obj 618 191 /~ 1;
#X obj 658 191 /~ 1;
#X obj 698 191 /~ 1;
#X obj 738 191 /~ 1;
#X obj 778 191 /~ 1;
#X obj 279 230 *~ 100;
#X obj 284 103 *;
#X obj 279 208 vcf~ 5;
#X obj 315 103 pow 0.25;
#X floatatom 322 279 5 0 0 0 - - -, f 5;
#X obj 322 257 env~ 16384;
#X obj 435 35 * 84;
#X msg 368 49 10000;
#X obj 396 69 sqrt;
#X connect 1 0 0 0;
#X connect 2 0 0 0;
#X connect 3 0 4 0;
#X connect 4 0 5 0;
#X connect 8 0 11 0;
#X connect 9 0 12 0;
#X connect 10 0 13 0;
#X connect 11 0 4 1;
#X connect 11 0 14 0;
#X connect 12 0 11 0;
#X connect 12 1 11 1;
#X connect 13 0 11 0;
#X connect 13 1 11 2;
#X connect 14 0 15 0;
#X connect 15 0 16 0;
#X connect 15 0 19 0;
#X connect 15 1 17 0;
#X connect 15 1 52 0;
#X connect 15 2 18 0;
#X connect 15 2 69 0;
#X connect 15 2 72 0;
#X connect 19 0 20 0;
#X connect 20 0 36 0;
#X connect 21 0 23 0;
#X connect 22 0 37 0;
#X connect 23 0 68 1;
#X connect 24 0 54 0;
#X connect 25 0 55 0;
#X connect 26 0 56 0;
#X connect 27 0 57 0;
#X connect 28 0 58 0;
#X connect 29 0 59 0;
#X connect 30 0 60 0;
#X connect 31 0 61 0;
#X connect 32 0 62 0;
#X connect 33 0 63 0;
#X connect 34 0 64 0;
#X connect 35 0 65 0;
#X connect 36 0 24 1;
#X connect 36 0 38 0;
#X connect 36 0 54 1;
#X connect 37 0 24 0;
#X connect 37 0 25 0;
#X connect 37 0 26 0;
#X connect 37 0 27 0;
#X connect 37 0 28 0;
#X connect 37 0 29 0;
#X connect 37 0 30 0;
#X connect 37 0 31 0;
#X connect 37 0 32 0;
#X connect 37 0 33 0;
#X connect 37 0 34 0;
#X connect 37 0 35 0;
#X connect 38 0 25 1;
#X connect 38 0 39 0;
#X connect 38 0 55 1;
#X connect 39 0 26 1;
#X connect 39 0 40 0;
#X connect 39 0 56 1;
#X connect 40 0 27 1;
#X connect 40 0 41 0;
#X connect 40 0 57 1;
#X connect 41 0 28 1;
#X connect 41 0 42 0;
#X connect 41 0 58 1;
#X connect 42 0 29 1;
#X connect 42 0 43 0;
#X connect 42 0 59 1;
#X connect 43 0 30 1;
#X connect 43 0 44 0;
#X connect 43 0 60 1;
#X connect 44 0 31 1;
#X connect 44 0 45 0;
#X connect 44 0 61 1;
#X connect 45 0 32 1;
#X connect 45 0 46 0;
#X connect 45 0 62 1;
#X connect 46 0 33 1;
#X connect 46 0 47 0;
#X connect 46 0 63 1;
#X connect 47 0 34 1;
#X connect 47 0 48 0;
#X connect 47 0 64 1;
#X connect 48 0 35 1;
#X connect 48 0 65 1;
#X connect 50 0 24 2;
#X connect 50 0 25 2;
#X connect 50 0 26 2;
#X connect 50 0 27 2;
#X connect 50 0 28 2;
#X connect 50 0 29 2;
#X connect 50 0 30 2;
#X connect 50 0 31 2;
#X connect 50 0 32 2;
#X connect 50 0 33 2;
#X connect 50 0 34 2;
#X connect 50 0 35 2;
#X connect 51 0 50 0;
#X connect 51 0 53 0;
#X connect 52 0 73 0;
#X connect 52 1 74 0;
#X connect 53 0 67 0;
#X connect 54 0 68 0;
#X connect 55 0 68 0;
#X connect 56 0 68 0;
#X connect 57 0 68 0;
#X connect 58 0 68 0;
#X connect 59 0 68 0;
#X connect 60 0 68 0;
#X connect 61 0 68 0;
#X connect 62 0 68 0;
#X connect 63 0 68 0;
#X connect 64 0 68 0;
#X connect 65 0 68 0;
#X connect 66 0 49 0;
#X connect 66 0 49 1;
#X connect 66 0 71 0;
#X connect 67 0 37 1;
#X connect 68 0 66 0;
#X connect 69 0 67 1;
#X connect 71 0 70 0;
#X connect 72 0 21 0;
#X connect 73 0 51 0;
#X connect 74 0 51 1;
_______________________________________________
[email protected] mailing list
UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
