Re: [PD] Pd and gpgpu
hi all i made a silly patch, that let's you control the volume of an audio signal using [colorRGB], just to illustrate, that audio can be processed on gpu by using gem and puredata. i'd be interested, if someone has a more meaningful application using this approach. people talked about doing convolution on gpu. what does it need to actually do it? http://romanhaefeli.net/software/pd/dsp_on_gpu.pd roman On Sun, 2007-11-11 at 09:09 -0600, chris clepper wrote: This chain is essentially the process although a few details prevent it from actually working properly. The audio needs to remain in floating point format and pix_texture and the read back from the screen are integer based. The solution is to use an offscreen framebuffer object (like gemframebuffer) to create a floating point rendering environment for the audio. This project would be best done as a custom object specific to audio rather than trying to use GEM. I don't think this will be very efficient for a single stream of audio since the data size is so small and the time to read back is so long. Perhaps it could enable something like computing an FIR on 16 or more channels at once or doing convolution to emulate an entire mixing console though. On 11/10/07, Roman Haefeli [EMAIL PROTECTED] wrote: audioIn~ | [pix_sig2pix~] | [pix_texture] | [doSomeStuff] | [pix_snap] | [pix_pix2sig~] | audioOut~ ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
Hallo! Max MSP/Jitter has bridge objects that convert from audio rate to matrices and back, which would be needed in PD land to readback from the GPU, and convert to an audio rate signal. Thats how this is done. Otherwise i have no idea how you would implement souch a patch. I think the easiest way to do it is simply in the C code of the audio externals (there exist already nice libraries for general computing on the GPU - as pointed to in the last link). However, of course it would be also interesting to do that at the patching level, but I don't know if that is possible ... LG Georg ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
hmm, I tried it, because I thought, that's it, but somehow it is not working. the problem is, that pix_snap is not working as expected: after do some stuff I am still on the graphics card, basically I created a texture, that I can reference as ID. and pix_snap, I don't know, does not create an output... [gemhead] | | [ID\ || [pix_texture] | [pix_snap] | [pix_pix2sig~ 8 8] | | | [dac~ 1 2 3] marius. Roman Haefeli wrote: yo, ignore me, if i am saying something absolutely stupid and unrelated, but isn't that exactly, what [pix_sig2pix~] and its counterpart [pix_pix2sig~] do? of course you still would have to [pix_texture] first in order to transfer the data to the gpu and then afterwards [pix_snap] it to bring it back from gpu to a pix. audioIn~ | [pix_sig2pix~] | [pix_texture] | [doSomeStuff] | [pix_snap] | [pix_pix2sig~] | audioOut~ from my experience, doing just all this transfers is pretty slow (especially [pix_snap], as others already mentioned) roman On Sat, 2007-11-10 at 13:59 -0500, vade wrote: Max MSP/Jitter has bridge objects that convert from audio rate to matrices and back, which would be needed in PD land to readback from the GPU, and convert to an audio rate signal. Thats how this is done. Otherwise i have no idea how you would implement souch a patch. On Nov 10, 2007, at 8:55 AM, Georg Holzmann wrote: Hallo! it is true that the gpu is unused most of the time and could be used to do fast calculations, but the main output for graphicscards is the screen and reading back into ram can be slow. (of course slow is a relative term). I also see a problem in the different formats for audio and images. most graphics cards are optimized for vec4 processings (4 parallel color channels). and audio... Well, there are already quite some papers/software for audio processing on the gpu (see e.g. http://www.gpgpu.org/cgi-bin/blosxom.cgi/Audio%20and%20Signal%20Processing/index.html) . But AFAIK nobody did this with pd up to now ... LG Georg ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ Der frühe Vogel fängt den Wurm. Hier gelangen Sie zum neuen Yahoo! Mail: http://mail.yahoo.de ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
This chain is essentially the process although a few details prevent it from actually working properly. The audio needs to remain in floating point format and pix_texture and the read back from the screen are integer based. The solution is to use an offscreen framebuffer object (like gemframebuffer) to create a floating point rendering environment for the audio. This project would be best done as a custom object specific to audio rather than trying to use GEM. I don't think this will be very efficient for a single stream of audio since the data size is so small and the time to read back is so long. Perhaps it could enable something like computing an FIR on 16 or more channels at once or doing convolution to emulate an entire mixing console though. On 11/10/07, Roman Haefeli [EMAIL PROTECTED] wrote: audioIn~ | [pix_sig2pix~] | [pix_texture] | [doSomeStuff] | [pix_snap] | [pix_pix2sig~] | audioOut~ ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
marius schebella wrote: hmm, I tried it, because I thought, that's it, but somehow it is not working. the problem is, that pix_snap is not working as expected: after do some stuff I am still on the graphics card, basically I created a texture, that I can reference as ID. and pix_snap, I don't well, [pix_snap] will crate a snapshot of the rendered-buffer, not of the current texture. you need to display the texture by putting it onto a geo... mfa.dr IOhannes ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
hi,
at some point I had sound and image, but then something broke again,
don't know what. also, the sound that I had was different than it should
be (could be a int/float problem or related to something completely
different), I also could not get the patch running at block~ 64, took
almost 200% of cpu, so if someone wants to carry this on...
and, I know, I attach the files as files, but they sometimes appear as
text, don't know why.
marius.
IOhannes m zmoelnig wrote:
marius schebella wrote:
hmm, I tried it, because I thought, that's it, but somehow it is not
working.
the problem is, that pix_snap is not working as expected:
after do some stuff I am still on the graphics card, basically I
created a texture, that I can reference as ID. and pix_snap, I don't
well, [pix_snap] will crate a snapshot of the rendered-buffer, not of
the current texture. you need to display the texture by putting it onto
a geo...
mfa.dr
IOhannes
uniform sampler2D texture;
void main()
{
gl_TexCoord[0] = gl_MultiTexCoord0;
// perform standard transform on vertex
gl_Position = ftransform();
}
uniform sampler2D texture;
uniform float volume;
void main()
{
vec4 B = texture2D(texture, gl_TexCoord[0].st);
gl_FragColor = B * volume;
}#N canvas 527 25 782 708 10;
#X obj 450 384 pix_texture;
#X obj 364 237 bang~;
#X obj 100 420 phasor~ 1000;
#X obj 204 327 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1
-1;
#N canvas 167 101 628 566 glsl 0;
#X obj 76 486 glsl_program;
#X obj 139 349 pack 0 0;
#X obj 159 329 t b f;
#X msg 139 369 link \$1 \$2;
#X msg 99 449 print;
#X obj 76 39 inlet;
#X obj 76 269 glsl_fragment;
#X obj 76 149 glsl_vertex;
#X obj 76 506 outlet;
#X obj 126 39 inlet;
#X obj 176 39 inlet;
#X obj 216 39 inlet;
#X obj 146 169 change;
#X obj 159 289 change;
#X obj 189 419 print linking;
#X obj 149 389 t b a a;
#X obj 99 429 delay 0;
#X obj 257 140 cnv 15 300 150 empty empty empty 20 12 0 14 -104052
-66577 0;
#X text 300 185 Inlet 1: GEMlist;
#X text 300 205 Inlet 2: vertex shader commands;
#X text 300 225 Inlet 3: fragment shader commands;
#X text 300 245 Inlet 4: glsl programm commands;
#X text 290 165 Inlets:;
#X connect 0 0 8 0;
#X connect 1 0 3 0;
#X connect 2 0 1 0;
#X connect 2 1 1 1;
#X connect 3 0 15 0;
#X connect 4 0 0 0;
#X connect 5 0 7 0;
#X connect 6 0 0 0;
#X connect 6 1 13 0;
#X connect 7 0 6 0;
#X connect 7 1 12 0;
#X connect 9 0 7 0;
#X connect 10 0 6 0;
#X connect 11 0 0 0;
#X connect 12 0 1 0;
#X connect 13 0 2 0;
#X connect 15 0 16 0;
#X connect 15 1 0 0;
#X connect 15 2 14 0;
#X connect 16 0 4 0;
#X restore 62 391 pd glsl;
#X obj 204 307 loadbang;
#X msg 154 363 open 01_basic.vert;
#X msg 298 363 open 01_volume.frag;
#X msg 109 363 print;
#X obj 62 485 pix_texture;
#X msg 474 359 mode \$1;
#X obj 524 339 tgl 15 0 empty empty empty 17 7 0 10 -262144 -1 -1 0
1;
#X obj 120 440 noise~;
#X obj 538 625 dac~;
#X obj 60 135 gemframebuffer;
#X floatatom 524 411 5 0 0 0 - - -;
#X floatatom 113 160 5 0 0 0 - - -;
#X obj 410 257 t b b b;
#X floatatom 208 168 5 0 0 0 - - -;
#X obj 450 517 pix_snap;
#X msg 449 494 snap;
#X msg 486 494 0 0;
#X obj 322 601 pix_info _;
#X floatatom 319 625 5 0 0 0 - - -;
#X floatatom 329 645 5 0 0 0 - - -;
#X msg 79 323 volume \$1;
#X floatatom 78 299 5 0 0 0 - - -;
#X floatatom 481 409 5 0 0 0 - - -;
#X msg 94 86 dim 8 8;
#X floatatom 129 513 5 0 0 0 - - -;
#X obj 450 450 square 4;
#X msg 110 108 dim 128 128;
#X msg 78 276 1;
#X obj 78 257 loadbang;
#X msg 519 493 8 8;
#X obj 125 283 cnv 15 30 30 empty empty empty 20 12 0 14 -81432 -66577
0;
#X obj 652 123 cnv 15 30 30 empty empty empty 20 12 0 14 -81432 -66577
0;
#X obj 361 88 cnv 15 30 30 empty empty empty 20 12 0 14 -81432 -66577
0;
#X msg 367 115 \; pd dsp \$1;
#X obj 367 96 tgl 15 0 empty empty empty 17 7 0 10 -262144 -1 -1 1
1;
#X obj 367 150 dsp;
#X floatatom 367 186 5 0 0 0 - - -;
#X obj 540 244 gemwin;
#X msg 589 122 buffer 1;
#X msg 589 141 1 \, bang;
#X msg 588 182 destroy;
#X msg 564 74 create;
#X msg 589 162 1;
#X msg 589 202 color 1 1 0;
#X obj 469 583 env~;
#X floatatom 470 604 5 0 0 0 - - -;
#X obj 63 508 square 1;
#X obj 61 185 translateXYZ 0 0 -1.01;
#X obj 450 472 t b a;
#X obj 451 553 pix_pix2sig~ 8 8;
#X obj 449 428 rotateXYZ 0 0 0;
#X obj 614 72 cnv 15 30 30 empty empty empty 20 12 0 14 -81432 -66577
0;
#X text 621 77 start here.;
#X text 653 125 then this;
#X text 133 286 play with vol;
#X text 353 70 be sure sound is on;
#X msg 595 235 dimen 8 8;
#X text 216 220 this seems to be too fast;
#X obj 409 235 metro 100;
#X obj 408 215 tgl 15 0 empty empty empty 17 7 0 10 -262144 -1 -1 1
1;
#X msg 595 270 dimen 400 400;
#X obj 451 312 tgl 15 0 empty empty empty 17 7 0 10 -262144 -1 -1 1
1;
#X msg 583 320 reset;
#X obj 87 46 tgl 15 0 empty empty empty 17 7 0 10 -262144 -1 -1 1 1
;
#X obj 61 66 gemhead 49;
#X obj 449 335 gemhead 50;
#X obj 62 458 pix_sig2pix~;
#X floatatom 541 375 5 0 0 0 - - -;
#X text 343 29 CAN SOMEONE GET THIS WORKING???;
#X
Re: [PD] Pd and gpgpu
On 12 Nov 2007, at 5:52 AM, marius schebella wrote:
and, I know, I attach the files as files, but they sometimes appear
as text, don't know why.
marius.
I looked into this a while ago - it depends on the 'format=flowed'
flag and the 'Content-Disposition' and 'Content-Type' flags on the
attachments - maybe your client has some settings to select these flags?
- in your mails each part has 'Content-Type: text/plain' and
'Content-Disposition: inline' and it is displayed that way on my
client - all plain text (it fails for the binary file, which is not
displayed - but that file certainly shouldn't be described as 'text/
plain')
--070400050902010409090204
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
hi,
at some point I had sound and image, but then something broke again,
etc ..
mfa.dr
IOhannes
--070400050902010409090204
Content-Type: text/plain; x-mac-type=0; x-mac-creator=0;
name=01_basic.vert
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename=01_basic.vert
uniform sampler2D texture;
void main()
{
gl_TexCoord[0] = gl_MultiTexCoord0;
// perform standard transform on vertex
gl_Position = ftransform();
}
--070400050902010409090204
Content-Type: text/plain; x-mac-type=0; x-mac-creator=0;
name=01_volume.frag
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename=01_volume.frag
dW5pZm9ybSBzYW1wbGVyMkQgdGV4dHVyZTsKdW5pZm9ybSBmbG9hdCB2b2x1bWU7CgoNCn
Zv
aWQgbWFpbigpDQp7Cgl2ZWM0IEIgPSB0ZXh0dXJlMkQodGV4dHVyZSwgZ2xfVGV4Q29vcm
Rb
MF0uc3QpOwoJZ2xfRnJhZ0NvbG9yID0gQiAqIHZvbHVtZTsNCn0=
--070400050902010409090204
Content-Type: text/plain;
name=test2.pd
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename=test2.pd
#N canvas 527 25 782 708 10;
this mail gets displayed as attachments placed in the flow of the
text - note 'Content-Disposition: attachment' on the relevant parts.
This works well for most clients - the main differences are that some
clients display the attachments (or their names or links to them) in
the intended place in the body of the email, while others only
display the list of attachments separately.
--Apple-Mail-30--380763525
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
format=flowed
test
test
test
--Apple-Mail-30--380763525
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
x-unix-mode=0644;
name=-notes.txt
Content-Disposition: attachment;
filename=-notes.txt
textfile
textfile
textfile
--Apple-Mail-30--380763525
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
delsp=yes;
format=flowed
test
test
test
test
--Apple-Mail-30--380763525
Content-Transfer-Encoding: 7bit
Content-Type: application/octet-stream;
x-unix-mode=0644;
name=colours.pd
Content-Disposition: attachment;
filename=colours.pd
#N canvas 385 152 626 609 10;
___
PD-list@iem.at mailing list
UNSUBSCRIBE and account-management -
http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
hi, no, I don't know how to use glsl for audio. I thought you were talking about graphical things. for audio you somehow would have to copy a buffer of audio data to the graphicscard (I don't know an object that could do that, but maybe there is one?...), then do your computations and then read it back into ram and then pass it to the soundcard. I also don't see, how you would time and synchronize your graphics card with the soundcard. it is true that the gpu is unused most of the time and could be used to do fast calculations, but the main output for graphicscards is the screen and reading back into ram can be slow. (of course slow is a relative term). I also see a problem in the different formats for audio and images. most graphics cards are optimized for vec4 processings (4 parallel color channels). and audio... OTOH I know that max/msp has some shaders that do audio computation. You could have a look at that and then maybe know how to do it in pd, too. although I think that is all related to jitter functionality, and pd can't really do that. marius. Julian Villegas wrote: Thank you Marius, Do I need to use GEM? I'm not doing anything with video, or graphics, I'm working only with audio. What I'd like to do in a first stage is to send to the GPU a set of data (harmonic peaks from live signals) and do some calculations there, then read back that into the program and proceed according to the results. In a second stage, the FFT and the spectral analyis could be performed there too. Chris mentioned that latency might be an issue since reading data from the GPU is slow, I don't know much about it but I think it shouldn't be since many cards can (and in fact must) use the main memory, so reading from memory should be as fast as it is when I read it with the CPU. Also, I'm working in Mac OS 10.5, using Miller's version of Pd. Have you have any problems with Leopard and your patches? Thank you again, and I'll read the patches you post to see if I can get more ideas from them... __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
Hallo! it is true that the gpu is unused most of the time and could be used to do fast calculations, but the main output for graphicscards is the screen and reading back into ram can be slow. (of course slow is a relative term). I also see a problem in the different formats for audio and images. most graphics cards are optimized for vec4 processings (4 parallel color channels). and audio... Well, there are already quite some papers/software for audio processing on the gpu (see e.g. http://www.gpgpu.org/cgi-bin/blosxom.cgi/Audio%20and%20Signal%20Processing/index.html). But AFAIK nobody did this with pd up to now ... LG Georg ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
Max MSP/Jitter has bridge objects that convert from audio rate to matrices and back, which would be needed in PD land to readback from the GPU, and convert to an audio rate signal. Thats how this is done. Otherwise i have no idea how you would implement souch a patch. On Nov 10, 2007, at 8:55 AM, Georg Holzmann wrote: Hallo! it is true that the gpu is unused most of the time and could be used to do fast calculations, but the main output for graphicscards is the screen and reading back into ram can be slow. (of course slow is a relative term). I also see a problem in the different formats for audio and images. most graphics cards are optimized for vec4 processings (4 parallel color channels). and audio... Well, there are already quite some papers/software for audio processing on the gpu (see e.g. http://www.gpgpu.org/cgi-bin/blosxom.cgi/Audio%20and%20Signal%20Processing/index.html) . But AFAIK nobody did this with pd up to now ... LG Georg ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu
yo, ignore me, if i am saying something absolutely stupid and unrelated, but isn't that exactly, what [pix_sig2pix~] and its counterpart [pix_pix2sig~] do? of course you still would have to [pix_texture] first in order to transfer the data to the gpu and then afterwards [pix_snap] it to bring it back from gpu to a pix. audioIn~ | [pix_sig2pix~] | [pix_texture] | [doSomeStuff] | [pix_snap] | [pix_pix2sig~] | audioOut~ from my experience, doing just all this transfers is pretty slow (especially [pix_snap], as others already mentioned) roman On Sat, 2007-11-10 at 13:59 -0500, vade wrote: Max MSP/Jitter has bridge objects that convert from audio rate to matrices and back, which would be needed in PD land to readback from the GPU, and convert to an audio rate signal. Thats how this is done. Otherwise i have no idea how you would implement souch a patch. On Nov 10, 2007, at 8:55 AM, Georg Holzmann wrote: Hallo! it is true that the gpu is unused most of the time and could be used to do fast calculations, but the main output for graphicscards is the screen and reading back into ram can be slow. (of course slow is a relative term). I also see a problem in the different formats for audio and images. most graphics cards are optimized for vec4 processings (4 parallel color channels). and audio... Well, there are already quite some papers/software for audio processing on the gpu (see e.g. http://www.gpgpu.org/cgi-bin/blosxom.cgi/Audio%20and%20Signal%20Processing/index.html) . But AFAIK nobody did this with pd up to now ... LG Georg ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list ___ Der frühe Vogel fängt den Wurm. Hier gelangen Sie zum neuen Yahoo! Mail: http://mail.yahoo.de ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] PD and GPGPU
Thank you Chris, I want to process audio. Can you elaborate more in the latency issue? were you working with audio too? the forum you mention is the one in gpgpu.org? There are many languages to do it, I know, many of them vendor dependent. Nvidia, Ati, and other are offering high level languages that eases the gpgpu implementation, there is even a company (rapidminds) that offers to compile your code to different platforms including the cell. Anyway, I don't think any of these solutions can be applied to extend my PD - audio external, and as I mentioned before, the shader language of GL (GLSL) seems to be a good option, do you have any comments about it? Bye, __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
Re: [PD] Pd and gpgpu (marius schebella)
Thank you Marius, Do I need to use GEM? I'm not doing anything with video, or graphics, I'm working only with audio. What I'd like to do in a first stage is to send to the GPU a set of data (harmonic peaks from live signals) and do some calculations there, then read back that into the program and proceed according to the results. In a second stage, the FFT and the spectral analyis could be performed there too. Chris mentioned that latency might be an issue since reading data from the GPU is slow, I don't know much about it but I think it shouldn't be since many cards can (and in fact must) use the main memory, so reading from memory should be as fast as it is when I read it with the CPU. Also, I'm working in Mac OS 10.5, using Miller's version of Pd. Have you have any problems with Leopard and your patches? Thank you again, and I'll read the patches you post to see if I can get more ideas from them... __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
[PD] Pd and gpgpu
Hi, I'm interested in reimplementing some externals that I've made in Pd. The idea is to take advantage of the graphic card power to deal with parallel data and boost the performance of the external. Have you guys done that before? I'm planning to use GLSL, the Open GL Shader Language, for compatibility. My problem is that I don't know how to do it, and I'm looking for some help or orientation of those among you who have some experience in this matter. Thanks in advance. PS: I'm posting this to pd-list and pd-dev-list, and I'm sorry for multiple instances of this email. Julian Villegas Me pregunto de un modo pensativo Que significa ser Colombiano? No se le respondi. Es un acto de fe JLB. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ PD-list@iem.at mailing list UNSUBSCRIBE and account-management - http://lists.puredata.info/listinfo/pd-list
