Re: [PD] Threading in Pd/libpd
Just the opposite, the idea is actually to avoid revising all existing objects, that's why I'd rather want to create variations on [block~] / [switch~] that support threading for all of the objects in the subpatch. Giulio From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Monday, 3 October 2016, 16:56 Subject: Re: [PD] Threading in Pd/libpd > Subpatches with the same or smaller block size as the parent patch should not > be threaded. > Subpatches with larger blocksize should be threaded, but it's left up to the > user to enable that. Do you have to revise every single signal object in order for this to work? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
> Subpatches with the same or smaller block size as the parent patch should not > be threaded. > Subpatches with larger blocksize should be threaded, but it's left up to the > user to enable that. Do you have to revise every single signal object in order for this to work? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
Subpatches with the same or smaller block size as the parent patch should not be threaded.Subpatches with larger blocksize should be threaded, but it's left up to the user to enable that. From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Saturday, 1 October 2016, 6:08 Subject: Re: [PD] Threading in Pd/libpd > The entire subpatch, which in principle can be used to wrap [fft~].> My plan > is to have a common way of wrapping these objects with threads so that > I do not have to re-write all of them. Would this potentially benefit any subpatch, or just ones that have a substantially larger block size than the parent patch? > Giulio From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Saturday, 1 October 2016, 1:43 Subject: Re: [PD] Threading in Pd/libpd > Yes, that's the plan, by default I'd set it to the number of samples > corresponding to the step determined by the specified overlap. What exactly gets computed in the separate thread? Is it only the revised [fft~] object? Or is it the entire subpatch? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
> The entire subpatch, which in principle can be used to wrap [fft~].> My plan > is to have a common way of wrapping these objects with threads so that > I do not have to re-write all of them. Would this potentially benefit any subpatch, or just ones that have a substantially larger block size than the parent patch? > Giulio From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Saturday, 1 October 2016, 1:43 Subject: Re: [PD] Threading in Pd/libpd > Yes, that's the plan, by default I'd set it to the number of samples > corresponding to the step determined by the specified overlap. What exactly gets computed in the separate thread? Is it only the revised [fft~] object? Or is it the entire subpatch? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
The entire subpatch, which in principle can be used to wrap [fft~].My plan is to have a common way of wrapping these objects with threads so that I do not have to re-write all of them. Giulio From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Saturday, 1 October 2016, 1:43 Subject: Re: [PD] Threading in Pd/libpd > Yes, that's the plan, by default I'd set it to the number of samples > corresponding to the step determined by the specified overlap. What exactly gets computed in the separate thread? Is it only the revised [fft~] object? Or is it the entire subpatch? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
> Yes, that's the plan, by default I'd set it to the number of samples > corresponding to the step determined by the specified overlap. What exactly gets computed in the separate thread? Is it only the revised [fft~] object? Or is it the entire subpatch? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
>> With a >> threaded implementation, you could add a delay for that specific subpatch >> instead of the whole of Pd. > I'm having trouble seeing how that would work in practice. > Would the user specify the delay in ms/blocks as an argument? Yes, that's the plan, by default I'd set it to the number of samples corresponding to the step determined by the specified overlap. Giulio > > From: Jonathan Wilkes <jancs...@yahoo.com> >To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> >Sent: Thursday, 29 September 2016, 2:33 >Subject: Re: [PD] Threading in Pd/libpd > >> So having a look at the source code I'd say what would happen is that > >> the second block prints the samples of the sound file. > >> I guess the point you are trying to make here is that a threaded version > >> of [fft~], the perform routing should block if the samples are not ready in > >> time. That sounds perfectly plausible (and doable) to me. >> Or am I missing your point? > >That's right. But on the broader scale I'm pointing out how easy it is to >overlook determinism in these cases. > >> I'd add that right now, if the system cannot compute an FFT on time and it > >> causes dropouts, you have to solve that by increasing Pd's delay. With a >> threaded implementation, you could add a delay for that specific subpatch >> instead of the whole of Pd. > >I'm having trouble seeing how that would work in practice. Would the >user specify the delay in ms/blocks as an argument? >> Giulio > > > >> >> From: Jonathan Wilkes <jancs...@yahoo.com> >>To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> >>Sent: Wednesday, 28 September 2016, 23:00 >>Subject: Re: [PD] Threading in Pd/libpd >> >> >> >>> Thanks Jonathan. >> >> >>> Also [readsf~] supports threading and so do [udpsend] and [udpreceive], for >>> obvious reasons involving system calls. >> >>>> Can you guarantee that the revisions you've implemented generate the same >>>> output as Pd Vanilla, for all cases? >> >> >>> I'd rather say it does not, in all cases. At the very least there is going >>> to be a delay involved. But, if this brings to a different behaviour, yet >>> still deterministic, would that be bad? After all, the above mentioned >>> objects are not deterministic themselves, yet they are widely used, with a >>> very high success rate. >> >> >>udpsend and udpreceive are special cases because the protocol itself rules >>out the kind of determinism we're discussing. >> >> >> >>So let's focus on readsf~ instead. >> >> >>Suppose readsf~ is reading a sound file and outputs 3 blocks-- block 1, block >>2, and block 3. The patch is outputting the sound file to the sound card, so >>we are >> >>listening to the sound file as Pd is running. >> >> >>Now, imagine this happens: >> >> >>1st block: readsf~ perform routine finishes in time to output a block to the >>soundcard on schedule. >> >>2nd block: perform routine takes longer to compute, and it misses the >>deadline set for the next block to be delivered to the soundcard. So we hear >>a dropout. >>3rd block: perform routine finishes in time to meet the deadline. >> >> >>Now, suppose we were debugging our patch by outputting each block of samples >> >>to the console. Here's the question, then: what gets printed for the 2nd >>block? >> >>Does Pd print out the samples from the sound file that missed the deadline, >>or >> >>does it print out 64 zeroes? >> >> >>-Jonathan >> >> >> >> >> >> > > > > > ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
> So having a look at the source code I'd say what would happen is that > the second block prints the samples of the sound file. > I guess the point you are trying to make here is that a threaded version > of [fft~], the perform routing should block if the samples are not ready in > time. That sounds perfectly plausible (and doable) to me.> Or am I missing > your point? That's right. But on the broader scale I'm pointing out how easy it is to overlook determinism in these cases. > I'd add that right now, if the system cannot compute an FFT on time and it > causes dropouts, you have to solve that by increasing Pd's delay. With a > threaded implementation, you could add a delay for that specific subpatch > instead of the whole of Pd. I'm having trouble seeing how that would work in practice. Would the user specify the delay in ms/blocks as an argument? > Giulio From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Wednesday, 28 September 2016, 23:00 Subject: Re: [PD] Threading in Pd/libpd > Thanks Jonathan. > Also [readsf~] supports threading and so do [udpsend] and [udpreceive], for > obvious reasons involving system calls. >> Can you guarantee that the revisions you've implemented generate the same >> output as Pd Vanilla, for all cases? > I'd rather say it does not, in all cases. At the very least there is going to > be a delay involved. But, if this brings to a different behaviour, yet still > deterministic, would that be bad? After all, the above mentioned objects are > not deterministic themselves, yet they are widely used, with a very high > success rate. udpsend and udpreceive are special cases because the protocol itself rules out the kind of determinism we're discussing. So let's focus on readsf~ instead. Suppose readsf~ is reading a sound file and outputs 3 blocks-- block 1, block 2, and block 3. The patch is outputting the sound file to the sound card, so we are listening to the sound file as Pd is running. Now, imagine this happens: 1st block: readsf~ perform routine finishes in time to output a block to the soundcard on schedule. 2nd block: perform routine takes longer to compute, and it misses the deadline set for the next block to be delivered to the soundcard. So we hear a dropout.3rd block: perform routine finishes in time to meet the deadline. Now, suppose we were debugging our patch by outputting each block of samples to the console. Here's the question, then: what gets printed for the 2nd block? Does Pd print out the samples from the sound file that missed the deadline, or does it print out 64 zeroes? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
So having a look at the source code I'd say what would happen is that the second block prints the samples of the sound file. I guess the point you are trying to make here is that a threaded version of [fft~], the perform routing should block if the samples are not ready in time. That sounds perfectly plausible (and doable) to me.Or am I missing your point? I'd add that right now, if the system cannot compute an FFT on time and it causes dropouts, you have to solve that by increasing Pd's delay. With a threaded implementation, you could add a delay for that specific subpatch instead of the whole of Pd. Giulio From: Jonathan Wilkes <jancs...@yahoo.com> To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> Sent: Wednesday, 28 September 2016, 23:00 Subject: Re: [PD] Threading in Pd/libpd > Thanks Jonathan. > Also [readsf~] supports threading and so do [udpsend] and [udpreceive], for > obvious reasons involving system calls. >> Can you guarantee that the revisions you've implemented generate the same >> output as Pd Vanilla, for all cases? > I'd rather say it does not, in all cases. At the very least there is going to > be a delay involved. But, if this brings to a different behaviour, yet still > deterministic, would that be bad? After all, the above mentioned objects are > not deterministic themselves, yet they are widely used, with a very high > success rate. udpsend and udpreceive are special cases because the protocol itself rules out the kind of determinism we're discussing. So let's focus on readsf~ instead. Suppose readsf~ is reading a sound file and outputs 3 blocks-- block 1, block 2, and block 3. The patch is outputting the sound file to the sound card, so we are listening to the sound file as Pd is running. Now, imagine this happens: 1st block: readsf~ perform routine finishes in time to output a block to the soundcard on schedule. 2nd block: perform routine takes longer to compute, and it misses the deadline set for the next block to be delivered to the soundcard. So we hear a dropout.3rd block: perform routine finishes in time to meet the deadline. Now, suppose we were debugging our patch by outputting each block of samples to the console. Here's the question, then: what gets printed for the 2nd block? Does Pd print out the samples from the sound file that missed the deadline, or does it print out 64 zeroes? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
> Thanks Jonathan. > Also [readsf~] supports threading and so do [udpsend] and [udpreceive], for > obvious reasons involving system calls. >> Can you guarantee that the revisions you've implemented generate the same >> output as Pd Vanilla, for all cases? > I'd rather say it does not, in all cases. At the very least there is going to > be a delay involved. But, if this brings to a different behaviour, yet still > deterministic, would that be bad? After all, the above mentioned objects are > not deterministic themselves, yet they are widely used, with a very high > success rate. udpsend and udpreceive are special cases because the protocol itself rules out the kind of determinism we're discussing. So let's focus on readsf~ instead. Suppose readsf~ is reading a sound file and outputs 3 blocks-- block 1, block 2, and block 3. The patch is outputting the sound file to the sound card, so we are listening to the sound file as Pd is running. Now, imagine this happens: 1st block: readsf~ perform routine finishes in time to output a block to the soundcard on schedule. 2nd block: perform routine takes longer to compute, and it misses the deadline set for the next block to be delivered to the soundcard. So we hear a dropout.3rd block: perform routine finishes in time to meet the deadline. Now, suppose we were debugging our patch by outputting each block of samples to the console. Here's the question, then: what gets printed for the 2nd block? Does Pd print out the samples from the sound file that missed the deadline, or does it print out 64 zeroes? -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
Thanks Jonathan. Also [readsf~] supports threading and so do [udpsend] and [udpreceive], for obvious reasons involving system calls. > Can you guarantee that the revisions you've implemented generate the same > output as Pd Vanilla, for all cases? I'd rather say it does not, in all cases. At the very least there is going to be a delay involved. But, if this brings to a different behaviour, yet still deterministic, would that be bad? After all, the above mentioned objects are not deterministic themselves, yet they are widely used, with a very high success rate. And what happens if you realize your [readsf~] glitches ? You change your code so that it sends the [open( message earlier on. As the objects I am talking about ( fft~, fiddle~, sigmund~) do not rely on system calls, I expect their behaviour to be more predictable than that of, e.g.: [readsf~]. I think I'll see if I can put together a [blockthread~] object which can do something useful. Best, Giulio > From: Jonathan Wilkes <jancs...@yahoo.com> >To: Giulio Moro <giuliom...@yahoo.it>; Pd-List <pd-list@lists.iem.at> >Sent: Tuesday, 27 September 2016, 18:35 >Subject: Re: [PD] Threading in Pd/libpd > > > >> So, probably this point has been discussed previously, I'd like to know: >> - are there any existing objects doing this already? > > >There is a creation argument to [coll] in Pd-l2ork that enables threading. > > > > > >> - what are the pitfalls that prevented such an approach from making its way >> into Pd? > > >The second biggest pitfall is that such an approach can easily (and subtly) >break determinism. > > >The biggest pitfall is overestimating the benefit of the performance gains to >the detriment of > >determinism. Can you guarantee that the revisions you've implemented generate >the same > >output as Pd Vanilla, for all cases? > > >- how can I help? > > >A good place to start might be regression tests for block~. I'd especially >look at cases that > >use vline~ in conjunction with it, using very small delays, and make sure that >you are getting > >the exact same samples output using your revised objects. > > >-Jonathan > > > > > > > >>>> >>From: Giulio Moro <giuliom...@yahoo.it> >>To: Pd-List <pd-list@lists.iem.at> >>Sent: Sunday, 18 September 2016, 2:23 >>Subject: Threading in Pd/libpd >> >> >> >>Hi all, >>if I understand correctly, using the [block~] and [switch~] objects to >>increase the blocksize for a given subpatch, means that the DSP computation >>for that subpatch is delayed until the moment when enough input samples have >>been collected, at which point the entire DSP stack for the subpatch is >>performed at once and the outputs are written to the output buffer. >>This means that the DSP load is not spread over time, rather it is >>concentrated in that single audio driver callback when the buffer for that >>subpatch happens to be ready to be processed. >> >> >>Now, if what I say makes sense, then this approach has the disadvantage that >>the CPU load is not spread evenly across audio callbacks, eventually causing >>dropouts if whatever computation takes too long in that one callback, forcing >>you to increase the internal buffering of Pd (``Delay'') to cope with this. >>At the same time, though, the CPU will be pretty much idle in all the other >>audio callbacks. >> >> >>If we could spread the load of the expensive, but occasional, computation >>(say fft) over multiple audio callbacks, then the CPU load would be more >>even, with no spikes and there would be no need to increase Pd's internal >>buffering. >>This would require to have the output of the fft available a few processing >>blocks after the one where it was started, while the current approach allows >>to have it immediately available. A fine tuning of the system would be >>required to understand how much this latency should be, and worst case it >>would be the number of overlap samples as set by [block~] (as in: if the >>system cannot process these blocks fast enough, then you should lower your >>requirements, as your system cannot provide the required throughput). Now >>this may seem a downside, but the actual overall roundtrip latency of the Pd >>subpatch would be not much larger than the one currently achievable (if at >>all larger), with the added advantage that the rest of Pd could work at >>smaller blocksizes, and with a ``Delay'' set to 0. >>The ultimate advantage would be to have a more responsive system, in terms
Re: [PD] Threading in Pd/libpd
> So, probably this point has been discussed previously, I'd like to know:> - > are there any existing objects doing this already? There is a creation argument to [coll] in Pd-l2ork that enables threading. > - what are the pitfalls that prevented such an approach from making its way > into Pd? The second biggest pitfall is that such an approach can easily (and subtly) break determinism. The biggest pitfall is overestimating the benefit of the performance gains to the detriment of determinism. Can you guarantee that the revisions you've implemented generate the same output as Pd Vanilla, for all cases? - how can I help? A good place to start might be regression tests for block~. I'd especially look at cases that use vline~ in conjunction with it, using very small delays, and make sure that you are getting the exact same samples output using your revised objects. -Jonathan ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Re: [PD] Threading in Pd/libpd
It looks like I am not very lucky in getting attention, so let me try to re-up this. Can we implement a threaded [block~] ? see details below From: Giulio Moro <giuliom...@yahoo.it> To: Pd-List <pd-list@lists.iem.at> Sent: Sunday, 18 September 2016, 2:23 Subject: Threading in Pd/libpd Hi all,if I understand correctly, using the [block~] and [switch~] objects to increase the blocksize for a given subpatch, means that the DSP computation for that subpatch is delayed until the moment when enough input samples have been collected, at which point the entire DSP stack for the subpatch is performed at once and the outputs are written to the output buffer.This means that the DSP load is not spread over time, rather it is concentrated in that single audio driver callback when the buffer for that subpatch happens to be ready to be processed. Now, if what I say makes sense, then this approach has the disadvantage that the CPU load is not spread evenly across audio callbacks, eventually causing dropouts if whatever computation takes too long in that one callback, forcing you to increase the internal buffering of Pd (``Delay'') to cope with this. At the same time, though, the CPU will be pretty much idle in all the other audio callbacks. If we could spread the load of the expensive, but occasional, computation (say fft) over multiple audio callbacks, then the CPU load would be more even, with no spikes and there would be no need to increase Pd's internal buffering.This would require to have the output of the fft available a few processing blocks after the one where it was started, while the current approach allows to have it immediately available. A fine tuning of the system would be required to understand how much this latency should be, and worst case it would be the number of overlap samples as set by [block~] (as in: if the system cannot process these blocks fast enough, then you should lower your requirements, as your system cannot provide the required throughput). Now this may seem a downside, but the actual overall roundtrip latency of the Pd subpatch would be not much larger than the one currently achievable (if at all larger), with the added advantage that the rest of Pd could work at smaller blocksizes, and with a ``Delay'' set to 0.The ultimate advantage would be to have a more responsive system, in terms of I/O roundtrip for most of the patch, except those subpatches where a longer latency is anyhow imposed by the algorithm. Think for instance of having a patch processing the live the sound of an instrument, which also uses [sigmund~] to detect its pitch to apply some adaptive effect. A low roundtrip latency could be used for the processed instrument while the latency imposed by [sigmund~] would only affect e.g.: the parameters of the effect. I see how this approach may be useful in many cases.Multi-core hardware would take extra advantage from this way of spreading the CPU usage. I am in the situation where I hacked together a threaded version of [sigmund~] for use with libpd on Bela which works fine and I am wondering if it is worth going down the route of making threaded versions of all objects with similar requirements (which I really would not want to do) or I should rather try to create some higher-level objects (say [blockThread~] ) that perform the threading strategy mentioned above.It may be that [pd~] could probably(?) provide the solution requested, but it seems to me there is lots of overhead associated with it, and I do not see how to easily integrate it with our use of libpd. So, probably this point has been discussed previously, I'd like to know:- are there any existing objects doing this already?- what are the pitfalls that prevented such an approach from making its way into Pd?- how can I help? Best,Giulio ___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
[PD] Threading in Pd/libpd
Hi all,if I understand correctly, using the [block~] and [switch~] objects to increase the blocksize for a given subpatch, means that the DSP computation for that subpatch is delayed until the moment when enough input samples have been collected, at which point the entire DSP stack for the subpatch is performed at once and the outputs are written to the output buffer.This means that the DSP load is not spread over time, rather it is concentrated in that single audio driver callback when the buffer for that subpatch happens to be ready to be processed. Now, if what I say makes sense, then this approach has the disadvantage that the CPU load is not spread evenly across audio callbacks, eventually causing dropouts if whatever computation takes too long in that one callback, forcing you to increase the internal buffering of Pd (``Delay'') to cope with this. At the same time, though, the CPU will be pretty much idle in all the other audio callbacks. If we could spread the load of the expensive, but occasional, computation (say fft) over multiple audio callbacks, then the CPU load would be more even, with no spikes and there would be no need to increase Pd's internal buffering.This would require to have the output of the fft available a few processing blocks after the one where it was started, while the current approach allows to have it immediately available. A fine tuning of the system would be required to understand how much this latency should be, and worst case it would be the number of overlap samples as set by [block~] (as in: if the system cannot process these blocks fast enough, then you should lower your requirements, as your system cannot provide the required throughput). Now this may seem a downside, but the actual overall roundtrip latency of the Pd subpatch would be <= the one currently achievable, with the added advantage that the rest of Pd could work at smaller blocksizes and without the additional ``Delay'' required in the current configuration.The ultimate advantage would be to have a more responsive system, in terms of I/O roundtrip for most of the patch, except those subpatches where a longer latency is imposed by the algorithm. Think for instance of having a patch processing the live the sound of an instrument, which also uses [sigmund~] to detect its pitch to apply some adaptive effect. A low roundtrip latency could be used for the processed instrument while the latency imposed by [sigmund~] would only affect e.g.: the parameters of the effect. I see how this approach may be useful in many cases.Multi-core hardware would take extra advantage from this way of spreading the CPU usage. I am in the situation where I hacked together a threaded version of [sigmund~] for use with libpd on Bela which works fine and I am wondering if it is worth going down the route of making threaded versions of all objects with similar requirements (which I really would not want to do) or I should rather try to create some higher-level objects (say [blockThread~] ) that perform the threading strategy mentioned above.It may be that [pd~] could probably(?) provide the solution requested, but it seems to me there is lots of overhead associated with it, and I do not see how to easily integrate it with our use of libpd. So, probably this point has been discussed previously, I'd like to know:- are there any existing objects doing this already?- what are the pitfalls that prevented such an approach from making its way into Pd?- how can I help? Best,Giulio___ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
