Hi Yury,
Thank you for reporting this, and for the script that helps reproduce it, which
is incredibly valuable. I have found a great module
(https://metacpan.org/pod/Test::LeakTrace) that is helping track down the
problems this highlights, and have just pushed a fix for the first one, a
memory-leak in “sclr”, detectable using this script:
use PDL;
use Test::LeakTrace;
no_leaks_ok { pdl(3)->sclr };
Similarly, your script can be reduced to this, since a single iteration will
still leak:
use PDL;
use Test::LeakTrace;
no_leaks_ok { zeroes(1,1,1)->interpND(pdl(1,1,1)) };
When I’ve squashed all of the problems revealed, there is likely to be a new
version of PDL out soon.
Best regards,
Ed
From: Yury Pakhomov<mailto:[email protected]>
Sent: 23 March 2023 22:19
To: [email protected]<mailto:[email protected]>
Subject: Re: [Pdl-general] PDL 2.082 released
Thanks!!!
But today I found a memory hole for v2.081 in interpND function. Within a
several minutes one-two gigabytes are spend. If I run 20 processes in parallel
then the calculations are interrupted.
You can check this code:
use strict;
use PDL;
use Memory::Usage;
my $mu = Memory::Usage->new();
my $data=zeroes(100,100,100);
$mu->record('starting work');
for(my $i=0; $i<100; $i++){
for(my $j=0;$j<100;$j++){
for(my $k=0; $k<100; $k++){
my $ind=pdl($i,$j,$k);
my $res=$data->interpND($ind);
}
}
$mu->record('step by i');
my $mem=$mu->state();
printf "%3i: RES=%.3f Mb\n",$i,($mem->[-1]->[3])/1024;
}
_________________________
0: RES=33.750 Mb
1: RES=40.375 Mb
2: RES=47.000 Mb
3: RES=53.625 Mb
4: RES=60.250 Mb
5: RES=66.875 Mb
6: RES=73.375 Mb
7: RES=80.000 Mb
....
6.6 Mb was spend for each step by i (10^4 calls of interpND).
Best wishes
Yuri
23.03.2023 06:31, Ed. пишет:
Dear PDL folks,
PDL 2.082 has just been released. Notable changes since 2.081:
1. allow OtherPars “nsize => n” to set index-size of not-otherwise-used index
2. fix [o] OtherPars to work if PMCode provided
3. fix nested loop() bug for Pars with e.g. (n,n)
4. add pp_add_typemaps
5. bug fixes: MatrixOps::det now always returns an ndarray, fixed an
intersect bug
Future plans, in something like intended order:
1. Finish PDL::OpenCV
2. Restructure the TriD stuff so there is a consistent API between OpenGL
and X3D/VRML
3. fix more open GitHub issues
4. add event-handling hooks for ndarrays
5. “loop fusion” techniques to maximise locality of computation, minimising
data’s trips through the “straw” between CPU and main RAM
6. finish the independent C interface for making PDL usable from e.g. Python
7. use OpenCL or other means to also utilise GPUs if available
The IRC channel (#pdl on irc.perl.org) is a great virtual place to come and ask
questions, or just watch the GitHub messages flow by.
As usual, please give the new release a try and report problems.
Best regards,
Ed
_______________________________________________
pdl-general mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/pdl-general
_______________________________________________
pdl-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pdl-general
