Thanks Ed.
Yes I can confirm the code snippet bugs as you describe on PDL-2.084
Good detective work and impressive use of git! Hope it is just a bug. I can
also confirm it works on 2.077 and bugs on 2.078
Karl
> On 14 Jan 2024, at 1:16 am, Ed . <ej...@hotmail.com> wrote:
>
> Hi Karl,
>
> Thank you for both reporting this issue, then doing this deep investigation
> as well.
>
> In order to track down exactly when this changed, I stripped down your repro
> code to:
> use PDL::LiteF; # not full PDL so can just build “make core”
> $x = sequence(4)+1;
> $x = $x->mv(-1,0)->slice("0:2");
> $x *= 100;
> $y=&PDL::_clump_int($x,-1);
> print "x=$x y=$y\n";
> $wrong = "$y" eq "[0 0 0]";
> exit $wrong;
>
> As you can see in that comment, if one comments out the multiplication, it
> all still works correctly. The “sequence” can be replaced with hardcoded “pdl
> [1..5]”. But if one changes either the “mv”, or the “slice”, or the inplace
> “*=” (including replacing with “$x = $x * 100” ), or the “_clump_int”, it
> doesn’t misbehave.
>
> Anyway, using the above code to tell me whether I’d found where it started
> failing, I did “git bisect” as follows (noted here to help anyone who wants
> to do this themselves; I’d forgotten and had to look up how):
> git bisect start
> git bisect bad # current “master” is bad
> git bisect good 2.025 # tell it it was working as of 2.025
> perl Makefile.PL && time make core && perl -Mblib repro-script; echo $? #
> kept running this, then:
> git bisect bad # if failing
> git bisect good # if working correctly
> git bisect reset # when finished, to close down the bisect
> Note the use of “make core” which takes about 2 mins from scratch on my
> system, vs about 6 to “make” everything, saving lots of time.
>
> It turns out it was this, which was released with 2.078:
> commit a4678091acf7e450c02a7b0feaf3c7578f37e53f
> Author: Ed J <moha...@users.noreply.github.com
> <mailto:moha...@users.noreply.github.com>>
> Date: Sun Apr 3 22:27:28 2022 +0100
>
> parents of non-flowing trans also track trans_children so can de-register
> on destroy
>
> Basic/Core/pdlapi.c | 46 +++++++++++++++++++---------------------------
> 1 file changed, 19 insertions(+), 27 deletions(-)
>
> I am now investigating a fix, which given how specific it is to trigger, will
> probably be small, and is surely related to book-keeping of parents vs
> children, and flowing transformations.
>
> Best regards,
> Ed
>
> From: Karl Glazebrook via pdl-general
> <mailto:pdl-general@lists.sourceforge.net>
> Sent: 07 January 2024 06:30
> To: perldl <mailto:pdl-general@lists.sourceforge.net>
> Subject: Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - rcols issue
>
> Seems I can’t help myself! I have now found the offending line in rcols() and
> can now reproduce this without rcols. That should at least make it easier to
> track down.
>
> use PDL;
> $x = sequence(100)+1; # This works
> $x = $x->mv(-1,0)->slice("0:3");
> print $x, "\n";
> $x *= 100;
> print $x, "\n";
> $y=&PDL::_clump_int($x,-1);
> print $y, "\n"; # prints [0 0 0 0]
> 1;
>
> It seems to be the particular combination of ->mv and ->slice (that was on
> line 693 of misc.pd). This is a 1D ndarray so the ->mv(-1,0) should do
> nothing. Removing it however makes the problem go away.
>
> Works fine in PDL-2.025. Some bug that has been introduced in ->mv ?
>
> Sorry to the stream of consciousness series of emails. I will stop looking
> now...
>
> Karl
>
>
>
>
>
> On 7 Jan 2024, at 4:32 pm, Karl Glazebrook <karlglazebr...@mac.com
> <mailto:karlglazebr...@mac.com>> wrote:
>
> OK here is some deeper diving in to the problem
>
>
> use PDL;
> $x = rcols 'tmp.dat'; # This does causes the error
> #$x = sequence(4)+1; # This works
> print $x, "\n";
> $x *= 100;
> print $x, "\n";
> $y=&PDL::_clump_int($x,-1);
> print $y, "\n"; # prints [0 0 0 0]
> ;
>
>
>
> looks like the error is happening in the internal pp routine _clump_int in
> slices.pd.
>
>
> I also found ->sever() had the same behaviour with rcols:
>
> pdl> $x = rcols 'tmp.dat'
>
> Reading data into ndarrays of type: [ Double ]
> Read in 4 elements.
>
> pdl> $x *= 100
>
>
> pdl> p $x
>
> [100 200 300 400]
> pdl> p $x->sever
>
> [0 0 0 0]
>
>
>
> Note this is not a general problem with dataflow, if I make a sequence and
> slice or index is then the ops work fine. It is just something weird on the
> ndarray produced by rcols.
>
> Karl
>
>
>
>
>
>
> On 7 Jan 2024, at 3:41 pm, Karl Glazebrook via pdl-general
> <pdl-general@lists.sourceforge.net
> <mailto:pdl-general@lists.sourceforge.net>> wrote:
>
> Ah! I believe the difference between medover and median is a clump(-1) to
> collapse the dimensions
>
> There does it indeed to be something wrong with clump too, so that is
> probably the underlying cause
>
> pdl> $x = rcols 'tmp.dat'
>
> Reading data into ndarrays of type: [ Double ]
> Read in 4 elements.
>
> pdl> p $x
>
> [1 2 3 4]
> pdl> $x *= 100
>
>
> pdl> p $x
>
> [100 200 300 400]
> pdl> p $x->clump(-1)
>
> [0 0 0 0]
>
>
> What could be happening in rcols() that produces an ndarray that behaves like
> that?
>
> Karl
>
>
>
> On 7 Jan 2024, at 12:48 pm, Luis Mochan <moc...@icf.unam.mx
> <mailto:moc...@icf.unam.mx>> wrote:
>
> I noticed that medover and maxover do work as expected in this case.
>
>
> On Sun, Jan 07, 2024 at 11:26:56AM +1100, Karl Glazebrook via pdl-general
> wrote:
>
> Hi all,
>
> This dinosaur just upgraded from PDL v2.025 to v.2.084 (yes, I know that is
> lame)
>
> I noticed a few things when running one of my complicated codes, I will start
> seperate email threads
>
> First there seems to be a serious rcols bug:
>
>
> e.g. create a file
>
> # tmp.dat
> 1
> 2
> 3
> 4
>
>
> Loaded PDL v2.084 (supports bad values)
> pdl> $x = rcols 'tmp.dat'
> Reading data into ndarrays of type: [ Double ]
> Read in 4 elements.
>
> pdl> p $x
> [1 2 3 4]
> pdl> $x *= 100
>
> pdl> p $x
> [100 200 300 400]
> pdl> p median($x)
> 0
> pdl> p $x
> [100 200 300 400]
>
>
> It seems the median function sees the values BEFORE the inplace
> multiplacation, whereas print does not. This is very bad. min() and max() are
> similar. No idea what is going on here! The behaviour or absent from v2.025
>
> Notes
> - making a $x->copy() removes the effect
> - creating $x using sequence also removes, so it is something to do with
> rcols() and not inplace in general?
>
> I’d be interested to know if others can reproduce this. It definitely needs a
> fix
>
> best
>
> Karl
>
>
>
>
>
>
> _______________________________________________
> pdl-general mailing list
> pdl-general@lists.sourceforge.net <mailto:pdl-general@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/pdl-general
> <https://lists.sourceforge.net/lists/listinfo/pdl-general>
>
>
> --
>
> o
> W. Luis Mochán, | tel:(52)(777)329-1734 /<(*)
> Instituto de Ciencias Físicas, UNAM | fax:(52)(777)317-5388 `>/ /\
> Av. Universidad s/n CP 62210 | (*)/\/ \
> Cuernavaca, Morelos, México | moc...@fis.unam.mx
> <mailto:moc...@fis.unam.mx> /\_/\__/
> GPG: 791EB9EB, C949 3F81 6D9B 1191 9A16 C2DF 5F0A C52B 791E B9EB
>
> _______________________________________________
> pdl-general mailing list
> pdl-general@lists.sourceforge.net <mailto:pdl-general@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/pdl-general
> <https://lists.sourceforge.net/lists/listinfo/pdl-general>
_______________________________________________
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
