Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - PDL::NiceSlice issue
Thanks Ed & Luis. I will chalk that one as being down to the limitations of
NiceSlice and update that particular code not to use it.
Karl
> On 14 Jan 2024, at 4:42 am, Ed . wrote:
>
> Hi Luis,
>
> Thank you for your help on both this and the “clump” issue.
>
> NiceSlice got changed (as a default) from the simpler but non-string-aware
> Filter::Util::Call implementation (which would mangle eg SQL strings in the
> code) to the string-aware but imperfect in other ways Filter::Simple
> implementation in 2.063_04. You can set the environment variable
> “PDL_NICESLICE_ENGINE” to Filter::Util::Call to use that, the previous
> default.
>
> Filter::Simple uses Text::Balanced, which got some sizable updates from me to
> improve its parsing of Perl code, but still makes some mistakes, as you can
> see. It works by finding all strings, comments, here-docs, and regexes, and
> removes them from the code before filtering, then puts them back after. The
> mistake here is that it misidentifies the text “$x/2/3” as having a regex in
> the middle, which gets removed from the “code” as you see.
>
> Similarly, Karl’s code which has “$y / (multi lines of stuff) / (other
> stuff)” is being misidentified as the “y /(multi lines)/” being a regex.
> Therefore it doesn’t go through the filter, therefore it retains its
> non-correct Perl status, and Perl gives a syntax error.
>
> A workaround is to avoid $y as a variable name, and possibly $x as well
> (since “x” is an operator and may throw off the parsing, for now). I will
> look at updating T:B to make it behave correctly in these cases, but that
> won’t happen today I’m afraid. NiceSlice continues to cause me agony out of
> all proportion to how “nice” it actually is. It’s a pity Perl doesn’t have a
> properly hookable parser yet (at least to my knowledge).
>
> Best regards,
> Ed
>
> From: Luis Mochan <mailto:moc...@icf.unam.mx>
> Sent: 07 January 2024 02:46
> To: Karl Glazebrook <mailto:karlglazebr...@mac.com>
> Cc: perldl <mailto:pdl-general@lists.sourceforge.net>
> Subject: Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 -
> PDL::NiceSlice issue
>
> I found some time back problems with NiceSlice. Actually, with
> Filter::Simple (used by default by NiceSlice, though it can use other
> options, controlled with environment variables). I vaguely recall that
> two slashes, as in two divisions consecutive, confuse it completely,
> as they seem as regular expression.
>
> I found this in my notes of 2022-11-05:
> ---
> Make a filter:
> package MyFilter;
> use v5.36;
> use Filter::Simple;
> FILTER_ONLY
> code_no_comments => sub { say "Code:\n$_" },
> all => sub { say "All:\n$_" };
> 1;
> And a program:
> use v5.36;
> use lib ".";
> use MyFilter;
> my $x=1;
> say $x/2/3;
>
> Execution yields:
>
> Code:
> my $x=1;
> say $x3;
>
> All:
> my $x=1;
> say $x/2/3;
>
> 0.167
>
> This means that /2/ is removed incorrectly. The name of the variable
> ($x and not $y) seems important.
> ---
>
> Regards,
> Luis
>
>
>
>
> On Sun, Jan 07, 2024 at 12:01:27PM +1100, Karl Glazebrook via pdl-general
> wrote:
> >
> >
> > > On 7 Jan 2024, at 11:55 am, Karl Glazebrook via pdl-general
> > > > > <mailto:pdl-general@lists.sourceforge.net>> wrote:
> > >
> > > (The mismatch of line numbers seems to be related to the use of hereto
> > > text earlier in the code, the debugger shows different line numbers)
> > >
> >
> >
> > PS just to prove this point given the line numbers issue:
> >
> >
> >
> >
> > syntax error at /Users/karl/Software/perl/KGB/Redshift.pm line 251, near
> > "$FOO("
> > Execution of /Users/karl/Software/perl/KGB/Redshift.pm aborted due to
> > compilation errors.
> >
> >
> >
> > You can see FOO triggers a compilation error and BAR does not, use
> > PDL::NiceSlice is way at the top of the module
> >
> > Karl
> >
>
>
> > ___
> > pdl-general mailing list
> > pdl-general@lists.sourceforge.net <mailto:pdl-general@lists.sourceforge.net>
> > https://lists.sourceforge.net/lists/listinfo/pdl-general
> > <https://lists.sourceforge.net/lists/listinfo/pdl-general>
>
>
> --
>
>
Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - PDL::NiceSlice issue
Happy 2024 PDL Users!
Nice to see the continued progress with PDL, Ed. Regarding the
Filter::Simple stuff, remember only perl can parse Perl. ;-)
--Chris
On 1/13/2024 12:42 PM, Ed . wrote:
Hi Luis,
Thank you for your help on both this and the “clump” issue.
NiceSlice got changed (as a default) from the simpler but
non-string-aware Filter::Util::Call implementation (which would mangle
eg SQL strings in the code) to the string-aware but imperfect in other
ways Filter::Simple implementation in 2.063_04. You can set the
environment variable “PDL_NICESLICE_ENGINE” to Filter::Util::Call to
use that, the previous default.
Filter::Simple uses Text::Balanced, which got some sizable updates
from me to improve its parsing of Perl code, but still makes some
mistakes, as you can see. It works by finding all strings, comments,
here-docs, and regexes, and removes them from the code before
filtering, then puts them back after. The mistake here is that it
misidentifies the text “$x/2/3” as having a regex in the middle, which
gets removed from the “code” as you see.
Similarly, Karl’s code which has “$y / (multi lines of stuff) / (other
stuff)” is being misidentified as the “y /(multi lines)/” being a
regex. Therefore it doesn’t go through the filter, therefore it
retains its non-correct Perl status, and Perl gives a syntax error.
A workaround is to avoid $y as a variable name, and possibly $x as
well (since “x” is an operator and may throw off the parsing, for
now). I will look at updating T:B to make it behave correctly in these
cases, but that won’t happen today I’m afraid. NiceSlice continues to
cause me agony out of all proportion to how “nice” it actually is.
It’s a pity Perl doesn’t have a properly hookable parser yet (at least
to my knowledge).
Best regards,
Ed
*From: *Luis Mochan <mailto:moc...@icf.unam.mx>
*Sent: *07 January 2024 02:46
*To: *Karl Glazebrook <mailto:karlglazebr...@mac.com>
*Cc: *perldl <mailto:pdl-general@lists.sourceforge.net>
*Subject: *Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 -
PDL::NiceSlice issue
I found some time back problems with NiceSlice. Actually, with
Filter::Simple (used by default by NiceSlice, though it can use other
options, controlled with environment variables). I vaguely recall that
two slashes, as in two divisions consecutive, confuse it completely,
as they seem as regular expression.
I found this in my notes of 2022-11-05:
---
Make a filter:
package MyFilter;
use v5.36;
use Filter::Simple;
FILTER_ONLY
code_no_comments => sub { say "Code:\n$_" },
all => sub { say "All:\n$_" };
1;
And a program:
use v5.36;
use lib ".";
use MyFilter;
my $x=1;
say $x/2/3;
Execution yields:
Code:
my $x=1;
say $x3;
All:
my $x=1;
say $x/2/3;
0.167
This means that /2/ is removed incorrectly. The name of the variable
($x and not $y) seems important.
---
Regards,
Luis
On Sun, Jan 07, 2024 at 12:01:27PM +1100, Karl Glazebrook via
pdl-general wrote:
>
>
> > On 7 Jan 2024, at 11:55 am, Karl Glazebrook via pdl-general
wrote:
> >
> > (The mismatch of line numbers seems to be related to the use of
hereto text earlier in the code, the debugger shows different line
numbers)
> >
>
>
> PS just to prove this point given the line numbers issue:
>
>
>
>
> syntax error at /Users/karl/Software/perl/KGB/Redshift.pm line 251,
near "$FOO("
> Execution of /Users/karl/Software/perl/KGB/Redshift.pm aborted due
to compilation errors.
>
>
>
> You can see FOO triggers a compilation error and BAR does not, use
PDL::NiceSlice is way at the top of the module
>
> Karl
>
> ___
> pdl-general mailing list
> pdl-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pdl-general
--
o
W. Luis Mochán, | tel:(52)(777)329-1734 /<(*)
Instituto de Ciencias Físicas, UNAM | fax:(52)(777)317-5388 `>/ /\
Av. Universidad s/n CP 62210 | (*)/\/ \
Cuernavaca, Morelos, México | moc...@fis.unam.mx /\_/\__/
GPG: 791EB9EB, C949 3F81 6D9B 1191 9A16 C2DF 5F0A C52B 791E B9EB
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - PDL::NiceSlice issue
Hi Luis,
Thank you for your help on both this and the “clump” issue.
NiceSlice got changed (as a default) from the simpler but non-string-aware
Filter::Util::Call implementation (which would mangle eg SQL strings in the
code) to the string-aware but imperfect in other ways Filter::Simple
implementation in 2.063_04. You can set the environment variable
“PDL_NICESLICE_ENGINE” to Filter::Util::Call to use that, the previous default.
Filter::Simple uses Text::Balanced, which got some sizable updates from me to
improve its parsing of Perl code, but still makes some mistakes, as you can
see. It works by finding all strings, comments, here-docs, and regexes, and
removes them from the code before filtering, then puts them back after. The
mistake here is that it misidentifies the text “$x/2/3” as having a regex in
the middle, which gets removed from the “code” as you see.
Similarly, Karl’s code which has “$y / (multi lines of stuff) / (other stuff)”
is being misidentified as the “y /(multi lines)/” being a regex. Therefore it
doesn’t go through the filter, therefore it retains its non-correct Perl
status, and Perl gives a syntax error.
A workaround is to avoid $y as a variable name, and possibly $x as well (since
“x” is an operator and may throw off the parsing, for now). I will look at
updating T:B to make it behave correctly in these cases, but that won’t happen
today I’m afraid. NiceSlice continues to cause me agony out of all proportion
to how “nice” it actually is. It’s a pity Perl doesn’t have a properly hookable
parser yet (at least to my knowledge).
Best regards,
Ed
From: Luis Mochan<mailto:moc...@icf.unam.mx>
Sent: 07 January 2024 02:46
To: Karl Glazebrook<mailto:karlglazebr...@mac.com>
Cc: perldl<mailto:pdl-general@lists.sourceforge.net>
Subject: Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 -
PDL::NiceSlice issue
I found some time back problems with NiceSlice. Actually, with
Filter::Simple (used by default by NiceSlice, though it can use other
options, controlled with environment variables). I vaguely recall that
two slashes, as in two divisions consecutive, confuse it completely,
as they seem as regular expression.
I found this in my notes of 2022-11-05:
---
Make a filter:
package MyFilter;
use v5.36;
use Filter::Simple;
FILTER_ONLY
code_no_comments => sub { say "Code:\n$_" },
all => sub { say "All:\n$_" };
1;
And a program:
use v5.36;
use lib ".";
use MyFilter;
my $x=1;
say $x/2/3;
Execution yields:
Code:
my $x=1;
say $x3;
All:
my $x=1;
say $x/2/3;
0.167
This means that /2/ is removed incorrectly. The name of the variable
($x and not $y) seems important.
---
Regards,
Luis
On Sun, Jan 07, 2024 at 12:01:27PM +1100, Karl Glazebrook via pdl-general wrote:
>
>
> > On 7 Jan 2024, at 11:55 am, Karl Glazebrook via pdl-general
> > wrote:
> >
> > (The mismatch of line numbers seems to be related to the use of hereto text
> > earlier in the code, the debugger shows different line numbers)
> >
>
>
> PS just to prove this point given the line numbers issue:
>
>
>
>
> syntax error at /Users/karl/Software/perl/KGB/Redshift.pm line 251, near
> "$FOO("
> Execution of /Users/karl/Software/perl/KGB/Redshift.pm aborted due to
> compilation errors.
>
>
>
> You can see FOO triggers a compilation error and BAR does not, use
> PDL::NiceSlice is way at the top of the module
>
> Karl
>
> ___
> pdl-general mailing list
> pdl-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pdl-general
--
o
W. Luis Mochán, | tel:(52)(777)329-1734 /<(*)
Instituto de Ciencias Físicas, UNAM | fax:(52)(777)317-5388 `>/ /\
Av. Universidad s/n CP 62210 | (*)/\/ \
Cuernavaca, Morelos, México | moc...@fis.unam.mx /\_/\__/
GPG: 791EB9EB, C949 3F81 6D9B 1191 9A16 C2DF 5F0A C52B 791E B9EB
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - PDL::NiceSlice issue
I found some time back problems with NiceSlice. Actually, with
Filter::Simple (used by default by NiceSlice, though it can use other
options, controlled with environment variables). I vaguely recall that
two slashes, as in two divisions consecutive, confuse it completely,
as they seem as regular expression.
I found this in my notes of 2022-11-05:
---
Make a filter:
package MyFilter;
use v5.36;
use Filter::Simple;
FILTER_ONLY
code_no_comments => sub { say "Code:\n$_" },
all => sub { say "All:\n$_" };
1;
And a program:
use v5.36;
use lib ".";
use MyFilter;
my $x=1;
say $x/2/3;
Execution yields:
Code:
my $x=1;
say $x3;
All:
my $x=1;
say $x/2/3;
0.167
This means that /2/ is removed incorrectly. The name of the variable
($x and not $y) seems important.
---
Regards,
Luis
On Sun, Jan 07, 2024 at 12:01:27PM +1100, Karl Glazebrook via pdl-general wrote:
>
>
> > On 7 Jan 2024, at 11:55 am, Karl Glazebrook via pdl-general
> > wrote:
> >
> > (The mismatch of line numbers seems to be related to the use of hereto text
> > earlier in the code, the debugger shows different line numbers)
> >
>
>
> PS just to prove this point given the line numbers issue:
>
>
>
>
> syntax error at /Users/karl/Software/perl/KGB/Redshift.pm line 251, near
> "$FOO("
> Execution of /Users/karl/Software/perl/KGB/Redshift.pm aborted due to
> compilation errors.
>
>
>
> You can see FOO triggers a compilation error and BAR does not, use
> PDL::NiceSlice is way at the top of the module
>
> Karl
>
> ___
> pdl-general mailing list
> pdl-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pdl-general
--
o
W. Luis Mochán, | tel:(52)(777)329-1734 /<(*)
Instituto de Ciencias Físicas, UNAM | fax:(52)(777)317-5388 `>/ /\
Av. Universidad s/n CP 62210 | (*)/\/ \
Cuernavaca, Morelos, México | moc...@fis.unam.mx /\_/\__/
GPG: 791EB9EB, C949 3F81 6D9B 1191 9A16 C2DF 5F0A C52B 791E B9EB
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
Re: [Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - PDL::NiceSlice issue
> On 7 Jan 2024, at 11:55 am, Karl Glazebrook via pdl-general
> wrote:
>
> (The mismatch of line numbers seems to be related to the use of hereto text
> earlier in the code, the debugger shows different line numbers)
>
PS just to prove this point given the line numbers issue:
syntax error at /Users/karl/Software/perl/KGB/Redshift.pm line 251, near "$FOO("
Execution of /Users/karl/Software/perl/KGB/Redshift.pm aborted due to
compilation errors.
You can see FOO triggers a compilation error and BAR does not, use
PDL::NiceSlice is way at the top of the module
Karl
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
[Pdl-general] Changes I noted PDL2.025 -> PDL2.084 - PDL::NiceSlice issue
Hi all,
This dinosaur just upgraded from PDL v2.025 to v.2.084 (yes, I know that is
lame)
I noticed a few things when running one of my complicated codes, I will start
seperate email threads
This last one - I had PDL::NiceSlice failing randomly halfway through a long
module?
So the module has ‘use PDL::NiceSlice’
then a line line this:
$FOO($ix) .= 1;
causes a syntax error (i.e. in compilation) if placed at line 247 but not at
line 249. Here is the surrounding code with line numbersso you can see it was
innocuous :
syntax error at /Users/karl/Software/perl/KGB/Redshift.pm line 251, near "$FOO("
Execution of /Users/karl/Software/perl/KGB/Redshift.pm aborted due to
compilation errors.
Compilation failed in require at /Users/karl/Software/bin/pdlredshift line 3.
(The mismatch of line numbers seems to be related to the use of hereto text
earlier in the code, the debugger shows different line numbers)
(I was just placing those no-op statements in to try and see what was going
on). As expected using ->slice instead causes the code to compile
I guess we should be cautious of the robustness of NiceSlice?
best
Karl
___
pdl-general mailing list
pdl-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pdl-general
