[perl #131914] [REGRESSION] Rakudo/Moar taking more memory at launch.

[jn...@jnthn.net via RT]

On Wed, 16 Aug 2017 14:53:05 -0700, alex.jakime...@gmail.com wrote:
> Also, it's not any different on HEAD:
> 
>  committable6test: 9658dd98c9dd8ec^^,9658dd98c9dd8ec,HEAD
> say
> "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024
>  AlexDaniel, ¦9658dd98c9dd8ec^^: «58628» ¦9658dd9:
> «138220»
> ¦HEAD(3e70d44): «136436»
> 
> On 2017-08-16 14:27:37, alex.jakime...@gmail.com wrote:
> > Note that the tests on 92bd7e4^ and 92bd7e4 are not very related to
> > this
> > particular issue.
> >
> > To put simply: it went from 58628 to 138220 because of changes
> > related
> > to this
> > commit (or maybe one or two commits before it):
> >
> https://github.com/rakudo/rakudo/commit/9658dd98c9dd8ecbefc606a08357e59718e328c2
> >
> > For Proc → Proc::Async issue see this ticket:
> > https://rt.perl.org/m/ticket/show?id=131915
> >
> > Bisect logs:
> > https://gist.github.com/6a7a6bfbc157bac3d06e9d2d58306846
> > https://gist.github.com/5cf477671560bad21251207895708d94
> >
> >
> > On 2017-08-16 14:21:25, scoli...@gmail.com wrote:
> > > After some test against various release of rakudo it appear that
> > > there
> > > is a huge step in the memory taken by rakudo/moar at launch. (after
> > > 2017.05)
> > > It's like up to 55Mb to 120Mb+
> > >
> > > It was tested by reading the memory info in /proc/pid/statm (the
> > > number is in Kbytes)
> > >
> > >  committable6test:
> > > 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec say
> > > "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024
> > >  AlexDaniel, ¦92bd7e4f54a9^: «55752» ¦92bd7e4:
> > > «55896» ¦9658dd98c9dd8ec^^: «58628» ¦9658dd9: «138220»

I can't reproduce such an sizable increase at HEAD or MoarVM/NQP/Rakduo:

$ ./perl6-m -e 'say "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 
1024'
79904

[perl #131914] [REGRESSION] Rakudo/Moar taking more memory at launch.

[jn...@jnthn.net via RT]

On Wed, 16 Aug 2017 14:53:05 -0700, alex.jakime...@gmail.com wrote:
> Also, it's not any different on HEAD:
> 
>  committable6test: 9658dd98c9dd8ec^^,9658dd98c9dd8ec,HEAD
> say
> "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024
>  AlexDaniel, ¦9658dd98c9dd8ec^^: «58628» ¦9658dd9:
> «138220»
> ¦HEAD(3e70d44): «136436»
> 
> On 2017-08-16 14:27:37, alex.jakime...@gmail.com wrote:
> > Note that the tests on 92bd7e4^ and 92bd7e4 are not very related to
> > this
> > particular issue.
> >
> > To put simply: it went from 58628 to 138220 because of changes
> > related
> > to this
> > commit (or maybe one or two commits before it):
> >
> https://github.com/rakudo/rakudo/commit/9658dd98c9dd8ecbefc606a08357e59718e328c2
> >
> > For Proc → Proc::Async issue see this ticket:
> > https://rt.perl.org/m/ticket/show?id=131915
> >
> > Bisect logs:
> > https://gist.github.com/6a7a6bfbc157bac3d06e9d2d58306846
> > https://gist.github.com/5cf477671560bad21251207895708d94
> >
> >
> > On 2017-08-16 14:21:25, scoli...@gmail.com wrote:
> > > After some test against various release of rakudo it appear that
> > > there
> > > is a huge step in the memory taken by rakudo/moar at launch. (after
> > > 2017.05)
> > > It's like up to 55Mb to 120Mb+
> > >
> > > It was tested by reading the memory info in /proc/pid/statm (the
> > > number is in Kbytes)
> > >
> > >  committable6test:
> > > 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec say
> > > "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024
> > >  AlexDaniel, ¦92bd7e4f54a9^: «55752» ¦92bd7e4:
> > > «55896» ¦9658dd98c9dd8ec^^: «58628» ¦9658dd9: «138220»

I can't reproduce such an sizable increase at HEAD or MoarVM/NQP/Rakduo:

$ ./perl6-m -e 'say "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 
1024'
79904

[perl #131857] [REGRESSION] Aliased named parameters broke in a recent MoarVM commit

[jn...@jnthn.net via RT]

On Mon, 07 Aug 2017 19:48:36 -0700, ddgr...@gmail.com wrote:
> [22:44]  m: sub foo(:color(:$colour)) { $colour + 1 }; my
> $s;
> for ^100 { $s += foo(:color($_)) }; say $s; say now - INIT now
> [22:44] <+camelia> rakudo-moar a91ad2: OUTPUT: «Unexpected named
> argument
> 'color' passed␤  in sub foo at  line 1␤  in block  at 
> line
> 1␤␤»
> [22:44]  bisectable6: sub foo(:color(:$colour)) { $colour
> + 1
> }; my $s; for ^100 { $s += foo(:color($_)) }; say $s;
> [22:44] <+bisectable6> MasterDuke, Bisecting by exit code (old=2015.12
> new=a91ad2d). Old exit code: 0
> [22:44] <+bisectable6> MasterDuke, bisect log:
> https://gist.github.com/651d00f77245537de7326c9316f4513e
> [22:44] <+bisectable6> MasterDuke, (2017-08-04)
> https://github.com/rakudo/rakudo/commit/8bed4a67b44acb6c0e9f057d11a2ab2f0ea5d8cd
> [22:45]  haw
> [22:45]  c: 2017.07 sub foo(:color(:$colour)) { $colour +
> 1 };
> my $s; for ^100 { $s += foo(:color($_)) }; say $s; say now - INIT
> now
> [22:45] <+committable6> MasterDuke, ¦2017.07: «5050␤1.4953885»

Fixed; test in S06-signature/named-parameters.t.

[perl #131857] [REGRESSION] Aliased named parameters broke in a recent MoarVM commit

[jn...@jnthn.net via RT]

On Mon, 07 Aug 2017 19:48:36 -0700, ddgr...@gmail.com wrote:
> [22:44]  m: sub foo(:color(:$colour)) { $colour + 1 }; my
> $s;
> for ^100 { $s += foo(:color($_)) }; say $s; say now - INIT now
> [22:44] <+camelia> rakudo-moar a91ad2: OUTPUT: «Unexpected named
> argument
> 'color' passed␤  in sub foo at  line 1␤  in block  at 
> line
> 1␤␤»
> [22:44]  bisectable6: sub foo(:color(:$colour)) { $colour
> + 1
> }; my $s; for ^100 { $s += foo(:color($_)) }; say $s;
> [22:44] <+bisectable6> MasterDuke, Bisecting by exit code (old=2015.12
> new=a91ad2d). Old exit code: 0
> [22:44] <+bisectable6> MasterDuke, bisect log:
> https://gist.github.com/651d00f77245537de7326c9316f4513e
> [22:44] <+bisectable6> MasterDuke, (2017-08-04)
> https://github.com/rakudo/rakudo/commit/8bed4a67b44acb6c0e9f057d11a2ab2f0ea5d8cd
> [22:45]  haw
> [22:45]  c: 2017.07 sub foo(:color(:$colour)) { $colour +
> 1 };
> my $s; for ^100 { $s += foo(:color($_)) }; say $s; say now - INIT
> now
> [22:45] <+committable6> MasterDuke, ¦2017.07: «5050␤1.4953885»

Fixed; test in S06-signature/named-parameters.t.

[perl #132016] Supply.merge and signals ( signal(SIGTERM).merge(signal(SIGINT)) )

[jn...@jnthn.net via RT]

On Fri, 01 Sep 2017 20:15:07 -0700, alex.jakime...@gmail.com wrote:
> Command:
> # run it and then send SIGINT (e.g. with Ctrl+C)
> perl6 -e 'react whenever signal(SIGINT).merge(signal(SIGTERM)) { say
> ‘hey!’; exit 0 }'
> 
> Result:
> hey!
> 
> 
> Command:
> # run it and then send SIGINT (e.g. with Ctrl+C)
> perl6 -e 'react whenever signal(SIGTERM).merge(signal(SIGINT)) { say
> ‘hey!’; exit 0 }'
> 
> Result:
> Unhandled exception in code scheduled on thread 10
> Cannot resolve caller postcircumfix:<[ ]>(Mu, Int); none of these
> signatures match:
> (\SELF, Any:U $type, |c is raw)
> (\SELF, int $pos)
> (\SELF, int $pos, Mu \assignee)
> (\SELF, int $pos, Mu :$BIND! is raw)
> (\SELF, int $pos, :$delete!, *%other)
> (\SELF, int $pos, :$exists!, *%other)
> (\SELF, int $pos, :$kv!, *%other)
> (\SELF, int $pos, :$p!, *%other)
> (\SELF, int $pos, :$k!, *%other)
> (\SELF, int $pos, :$v!, *%other)
> (\SELF, Int:D $pos)
> (\SELF, Int:D $pos, Mu \assignee)
> (\SELF, Int:D $pos, Mu :$BIND! is raw)
> (\SELF, Int:D $pos, :$delete!, *%other)
> (\SELF, Int:D $pos, :$exists!, *%other)
> (\SELF, Int:D $pos, :$kv!, *%other)
> (\SELF, Int:D $pos, :$p!, *%other)
> (\SELF, Int:D $pos, :$k!, *%other)
> (\SELF, Int:D $pos, :$v!, *%other)
> (\SELF, Any:D \pos)
> (\SELF, Any:D \pos, Mu \assignee)
> (\SELF, Any:D \pos, Mu :$BIND! is raw)
> (\SELF, Any:D \pos, :$delete!, *%other)
> (\SELF, Any:D \pos, :$exists!, *%other)
> (\SELF, Any:D \pos, :$kv!, *%other)
> (\SELF, Any:D \pos, :$p!, *%other)
> (\SELF, Any:D \pos, :$k!, *%other)
> (\SELF, Any:D \pos, :$v!, *%other)
> (\SELF, Iterable:D \pos)
> (\SELF, Iterable:D \pos, Mu \val)
> (\SELF, Iterable:D \pos, :$BIND!)
> (\SELF, Iterable:D \pos, :$delete!, *%other)
> (\SELF, Iterable:D \pos, :$exists!, *%other)
> (\SELF, Iterable:D \pos, :$kv!, *%other)
> (\SELF, Iterable:D \pos, :$p!, *%other)
> (\SELF, Iterable:D \pos, :$k!, *%other)
> (\SELF, Iterable:D \pos, :$v!, *%other)
> (\SELF, Callable:D $block)
> (\SELF, Callable:D $block, Mu \assignee)
> (\SELF, Callable:D $block, :$BIND!)
> (\SELF, Callable:D $block, :$delete!, *%other)
> (\SELF, Callable:D $block, :$exists!, *%other)
> (\SELF, Callable:D $block, :$kv!, *%other)
> (\SELF, Callable:D $block, :$p!, *%other)
> (\SELF, Callable:D $block, :$k!, *%other)
> (\SELF, Callable:D $block, :$v!, *%other)
> (\SELF, Whatever:D)
> (\SELF, Whatever:D, Mu \assignee)
> (\SELF, Whatever:D, :$BIND!)
> (\SELF, Whatever:D, :$delete!, *%other)
> (\SELF, Whatever:D, :$exists!, *%other)
> (\SELF, Whatever:D, :$kv!, *%other)
> (\SELF, Whatever:D, :$p!, *%other)
> (\SELF, Whatever:D, :$k!, *%other)
> (\SELF, Whatever:D, :$v!, *%other)
> (\SELF, HyperWhatever:D, *%adv)
> (\SELF, HyperWhatever:D, Mu \assignee)
> (\SELF, :$BIND!)
> (\SELF, :$delete!, *%other)
> (\SELF, :$exists!, *%other)
> (\SELF, :$kv!, *%other)
> (\SELF, :$p!, *%other)
> (\SELF, :$k!, *%other)
> (\SELF, :$v!, *%other)
> (\SELF, *%other)
> 
> 
> 
> I think there's something wrong.

Fixed in Rakudo commits 1f411693bd and 13b6a33c20. Tagging as test needed.

/jnthn

[perl #132016] Supply.merge and signals ( signal(SIGTERM).merge(signal(SIGINT)) )

[jn...@jnthn.net via RT]

On Fri, 01 Sep 2017 20:15:07 -0700, alex.jakime...@gmail.com wrote:
> Command:
> # run it and then send SIGINT (e.g. with Ctrl+C)
> perl6 -e 'react whenever signal(SIGINT).merge(signal(SIGTERM)) { say
> ‘hey!’; exit 0 }'
> 
> Result:
> hey!
> 
> 
> Command:
> # run it and then send SIGINT (e.g. with Ctrl+C)
> perl6 -e 'react whenever signal(SIGTERM).merge(signal(SIGINT)) { say
> ‘hey!’; exit 0 }'
> 
> Result:
> Unhandled exception in code scheduled on thread 10
> Cannot resolve caller postcircumfix:<[ ]>(Mu, Int); none of these
> signatures match:
> (\SELF, Any:U $type, |c is raw)
> (\SELF, int $pos)
> (\SELF, int $pos, Mu \assignee)
> (\SELF, int $pos, Mu :$BIND! is raw)
> (\SELF, int $pos, :$delete!, *%other)
> (\SELF, int $pos, :$exists!, *%other)
> (\SELF, int $pos, :$kv!, *%other)
> (\SELF, int $pos, :$p!, *%other)
> (\SELF, int $pos, :$k!, *%other)
> (\SELF, int $pos, :$v!, *%other)
> (\SELF, Int:D $pos)
> (\SELF, Int:D $pos, Mu \assignee)
> (\SELF, Int:D $pos, Mu :$BIND! is raw)
> (\SELF, Int:D $pos, :$delete!, *%other)
> (\SELF, Int:D $pos, :$exists!, *%other)
> (\SELF, Int:D $pos, :$kv!, *%other)
> (\SELF, Int:D $pos, :$p!, *%other)
> (\SELF, Int:D $pos, :$k!, *%other)
> (\SELF, Int:D $pos, :$v!, *%other)
> (\SELF, Any:D \pos)
> (\SELF, Any:D \pos, Mu \assignee)
> (\SELF, Any:D \pos, Mu :$BIND! is raw)
> (\SELF, Any:D \pos, :$delete!, *%other)
> (\SELF, Any:D \pos, :$exists!, *%other)
> (\SELF, Any:D \pos, :$kv!, *%other)
> (\SELF, Any:D \pos, :$p!, *%other)
> (\SELF, Any:D \pos, :$k!, *%other)
> (\SELF, Any:D \pos, :$v!, *%other)
> (\SELF, Iterable:D \pos)
> (\SELF, Iterable:D \pos, Mu \val)
> (\SELF, Iterable:D \pos, :$BIND!)
> (\SELF, Iterable:D \pos, :$delete!, *%other)
> (\SELF, Iterable:D \pos, :$exists!, *%other)
> (\SELF, Iterable:D \pos, :$kv!, *%other)
> (\SELF, Iterable:D \pos, :$p!, *%other)
> (\SELF, Iterable:D \pos, :$k!, *%other)
> (\SELF, Iterable:D \pos, :$v!, *%other)
> (\SELF, Callable:D $block)
> (\SELF, Callable:D $block, Mu \assignee)
> (\SELF, Callable:D $block, :$BIND!)
> (\SELF, Callable:D $block, :$delete!, *%other)
> (\SELF, Callable:D $block, :$exists!, *%other)
> (\SELF, Callable:D $block, :$kv!, *%other)
> (\SELF, Callable:D $block, :$p!, *%other)
> (\SELF, Callable:D $block, :$k!, *%other)
> (\SELF, Callable:D $block, :$v!, *%other)
> (\SELF, Whatever:D)
> (\SELF, Whatever:D, Mu \assignee)
> (\SELF, Whatever:D, :$BIND!)
> (\SELF, Whatever:D, :$delete!, *%other)
> (\SELF, Whatever:D, :$exists!, *%other)
> (\SELF, Whatever:D, :$kv!, *%other)
> (\SELF, Whatever:D, :$p!, *%other)
> (\SELF, Whatever:D, :$k!, *%other)
> (\SELF, Whatever:D, :$v!, *%other)
> (\SELF, HyperWhatever:D, *%adv)
> (\SELF, HyperWhatever:D, Mu \assignee)
> (\SELF, :$BIND!)
> (\SELF, :$delete!, *%other)
> (\SELF, :$exists!, *%other)
> (\SELF, :$kv!, *%other)
> (\SELF, :$p!, *%other)
> (\SELF, :$k!, *%other)
> (\SELF, :$v!, *%other)
> (\SELF, *%other)
> 
> 
> 
> I think there's something wrong.

Fixed in Rakudo commits 1f411693bd and 13b6a33c20. Tagging as test needed.

/jnthn

[perl #131961] Built-in decoder is dropping bytes on the floor

[jn...@jnthn.net via RT]

On Fri, 25 Aug 2017 18:56:37 -0700, alex.jakime...@gmail.com wrote:
> The input file for this problem is ≈15 MB so please bear with external
> link:
> https://files.progarm.org/golfed.gz (1.6 MB compressed)
> 
> Command:
> perl6 -ne 'say $++' golfed
> # or
> perl6 -ne 'say $++' < golfed
> 
> Result:
> 0
> 1
> 2
> … … …
> 257568
> 257569
> 257570
> Malformed UTF-8
>   in block  at -e line 1
> 
> 
> There's no malformed UTF-8 in the file. And if you don't believe me,
> try this:
> 
> cat golfed | perl6 -ne 'say $++'
> 
> There are at least three possible outcomes (it is not as stable as
> previous examples):
> (*) Fails after 257570, just like in the previous example
> (*) Fails after 121712
> (*) No error, goes through the whole file just fine
> 
> 
>  sounds more likely to be I/O related than unicode related
>  like it's dropping bytes on the floor and if the utf8
> decoder was in (or lands in) the middle of a sequence, boom
> 
> 
> IRC log: https://irclog.perlgeek.de/perl6-dev/2017-08-26#i_15071860
> 
> This issue may be related:
> https://gist.github.com/coke/3feef738886b1e5af79a1ca636146075

Was actually the decoder itself dropping the bytes in a fast path -> slow path 
transition. Fixed, and test added in S32-io/io-handle.t.

[perl #131961] Built-in decoder is dropping bytes on the floor

[jn...@jnthn.net via RT]

On Fri, 25 Aug 2017 18:56:37 -0700, alex.jakime...@gmail.com wrote:
> The input file for this problem is ≈15 MB so please bear with external
> link:
> https://files.progarm.org/golfed.gz (1.6 MB compressed)
> 
> Command:
> perl6 -ne 'say $++' golfed
> # or
> perl6 -ne 'say $++' < golfed
> 
> Result:
> 0
> 1
> 2
> … … …
> 257568
> 257569
> 257570
> Malformed UTF-8
>   in block  at -e line 1
> 
> 
> There's no malformed UTF-8 in the file. And if you don't believe me,
> try this:
> 
> cat golfed | perl6 -ne 'say $++'
> 
> There are at least three possible outcomes (it is not as stable as
> previous examples):
> (*) Fails after 257570, just like in the previous example
> (*) Fails after 121712
> (*) No error, goes through the whole file just fine
> 
> 
>  sounds more likely to be I/O related than unicode related
>  like it's dropping bytes on the floor and if the utf8
> decoder was in (or lands in) the middle of a sequence, boom
> 
> 
> IRC log: https://irclog.perlgeek.de/perl6-dev/2017-08-26#i_15071860
> 
> This issue may be related:
> https://gist.github.com/coke/3feef738886b1e5af79a1ca636146075

Was actually the decoder itself dropping the bytes in a fast path -> slow path 
transition. Fixed, and test added in S32-io/io-handle.t.

[perl #132030] [REGRESSION] Broken Text::CSV tests and possibly other ecosystem fallout

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 00:54:27 -0700, alex.jakime...@gmail.com wrote:
> <[Tux]> CSV tests started to fail
> <[Tux]> # at t/90_csv.t line 260
> <[Tux]> # expected: $[["1", "2", "3"], ["2", "a b", ""]]
> <[Tux]> #  got: $[["1", "2", "3"],]
> <[Tux]> # Failed test 'AOH parse out'
> 
> 
> Bisectable points to
> https://github.com/rakudo/rakudo/commit/4b02b8aadcb47072bc87fb8be8069177b74cd59d
> 
> bisect log: https://gist.github.com/62a876b09bfecc9aa305061e384f43ce
> double-check with committable:
> https://gist.github.com/50bd16934e6bc93ad49a1659cf31bf06
> 
> 
> I'm suspecting that Whateverable issues that I'm seeing right now are
> also associated with this.

Text::CSV is failing to close an output handle on all codepaths in one of its 
methods, and can be fixed by a patch to do that:

https://gist.github.com/jnthn/a999df1f89d24cdc6ffac42ca55be806

Failing to close output handles has been clearly documented (and yes, 
documented well before the recent buffering change) as something that can cause 
data loss. Default output buffering just makes it quite a lot more likely to 
show up.

While there will be some ecosystem fallout like this, unfortunately I don't 
think it's avoidable. If we whip out the patch that turns output buffering on 
by default for non-TTYs for this release, then when will we include it? The 
longer we leave it, the more painful it will be, because more code will be 
written that is careless with handles.

I don't think "leave it off by default" is a good option either, otherwise we 
get to spend the next decade hearing "Perl 6 I/O is slow" because it'd be one 
of the only languages that doesn't buffer its output without an explicit flag 
being passed to enable that (which nearly nobody doing quick benchmarks will 
know to use).

In summary, if we had a time machine, going back and implementing this when the 
ecosystem was smaller would be a good use of it. But we don't, and we won't in 
the future, so I suspect the best thing to do is take the hit now.

Will leave this ticket open a little longer for further comments/discussion, 
but my intention is to reject it.

/jnthn

[perl #132030] [REGRESSION] Broken Text::CSV tests and possibly other ecosystem fallout

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 00:54:27 -0700, alex.jakime...@gmail.com wrote:
> <[Tux]> CSV tests started to fail
> <[Tux]> # at t/90_csv.t line 260
> <[Tux]> # expected: $[["1", "2", "3"], ["2", "a b", ""]]
> <[Tux]> #  got: $[["1", "2", "3"],]
> <[Tux]> # Failed test 'AOH parse out'
> 
> 
> Bisectable points to
> https://github.com/rakudo/rakudo/commit/4b02b8aadcb47072bc87fb8be8069177b74cd59d
> 
> bisect log: https://gist.github.com/62a876b09bfecc9aa305061e384f43ce
> double-check with committable:
> https://gist.github.com/50bd16934e6bc93ad49a1659cf31bf06
> 
> 
> I'm suspecting that Whateverable issues that I'm seeing right now are
> also associated with this.

Text::CSV is failing to close an output handle on all codepaths in one of its 
methods, and can be fixed by a patch to do that:

https://gist.github.com/jnthn/a999df1f89d24cdc6ffac42ca55be806

Failing to close output handles has been clearly documented (and yes, 
documented well before the recent buffering change) as something that can cause 
data loss. Default output buffering just makes it quite a lot more likely to 
show up.

While there will be some ecosystem fallout like this, unfortunately I don't 
think it's avoidable. If we whip out the patch that turns output buffering on 
by default for non-TTYs for this release, then when will we include it? The 
longer we leave it, the more painful it will be, because more code will be 
written that is careless with handles.

I don't think "leave it off by default" is a good option either, otherwise we 
get to spend the next decade hearing "Perl 6 I/O is slow" because it'd be one 
of the only languages that doesn't buffer its output without an explicit flag 
being passed to enable that (which nearly nobody doing quick benchmarks will 
know to use).

In summary, if we had a time machine, going back and implementing this when the 
ecosystem was smaller would be a good use of it. But we don't, and we won't in 
the future, so I suspect the best thing to do is take the hit now.

Will leave this ticket open a little longer for further comments/discussion, 
but my intention is to reject it.

/jnthn

[perl #132029] Floppage of atomic tests

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 00:36:34 -0700, n...@detonation.org wrote:
> A golfed version that reliably fails:
> 
> {
> my Int $test-cont = 42;
> ⚛$test-cont;
> }
> {
> my atomicint $set = 0;
> start { sleep 1; $set ⚛= 1 };
> until ⚛$set { }
> }
> 
> The important bit is the Int being read atomicly before reading the
> atomicint. The sleep makes it fail reliably.

And that in turn got it down to being a spesh bug (this golf reliably worked 
with MVM_SPESH_DISABLE=1), which was that it mis-optimized the call to the 
wrong multi candidate, due to some bogus logging. Fixed with 
https://github.com/MoarVM/MoarVM/commit/5bf652edbb7f3eb067e0e753d945109282051c62
 (though the spesh output it isn't too great, so I'll now see if I can do some 
further tweaks to get it inlining the prefix:<⚛> for atomicint).

/jnthn

[perl #132029] Floppage of atomic tests

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 00:36:34 -0700, n...@detonation.org wrote:
> A golfed version that reliably fails:
> 
> {
> my Int $test-cont = 42;
> ⚛$test-cont;
> }
> {
> my atomicint $set = 0;
> start { sleep 1; $set ⚛= 1 };
> until ⚛$set { }
> }
> 
> The important bit is the Int being read atomicly before reading the
> atomicint. The sleep makes it fail reliably.

And that in turn got it down to being a spesh bug (this golf reliably worked 
with MVM_SPESH_DISABLE=1), which was that it mis-optimized the call to the 
wrong multi candidate, due to some bogus logging. Fixed with 
https://github.com/MoarVM/MoarVM/commit/5bf652edbb7f3eb067e0e753d945109282051c62
 (though the spesh output it isn't too great, so I'll now see if I can do some 
further tweaks to get it inlining the prefix:<⚛> for atomicint).

/jnthn

[perl #132030] [REGRESSION] Broken Text::CSV tests and possibly other ecosystem fallout

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 05:29:00 -0700, 1parr...@gmail.com wrote:
> Perl 5 programmers are used to being casual about closing file
> handles. Obviously, 6 requires a change of habits. A cultural shift,
> if that's not too pretentious a term. Perhaps it needs to be mentioned
> in large, friendly letters somewhere in the documentation?
> 
It's called out in the open docs here:

https://docs.perl6.org/routine/open#(IO::Handle)_method_open

But it would be worth a mention here also:

https://docs.perl6.org/language/5to6-perlfunc#open

I've filed a docs issue about that:

https://github.com/perl6/doc/issues/1536

[perl #132030] [REGRESSION] Broken Text::CSV tests and possibly other ecosystem fallout

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 05:29:00 -0700, 1parr...@gmail.com wrote:
> Perl 5 programmers are used to being casual about closing file
> handles. Obviously, 6 requires a change of habits. A cultural shift,
> if that's not too pretentious a term. Perhaps it needs to be mentioned
> in large, friendly letters somewhere in the documentation?
> 
It's called out in the open docs here:

https://docs.perl6.org/routine/open#(IO::Handle)_method_open

But it would be worth a mention here also:

https://docs.perl6.org/language/5to6-perlfunc#open

I've filed a docs issue about that:

https://github.com/perl6/doc/issues/1536

[perl #131985] [CONC] Curly-brace string interpolation is not thread safe

[jn...@jnthn.net via RT]

On Mon, 28 Aug 2017 17:59:39 -0700, j.david.l...@apple.com wrote:
> This short program behaves strangely on my system:
> 
> ```
> #!/usr/bin/env perl6
> 
> use v6.c;
> 
> await((1 .. 5).map: -> $tid {
>start {
>   for (1 .. 100) -> $index {
>  my $s-tid   = "{$tid}";
>  my $s-index = "{$index}";
> 
> say "thread $tid   != stringified $s-tid" unless $tid ~~ $s-tid;
> say "index  $index != stringified $s-index" unless $index ~~ $s-index;
> }
> }
> });
> ```
> 
> ... producing output like:
> 
> 
> ```
> $ ./bug
> thread 2   != stringified 1
> index  54 != stringified 11
> thread 5   != stringified 4
> thread 3   != stringified 2
> index  61 != stringified 39
> index  42 != stringified 16
> thread 5   != stringified 2
> thread 2   != stringified 1
> index  66 != stringified 53
> index  57 != stringified 79
> thread 1   != stringified 3
> index  59 != stringified 70
> thread 4   != stringified 1
> index  36 != stringified 71
> index  39 != stringified 76
> index  41 != stringified 97
> thread 2   != stringified 5
> 
> $ ./bug
> thread 2   != stringified 5
> index  26 != stringified 35
> index  47 != stringified 58
> thread 4   != stringified 2
> index  19 != stringified 33
> ```
> 
> It's hard to imagine that this could be correct.
> 
> More information:
> 
> ```
> $ perl6 --version
> This is Rakudo version 2017.07 built on MoarVM version 2017.07
> implementing Perl 6.c.
> ```

Fixed, and test added to S17-promise/start.t.

[perl #131985] [CONC] Curly-brace string interpolation is not thread safe

[jn...@jnthn.net via RT]

On Mon, 28 Aug 2017 17:59:39 -0700, j.david.l...@apple.com wrote:
> This short program behaves strangely on my system:
> 
> ```
> #!/usr/bin/env perl6
> 
> use v6.c;
> 
> await((1 .. 5).map: -> $tid {
>start {
>   for (1 .. 100) -> $index {
>  my $s-tid   = "{$tid}";
>  my $s-index = "{$index}";
> 
> say "thread $tid   != stringified $s-tid" unless $tid ~~ $s-tid;
> say "index  $index != stringified $s-index" unless $index ~~ $s-index;
> }
> }
> });
> ```
> 
> ... producing output like:
> 
> 
> ```
> $ ./bug
> thread 2   != stringified 1
> index  54 != stringified 11
> thread 5   != stringified 4
> thread 3   != stringified 2
> index  61 != stringified 39
> index  42 != stringified 16
> thread 5   != stringified 2
> thread 2   != stringified 1
> index  66 != stringified 53
> index  57 != stringified 79
> thread 1   != stringified 3
> index  59 != stringified 70
> thread 4   != stringified 1
> index  36 != stringified 71
> index  39 != stringified 76
> index  41 != stringified 97
> thread 2   != stringified 5
> 
> $ ./bug
> thread 2   != stringified 5
> index  26 != stringified 35
> index  47 != stringified 58
> thread 4   != stringified 2
> index  19 != stringified 33
> ```
> 
> It's hard to imagine that this could be correct.
> 
> More information:
> 
> ```
> $ perl6 --version
> This is Rakudo version 2017.07 built on MoarVM version 2017.07
> implementing Perl 6.c.
> ```

Fixed, and test added to S17-promise/start.t.

[perl #130855] [SPESH][OPTIMIZER] glitch with anon vars causes incorrect `Cannot assign to an immutable value` error

[jn...@jnthn.net via RT]

On Fri, 24 Feb 2017 15:02:23 -0800, c...@zoffix.com wrote:
> Expected no errors, but in the first case it complains about
> assignment to immutable value even though none exist in the code.
> 
> zoffix@VirtualBox:~$ perl6 -e 'm: sub foo () {$ = 42}; for ^200 {
> $ = foo }; say now - INIT now'
> Cannot assign to an immutable value
>   in block  at -e line 1
> 
Was fixed in spesh work last month or so; test now added to 
S04-declarations/state.t so we don't regress.

[perl #130855] [SPESH][OPTIMIZER] glitch with anon vars causes incorrect `Cannot assign to an immutable value` error

[jn...@jnthn.net via RT]

On Fri, 24 Feb 2017 15:02:23 -0800, c...@zoffix.com wrote:
> Expected no errors, but in the first case it complains about
> assignment to immutable value even though none exist in the code.
> 
> zoffix@VirtualBox:~$ perl6 -e 'm: sub foo () {$ = 42}; for ^200 {
> $ = foo }; say now - INIT now'
> Cannot assign to an immutable value
>   in block  at -e line 1
> 
Was fixed in spesh work last month or so; test now added to 
S04-declarations/state.t so we don't regress.

[perl #132035] [LTA] temp on an undeclared dynamic variable produces a not very helpful error

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 11:02:59 -0700, allber...@gmail.com wrote:
> Or maybe it's not that tricky after all; seems the Failure is actually a
> useful one (X::Dynamic:NotFound), so introspection of the Failure would
> reveal the issue.
> 
Indeed. I added code to check if the thing we were trying to temporize was a 
Failure and just throw that instead of the more generic error, which improves 
the error reporting and has no ill effects on the spectest. Also added a test 
in S04-blocks-and-statements/temp.t to cover the improvement.

[perl #132035] [LTA] temp on an undeclared dynamic variable produces a not very helpful error

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 11:02:59 -0700, allber...@gmail.com wrote:
> Or maybe it's not that tricky after all; seems the Failure is actually a
> useful one (X::Dynamic:NotFound), so introspection of the Failure would
> reveal the issue.
> 
Indeed. I added code to check if the thing we were trying to temporize was a 
Failure and just throw that instead of the more generic error, which improves 
the error reporting and has no ill effects on the spectest. Also added a test 
in S04-blocks-and-statements/temp.t to cover the improvement.

[perl #132030] [REGRESSION] Broken Text::CSV tests and possibly other ecosystem fallout

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 09:11:19 -0700, allber...@gmail.com wrote:
> On Tue, Sep 5, 2017 at 5:40 AM, jn...@jnthn.net via RT <
> perl6-bugs-follo...@perl.org> wrote:
> 
> > Failing to close output handles has been clearly documented (and yes,
> > documented well before the recent buffering change) as something that can
> > cause data loss. Default output buffering just makes it quite a lot more
> > likely to show up.
> >
> > While there will be some ecosystem fallout like this, unfortunately I
> > don't think it's avoidable. If we whip out the patch that turns output
> > buffering on by default for non-TTYs for this release, then when will we
> > include it? The longer we leave it, the more painful it will be, because
> > more code will be written that is careless with handles.
> >
> > I don't think "leave it off by default" is a good option either, otherwise
> > we get to spend the next decade hearing "Perl 6 I/O is slow" because it'd
> > be one of the only languages that doesn't buffer its output without an
> > explicit flag being passed to enable that (which nearly nobody doing quick
> > benchmarks will know to use).
> >
> 
> Are we missing something to flush/close handles at exit? Leaving it to a GC
> that may not finalize before exit is not really an option.
> 
To recap the IRC discussion yesterday: no, we haven't had this so far (except 
for stdout/stderr), and have gotten away with it due to the lack of output 
buffering. At present, we can either choose between:

1) Start keeping a list of open files, and at exit close them (flushing is 
already part of closing). This can be done at Perl 6 level, in the same place 
we make sure to run END blocks.

2) Having unclosed handles possible to GC, and closing them if/when they get 
GC'd.

Today we are doing #2. We could switch to doing #1. We can't currently do both, 
because the moment we start keeping a list of open handles then they can't be 
GC'd, and so #2 can't happen.

My initial inclination was to preserve behavior #2, though others have pointed 
out that behavior #1 is more useful for debugging in that it ensures log files, 
for example, will be written in the event of a crash, and a program relying on 
behavior #2 could already run out of handles today anyway if it were less lucky 
with GC timing. This is a fair argument, and the automatic close at exit might 
be softer on the ecosystem too (but would have done nothing for the Text::CSV 
case, which is the original subject of this ticket, because it wrote a file, 
didn't close it, then separately opened it for reading).

There's probably enough consensus to switch to option #1, and lizmat++ 
mentioned maybe looking into a patch to do that.

In the longer run, we can have both, but it depends on implementing weak 
references. In terms of backend support, the JVM does have them, and it seems 
there's an npm package [1] exposing v8 weak refs so a Node.js backend could 
support that also. I'm OK with adding them to MoarVM in the future, but both 
doing that and exposing weak references at Perl 6 level would be a non-small, 
and certainly non-trivial, task.

[perl #132030] [REGRESSION] Broken Text::CSV tests and possibly other ecosystem fallout

[jn...@jnthn.net via RT]

On Tue, 05 Sep 2017 09:11:19 -0700, allber...@gmail.com wrote:
> On Tue, Sep 5, 2017 at 5:40 AM, jn...@jnthn.net via RT <
> perl6-bugs-follo...@perl.org> wrote:
> 
> > Failing to close output handles has been clearly documented (and yes,
> > documented well before the recent buffering change) as something that can
> > cause data loss. Default output buffering just makes it quite a lot more
> > likely to show up.
> >
> > While there will be some ecosystem fallout like this, unfortunately I
> > don't think it's avoidable. If we whip out the patch that turns output
> > buffering on by default for non-TTYs for this release, then when will we
> > include it? The longer we leave it, the more painful it will be, because
> > more code will be written that is careless with handles.
> >
> > I don't think "leave it off by default" is a good option either, otherwise
> > we get to spend the next decade hearing "Perl 6 I/O is slow" because it'd
> > be one of the only languages that doesn't buffer its output without an
> > explicit flag being passed to enable that (which nearly nobody doing quick
> > benchmarks will know to use).
> >
> 
> Are we missing something to flush/close handles at exit? Leaving it to a GC
> that may not finalize before exit is not really an option.
> 
To recap the IRC discussion yesterday: no, we haven't had this so far (except 
for stdout/stderr), and have gotten away with it due to the lack of output 
buffering. At present, we can either choose between:

1) Start keeping a list of open files, and at exit close them (flushing is 
already part of closing). This can be done at Perl 6 level, in the same place 
we make sure to run END blocks.

2) Having unclosed handles possible to GC, and closing them if/when they get 
GC'd.

Today we are doing #2. We could switch to doing #1. We can't currently do both, 
because the moment we start keeping a list of open handles then they can't be 
GC'd, and so #2 can't happen.

My initial inclination was to preserve behavior #2, though others have pointed 
out that behavior #1 is more useful for debugging in that it ensures log files, 
for example, will be written in the event of a crash, and a program relying on 
behavior #2 could already run out of handles today anyway if it were less lucky 
with GC timing. This is a fair argument, and the automatic close at exit might 
be softer on the ecosystem too (but would have done nothing for the Text::CSV 
case, which is the original subject of this ticket, because it wrote a file, 
didn't close it, then separately opened it for reading).

There's probably enough consensus to switch to option #1, and lizmat++ 
mentioned maybe looking into a patch to do that.

In the longer run, we can have both, but it depends on implementing weak 
references. In terms of backend support, the JVM does have them, and it seems 
there's an npm package [1] exposing v8 weak refs so a Node.js backend could 
support that also. I'm OK with adding them to MoarVM in the future, but both 
doing that and exposing weak references at Perl 6 level would be a non-small, 
and certainly non-trivial, task.

[perl #132045] [BUG] Cannot use [] to parameterize a role unless it's known at compile time and it's a bareword.

[jn...@jnthn.net via RT]

On Thu, 07 Sep 2017 14:53:44 -0700, ben-goldb...@hotmail.com wrote:
> The following code samples work as expected:
> 
>  m: use MONKEY; my $m = 'foo'; my $p = EVAL qq!role ::
> [\$value] \{ method $m \{ \$value } }!; say $p.^parameterize(42).foo;
> <+camelia> rakudo-moar e7a588: OUTPUT: «42␤»
> 
>  m: use MONKEY; constant $p = do { my $m = 'foo'; EVAL
> qq!role
> :: [\$value] \{ method $m \{ \$value } }! }; say
> $p.^parameterize(42).foo
>  rakudo-moar e7a588: OUTPUT: «42␤»
> 
>  m: use MONKEY; constant p = do { my $m = 'foo'; EVAL
> qq!role
> :: [\$value] \{ method $m \{ \$value } }! }; say p[42].foo
>  rakudo-moar e7a588: OUTPUT: «42␤»
> 
> The following, slightly different code samples do not work as
> expected:
> 
> m: use MONKEY; my $m = 'foo'; my $p = EVAL qq!role :: [\$value] \{
> method $m
> \{ \$value } }!; say $p[42].foo;
> <+camelia> rakudo-moar e7a588: OUTPUT: «Could not instantiate role
> '':␤Too few positionals passed; expected 2 arguments
> but got
> 1␤  in any protect at gen/moar/stage2/NQPCORE.setting line 1033␤  in
> block
>  at  line 1␤␤»
> 
>  m: use MONKEY; constant $p = do { my $m = 'foo'; EVAL
> qq!role
> :: [\$value] \{ method $m \{ \$value } }! }; say $p[42].foo
>  rakudo-moar e7a588: OUTPUT: «Could not instantiate role
> '':␤Too few positionals passed; expected 2 arguments
> but got
> 1␤  in any protect at gen/moar/stage2/NQPCORE.setting line 1033␤  in
> block
>  at  line 1␤␤»
> 
>  m: use MONKEY; my \p = do { my $m = 'foo'; EVAL qq!role
> ::
> [\$value] \{ method $m \{ \$value } }! }; say p[42].foo
>  rakudo-moar e7a588: OUTPUT: «Could not instantiate role
> '':␤Too few positionals passed; expected 2 arguments
> but got
> 1␤  in any protect at gen/moar/stage2/NQPCORE.setting line 1033␤  in
> block
>  at  line 1␤␤»
> 

This is expected behavior. The decision of whether we're looking at a 
parametric type or an array indexing operation is made at parse time, meaning 
it's both a syntactic decision *and* depends on knowing what we've got at 
compile time is a type. Which is precisely what you're observing.

[perl #132045] [BUG] Cannot use [] to parameterize a role unless it's known at compile time and it's a bareword.

[jn...@jnthn.net via RT]

On Thu, 07 Sep 2017 14:53:44 -0700, ben-goldb...@hotmail.com wrote:
> The following code samples work as expected:
> 
>  m: use MONKEY; my $m = 'foo'; my $p = EVAL qq!role ::
> [\$value] \{ method $m \{ \$value } }!; say $p.^parameterize(42).foo;
> <+camelia> rakudo-moar e7a588: OUTPUT: «42␤»
> 
>  m: use MONKEY; constant $p = do { my $m = 'foo'; EVAL
> qq!role
> :: [\$value] \{ method $m \{ \$value } }! }; say
> $p.^parameterize(42).foo
>  rakudo-moar e7a588: OUTPUT: «42␤»
> 
>  m: use MONKEY; constant p = do { my $m = 'foo'; EVAL
> qq!role
> :: [\$value] \{ method $m \{ \$value } }! }; say p[42].foo
>  rakudo-moar e7a588: OUTPUT: «42␤»
> 
> The following, slightly different code samples do not work as
> expected:
> 
> m: use MONKEY; my $m = 'foo'; my $p = EVAL qq!role :: [\$value] \{
> method $m
> \{ \$value } }!; say $p[42].foo;
> <+camelia> rakudo-moar e7a588: OUTPUT: «Could not instantiate role
> '':␤Too few positionals passed; expected 2 arguments
> but got
> 1␤  in any protect at gen/moar/stage2/NQPCORE.setting line 1033␤  in
> block
>  at  line 1␤␤»
> 
>  m: use MONKEY; constant $p = do { my $m = 'foo'; EVAL
> qq!role
> :: [\$value] \{ method $m \{ \$value } }! }; say $p[42].foo
>  rakudo-moar e7a588: OUTPUT: «Could not instantiate role
> '':␤Too few positionals passed; expected 2 arguments
> but got
> 1␤  in any protect at gen/moar/stage2/NQPCORE.setting line 1033␤  in
> block
>  at  line 1␤␤»
> 
>  m: use MONKEY; my \p = do { my $m = 'foo'; EVAL qq!role
> ::
> [\$value] \{ method $m \{ \$value } }! }; say p[42].foo
>  rakudo-moar e7a588: OUTPUT: «Could not instantiate role
> '':␤Too few positionals passed; expected 2 arguments
> but got
> 1␤  in any protect at gen/moar/stage2/NQPCORE.setting line 1033␤  in
> block
>  at  line 1␤␤»
> 

This is expected behavior. The decision of whether we're looking at a 
parametric type or an array indexing operation is made at parse time, meaning 
it's both a syntactic decision *and* depends on knowing what we've got at 
compile time is a type. Which is precisely what you're observing.

[perl #132044] returning inside whenever block causes MoarVM panic

[jn...@jnthn.net via RT]

On Thu, 07 Sep 2017 13:33:30 -0700, nlo...@gmail.com wrote:
> The code below almost always results in "MoarVM panic: Internal error:
> Unwound entire stack and missed handler"
> ```
> sub ran-ok() {
> react {
> my $proc = Proc::Async.new("echo", "1”);
> whenever $proc.start { return 1; }
> }
> }
> ran-ok()
> ```

Odd. It should really be complaining about it being a return outside of the 
dynamic scope of the routine, since the whenever block will run on a different 
thread.

I wonder if we can make this magically do what you wanted it to. I think maybe 
we can, but it's non-trivial since essentially we'd need to be having the 
return control exception caught, cause the react block to be torn down, and 
then re-throw it back on the thread that did the react. Also that would break 
the current "react is just like supply except it blocks" thing.

[perl #132053] [BUG] Autovivification on element with [] or {} as default causes "secret" keys

[jn...@jnthn.net via RT]

On Sat, 09 Sep 2017 19:53:53 -0700, tomentiru...@gmail.com wrote:
> > my @h is default([])
> []
> > @h[0].push: 3
> [3]
> > @h
> []
> > @h[0]
> [3]
> 
> > my %h is default({})
> {}
> > %h = 3
> 3
> > %h
> {}
> > %h
> {b => 3}
> 
> The array/hash variable's default is an empty array/hash literal. If
> an element is autovivified, it can be accessed directly, but isn’t
> findable from the variable.

While I suspect this can be fixed (and is missing setup of the auto-viv 
callback on the codepath for setting up a default), note that `is default` is a 
trait, it is evaluated at compile time. Thus there is one single array that 
will be shared globally and assigned into all elements, which probably isn't 
what you wanted.

/jnthn

[perl #132053] [BUG] Autovivification on element with [] or {} as default causes "secret" keys

[jn...@jnthn.net via RT]

On Sat, 09 Sep 2017 19:53:53 -0700, tomentiru...@gmail.com wrote:
> > my @h is default([])
> []
> > @h[0].push: 3
> [3]
> > @h
> []
> > @h[0]
> [3]
> 
> > my %h is default({})
> {}
> > %h = 3
> 3
> > %h
> {}
> > %h
> {b => 3}
> 
> The array/hash variable's default is an empty array/hash literal. If
> an element is autovivified, it can be accessed directly, but isn’t
> findable from the variable.

While I suspect this can be fixed (and is missing setup of the auto-viv 
callback on the codepath for setting up a default), note that `is default` is a 
trait, it is evaluated at compile time. Thus there is one single array that 
will be shared globally and assigned into all elements, which probably isn't 
what you wanted.

/jnthn

[perl #131387] `is default` value not applied to attributes by default

[jn...@jnthn.net via RT]

On Sat, 27 May 2017 19:35:53 -0700, c...@zoffix.com wrote:
> $z is `Nil` without my having to assign anything, while `$!z` is Any.
> To make `$!z` Nil I also have to assign Nil to it:
> 
>  m: my $z is default(Nil); dd $z;class Foo { has $!z is
> default(Nil); method x { dd $!z } }.new.x
>  rakudo-moar 0c5fe5: OUTPUT: «Nil $z = Nil␤Any $!z = Any␤»

Seems fixed, test needed.

13:56 < jnthn> m: class Foo { has $!z is default(Nil); method x { dd $!z } 
}.new.x
13:56 <+camelia> rakudo-moar 760530: OUTPUT: «Nil $!z = Nil␤»

[perl #131387] `is default` value not applied to attributes by default

[jn...@jnthn.net via RT]

On Sat, 27 May 2017 19:35:53 -0700, c...@zoffix.com wrote:
> $z is `Nil` without my having to assign anything, while `$!z` is Any.
> To make `$!z` Nil I also have to assign Nil to it:
> 
>  m: my $z is default(Nil); dd $z;class Foo { has $!z is
> default(Nil); method x { dd $!z } }.new.x
>  rakudo-moar 0c5fe5: OUTPUT: «Nil $z = Nil␤Any $!z = Any␤»

Seems fixed, test needed.

13:56 < jnthn> m: class Foo { has $!z is default(Nil); method x { dd $!z } 
}.new.x
13:56 <+camelia> rakudo-moar 760530: OUTPUT: «Nil $!z = Nil␤»

[perl #130417] [CONC] Take care of cas

[jn...@jnthn.net via RT]

On Tue, 27 Dec 2016 12:32:50 -0800, alex.jakime...@gmail.com wrote:
> https://github.com/rakudo/rakudo/blob/9594a3ece38ca212023cf3ece7405d955303db13/src/core/asyncops.pm#L50
> 
>  I thought I killed that off once
>  I did in 539b5f44ef3f pre-xmas
>  Because I thought it was pretty ridiculous having a non-atomic
> CAS in there :/
>  I made the mistake of tossing it with something else and it
> seems it got restored later as if I did it by accident rather than by
> quiet sleight of hand :P
>  jnthn: can you take care of it now? :)
>  AlexDaniel: Well, we could actually do with a working cas...
> :P
>  (A few things I'd like to do could use it)
>  So I'll probably just fix it sometime next year :)
>  AlexDaniel: Feel free, and mark it [CONC]; I watch the CONC
> list for stuff to do :)

This got sorted out. There's now lots of tests that cas really does an atomic 
compare and swap (e.g. S17-lowlevel/cas.t).

[perl #130417] [CONC] Take care of cas

[jn...@jnthn.net via RT]

On Tue, 27 Dec 2016 12:32:50 -0800, alex.jakime...@gmail.com wrote:
> https://github.com/rakudo/rakudo/blob/9594a3ece38ca212023cf3ece7405d955303db13/src/core/asyncops.pm#L50
> 
>  I thought I killed that off once
>  I did in 539b5f44ef3f pre-xmas
>  Because I thought it was pretty ridiculous having a non-atomic
> CAS in there :/
>  I made the mistake of tossing it with something else and it
> seems it got restored later as if I did it by accident rather than by
> quiet sleight of hand :P
>  jnthn: can you take care of it now? :)
>  AlexDaniel: Well, we could actually do with a working cas...
> :P
>  (A few things I'd like to do could use it)
>  So I'll probably just fix it sometime next year :)
>  AlexDaniel: Feel free, and mark it [CONC]; I watch the CONC
> list for stuff to do :)

This got sorted out. There's now lots of tests that cas really does an atomic 
compare and swap (e.g. S17-lowlevel/cas.t).

[perl #129941] [PERF] [IO] Perl 6 text file line read is much slower than Perl 5

[jn...@jnthn.net via RT]

On Mon, 24 Oct 2016 03:27:55 -0700, tbrowder wrote:
> On Sat Oct 22 04:24:15 2016, tbrowder wrote:
> > See  for a suite
> > of tests that show the differences.
> 
> Suite has been updated considerably.

In a benchmark on my local machine, after many improvements, I now see Perl 6 
coming out slightly ahead of Perl 5 when the UTF-8 encoding is being used:

$ time perl6 -e 'my $fh = open "longfile"; my $chars = 0; for $fh.lines { 
$chars = $chars + .chars }; $fh.close; say $chars'
6000

real0m1.081s
user0m1.168s
sys 0m0.032s

$ time perl -e 'open my $fh, "<:encoding(UTF-8)", "longfile"; my $chars = 0; 
while ($_ = <$fh>) { chomp; $chars = $chars + length($_) }; close $fh; print 
"$chars\n"'
6000

real0m1.110s
user0m1.088s
sys 0m0.020s

The situation with ASCII/latin-1 is still not quite so rosy:

$ time perl -e 'open my $fh, "<", "longfile"; my $chars = 0; while ($_ = <$fh>) 
{ chomp; $chars = $chars + length($_) }; close $fh; print "$chars\n"'
6000

real0m0.277s
user0m0.260s
sys 0m0.016s

$ time ./perl6-m -e 'my $fh = open "longfile", :enc; my $chars = 0; for 
$fh.lines { $chars = $chars + .chars }; $fh.close; say $chars'
6000

real0m0.988s
user0m1.028s
sys 0m0.068s

Though that's now down to a factor of 3.5x, which is hugely better than the 
factor of 9 or 10 before.

What are the conditions for resolving this issue? Clearly the UTF-8 case is 
good enough because Perl 6 is winning there, but "much slower" is a bit 
subjective, so hard to know when we're there (unless we somehow manage to win 
in the ASCII case too...) :-)

/jnthn

[perl #129941] [PERF] [IO] Perl 6 text file line read is much slower than Perl 5

[jn...@jnthn.net via RT]

On Mon, 24 Oct 2016 03:27:55 -0700, tbrowder wrote:
> On Sat Oct 22 04:24:15 2016, tbrowder wrote:
> > See  for a suite
> > of tests that show the differences.
> 
> Suite has been updated considerably.

In a benchmark on my local machine, after many improvements, I now see Perl 6 
coming out slightly ahead of Perl 5 when the UTF-8 encoding is being used:

$ time perl6 -e 'my $fh = open "longfile"; my $chars = 0; for $fh.lines { 
$chars = $chars + .chars }; $fh.close; say $chars'
6000

real0m1.081s
user0m1.168s
sys 0m0.032s

$ time perl -e 'open my $fh, "<:encoding(UTF-8)", "longfile"; my $chars = 0; 
while ($_ = <$fh>) { chomp; $chars = $chars + length($_) }; close $fh; print 
"$chars\n"'
6000

real0m1.110s
user0m1.088s
sys 0m0.020s

The situation with ASCII/latin-1 is still not quite so rosy:

$ time perl -e 'open my $fh, "<", "longfile"; my $chars = 0; while ($_ = <$fh>) 
{ chomp; $chars = $chars + length($_) }; close $fh; print "$chars\n"'
6000

real0m0.277s
user0m0.260s
sys 0m0.016s

$ time ./perl6-m -e 'my $fh = open "longfile", :enc; my $chars = 0; for 
$fh.lines { $chars = $chars + .chars }; $fh.close; say $chars'
6000

real0m0.988s
user0m1.028s
sys 0m0.068s

Though that's now down to a factor of 3.5x, which is hugely better than the 
factor of 9 or 10 before.

What are the conditions for resolving this issue? Clearly the UTF-8 case is 
good enough because Perl 6 is winning there, but "much slower" is a bit 
subjective, so hard to know when we're there (unless we somehow manage to win 
in the ASCII case too...) :-)

/jnthn

[perl #126692] [RFC] wrong line number in error message using Promise

[jn...@jnthn.net via RT]

On Fri, 20 Nov 2015 08:19:56 -0800, mt1...@gmail.com wrote:
> Hi,
> 
> When using 'my $p = Promise.start($code)' and die in that code for some 
> reason, I will get the die error message but the line number is of the 
> 'await $p' line. It would be more helpful to have the line number of the 
> original die statement.
> 
> perl6 version 2015.11-5-g08a804c built on MoarVM version 2015.11
> 
This was long ago improved. It now looks like this:

$ cat x.p6 
my $p = start {
die "oops";
}
await $p;

$ ./perl6-m x.p6 
Tried to get the result of a broken Promise
  in block  at x.p6 line 4

Original exception:
oops
  in block  at x.p6 line 2

And I remember putting in tests for it, so resolving.

[perl #126692] [RFC] wrong line number in error message using Promise

[jn...@jnthn.net via RT]

On Fri, 20 Nov 2015 08:19:56 -0800, mt1...@gmail.com wrote:
> Hi,
> 
> When using 'my $p = Promise.start($code)' and die in that code for some 
> reason, I will get the die error message but the line number is of the 
> 'await $p' line. It would be more helpful to have the line number of the 
> original die statement.
> 
> perl6 version 2015.11-5-g08a804c built on MoarVM version 2015.11
> 
This was long ago improved. It now looks like this:

$ cat x.p6 
my $p = start {
die "oops";
}
await $p;

$ ./perl6-m x.p6 
Tried to get the result of a broken Promise
  in block  at x.p6 line 4

Original exception:
oops
  in block  at x.p6 line 2

And I remember putting in tests for it, so resolving.

[perl #125230] [BUG] Reading from $*IN in a sub process' start {} hangs

[jn...@jnthn.net via RT]

On Thu, 21 May 2015 12:50:18 -0700, tony.od...@live.com wrote:
> Encountered the following problem implementing Event::Emitter between
> processes.  This code will run forever as .read(1) forever reads
> nothing from $*IN
> 
> 
> #!/usr/bin/env perl6
> 
> 
> 
> 
> my $p = Proc::Async.new('perl6', '-e', '
>await start {
> 
> my $d = $*IN.read(1);
> 
> while $d.elems == 0 {
> 
> $d = $*IN.read(1);
> 
> }
> 
> $d.say;
> 
> };', :w);
> 
> 
> 
> my $r = $p.start;
> 
> $p.write("0".encode);
> 
> await $r;
> 
> 
> --
> 
> ACTUAL:
> Sub process 'reads' an empty Buf forever
> 
> EXPECTED:
> Sub process eventually 'reads' a Buf containing 1 byte
> 

This was one of the many cases of I/O handles not working across different 
threads, which was fixed earlier on this year. Output is now:

Buf[uint8]:0x<30>

Various tests were added covering this, as it showed up in various other 
tickets, so will mark this issue resolved.

[perl #125230] [BUG] Reading from $*IN in a sub process' start {} hangs

[jn...@jnthn.net via RT]

On Thu, 21 May 2015 12:50:18 -0700, tony.od...@live.com wrote:
> Encountered the following problem implementing Event::Emitter between
> processes.  This code will run forever as .read(1) forever reads
> nothing from $*IN
> 
> 
> #!/usr/bin/env perl6
> 
> 
> 
> 
> my $p = Proc::Async.new('perl6', '-e', '
>await start {
> 
> my $d = $*IN.read(1);
> 
> while $d.elems == 0 {
> 
> $d = $*IN.read(1);
> 
> }
> 
> $d.say;
> 
> };', :w);
> 
> 
> 
> my $r = $p.start;
> 
> $p.write("0".encode);
> 
> await $r;
> 
> 
> --
> 
> ACTUAL:
> Sub process 'reads' an empty Buf forever
> 
> EXPECTED:
> Sub process eventually 'reads' a Buf containing 1 byte
> 

This was one of the many cases of I/O handles not working across different 
threads, which was fixed earlier on this year. Output is now:

Buf[uint8]:0x<30>

Various tests were added covering this, as it showed up in various other 
tickets, so will mark this issue resolved.

[perl #124005] [CONC] Cannot read from STDIN in a thread on windows console

[jn...@jnthn.net via RT]

On Fri, 19 Aug 2016 12:04:15 -0700, c...@zoffix.com wrote:
> Tests appear to exist in t/spec/S16-io/handles-between-threads.t

And have been fully passing since earlier this year when the issue of handles 
between threads was addressed; resolving.

[perl #124005] [CONC] Cannot read from STDIN in a thread on windows console

[jn...@jnthn.net via RT]

On Fri, 19 Aug 2016 12:04:15 -0700, c...@zoffix.com wrote:
> Tests appear to exist in t/spec/S16-io/handles-between-threads.t

And have been fully passing since earlier this year when the issue of handles 
between threads was addressed; resolving.

[perl #122790] [CONC] Tapping multiple signals fails strangely

[jn...@jnthn.net via RT]

On Tue, 16 Sep 2014 07:51:34 -0700, elizabeth wrote:
> 16:48:20]   jnthn: sanity check: we should allow more than
> one signal to be tapped at a time, right ?
> [16:49:10]lizmat: aye
> [16:49:32]   putting this in a file:
> [16:49:34]   signal(SIGHUP).act: -> { };
> signal(SIGINT).act: -> { }; sleep 5;
> [16:49:50]   running it and pressing CTRL-c gives a very
> strange backtrace
> [16:50:03]   Unhandled exception in code scheduled on
> thread 4527849472
> [16:50:03]   Cannot call 'postcircumfix:<[ ]>'; none of
> these signatures match:
> [16:50:22]   no line numbers, not even with --ll-exception
> [16:51:13]Odd
> [16:52:06]   rakudobug it?
> [16:54:14]yeah

This was duplicated in RT #132016, which was recently fixed. Resolving.

[perl #122790] [CONC] Tapping multiple signals fails strangely

[jn...@jnthn.net via RT]

On Tue, 16 Sep 2014 07:51:34 -0700, elizabeth wrote:
> 16:48:20]   jnthn: sanity check: we should allow more than
> one signal to be tapped at a time, right ?
> [16:49:10]lizmat: aye
> [16:49:32]   putting this in a file:
> [16:49:34]   signal(SIGHUP).act: -> { };
> signal(SIGINT).act: -> { }; sleep 5;
> [16:49:50]   running it and pressing CTRL-c gives a very
> strange backtrace
> [16:50:03]   Unhandled exception in code scheduled on
> thread 4527849472
> [16:50:03]   Cannot call 'postcircumfix:<[ ]>'; none of
> these signatures match:
> [16:50:22]   no line numbers, not even with --ll-exception
> [16:51:13]Odd
> [16:52:06]   rakudobug it?
> [16:54:14]yeah

This was duplicated in RT #132016, which was recently fixed. Resolving.

[perl #130692] [BUG] IO::Socket::Async broken in 6.d.PREVIEW

[jn...@jnthn.net via RT]

On Wed, 01 Feb 2017 13:11:21 -0800, j...@gellyfish.co.uk wrote:
> With 6.d.PREVIEW the simplest code for IO::Socket::Async mis-behaves:
> 
> use v6.d.PREVIEW;
> 
> react {
> whenever IO::Socket::Async.listen('localhost', ) -> $conn {
> whenever $conn.Supply(:bin) -> $buf {
> await $conn.write: $buf;
> $conn.close;
> }
> }
> }
> 
> Gives rise to:
> 
> Unhandled exception in code scheduled on thread 7
> Attempt to unlock mutex by thread not holding it
> 
> After the second connection every time.  I noticed this when testing
> HTTP::Server::Tiny.
> 
> This is Rakudo version 2017.01-132-g97359ae built on MoarVM version
> 2017.01-25-g70d4bd5

Fixed now, and test coverage of this, and the deeper underlying issue, is in 
S17-promise/nonblocking-await.t.

[perl #130692] [BUG] IO::Socket::Async broken in 6.d.PREVIEW

[jn...@jnthn.net via RT]

On Wed, 01 Feb 2017 13:11:21 -0800, j...@gellyfish.co.uk wrote:
> With 6.d.PREVIEW the simplest code for IO::Socket::Async mis-behaves:
> 
> use v6.d.PREVIEW;
> 
> react {
> whenever IO::Socket::Async.listen('localhost', ) -> $conn {
> whenever $conn.Supply(:bin) -> $buf {
> await $conn.write: $buf;
> $conn.close;
> }
> }
> }
> 
> Gives rise to:
> 
> Unhandled exception in code scheduled on thread 7
> Attempt to unlock mutex by thread not holding it
> 
> After the second connection every time.  I noticed this when testing
> HTTP::Server::Tiny.
> 
> This is Rakudo version 2017.01-132-g97359ae built on MoarVM version
> 2017.01-25-g70d4bd5

Fixed now, and test coverage of this, and the deeper underlying issue, is in 
S17-promise/nonblocking-await.t.

[perl #132091] await handles Slip differently between v6.c and v6.d.PREVIEW

[jn...@jnthn.net via RT]

On Thu, 14 Sep 2017 13:30:46 -0700, timo wrote:
> I'm using two start blocks that return Slips to generate a long list in
> parallel, then i iterate over the result in the Moar Heap Analyzer.
> 
> Trying to take advantage of v6.d's nonblocking await gives me a change
> in semantics, though:
> 
>     use v6.d.PREVIEW;
>     use Test;
>     my @foo = do {
>     await start { do for ^2 { my uint64 @ = 9, 9; }.Slip },
>   start { do for ^2 { my uint64 @ = 1, 2; }.Slip };
>     };
>     is @foo.elems, 4, "slips awaited over get flattened out";
>     say @foo.perl
> 
>     # -> not ok 1 - slips awaited over get flattened out
>     # -> [slip(array[uint64].new(9, 9), array[uint64].new(9, 9)),
> slip(array[uint64].new(1, 2), array[uint64].new(1, 2))]
> 
>     use Test;
>     my @foo = do {
>     await start { do for ^2 { my uint64 @ = 9, 9; }.Slip },
>   start { do for ^2 { my uint64 @ = 1, 2; }.Slip };
>     };
>     is @foo.elems, 4, "slips awaited over get flattened out";
>     say @foo.perl
> 
>     # -> ok 1 - slips awaited over get flattened out
>     # -> [array[uint64].new(9, 9), array[uint64].new(9, 9),
> array[uint64].new(1, 2), array[uint64].new(1, 2)]

Fixed, and test coverage in S17-promise/nonblocking-await.t.

[perl #132091] await handles Slip differently between v6.c and v6.d.PREVIEW

[jn...@jnthn.net via RT]

On Thu, 14 Sep 2017 13:30:46 -0700, timo wrote:
> I'm using two start blocks that return Slips to generate a long list in
> parallel, then i iterate over the result in the Moar Heap Analyzer.
> 
> Trying to take advantage of v6.d's nonblocking await gives me a change
> in semantics, though:
> 
>     use v6.d.PREVIEW;
>     use Test;
>     my @foo = do {
>     await start { do for ^2 { my uint64 @ = 9, 9; }.Slip },
>   start { do for ^2 { my uint64 @ = 1, 2; }.Slip };
>     };
>     is @foo.elems, 4, "slips awaited over get flattened out";
>     say @foo.perl
> 
>     # -> not ok 1 - slips awaited over get flattened out
>     # -> [slip(array[uint64].new(9, 9), array[uint64].new(9, 9)),
> slip(array[uint64].new(1, 2), array[uint64].new(1, 2))]
> 
>     use Test;
>     my @foo = do {
>     await start { do for ^2 { my uint64 @ = 9, 9; }.Slip },
>   start { do for ^2 { my uint64 @ = 1, 2; }.Slip };
>     };
>     is @foo.elems, 4, "slips awaited over get flattened out";
>     say @foo.perl
> 
>     # -> ok 1 - slips awaited over get flattened out
>     # -> [array[uint64].new(9, 9), array[uint64].new(9, 9),
> array[uint64].new(1, 2), array[uint64].new(1, 2)]

Fixed, and test coverage in S17-promise/nonblocking-await.t.

[perl #126842] [BUG] New react + chain syntax locks sometimes

[jn...@jnthn.net via RT]

On Tue, 08 Dec 2015 05:40:29 -0800, pub.clau...@gmail.com wrote:
> The replacement of the 'whatever' syntax as sugar for consuming Channels
> through 'react' locks very often (5-10% of the time). The thread seems to
> be stuck on 'shift', probably from the Channel). When rewriting the exact
> same functionality with the channel.poll syntax, the problem does not occur.
> 
> Attached you'll find 2 scripts: loop_channelreact.p6 reproduces the
> problem, while loop_channelpoll uses the old tried and tested approach.
> 
All kinds of things have been fixed that could have been to blame since this 
ticket was field; either way, it works out. I remove the sleep so it'd go 
faster and ran it for 250,000 iterations without trouble.

Added this example as a test in S17-supply/syntax.t to make sure it doesn't 
come up again.

[perl #126842] [BUG] New react + chain syntax locks sometimes

[jn...@jnthn.net via RT]

On Tue, 08 Dec 2015 05:40:29 -0800, pub.clau...@gmail.com wrote:
> The replacement of the 'whatever' syntax as sugar for consuming Channels
> through 'react' locks very often (5-10% of the time). The thread seems to
> be stuck on 'shift', probably from the Channel). When rewriting the exact
> same functionality with the channel.poll syntax, the problem does not occur.
> 
> Attached you'll find 2 scripts: loop_channelreact.p6 reproduces the
> problem, while loop_channelpoll uses the old tried and tested approach.
> 
All kinds of things have been fixed that could have been to blame since this 
ticket was field; either way, it works out. I remove the sleep so it'd go 
faster and ran it for 250,000 iterations without trouble.

Added this example as a test in S17-supply/syntax.t to make sure it doesn't 
come up again.

[perl #128717] [LTA] react whenever throws a cryptic error

[jn...@jnthn.net via RT]

On Sat, 23 Jul 2016 20:36:59 -0700, c...@zoffix.com wrote:
> Blockless `react` seems to be unsupported and the error message it
> throws is cryptic and unrelated to the actual issue. Best would be if
> it actually worked, but at least the error message should be improved.
> 
> $ perl6 -e 'react whenever Supply.interval: 1 { .say }'
> 0
> Method 'phasers' not found for invocant of class 'Code'
>   in block  at -e line 1

Was fixed a while back; regression test added to S17-supply/syntax.t.

[perl #128717] [LTA] react whenever throws a cryptic error

[jn...@jnthn.net via RT]

On Sat, 23 Jul 2016 20:36:59 -0700, c...@zoffix.com wrote:
> Blockless `react` seems to be unsupported and the error message it
> throws is cryptic and unrelated to the actual issue. Best would be if
> it actually worked, but at least the error message should be improved.
> 
> $ perl6 -e 'react whenever Supply.interval: 1 { .say }'
> 0
> Method 'phasers' not found for invocant of class 'Code'
>   in block  at -e line 1

Was fixed a while back; regression test added to S17-supply/syntax.t.

[perl #132114] [ANNOYING] t\04-nativecall\21-callback-other-thread.t is flapping on appveyor and travis

[jn...@jnthn.net via RT]

On Sun, 17 Sep 2017 14:04:02 -0700, alex.jakime...@gmail.com wrote:
> It happens often and that's why we sometimes get false-red CI status.
> See
> https://ci.appveyor.com/project/rakudo/rakudo/build/1.0.130/job/yqmgs4uo0yim1bxx

Fixed the MoarVM bug that caused this, and the revision bump took place. 
Resolving; re-open if this happens again, but I'm pretty sure it's nailed.

[perl #132114] [ANNOYING] t\04-nativecall\21-callback-other-thread.t is flapping on appveyor and travis

[jn...@jnthn.net via RT]

On Sun, 17 Sep 2017 14:04:02 -0700, alex.jakime...@gmail.com wrote:
> It happens often and that's why we sometimes get false-red CI status.
> See
> https://ci.appveyor.com/project/rakudo/rakudo/build/1.0.130/job/yqmgs4uo0yim1bxx

Fixed the MoarVM bug that caused this, and the revision bump took place. 
Resolving; re-open if this happens again, but I'm pretty sure it's nailed.

[perl #130716] [CONC] unbounded supply {} + react {} = pseudo-hang

[jn...@jnthn.net via RT]

On Wed, 15 Feb 2017 08:35:09 -0800, jn...@jnthn.net wrote:
> Hope this helps, and I'll keep the issue under consideration.

So the time came to tackle getting supply/react/whenever syntax capable of 
playing nice with non-blocking await, and I decided as part of those changes to 
look at both this problem and also the more general problem of lack of good 
back-pressure and, related to that, lack of fairness when using the 
supply/react syntax.

To recap, until very recently, a `supply` or `react` block instance had its own 
processing queue. If it was empty, the emitting thread would enter and run the 
required code. If any messages were emitted to it in the meantime, they would 
be queued asynchronously. When the sender of the currently-being-processed 
message was done, it would check if there was anything added to the queue in 
the meantime, and if so it would process those too. This mechanism also handled 
recursive messages by queuing them up (this occurs when some code running in a 
`supply` block instance results in another emit being sent to the same `supply` 
block instance). The asynchronous queuing, however, meant that the cost of 
processing a message didn't push back on senders as it should have.

I've just finished (I hope :-)) re-working the Supply internals to instead use 
asynchronous locking throughout. An asynchronous lock returns a Promise that 
will already be Kept if the lock is already available, or will be Kept once it 
is available. Multiple contenders queue for the lock. This, in combination with 
non-blocking `await` of the Promise, forms the new supply concurrency control 
model, used consistently in `supply` blocks and elsewhere in the supplies 
implementation (previously, the code elsewhere used a real mutex, which gave 
its own set of issues).

On its own this cannot replace the previous mechanism, however, because the 
queuing was used in preventing various forms of deadlock, especially recursion. 
It also would cause problems for any `whenever` tapping a `supply` that emitted 
values synchronously after being tapped (as in your case). The former is 
resolved by detecting lock recursion and, in that case falling back to queuing 
the work to run later, using the thread pool. The latter is resolved with a 
custom Awaiter implementation: if anything during the processing of setting up 
a `whenever` block does an `await`, a continuation is taken, and then - after 
the setup work of the `supply` block is completed - the continuations are 
invoked.

This latter case is relevant to the original subject to this ticket, because 
with the supply concurrency control mechanism now being asynchronous locking, 
the outcome of the `emit` that previously queued endlessly is now an `await` 
instead. Thus the setup of the consumer is allowed to complete, before the 
producer is resumed. Any further awaits are also collected and handled in the 
same way, until we run out of them. The effect is that if we rewrite the 
original code (to use the CLOSE phaser, not a hack with a role):

sub make-supply() {
supply { 
until my $done {
say "Emitting ...";
emit(++$);
}   
CLOSE $done = True;
} 
}  

my $s2 = make-supply;
react {
whenever $s2 -> $n {
say "Received $n";
done if $n >= 5;
}   
}

Then it will produce:

Emitting ...
Received 1
Emitting ...
Received 2
Emitting ...
Received 3
Emitting ...
Received 4
Emitting ...
Received 5

Furthermore, if it is written as just:

sub make-supply() {
supply { 
loop {
say "Emitting ...";
emit(++$);
}   
}
}   

my $s2 = make-supply;
react {
whenever $s2 -> $n {
say "Received $n";
done if $n >= 5;
}   
}   

The output is similar:

Emitting ...
Received 1
Emitting ...
Received 2
Emitting ...
Received 3
Emitting ...
Received 4
Emitting ...
Received 5
Emitting ...

Note the one extra "Emitting ...". The `emit` operation will now check if the 
supply block is still active; in this case, it was closed by its consumer, so 
it won't bother emitting and won't bother resuming either (emit is a control 
exception, which is why we can unwind the stack and thus exit the loop).

Finally, this model means that:

sub make-supply() {
supply { 
my $i = 0;
loop {
say "Emitting 2000 messages" if $i %% 2000;
emit(++$i);
}   
}   
}   

my $s2 = make-supply;
react {
my $received = 0;
whenever $s2 -> $n {
$received++;
}   
whenever Promise.in(1) {
say "Received $received messages";
done;
}
}   

Works - as in, the second `whenever` block gets its fair chance to have a 
message processed too, so the output is something like:

Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Receive

[perl #130716] [CONC] unbounded supply {} + react {} = pseudo-hang

[jn...@jnthn.net via RT]

On Wed, 15 Feb 2017 08:35:09 -0800, jn...@jnthn.net wrote:
> Hope this helps, and I'll keep the issue under consideration.

So the time came to tackle getting supply/react/whenever syntax capable of 
playing nice with non-blocking await, and I decided as part of those changes to 
look at both this problem and also the more general problem of lack of good 
back-pressure and, related to that, lack of fairness when using the 
supply/react syntax.

To recap, until very recently, a `supply` or `react` block instance had its own 
processing queue. If it was empty, the emitting thread would enter and run the 
required code. If any messages were emitted to it in the meantime, they would 
be queued asynchronously. When the sender of the currently-being-processed 
message was done, it would check if there was anything added to the queue in 
the meantime, and if so it would process those too. This mechanism also handled 
recursive messages by queuing them up (this occurs when some code running in a 
`supply` block instance results in another emit being sent to the same `supply` 
block instance). The asynchronous queuing, however, meant that the cost of 
processing a message didn't push back on senders as it should have.

I've just finished (I hope :-)) re-working the Supply internals to instead use 
asynchronous locking throughout. An asynchronous lock returns a Promise that 
will already be Kept if the lock is already available, or will be Kept once it 
is available. Multiple contenders queue for the lock. This, in combination with 
non-blocking `await` of the Promise, forms the new supply concurrency control 
model, used consistently in `supply` blocks and elsewhere in the supplies 
implementation (previously, the code elsewhere used a real mutex, which gave 
its own set of issues).

On its own this cannot replace the previous mechanism, however, because the 
queuing was used in preventing various forms of deadlock, especially recursion. 
It also would cause problems for any `whenever` tapping a `supply` that emitted 
values synchronously after being tapped (as in your case). The former is 
resolved by detecting lock recursion and, in that case falling back to queuing 
the work to run later, using the thread pool. The latter is resolved with a 
custom Awaiter implementation: if anything during the processing of setting up 
a `whenever` block does an `await`, a continuation is taken, and then - after 
the setup work of the `supply` block is completed - the continuations are 
invoked.

This latter case is relevant to the original subject to this ticket, because 
with the supply concurrency control mechanism now being asynchronous locking, 
the outcome of the `emit` that previously queued endlessly is now an `await` 
instead. Thus the setup of the consumer is allowed to complete, before the 
producer is resumed. Any further awaits are also collected and handled in the 
same way, until we run out of them. The effect is that if we rewrite the 
original code (to use the CLOSE phaser, not a hack with a role):

sub make-supply() {
supply { 
until my $done {
say "Emitting ...";
emit(++$);
}   
CLOSE $done = True;
} 
}  

my $s2 = make-supply;
react {
whenever $s2 -> $n {
say "Received $n";
done if $n >= 5;
}   
}

Then it will produce:

Emitting ...
Received 1
Emitting ...
Received 2
Emitting ...
Received 3
Emitting ...
Received 4
Emitting ...
Received 5

Furthermore, if it is written as just:

sub make-supply() {
supply { 
loop {
say "Emitting ...";
emit(++$);
}   
}
}   

my $s2 = make-supply;
react {
whenever $s2 -> $n {
say "Received $n";
done if $n >= 5;
}   
}   

The output is similar:

Emitting ...
Received 1
Emitting ...
Received 2
Emitting ...
Received 3
Emitting ...
Received 4
Emitting ...
Received 5
Emitting ...

Note the one extra "Emitting ...". The `emit` operation will now check if the 
supply block is still active; in this case, it was closed by its consumer, so 
it won't bother emitting and won't bother resuming either (emit is a control 
exception, which is why we can unwind the stack and thus exit the loop).

Finally, this model means that:

sub make-supply() {
supply { 
my $i = 0;
loop {
say "Emitting 2000 messages" if $i %% 2000;
emit(++$i);
}   
}   
}   

my $s2 = make-supply;
react {
my $received = 0;
whenever $s2 -> $n {
$received++;
}   
whenever Promise.in(1) {
say "Received $received messages";
done;
}
}   

Works - as in, the second `whenever` block gets its fair chance to have a 
message processed too, so the output is something like:

Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Emitting 2000 messages
Receive

[perl #132170] fork + react + IO::Socket::Async issue

[jn...@jnthn.net via RT]

On Tue, 26 Sep 2017 12:33:21 -0700, bdug...@matatu.org wrote:
> When I export a function that does a fork() and then
> a react + whenever + IO::Socket::Async.listen in the
> child process, the process only seems to be listening
> on the socket the second time I run the code, i.e. only
> after .precomp has been written.
> 
We don't support calling fork(), because there's no way we reasonably can in 
combination with providing the thread support that we do (and rely on 
internally).

The immediate reason for what you're seeing is probably that precompilation 
uses Proc::Async, which in the turn starts some background threads. A fork() 
call doesn't clone anything other than the thread that forks. So after the 
fork(), the VM is suddenly missing a bunch of its threads, and will very 
possibly deadlock at the next GC run. The lack of a listener is almost 
certainly that the eventloop thread (used for async procs and sockets) was also 
started prior to the fork() and so also goes missing.

Even if that wasn't the case, modern VMs like MoarVM and the JVM start their 
own internal threads for things like optimization and GC, so even a far simpler 
program is very unlikely to successfully fork(), since those internal threads 
will also go missing.

In summary, we can't usefully support fork() in combination with the other 
things we support. What I have done is updated the docs:

https://github.com/perl6/doc/commit/8f9443c3ac8c8d8c33e82457f64a1718f965c769

To not send people down the dead-end road of NativeCalling fork.

[perl #132170] fork + react + IO::Socket::Async issue

[jn...@jnthn.net via RT]

On Tue, 26 Sep 2017 12:33:21 -0700, bdug...@matatu.org wrote:
> When I export a function that does a fork() and then
> a react + whenever + IO::Socket::Async.listen in the
> child process, the process only seems to be listening
> on the socket the second time I run the code, i.e. only
> after .precomp has been written.
> 
We don't support calling fork(), because there's no way we reasonably can in 
combination with providing the thread support that we do (and rely on 
internally).

The immediate reason for what you're seeing is probably that precompilation 
uses Proc::Async, which in the turn starts some background threads. A fork() 
call doesn't clone anything other than the thread that forks. So after the 
fork(), the VM is suddenly missing a bunch of its threads, and will very 
possibly deadlock at the next GC run. The lack of a listener is almost 
certainly that the eventloop thread (used for async procs and sockets) was also 
started prior to the fork() and so also goes missing.

Even if that wasn't the case, modern VMs like MoarVM and the JVM start their 
own internal threads for things like optimization and GC, so even a far simpler 
program is very unlikely to successfully fork(), since those internal threads 
will also go missing.

In summary, we can't usefully support fork() in combination with the other 
things we support. What I have done is updated the docs:

https://github.com/perl6/doc/commit/8f9443c3ac8c8d8c33e82457f64a1718f965c769

To not send people down the dead-end road of NativeCalling fork.

[perl #132148] [RFC] native 'str' type unspecced, undocumented, and ill-defined

[jn...@jnthn.net via RT]

On Fri, 22 Sep 2017 16:40:35 -0700, b...@abrij.org wrote:
> 
> We've had a native 'str' type for a while, and still have one even
> though NativeCall decided to go with Str and 'is encoded'.
> 
It's not native in the sense of same sense that NativeCall uses the word.

> Currently it seems to just be a Str under the hood which has very
> few restrictions, other than not allowing binding to a Str.
> 
That's the wrong way around. A Str boxes a str under the hood, much like Num 
boxes a num.

> What exactly is a 'str'?  Is it null terminated or does it know
> its length?  Does it have an encoding?
> 
Str uses the P6opaque representation. It's defined as something like:

class Str {
has str $!value;
...
}

This means it can support things like mixins. The `str` instance is what 
actually holds the string data buffer. It is always in NFG. Its representation 
is implementation-defined, but no implementation that provides it supports 
mixing in to it, which a high-level type like Str should support.

So, much like num is an unboxed Num, str is an unboxed Str. It's used quite a 
bit inside of Rakudo, as it's what all the nqp:: ops for str use. However, 
there are user-level places one might wish to have it if dealing with a large 
number of strings. For example:

my str @loads-of-strings;

Will be far more compact in memory than:

my Str @loads-of-strings;

Since the latter stores a Scalar pointing to Str pointing to str, while the 
former is just a bunch of str.

> Roast indicates it knows how to take an assignment from a Str literal
> (but no literal is used that pushes any boundaries) and
> can be used in multi-dispatch and... that's really about all.
> 
I'm sure there's room for more tests, though given that it's on the inside of 
every Str, then its functionality is largely covered implicitly (and any method 
call on a str will box it to the Str object type to actually call the method).

[perl #132148] [RFC] native 'str' type unspecced, undocumented, and ill-defined

[jn...@jnthn.net via RT]

On Fri, 22 Sep 2017 16:40:35 -0700, b...@abrij.org wrote:
> 
> We've had a native 'str' type for a while, and still have one even
> though NativeCall decided to go with Str and 'is encoded'.
> 
It's not native in the sense of same sense that NativeCall uses the word.

> Currently it seems to just be a Str under the hood which has very
> few restrictions, other than not allowing binding to a Str.
> 
That's the wrong way around. A Str boxes a str under the hood, much like Num 
boxes a num.

> What exactly is a 'str'?  Is it null terminated or does it know
> its length?  Does it have an encoding?
> 
Str uses the P6opaque representation. It's defined as something like:

class Str {
has str $!value;
...
}

This means it can support things like mixins. The `str` instance is what 
actually holds the string data buffer. It is always in NFG. Its representation 
is implementation-defined, but no implementation that provides it supports 
mixing in to it, which a high-level type like Str should support.

So, much like num is an unboxed Num, str is an unboxed Str. It's used quite a 
bit inside of Rakudo, as it's what all the nqp:: ops for str use. However, 
there are user-level places one might wish to have it if dealing with a large 
number of strings. For example:

my str @loads-of-strings;

Will be far more compact in memory than:

my Str @loads-of-strings;

Since the latter stores a Scalar pointing to Str pointing to str, while the 
former is just a bunch of str.

> Roast indicates it knows how to take an assignment from a Str literal
> (but no literal is used that pushes any boundaries) and
> can be used in multi-dispatch and... that's really about all.
> 
I'm sure there's room for more tests, though given that it's on the inside of 
every Str, then its functionality is largely covered implicitly (and any method 
call on a str will box it to the Str object type to actually call the method).

[perl #122709] [CONC][BUG] `await`ing a Promise in a different thread sometimes hangs

[jn...@jnthn.net via RT]

On Wed, 03 Aug 2016 12:23:32 -0700, jn...@jnthn.net wrote:
> I can still reproduce it too; seems that occasionally the scheduler
> starts too few threads (in fact, only one, and that is tied up with
> the await, and so nothing processes the incoming message saying the
> process ended).

The new scheduler merged a couple weeks back addresses this; added a test in 
S17-procasync/stress.t.

[perl #122709] [CONC][BUG] `await`ing a Promise in a different thread sometimes hangs

[jn...@jnthn.net via RT]

On Wed, 03 Aug 2016 12:23:32 -0700, jn...@jnthn.net wrote:
> I can still reproduce it too; seems that occasionally the scheduler
> starts too few threads (in fact, only one, and that is tied up with
> the await, and so nothing processes the incoming message saying the
> process ended).

The new scheduler merged a couple weeks back addresses this; added a test in 
S17-procasync/stress.t.

[perl #125523] [LTA] num64 not properly defaulting

[jn...@jnthn.net via RT]

On Mon, 02 Oct 2017 18:09:45 -0700, alex.jakime...@gmail.com wrote:
> Right, because it's a Rat.
> 
> sub foo(num64 $scale = 1.0) {}; say foo # This type cannot unbox to a
> native
> number: P6opaque, Rat
> sub foo(num64 $scale = 1.0.Num) {}; say foo # Nil
> 
No need for .Num, just write it as 1e0 so it's a Num literal.

> I don't know what is the consensus on this one, but having to .Num
> your values
> is rather reasonable.
> 
The rule is simple: types in Perl 6 are constraints. We do go as far as letting 
literals play the role of both a native and a boxed (int/Int, num/Num), but no 
further.

> The error message is LTA though.
> 
What would you like it to say? It seems to contain all relevant information we 
can reasonably provide in a runtime error (what we tried to unbox it to, and 
what type it was); the mention of P6opauqe could go away, though, as it doesn't 
really add anything.

[perl #125523] [LTA] num64 not properly defaulting

[jn...@jnthn.net via RT]

On Mon, 02 Oct 2017 18:09:45 -0700, alex.jakime...@gmail.com wrote:
> Right, because it's a Rat.
> 
> sub foo(num64 $scale = 1.0) {}; say foo # This type cannot unbox to a
> native
> number: P6opaque, Rat
> sub foo(num64 $scale = 1.0.Num) {}; say foo # Nil
> 
No need for .Num, just write it as 1e0 so it's a Num literal.

> I don't know what is the consensus on this one, but having to .Num
> your values
> is rather reasonable.
> 
The rule is simple: types in Perl 6 are constraints. We do go as far as letting 
literals play the role of both a native and a boxed (int/Int, num/Num), but no 
further.

> The error message is LTA though.
> 
What would you like it to say? It seems to contain all relevant information we 
can reasonably provide in a runtime error (what we tried to unbox it to, and 
what type it was); the mention of P6opauqe could go away, though, as it doesn't 
really add anything.

[perl #131915] [REGRESSION] Proc using more memory since being a Proc::Async

[jn...@jnthn.net via RT]

On Wed, 16 Aug 2017 14:24:48 -0700, scoli...@gmail.com wrote:
> Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
> that there is a huge increase of memory usage after running a simple
> echo command.
> Result are in Kbytes.
> 
>  commit:
> 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem {
> "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
> mem(); run 'echo'; say mem()
>  AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820»
> ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660»
> ¦9658dd9: «139024␤␤305320»

We won't get it back to where it was, because threads cost something. And we'd 
not really want to go back to where we were, because Proc had all kinds of ways 
of not working. It's easy to be cheaper if you can be brokener too.

The cost is now very significantly lower, however, thanks to:

* The new scheduler, which doesn't wastefully start a load of threads for a 
single Proc::Async usage. This was the major win.
* Smarter handling of memory for new threads in MoarVM; now they start with a 
128KB nursery and grow it as needed, as opposed to starting with a 4MB nursery.

So rather than it causing a 3.8 times growth in program memory size, I now 
measure it as less than 1.5 (and, better, it won't increase with follow-up 
calls to run the way it used to either).

[perl #131915] [REGRESSION] Proc using more memory since being a Proc::Async

[jn...@jnthn.net via RT]

On Wed, 16 Aug 2017 14:24:48 -0700, scoli...@gmail.com wrote:
> Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
> that there is a huge increase of memory usage after running a simple
> echo command.
> Result are in Kbytes.
> 
>  commit:
> 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem {
> "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
> mem(); run 'echo'; say mem()
>  AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820»
> ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660»
> ¦9658dd9: «139024␤␤305320»

We won't get it back to where it was, because threads cost something. And we'd 
not really want to go back to where we were, because Proc had all kinds of ways 
of not working. It's easy to be cheaper if you can be brokener too.

The cost is now very significantly lower, however, thanks to:

* The new scheduler, which doesn't wastefully start a load of threads for a 
single Proc::Async usage. This was the major win.
* Smarter handling of memory for new threads in MoarVM; now they start with a 
128KB nursery and grow it as needed, as opposed to starting with a 4MB nursery.

So rather than it causing a 3.8 times growth in program memory size, I now 
measure it as less than 1.5 (and, better, it won't increase with follow-up 
calls to run the way it used to either).

[perl #130370] [CONC] Tapping on a very active .out of Proc::Async wrecks the work queue

[jn...@jnthn.net via RT]

On Tue, 20 Dec 2016 05:07:57 -0800, jn...@jnthn.net wrote:
> On Sun, 18 Dec 2016 06:24:38 -0800, alex.jakime...@gmail.com wrote:
> > This is probably best demonstrated by a code snippet.
> >
> > Code:
> >
> > my $out = Channel.new;
> > #my $proc = Proc::Async.new(‘perl6’, ‘-e’, ‘sleep ∞’); # ← this works
> > my $proc = Proc::Async.new(‘perl6’, ‘-e’, ‘.say for 0..0x1’); # ←
> > this doesn't
> > $proc.stdout.tap({ $out.send: 1 });
> >
> > my $promise = $proc.start;
> > say ‘let's await  ’, now;
> > await Promise.anyof(Promise.in(1), $promise);
> > say ‘let's close  ’, now;
> > $proc.kill;
> > await $promise;
> > $out.close;
> > say ‘that's it ’, now;
> >
> >
> > The idea is that we want to kill our process if it is still working
> > after 1 second. This works fine, unless the process is writing a lot
> > of stuff to stdout.
> >
> At present, the default scheduler has a single work queue, and if the
> output from the process is not being processed at the rate it's coming
> in then the queue will end up with a backlog, and the timer event -
> placed into the same queue - will be delayed. In the future, we'll get
> around to having a smarter scheduler. In the meantime, I suggest
> creating a second scheduler to use for timer (or Proc::Async) events.
> 

The new scheduler has a separate queue for time-based events, and seems to do 
notably better at this.

[perl #130370] [CONC] Tapping on a very active .out of Proc::Async wrecks the work queue

[jn...@jnthn.net via RT]

On Tue, 20 Dec 2016 05:07:57 -0800, jn...@jnthn.net wrote:
> On Sun, 18 Dec 2016 06:24:38 -0800, alex.jakime...@gmail.com wrote:
> > This is probably best demonstrated by a code snippet.
> >
> > Code:
> >
> > my $out = Channel.new;
> > #my $proc = Proc::Async.new(‘perl6’, ‘-e’, ‘sleep ∞’); # ← this works
> > my $proc = Proc::Async.new(‘perl6’, ‘-e’, ‘.say for 0..0x1’); # ←
> > this doesn't
> > $proc.stdout.tap({ $out.send: 1 });
> >
> > my $promise = $proc.start;
> > say ‘let's await  ’, now;
> > await Promise.anyof(Promise.in(1), $promise);
> > say ‘let's close  ’, now;
> > $proc.kill;
> > await $promise;
> > $out.close;
> > say ‘that's it ’, now;
> >
> >
> > The idea is that we want to kill our process if it is still working
> > after 1 second. This works fine, unless the process is writing a lot
> > of stuff to stdout.
> >
> At present, the default scheduler has a single work queue, and if the
> output from the process is not being processed at the rate it's coming
> in then the queue will end up with a backlog, and the timer event -
> placed into the same queue - will be delayed. In the future, we'll get
> around to having a smarter scheduler. In the meantime, I suggest
> creating a second scheduler to use for timer (or Proc::Async) events.
> 

The new scheduler has a separate queue for time-based events, and seems to do 
notably better at this.

[perl #132242] [BUG] Proc::Async running with yes command returns superfluous output and hangs

[jn...@jnthn.net via RT]

On Sat, 07 Oct 2017 19:33:40 -0700, allber...@gmail.com wrote:
> On Sat, Oct 7, 2017 at 10:21 PM, Itsuki Toyota  > wrote:
> 
> > See the following result:
> >
> > $ perl6 -e 'my $proc = Proc::Async.new("yes");
> > $proc.stdout.head(1).tap(&say); await $proc.start;'
> >
> > y
> > y
> > y
> > y
> > y
> > y
> > y
> > y
> > ...
> >
> >
> > I think tap(&say) should output just a 'y' char and completes its work.
> >
> 
> No guarantee it's line based, nor do pipes deliver output in lines. You
> wanted a .lines in there to 'rechunk' the output; otherwise I suspect you
> get blocks based on the size of am OS pipe buffer, or similar.
> 
> https://docs.perl6.org/type/Supply#method_lines
> 

Indeed; the Supply objects returned from Proc::Async just pass on the data that 
arrives, as it arrives, in the chunks that it arrives in. Often one wants all 
of the output, so parsing it into lines by default would just add a load of 
overhead to that case. The `.lines` method is the correct way for those wanting 
line-by-line output.

[perl #132242] [BUG] Proc::Async running with yes command returns superfluous output and hangs

[jn...@jnthn.net via RT]

On Sat, 07 Oct 2017 19:33:40 -0700, allber...@gmail.com wrote:
> On Sat, Oct 7, 2017 at 10:21 PM, Itsuki Toyota  > wrote:
> 
> > See the following result:
> >
> > $ perl6 -e 'my $proc = Proc::Async.new("yes");
> > $proc.stdout.head(1).tap(&say); await $proc.start;'
> >
> > y
> > y
> > y
> > y
> > y
> > y
> > y
> > y
> > ...
> >
> >
> > I think tap(&say) should output just a 'y' char and completes its work.
> >
> 
> No guarantee it's line based, nor do pipes deliver output in lines. You
> wanted a .lines in there to 'rechunk' the output; otherwise I suspect you
> get blocks based on the size of am OS pipe buffer, or similar.
> 
> https://docs.perl6.org/type/Supply#method_lines
> 

Indeed; the Supply objects returned from Proc::Async just pass on the data that 
arrives, as it arrives, in the chunks that it arrives in. Often one wants all 
of the output, so parsing it into lines by default would just add a load of 
overhead to that case. The `.lines` method is the correct way for those wanting 
line-by-line output.

[perl #132248] Z Metaoperator bug

[jn...@jnthn.net via RT]

On Sun, 08 Oct 2017 19:13:34 -0700, ipatrol6...@yahoo.com wrote:
> As per a discussion on the IRC channel, I am requesting that a
> regression bug be filed regarding the incorrect handling of variables
> referenced to packages by the Z and X meta-operators.
> 
> The behavior over time is detailed here:
> https://gist.github.com/Whateverable/83ce4a1ea73a429131713367ee23542e
> 
> Of those, the result given from 2015.09 to 2015.11 is the correct and
> desired behavior, considering the design documents.
> 
> Doing a bisection:
> https://gist.github.com/Whateverable/0a73f0433cee989d7a82ce8a601bd465
> 
> The regression revision appears to be:
> https://github.com/rakudo/rakudo/commit/234287170cb7dd234ffc90271d762b1c106bc57f
> 
I think Z and X are doing the right thing: treating something in a Scalar 
container as a single item. This is done consistently across the language, so 
you'll observe this behavior in many more constructs.

The surprise here is that we get a Scalar container in the first place:

$ perl6-m -e 'my @A::B = (1..8); say @A::B.VAR.WHAT; my @B = (1..8); say 
@B.VAR.WHAT'
(Scalar)
(Array)

I suspect that's happening because a package's symbol table is a Stash, which 
is just a subclass of Hash, and this code is relying on autovivification, which 
always involves a Scalar container. It probably should instead, somehow, be 
arranging than an Array is bound directly into the Stash. Then Z, X, and plenty 
more things would work less surprisingly. A `my %A::B` is likely exhibiting the 
same kind of problem at the moment too.

[perl #132248] Z Metaoperator bug

[jn...@jnthn.net via RT]

On Sun, 08 Oct 2017 19:13:34 -0700, ipatrol6...@yahoo.com wrote:
> As per a discussion on the IRC channel, I am requesting that a
> regression bug be filed regarding the incorrect handling of variables
> referenced to packages by the Z and X meta-operators.
> 
> The behavior over time is detailed here:
> https://gist.github.com/Whateverable/83ce4a1ea73a429131713367ee23542e
> 
> Of those, the result given from 2015.09 to 2015.11 is the correct and
> desired behavior, considering the design documents.
> 
> Doing a bisection:
> https://gist.github.com/Whateverable/0a73f0433cee989d7a82ce8a601bd465
> 
> The regression revision appears to be:
> https://github.com/rakudo/rakudo/commit/234287170cb7dd234ffc90271d762b1c106bc57f
> 
I think Z and X are doing the right thing: treating something in a Scalar 
container as a single item. This is done consistently across the language, so 
you'll observe this behavior in many more constructs.

The surprise here is that we get a Scalar container in the first place:

$ perl6-m -e 'my @A::B = (1..8); say @A::B.VAR.WHAT; my @B = (1..8); say 
@B.VAR.WHAT'
(Scalar)
(Array)

I suspect that's happening because a package's symbol table is a Stash, which 
is just a subclass of Hash, and this code is relying on autovivification, which 
always involves a Scalar container. It probably should instead, somehow, be 
arranging than an Array is bound directly into the Stash. Then Z, X, and plenty 
more things would work less surprisingly. A `my %A::B` is likely exhibiting the 
same kind of problem at the moment too.

[perl #132292] [REGRESSION] Recursively .emit-ing from the tap of the same supply bails out

[jn...@jnthn.net via RT]

On Fri, 13 Oct 2017 20:43:42 -0700, alex.jakime...@gmail.com wrote:
> Code:
> my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ <
> 5; say "here" }; $s1.emit(1)
> 
> 
> ¦2017.06:
> 1
> 2
> 2
> 2
> 2
> 2
> here
> here
> here
> here
> here
> here
> 
> ¦HEAD(012c80f):
> 1
> here
> 2
> 
> 
> Possible IRC discussion: https://irclog.perlgeek.de/perl6-dev/2017-09-
> 21#i_15197905
> 
> The behavior change two times, first it started hanging after
> (2017-09-18)
> https://github.com/rakudo/rakudo/commit/4a8038c2956e863bc661a2a00e8371eb98002608
> And then the hang was resolved (incorrectly?) in
> (2017-09-22)
> https://github.com/rakudo/rakudo/commit/547839200a772e26ea164e9d1fd8c9cd4a5c2d9f
> 
> 
> I think the output on 2017.06 makes more sense.

Actually the 2017.06 behavior was clearly wrong, because it violates the 
principle that a Supply chain will process a message at a time. That every 
"here" comes out at the end illustrates that the tap block was reentered. That 
was not an intended behavior, but rather an accident resulting through use of a 
reentrant mutex for some (not all) Supply concurrency management.

The commits in question and those around them introduced a unified concurrency 
model for all Supply operations, including `supply` blocks, based around 
Lock::Async. The changes fixed many other issues, but also forced a revisit of 
the question of recursion - effectively, a supply sending a message to itself. 
This is a tricky problem, because there's some competing design goals around 
supplies:

1. Serial message processing (as mentioned above)
2. Back-pressure: those who emit into a Supply chain pay the cost of the 
message processing.
3. Fairness: Messages are processed in the order they arrive.
4. No concurrency unless requested

I think 1 is pretty non-negotiable, because it's hard to write reliable 
concurrent code if you don't know what your "transaction scope" is. I really 
should have noticed the issue with reentrant mutexes sooner, though I guess 
that's my own usage biases to blame: I very rarely use `.tap` and instead use 
supply/react/whenever where this issue could never happen (but - big issue - 
the old supply block mechanism violated goal 2 and arguably 3).

So the interesting question is how many we can have out of 2, 3, and 4. The 
current solution, on recursion, is to schedule the recursive message handling 
using the current $*SCHEDULER. This makes sure that we get 1 and 3. 
Unfortunately, it violates 2 (if we expect it to be transitively applied) and 
4, and it's 4 that results in the effects reported here.

Note that if it's rewritten as:

my $s1 = Supplier.new; react { whenever $s1 { say $_; $++ < 5 ?? $s1.emit(2) !! 
$s1.done; say "here" }; $s1.emit(1) }

Then it works fine:

1
here
2
here
2
here
2
here
2
here
2
here

Also if you sleep a bit after the original:

my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ < 5; say 
"here" }; $s1.emit(1); sleep 1

Then the output is the same as the above also (of course, the react block way 
is the correct one, not sleeping!)

So the question is if we can find a way to have 2 and 4, while retaining 1 and 
3, and at what cost.

[perl #132292] [REGRESSION] Recursively .emit-ing from the tap of the same supply bails out

[jn...@jnthn.net via RT]

On Fri, 13 Oct 2017 20:43:42 -0700, alex.jakime...@gmail.com wrote:
> Code:
> my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ <
> 5; say "here" }; $s1.emit(1)
> 
> 
> ¦2017.06:
> 1
> 2
> 2
> 2
> 2
> 2
> here
> here
> here
> here
> here
> here
> 
> ¦HEAD(012c80f):
> 1
> here
> 2
> 
> 
> Possible IRC discussion: https://irclog.perlgeek.de/perl6-dev/2017-09-
> 21#i_15197905
> 
> The behavior change two times, first it started hanging after
> (2017-09-18)
> https://github.com/rakudo/rakudo/commit/4a8038c2956e863bc661a2a00e8371eb98002608
> And then the hang was resolved (incorrectly?) in
> (2017-09-22)
> https://github.com/rakudo/rakudo/commit/547839200a772e26ea164e9d1fd8c9cd4a5c2d9f
> 
> 
> I think the output on 2017.06 makes more sense.

Actually the 2017.06 behavior was clearly wrong, because it violates the 
principle that a Supply chain will process a message at a time. That every 
"here" comes out at the end illustrates that the tap block was reentered. That 
was not an intended behavior, but rather an accident resulting through use of a 
reentrant mutex for some (not all) Supply concurrency management.

The commits in question and those around them introduced a unified concurrency 
model for all Supply operations, including `supply` blocks, based around 
Lock::Async. The changes fixed many other issues, but also forced a revisit of 
the question of recursion - effectively, a supply sending a message to itself. 
This is a tricky problem, because there's some competing design goals around 
supplies:

1. Serial message processing (as mentioned above)
2. Back-pressure: those who emit into a Supply chain pay the cost of the 
message processing.
3. Fairness: Messages are processed in the order they arrive.
4. No concurrency unless requested

I think 1 is pretty non-negotiable, because it's hard to write reliable 
concurrent code if you don't know what your "transaction scope" is. I really 
should have noticed the issue with reentrant mutexes sooner, though I guess 
that's my own usage biases to blame: I very rarely use `.tap` and instead use 
supply/react/whenever where this issue could never happen (but - big issue - 
the old supply block mechanism violated goal 2 and arguably 3).

So the interesting question is how many we can have out of 2, 3, and 4. The 
current solution, on recursion, is to schedule the recursive message handling 
using the current $*SCHEDULER. This makes sure that we get 1 and 3. 
Unfortunately, it violates 2 (if we expect it to be transitively applied) and 
4, and it's 4 that results in the effects reported here.

Note that if it's rewritten as:

my $s1 = Supplier.new; react { whenever $s1 { say $_; $++ < 5 ?? $s1.emit(2) !! 
$s1.done; say "here" }; $s1.emit(1) }

Then it works fine:

1
here
2
here
2
here
2
here
2
here
2
here

Also if you sleep a bit after the original:

my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ < 5; say 
"here" }; $s1.emit(1); sleep 1

Then the output is the same as the above also (of course, the react block way 
is the correct one, not sleeping!)

So the question is if we can find a way to have 2 and 4, while retaining 1 and 
3, and at what cost.

[perl #132292] [REGRESSION] Recursively .emit-ing from the tap of the same supply bails out

[jn...@jnthn.net via RT]

On Mon, 16 Oct 2017 07:42:06 -0700, jn...@jnthn.net wrote:
> So the question is if we can find a way to have 2 and 4, while
> retaining 1 and 3, and at what cost.

Also noting that in order to preserve 3, then in a situation like:

* "Thread" 1 sends message A
* Handler for message A starts running
* "Thread" 2 sends message B
* Handler for message A emits a recursive message C
* Handler for message A completes

Then at this point, the next thing that needs to happen is for message B to be 
processed, for fairness. This means that "Thread" 1 needs to (non-blockingly, 
if in the pool) wait until message B has been processed, before it can do 
message C. If we can arrange for that to happen then I guess things would work 
out OK enough: the sender of A will have to wait a while, but the recursive 
message send of C was "its fault" for sending A.

[perl #132287] [REGRESSION][CONC] stall/block in async heavy code

[jn...@jnthn.net via RT]

On Fri, 13 Oct 2017 10:13:36 -0700, jdv79 wrote:
> The symptom is that sometimes (90%+ on my box and ungolfed) the program
> seems to stall out on the qqx line.  The last thing that is printed is
> "before".  But on earlier commits (that actually build) no such 
> stallage
> happened for a couple minutes run time.
> 
So if I follow correctly, the changes in the commit mentioned make an existing 
problem manifest sooner, rather than introduce the problem?

[perl #132287] [REGRESSION][CONC] stall/block in async heavy code

[jn...@jnthn.net via RT]

On Fri, 13 Oct 2017 10:13:36 -0700, jdv79 wrote:
> The symptom is that sometimes (90%+ on my box and ungolfed) the program
> seems to stall out on the qqx line.  The last thing that is printed is
> "before".  But on earlier commits (that actually build) no such 
> stallage
> happened for a couple minutes run time.
> 
So if I follow correctly, the changes in the commit mentioned make an existing 
problem manifest sooner, rather than introduce the problem?

[perl #129234] [CONC] `.hyper` and `.race` resume after exceptions

[jn...@jnthn.net via RT]

On Thu, 08 Sep 2016 12:55:10 -0700, sml...@gmail.com wrote:
> If you `die` inside a `map/for` that is being `hyper/race`d...
> 
> for (1..1).hyper { die };  sleep 1;  say "Alive";
> 
> ...it prints the exception's backtrace, but then resumes the program
> as if nothing had happened:
> 
> Died
>   in block  at -e line 1
> 
> Alive

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t. The new error reporting shows the backtrace of both the 
thing that went wrong in a worker thread, and the place that triggered the 
iteration.

[perl #129234] [CONC] `.hyper` and `.race` resume after exceptions

[jn...@jnthn.net via RT]

On Thu, 08 Sep 2016 12:55:10 -0700, sml...@gmail.com wrote:
> If you `die` inside a `map/for` that is being `hyper/race`d...
> 
> for (1..1).hyper { die };  sleep 1;  say "Alive";
> 
> ...it prints the exception's backtrace, but then resumes the program
> as if nothing had happened:
> 
> Died
>   in block  at -e line 1
> 
> Alive

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t. The new error reporting shows the backtrace of both the 
thing that went wrong in a worker thread, and the place that triggered the 
iteration.

[perl #128084] [CONC][BUG] `.hyper/.race.map(&f)` produces an empty sequence if `&f` is a multi-sub

[jn...@jnthn.net via RT]

On Fri, 06 May 2016 08:37:20 -0700, sml...@gmail.com wrote:
> ➜  sub f ($a) { $a**2 };  say (^10).hyper.map(&f).list;
> (0 1 4 9 16 25 36 49 64 81)
> 
> ➜  multi f ($a) { $a**2 };  say (^10).hyper.map(&f).list;
> ()
> 
> The expected behavior would be for both examples to produce the same
> (non-empty) list.
> 
> (This was first reported by Alexander Moquin in a comment to ticket
> #126597, but I think it should have its own ticket.)

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #128084] [CONC][BUG] `.hyper/.race.map(&f)` produces an empty sequence if `&f` is a multi-sub

[jn...@jnthn.net via RT]

On Fri, 06 May 2016 08:37:20 -0700, sml...@gmail.com wrote:
> ➜  sub f ($a) { $a**2 };  say (^10).hyper.map(&f).list;
> (0 1 4 9 16 25 36 49 64 81)
> 
> ➜  multi f ($a) { $a**2 };  say (^10).hyper.map(&f).list;
> ()
> 
> The expected behavior would be for both examples to produce the same
> (non-empty) list.
> 
> (This was first reported by Alexander Moquin in a comment to ticket
> #126597, but I think it should have its own ticket.)

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #131865] [REGRESSION] Looping over a HyperSeq in sink context does nothing (for .hyper { say 2 })

[jn...@jnthn.net via RT]

On Tue, 08 Aug 2017 18:40:48 -0700, alex.jakime...@gmail.com wrote:
> Code:
> for .hyper { say 2 }
> 
> Result (2015.09-2015.06):
> 2
> 2
> 2
> 
> Result (9b0b9effe5,2017.07,HEAD(6745517)):
> (no output)
> 
> 
> There seems to be nothing wrong with using a for loop on .hyper or
> .race, so it should work. It starts working if you somehow use the
> value, for example:
> 
> Code:
> do for .hyper { say 2 }
> 
> Result (9b0b9effe5,2017.07,HEAD(6745517)):
> 2
> 2
> 2
> 
> 
> Bisectable points to
> https://github.com/rakudo/rakudo/commit/9b0b9effe5fee1f35497cf97a5e7bda9bb083507

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #131865] [REGRESSION] Looping over a HyperSeq in sink context does nothing (for .hyper { say 2 })

[jn...@jnthn.net via RT]

On Tue, 08 Aug 2017 18:40:48 -0700, alex.jakime...@gmail.com wrote:
> Code:
> for .hyper { say 2 }
> 
> Result (2015.09-2015.06):
> 2
> 2
> 2
> 
> Result (9b0b9effe5,2017.07,HEAD(6745517)):
> (no output)
> 
> 
> There seems to be nothing wrong with using a for loop on .hyper or
> .race, so it should work. It starts working if you somehow use the
> value, for example:
> 
> Code:
> do for .hyper { say 2 }
> 
> Result (9b0b9effe5,2017.07,HEAD(6745517)):
> 2
> 2
> 2
> 
> 
> Bisectable points to
> https://github.com/rakudo/rakudo/commit/9b0b9effe5fee1f35497cf97a5e7bda9bb083507

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #130576] [CONC] .race & .hyper break [+] (1..100)

[jn...@jnthn.net via RT]

On Tue, 17 Jan 2017 10:10:44 -0800, coke wrote:
> The presence of a hyper or race causes incorrect output in some cases.
> Originally found with the .grep in the pipeline, but seems to impact
> even the simple case.
> 
> $ perl6 -e 'say [+] (1..100)'
> 5050
> 
> $ perl6 -e 'say [+] (1..100).race'
> 0
> 
>  $ perl6 -e 'say [+] (1..100).grep(* != 22)'
> 5028
> 
>  $ perl6 -e 'say [+] (1..100).race.grep(* != 22)'
> 0
> 
> $ perl6 -e 'say [+] (1..100).hyper.grep(* != 22)'
> 0
> 
> $ perl6 -e 'say [+] (1..100).grep(* != 22).race'
> 0
> 
> $ perl6 -e 'say [+] (1..100).grep(* != 22).hyper'
> 0
> 

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #130576] [CONC] .race & .hyper break [+] (1..100)

[jn...@jnthn.net via RT]

On Tue, 17 Jan 2017 10:10:44 -0800, coke wrote:
> The presence of a hyper or race causes incorrect output in some cases.
> Originally found with the .grep in the pipeline, but seems to impact
> even the simple case.
> 
> $ perl6 -e 'say [+] (1..100)'
> 5050
> 
> $ perl6 -e 'say [+] (1..100).race'
> 0
> 
>  $ perl6 -e 'say [+] (1..100).grep(* != 22)'
> 5028
> 
>  $ perl6 -e 'say [+] (1..100).race.grep(* != 22)'
> 0
> 
> $ perl6 -e 'say [+] (1..100).hyper.grep(* != 22)'
> 0
> 
> $ perl6 -e 'say [+] (1..100).grep(* != 22).race'
> 0
> 
> $ perl6 -e 'say [+] (1..100).grep(* != 22).hyper'
> 0
> 

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #126597] [CONC][BUG] .race.grep(...) and .hyper.grep(...) produce an empty sequence

[jn...@jnthn.net via RT]

On Mon, 09 Nov 2015 06:36:49 -0800, matt.oa...@congenica.com wrote:
> The following has an empty result when using race or hyper in perl6
> version 2015.10-233-gac750a9 built on MoarVM version 2015.10-61-
> g624d504
> 
> $ perl6
> > (1..10).grep(* > 3)
> (4 5 6 7 8 9 10)
> > (1..10).race().grep(* > 3)
> ()
> > (1..10).hyper().grep(* > 3)
> ()
> 
> Looks specifically like the methods after the hyper/race never get
> called:
> 
> > (1..10).hyper().grep(* > 3).map({$_.say; $_})
> ()
> > (1..10).map({$_.say; $_}).hyper().grep(* > 3)
> 1
> 2
> 3
> 4
> 5
> 6
> 7
> 8
> 9
> 10
> ()

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #126597] [CONC][BUG] .race.grep(...) and .hyper.grep(...) produce an empty sequence

[jn...@jnthn.net via RT]

On Mon, 09 Nov 2015 06:36:49 -0800, matt.oa...@congenica.com wrote:
> The following has an empty result when using race or hyper in perl6
> version 2015.10-233-gac750a9 built on MoarVM version 2015.10-61-
> g624d504
> 
> $ perl6
> > (1..10).grep(* > 3)
> (4 5 6 7 8 9 10)
> > (1..10).race().grep(* > 3)
> ()
> > (1..10).hyper().grep(* > 3)
> ()
> 
> Looks specifically like the methods after the hyper/race never get
> called:
> 
> > (1..10).hyper().grep(* > 3).map({$_.say; $_})
> ()
> > (1..10).map({$_.say; $_}).hyper().grep(* > 3)
> 1
> 2
> 3
> 4
> 5
> 6
> 7
> 8
> 9
> 10
> ()

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t.

[perl #127452] [CONC] hyper is very broken, *sometimes* it returns nothing

[jn...@jnthn.net via RT]

On Mon, 01 Feb 2016 21:59:33 -0800, alex.jakime...@gmail.com wrote:
> Let's try running the same code twice:
> 
>  m: my @x = ^10; my @y = @x.hyper(:3batch, :5degree).map: {
> sleep rand / 10; $_ + 1 }; say @y
>  rakudo-moar a5fe34: OUTPUT«[]␤»
>  m: my @x = ^10; my @y = @x.hyper(:3batch, :5degree).map: {
> sleep rand / 10; $_ + 1 }; say @y
>  rakudo-moar a5fe34: OUTPUT«[1 2 3 4 5 6 7 8 9 10]␤»
> 
> The thing is that sometimes ｢hyper｣ decides to return an empty list. This
> is a real problem because it basically renders it useless (too unreliable).
> 
> If you have problems reproducing it, try changing ｢/ 10｣ to something
> else. This delay is not really relevant but it seems like it affects the
> chances to get one result or another.
> 
> Here's also a golfed down version:
> 
> Code:
> dd @(^10 .hyper(:1batch).map: { sleep rand; $_ })
> 
> Result:
> ()
> OR
> (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
> 
> According to the execution time ｢hyper｣ does not even bother to compute the
> result in these problematic cases (it just returns an empty list without
> trying to process the data). That being said, there is a chance that it
> fails half way through, see next example.
> 
> 
> Another interesting issue (probably the same one):
> 
> Code:
> .say for (^10).hyper(:1batch).map: { sleep rand; $_ }
> 
> Result:
> 1
> OR
> 2
> OR
> 3
> OR
> 0
> 3
> 2
> 1
> 5
> 8
> 7
> 4
> 6
> 9
> OR etc.
> 
> So not only it is out of order (#127099), but it is also extremely broken.
> 
> It looks like there is no such problem with ｢race｣.
> 
> IRC log (nothing useful there though):
> http://irclog.perlgeek.de/perl6/2016-02-02#i_11975794

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t (for original post and other examples that I found in the 
discussion of this ticket).

[perl #127452] [CONC] hyper is very broken, *sometimes* it returns nothing

[jn...@jnthn.net via RT]

On Mon, 01 Feb 2016 21:59:33 -0800, alex.jakime...@gmail.com wrote:
> Let's try running the same code twice:
> 
>  m: my @x = ^10; my @y = @x.hyper(:3batch, :5degree).map: {
> sleep rand / 10; $_ + 1 }; say @y
>  rakudo-moar a5fe34: OUTPUT«[]␤»
>  m: my @x = ^10; my @y = @x.hyper(:3batch, :5degree).map: {
> sleep rand / 10; $_ + 1 }; say @y
>  rakudo-moar a5fe34: OUTPUT«[1 2 3 4 5 6 7 8 9 10]␤»
> 
> The thing is that sometimes ｢hyper｣ decides to return an empty list. This
> is a real problem because it basically renders it useless (too unreliable).
> 
> If you have problems reproducing it, try changing ｢/ 10｣ to something
> else. This delay is not really relevant but it seems like it affects the
> chances to get one result or another.
> 
> Here's also a golfed down version:
> 
> Code:
> dd @(^10 .hyper(:1batch).map: { sleep rand; $_ })
> 
> Result:
> ()
> OR
> (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
> 
> According to the execution time ｢hyper｣ does not even bother to compute the
> result in these problematic cases (it just returns an empty list without
> trying to process the data). That being said, there is a chance that it
> fails half way through, see next example.
> 
> 
> Another interesting issue (probably the same one):
> 
> Code:
> .say for (^10).hyper(:1batch).map: { sleep rand; $_ }
> 
> Result:
> 1
> OR
> 2
> OR
> 3
> OR
> 0
> 3
> 2
> 1
> 5
> 8
> 7
> 4
> 6
> 9
> OR etc.
> 
> So not only it is out of order (#127099), but it is also extremely broken.
> 
> It looks like there is no such problem with ｢race｣.
> 
> IRC log (nothing useful there though):
> http://irclog.perlgeek.de/perl6/2016-02-02#i_11975794

Fixed in new hyper/race implementation; tests in S07-hyperrace/hyper.t and 
S07-hyperrace/race.t (for original post and other examples that I found in the 
discussion of this ticket).

[perl #62262] [BUG] initialiazation of parent attributes

[jn...@jnthn.net via RT]

On Mon Jan 12 14:44:35 2009, moritz wrote:
> I've added some tests to t/spec/S12-construction/new.t which boil down to:
> 
> #+snip
> class Parent {
> has $.x;
> }
> 
> class Child is Parent {
> has $.y;
> }
> 
> my $o;
> ...
> 
> lives_ok { $o = Child.new( :y(4), Parent{ :x<5> }) },
>  'can instanticate class with explicit specification of parent
> attrib';
> 
> #?rakudo todo 'OO initialization bug'
> is $o.y, 4, '... worked for the child';
> is $o.x, 5, '... worked for the parent';
> #-snip
> 
> Here the initialization of Parent{ :x<5> } worked, but that of the Child
> failed ($o.y is undef).
> 
I think this is a dupe of an already resolved ticket (I remember fixing
a bug like this, and further we already have got these tests unfudged
and that file is in spectest.data). So resolving ticket.

Thanks,

Jonathan

[perl #62966] Rakudo reports a type mismatch in assignment when typing an attribute with a role

[jn...@jnthn.net via RT]

On Sun Feb 01 02:19:22 2009, masak wrote:
> With Rakudo 153a4a8 on Parrot r36239, I get the following:
> 
> $ perl6 -e 'role A {}; class B { has A $!a }; B.new'
> Type mismatch in assignment.
> [...]
> 
> I see nothing wrong with the above, so I don't think the error is 
> justified.

Aye, there were a range of quirks/issues in role type checking, which
I've now fixed in git 88b6c25. Plus added some tests for roles being
used as type constraints.

Thanks,

Jonathan

[perl #63568] Null PMC access when calling *any* method on a multi sub in Rakudo

[jn...@jnthn.net via RT]

On Tue Mar 10 13:45:27 2009, moritz wrote:
> On Sun Mar 01 02:44:17 2009, masak wrote:
> >  oh and is arity defined on multi subs ?
> >  rakudo: sub f($z) {}; &f.arity.say
> >  rakudo af4b73: OUTPUT«1␤»
> >  rakudo: multi sub f($z) {}; &f.arity.say
> >  rakudo af4b73: OUTPUT«Null PMC access in find_method()␤ [...]
> > * masak submits rakudobug
> >  szabgab++
> 
> Actually calling *any* method on &f gives the Null PMC access, if f() is
> a multi:
> 
> ./perl6 -e 'multi f { ... }; say &f.WHAT'
> Null PMC access in find_method()
> 

Fixed this somewhat. Notes:

* .WHAT now reports Multi and this is spectest'd.
* .candidates can be called to get the set of candidates. You can then
use .signature and .arity on those (.candidates is not quite spec yet,
but probably fine and reasonable to go in to the spec - someone has
agreed to do that in the next day or so)
* .signature and .arity for now are not defined on Multi - it's not
spec'd and I'm not so sure exactly what it'd do, though an any junction
seems easiest. But maybe better to make people write
any(&foo.candidates).signature to get that (this produces an any
junction of signatures).

See 76b2652.

Thanks,

Jonathan

[perl #63812] MMD bug with one candidate, who have where trait

[jn...@jnthn.net via RT]

On Thu Mar 12 06:28:03 2009, ihrd wrote:
> class C {
> proto method Foo ($any) { ":)".say }
> multi method Foo ($foo where { $_ eq "foo"}) { "$foo".say }
> }
> C.Foo("foo"); # foo
> C.Foo("bar"); # should smile but
> # die with "Parameter type check failed for $foo
> in call to Foo"
> 

Fixed in git e8a8fb6 and added something very like the above as a spectest.

Thanks!

Jonathan

[perl #63594] [BUG] memory leaks

[jn...@jnthn.net via RT]

On Mon Mar 02 01:04:52 2009, ml...@physik.uni-wuerzburg.de wrote:
> It seems that even the simplest programs leak memory:
> 
> $ perl6 -e 'while 1 { }'
> 
> Watching this in top (1) shows that it leaks about 1m per second on my
> machine (linux amd64).
> 
It's interesting to note that while this leaks, this one:

perl6 -e '42 while 1'

Does not.

Jonathan

[perl #63570] Null PMC access in &f.signature.params in Rakudo

[jn...@jnthn.net via RT]

On Sun Mar 01 02:51:32 2009, masak wrote:
>  rakudo: sub f($z) {}; &f.signature.params
>  rakudo af4b73: RESULT«Null PMC access in find_method()␤ [...]
> * masak submits rakudobug

Fixed in git 41267fd.

Thanks,

Jonathan

[perl #63872] [PATCH] implement $*PROGRAM_NAME (S28)

[jn...@jnthn.net via RT]

On Sat Mar 14 11:07:14 2009, mbere...@autoexec.demon.nl wrote:
> Just a one liner to copy the full pathname of the running Perl script, or
> otherwise '-e' into $*PROGRAM_NAME. (The * twigil does not appear in the
> patch).
> 
Thanks, applied as git 6d97951. Not sure what it should do in -e, but
will leave that for particle who will be doing a larger review of
command line things anyway.

Jonathan

[perl #56976] [TODO] implement state variables

[jn...@jnthn.net via RT]

On Wed Jul 16 01:47:29 2008, mor...@casella.verplant.org wrote:
> State variables need to be implemented, the tests are already there in
> S04-declarations/state.t
> 
> As a side node (and related to RT #56748), rakudo as of r29490 gives a
> bogus error message when encountering a state declaration:
> 
> $ ../../parrot perl6.pbc -e 'sub foo { state $x }; 1'
> Null PMC access in type()
> 
State variables are now implemented as of git cc85a31. :-) I'm now
passing everything in state.t that I can by just implementing state
variables (other failures are due to other missing features), so I think
that's enough to resolve this ticket. :-)

Thanks,

Jonathan

[perl #56976] [TODO] implement state variables

[jn...@jnthn.net via RT]

On Wed Jul 16 01:47:29 2008, mor...@casella.verplant.org wrote:
> State variables need to be implemented, the tests are already there in
> S04-declarations/state.t
> 
> As a side node (and related to RT #56748), rakudo as of r29490 gives a
> bogus error message when encountering a state declaration:
> 
> $ ../../parrot perl6.pbc -e 'sub foo { state $x }; 1'
> Null PMC access in type()
> 
State variables are now implemented as of git cc85a31. :-) I'm now
passing everything in state.t that I can by just implementing state
variables (other failures are due to other missing features), so I think
that's enough to resolve this ticket. :-)

Thanks,

Jonathan

[perl #62184] Rakudo doesn't DTRT with 'state' variables and cloned blocks

[jn...@jnthn.net via RT]

On Sun Jan 11 04:59:51 2009, masak wrote:
> Rakudo r35404 doesn't work according to spec in the area of 'state'
> variables and cloned blocks.
> 
> $ perl6 -e 'my $i = 0; my $func = { state $x = $i++; say $x }; my ($a,
> $b) = $func.clone, $func.clone; $a(); $b(); $a()'
> 0
> 1
> 2
> 
> According to S04, this should be equivalent to
> 
> $func = { state $x will start { $x = $i++ }; say $i };
> 
> and thus (in my understanding) produce
> 
> 0
> 1
> 1

Actually expected output is:

0
1
0

Or at least I talked at least two other people into #perl6 into agreeing
with me that it should be. ;-)

Added this with 0/1/0 as expected output to the spectests, and it's passing.

Thanks,

Jonathan

[perl #63824] [PATCH] build/gen_metaop_pir.pl: added missing backslash in \u00bb

[jn...@jnthn.net via RT]

On Thu Mar 12 15:29:54 2009, kru...@gmail.com wrote:
> diff --git a/build/gen_metaop_pir.pl b/build/gen_metaop_pir.pl
> index 3f858a3..78e4fce 100644
> --- a/build/gen_metaop_pir.pl
> +++ b/build/gen_metaop_pir.pl
> @@ -131,7 +131,7 @@ while (@ops) {
>  .end\n);
> 
>  # LHS-dwimming hyper ops.
> -$hypername = qq(unicode:"infix:\u00bb$opname\\u00bb");
> +$hypername = qq(unicode:"infix:\\u00bb$opname\\u00bb");
>  push @gtokens, sprintf($hyper_no_dwim_fmt, $hypername, $opname,
> "<<$opname<<", $opname, $hypername);
>  push @code, qq(
>  .sub $hypername

Hey, good catch! Applied as git 285279b (though IIRC thanks to a Parrot
hash/unicode interaction bug unicode forms of hyper ops currently don't
work...that may be out of date though)

Jonathan

[perl #63906] Matching against Junction fails

[jn...@jnthn.net via RT]

On Sun Mar 15 12:12:51 2009, richardh wrote:
> perl6
>  > my @x=1|11,2,3;my $s=[+] @x; $s.perl.say; $s ~~ Junction and say 'hi'
> any(6, 16)
> Null PMC access in get_integer()
> 
Fixed in git 90c8b8e.

> my @x=1|11,2,3;my $s=[+] @x; $s.perl.say; $s ~~ Junction and say 'hi'
any(6, 16)
hi

Thanks,

Jonathan

[perl #63908] ~$stuff.WHAT should end in ()

[jn...@jnthn.net via RT]

On Sun Mar 15 15:27:41 2009, moritz wrote:
> S12-objects.pod
> 1581:WHATthe protoobject of the type, stringifies to short
> name ~ '()'
> 
> Currently it stringfies to short name without trailing ().
> 
> If somebody submits a patch for it, I'll fix the test suite (which has
> this wrong in many places, I fear; used to be different).
> 
I'm happy to put one in after the next release. However, since this
affects compiler toolkit level stuff too (at least in the obvious way
that we'd do it), do we need a deprecation notice/cycle in Parrot for
this? Ideally people won't have been relying on .WHAT to do string
comparisons to check what things are since it's a Really Bad Idea, so
IMO we don't...

Jonathan

[perl #63918] eigenstates method on Object

[jn...@jnthn.net via RT]

On Mon Mar 16 00:57:05 2009, richardh wrote:
> An .eigenstates method is needed for Object.
> ...
> Larry Wall wrote:
> I see no particular reason not to add an .eigenstates method to Object
> that returns a list of the object itself . 
> 

And so it is added in git 1ea27f2, plus tests.

Thanks!

Jonathan