In perl.git, the branch smoke-me/lexsub has been created
<http://perl5.git.perl.org/perl.git/commitdiff/5b9c931d372e53ca0f4fa0239c0dacceed8f1efd?hp=0000000000000000000000000000000000000000>
at 5b9c931d372e53ca0f4fa0239c0dacceed8f1efd (commit)
- Log -----------------------------------------------------------------
commit 5b9c931d372e53ca0f4fa0239c0dacceed8f1efd
Author: Father Chrysostomos <[email protected]>
Date: Tue Sep 11 13:58:24 2012 -0700
Document lexical subs
M lib/experimental.pm
M pod/perlexperiment.pod
M pod/perlsub.pod
commit ca355f061742d27b11213d799ca12352aa99f910
Author: Father Chrysostomos <[email protected]>
Date: Tue Sep 11 13:24:59 2012 -0700
Increase $feature::VERSION to 1.30
M lib/feature.pm
M regen/feature.pl
commit fe144bb232b068f765f117005514858a3909bf87
Author: Father Chrysostomos <[email protected]>
Date: Tue Sep 11 13:24:06 2012 -0700
Disable lexsubs outside of experimental.pm
M feature.h
M lib/experimental.pm
M lib/feature.pm
M pod/perldiag.pod
M regen/feature.pl
M t/cmd/lexsub.t
M t/lib/croak/toke
M t/lib/warnings/toke
M toke.c
commit 728a0dd20001c526a9469d2b4f7700f22167fb3e
Author: Father Chrysostomos <[email protected]>
Date: Tue Sep 11 13:11:05 2012 -0700
experimental.pm
M MANIFEST
M Porting/Maintainers.pl
A lib/experimental.pm
A lib/experimental.t
M lib/feature.pm
M regen/feature.pl
commit 5b22f69cd8578e4e26445645296c4b34af1c458a
Author: Father Chrysostomos <[email protected]>
Date: Mon Sep 10 23:44:11 2012 -0700
perlsub: Document state variables better
M pod/perlsub.pod
commit dbf6a4cb2975e082c2222fae6226eff9ac29fb8b
Author: Father Chrysostomos <[email protected]>
Date: Mon Sep 10 22:29:15 2012 -0700
Allow lexical sub redefinition inside eval
For non-clonable state subs, this already happened to work.
For any clonable subs, we need to clone the sub as soon as it
is defined.
For redefined state subs, we need to apply the new sub to all recur-
sion levels, as state subs are shared.
M op.c
M t/cmd/lexsub.t
commit 0ec56a544cc98a9032e44843bb25066a10843fe4
Author: Father Chrysostomos <[email protected]>
Date: Mon Sep 10 21:59:51 2012 -0700
Move my sub prototype CVs to the pad names
my subs are cloned on scope entry. To make closures work, a stub
stored in the pad (and closed over elsewhere) is cloned into.
But we need somewhere to store the prototype from which the clone is
made. I was attaching the prototype via magic to the stub in the pad,
since the pad is available at run time, but not the pad names.
That leads to lots of little games all over the place to make sure
the prototype isnât lost when the pad is swiped on scope exit
(SAVEt_CLEARSV in scope.c). We also run the risk of losing it if an
XS module replaces the sub with another.
Instead, we should be storing it with the pad name. The previous com-
mit made the pad names available at run time, so we can move it there
(still stuffed inside a magic box) and delete much code.
This does mean that newMYSUB cannot rely on the behaviour of non-clon-
able subs that close over variables (or subs) immediately. Previ-
ously, we would dig through outer scopes to find the stub in cases
like this:
sub y {
my sub foo;
sub x {
sub {
sub foo { ... }
}
}
}
We would stop at x, which happens to have yâs stub in its pad, so
thatâs no problem.
If we attach it to the pad name, we definitely have to dig past x to
get to the pad name in yâs pad.
Usually, immediate closures do not store the parent pad index, since
it will never be used. But now we do need to use it, so we modify the
code in pad.c:S_pad_findlex to set it always for my/state.
M op.c
M pad.c
M pp.c
M scope.c
commit 367a6f7894981acc81feb22ce65868f3f693c458
Author: Father Chrysostomos <[email protected]>
Date: Mon Sep 10 16:07:30 2012 -0700
Set PL_comppad_name on sub entry
M pad.h
M scope.c
M scope.h
M sv.c
commit 72a7a99f32ea558de51fcbfe57b98ff530807544
Author: Father Chrysostomos <[email protected]>
Date: Sun Sep 9 11:09:53 2012 -0700
lexsub.t: Test state sub defined inside eval
M t/cmd/lexsub.t
commit c47b3454938fdcd717be617a9bb87bb595c55a95
Author: Father Chrysostomos <[email protected]>
Date: Sat Sep 8 19:28:00 2012 -0700
Honour lexical prototypes
newCVREF is changed to return a PADCV op, not an RV2CV with a PADCV
kid, to keep the rv2cv_op_cv changes to a minimum. (For some reason,
if newCVREF returns an RV2CV, we end up with two inside each other.)
I also added a test for recursion, since I nearly broke it.
M op.c
M t/cmd/lexsub.t
commit e8d69e0f4d4367a30d737797848a3303314e26fa
Author: Father Chrysostomos <[email protected]>
Date: Thu Sep 6 22:57:50 2012 -0700
Donât mention pkg in proto warnings for lex subs
M t/lib/warnings/toke
M toke.c
commit 474f699ce2b305663e65e0db554b793146bfad46
Author: Father Chrysostomos <[email protected]>
Date: Thu Sep 6 22:11:36 2012 -0700
pad.c: Put unavailability warning in one spot
M pad.c
commit 482acb3742c39a54d8d177d971a3a109ce39169d
Author: Father Chrysostomos <[email protected]>
Date: Thu Sep 6 20:32:47 2012 -0700
Use the same outside logic for mysubs and formats
By using find_runcv_where both for formats and my subs nested in inner
clonable subs, we can simplify the code.
It happens to make this work ($x is visible):
use 5.01;
sub not_lexical8 {
my sub foo;
foo();
sub not_lexical9 {
my sub bar {
my $x = 'khaki car keys for the khaki car';
not_lexical8();
sub foo { warn $x }
}
bar()
}
}
not_lexical9();
This is definitely iffy code, but if making it work makes the imple-
mentation simpler, so why not?
M pad.c
M t/cmd/lexsub.t
commit 4273103bb3ed134940357a89c0d856185f71a2fa
Author: Father Chrysostomos <[email protected]>
Date: Thu Sep 6 18:05:35 2012 -0700
Fix subroutine unavailability during cloning
sub foo {
my $x;
format =
@
$x||'#'
.
}
write;
__END__
Variable "$x" is not available at - line 9.
That oneâs OK.
sub foo {
my sub x {};
format =
@
&x
.
}
write;
__END__
Variable "&x" is not available at - line 9.
Assertion failed: (SvTYPE(_svmagic) >= SVt_PVMG), function
S_mg_findext_flags, file mg.c, line 404.
Abort trap
That should say âSubroutineâ. And it shouldnât crash.
The my-sub-cloning code was not taking this case into account. The
value in the proto pad is an undef scalar.
M pad.c
M t/cmd/lexsub.t
commit 95668a08bcf48920cc1d82dcdd82a2a8e4931923
Author: Father Chrysostomos <[email protected]>
Date: Thu Sep 6 16:03:20 2012 -0700
âSubroutine "&x" is not availableâ during compilation
sub {
my $x;
sub { eval '$x' }
}->()()
__END__
Variable "$x" is not available at (eval 1) line 2.
That oneâs OK (though I wonder about the line number).
sub {
my sub x {};
sub { eval '\&x' }
}->()()
__END__
Variable "&x" is not available at (eval 1) line 1.
That should say âSubroutineâ.
M pad.c
M pod/perldiag.pod
M t/cmd/lexsub.t
M t/porting/diag.t
commit 1b901749a0a207bd94ad15821580b6c98af25196
Author: Father Chrysostomos <[email protected]>
Date: Tue Sep 4 10:24:57 2012 -0700
In cv_clone, use pad ID to identify mysub outside
This code prints ARRAY(0x802e10), whereas it should print
SCALAR(0xfedbee):
undef &bar;
eval 'sub bar { my @x }';
{
my sub foo;
foo();
sub bar {
CORE::state $x;
sub foo { warn \$x }
}
}
The foo sub has a strong CvOUTSIDE pointer, but what it points to
can still be undefined and redefined. So we need to identify it
by its pad.
M pad.c
M t/cmd/lexsub.t
commit 05a14df3feb69a52ec79fef46b77dacf84fd01ae
Author: Father Chrysostomos <[email protected]>
Date: Mon Sep 3 21:26:37 2012 -0700
CvOUTSIDE should be strong for lexsub declared in inner pack sub
PadnameOUTER (SvFAKE) entries in pads of clonable subs contain the
offset in the parent pad where the closed-over entry is to be found.
The pad itself does not reference the outer lexical until the sub is
cloned at run time.
newMYSUB had to account for that by following CvOUTSIDE for
PadnameOUTER entries, to account for cases like this:
my sub foo;
my sub bar { sub foo {} }
The sub foo{} definition would have to find the my sub foo declaration
from outside and store the sub there.
That code was not accounting for named package subs, which close over
variables at compile time, so they donât need (and donât) store a par-
ent offset.
So outcv would point to bar in this case:
my sub foo;
sub bar { sub foo {} }
If outcv matched CvOUTSIDE(foo), then CvOUTSIDE was made weak.
That does not help in cases like this:
undef *bar;
{
my sub foo;
sub bar { sub foo {} }
}
If foo has a weak CvOUTSIDE pointer, then it will still point to bar
after bar is freed, which does not help when the sub is cloned and
tries to look at CvROOT(CvOUTSIDE).
If the pad name is marked PadnameOUTER, even if it has no parent pad
index, newMYSUB needs to leave the CvOUTSIDE pointer strongc.
Also, pad_fixup_inner_anons did not account for subs with strong
CvOUTSIDE pointers whose CvOUTSIDE point to the sub whose pad is being
iterated through.
M op.c
M pad.c
M t/cmd/lexsub.t
commit fea4628f99b1f3605a50dc67fcc1ecff46a55f4e
Author: Father Chrysostomos <[email protected]>
Date: Tue Aug 14 18:10:40 2012 -0700
Use the right outside for my subs defined in inner subs
In this example,
{
my sub foo;
sub bar {
sub foo { }
}
}
the foo sub is cloned when the scope containing the âmy subâ declara-
tion is entered, but fooâs CvOUTSIDE pointer points to something other
than the active sub. cv_clone assumes that the currently-running sub
is the right sub to close over (at least for subs; formats are another
matter). That was true in the absence of my subs. This commit
changes it to account.
I had to tweak the test, which was wrong, because sub foo was closing
over a stale var.
M pad.c
M t/cmd/lexsub.t
commit 133424ae670d8a83eb535a76037abe0e3f55833a
Author: Father Chrysostomos <[email protected]>
Date: Tue Aug 14 12:24:43 2012 -0700
Fix Peek.t
M ext/Devel-Peek/t/Peek.t
commit a78dc69935a3ead652bf95d5d6ce02d9f14e1342
Author: Father Chrysostomos <[email protected]>
Date: Mon Aug 13 22:56:05 2012 -0700
Preserve outside pointers of my subs with string eval
The CvHASEVAL flag lets cv_clone know that the clone needs to have its
CvOUTSIDE pointer set, for the sake of string evalsâ being able to
look up variables.
It was only being set on anonymous subs. It should be set for all
clonable subs. It doesnât actually hurt to set it on all types of
subs, whether clonable or not, since it has no effect on non-clon-
able subs.
M pad.c
M t/cmd/lexsub.t
commit 31805d335c1e394ce6dd0dabf6a05a01172b4eca
Author: Father Chrysostomos <[email protected]>
Date: Sun Aug 12 17:57:35 2012 -0700
Fix up outside pointers for my subs
I had not yet fixed Perl_pad_fixup_inner_anons to account for the
fact that my sub prototype CVs are stored in magic attached to
the SV slot in the pad, rather than directly in the pad. It also
did not like & entries that close over subs defined in outer
or inner subs (âmy sub foo; sub bar; sub bar { &foo } }â and
âsub bar; sub bar { my sub foo; sub { sub foo { } } }â respectively).
This was resulting in assertion failures, unsurprisingly.
Some of the tests I added, which were causing assertion failures, are
now failing for other reasons, and are marked as to-do.
M pad.c
M t/cmd/lexsub.t
commit ac7375dea0382b5e69d84f1883f664b550291245
Author: Father Chrysostomos <[email protected]>
Date: Fri Aug 3 18:01:06 2012 -0700
perly.y: Remove MYSUB
This token is not used any more.
M perly.act
M perly.h
M perly.tab
M perly.y
M toke.c
commit 8a42704a9cd368cf274b51543f13b9de470b36cd
Author: Father Chrysostomos <[email protected]>
Date: Fri Aug 3 12:41:11 2012 -0700
CvNAME_HEK_set
M cv.h
M op.c
M pad.c
M scope.c
commit a422885d783f515675168a730bfd19bd292837c8
Author: Father Chrysostomos <[email protected]>
Date: Fri Aug 3 09:23:15 2012 -0700
Clone my subs on scope entry
The pad slot for a my sub now holds a stub with a prototype CV
attached to it by proto magic.
The prototype is cloned on scope entry. The stub in the pad is used
when cloning, so any code that references the sub before scope entry
will be able to see that stub become defined, making these behave
similarly:
our $x;
BEGIN { $x = \&foo }
sub foo { }
our $x;
my sub foo { }
BEGIN { $x = \&foo }
Constants are currently not cloned, but that may cause bugs in
pad_push. Iâll have to look into that.
On scope exit, lexical CVs go through leave_scopeâs SAVEt_CLEARSV sec-
tion, like lexical variables. If the sub is referenced elsewhere, it
is abandoned, and its proto magic is stolen and attached to a new stub
stored in the pad. If the sub is not referenced elsewhere, it is
undefined via cv_undef.
To clone my subs on scope entry, we create a sequence of introcv and
clonecv ops. See the huge comment in block_end that explains why we
need two separate ops for each CV.
To allow my subs to be defined in inner subs (my sub foo; sub { sub
foo {} }), pad_add_name_pvn and S_pad_findlex now upgrade the entry
for a my sub to a CV to begin with, so that fake entries added to pads
(fake entries are those that reference outer pads) can share the same
CV. Otherwise newMYSUB would have to add the CV to every pad that
closes over the âmy subâ declaration. newMYSUB no longer throws away
the initial value replacing it with a new one.
Prototypes are not currently visible to sub calls at compile time,
because the lexer sees the empty stub. A future commit will
solve that.
When I added name heks to CVâs I made mistakes in a few places, by not
turning on the CVf_NAMED flag, or by not clearing the field when free-
ing the hek. Those code paths were not exercised enough by state
subs, so the problems did not show up till now. So this commit fixes
those, too.
One of the tests in lexsub.t, involving foreach loops, was incorrect,
and has been fixed. Another test has been added to the end for a par-
ticular case of state subs closing over my subs that I broke when ini-
tially trying to get sibling my subs to close over each other, before
I had separate introcv and clonecv ops.
M embed.fnc
M embed.h
M op.c
M pad.c
M perly.act
M perly.h
M perly.tab
M perly.y
M pp.c
M proto.h
M scope.c
M t/cmd/lexsub.t
commit 16a51d5b0f7c37aee1024d4b76b1cb880ac2d863
Author: Father Chrysostomos <[email protected]>
Date: Fri Aug 3 09:29:38 2012 -0700
cv_clone: panic for no pad
cv_clone has serendipitously gained the ability to clone CVs without
pads. It is not clear that we want to add this ability to this API
function, because we would be stuck supporting it, even if we came up
with a better interface. It used to crash or fail an assertion if
there was no pad.
M pad.c
commit 243ee23936a7e5b7ad1682a45705a514eed7ab2d
Author: Father Chrysostomos <[email protected]>
Date: Thu Aug 2 13:45:31 2012 -0700
pad.c: Let S_cv_clone clone stubs
This will be used by cv_clone_into (which does not exist yet) in a
later commit. pp_clonecv will use cv_clone_into.
Teasing out the pad-related and non-pad-related parts of cv_clone
was the easiest way to do this. Now the pad stuff is in a separate
function.
M pad.c
commit a56f2d0cbf3306837374ca3825a0c2a965c72a99
Author: Father Chrysostomos <[email protected]>
Date: Sun Jul 29 18:47:48 2012 -0700
op.c: Remove proto storage optimisation for lex subs
It was already #if 0âd out. This optimisation, copied from package
subs, only makes sense when there is autoloading, which lexical subs
donât do. Hence, lexical stubs will be rare indeed, so having an
optimisation for those just creates more nooks to hide bugs.
M op.c
commit da9a2abdb9a703a14985bb8f28d5ee8b19c68379
Author: Father Chrysostomos <[email protected]>
Date: Thu Aug 2 22:11:08 2012 -0700
Add clonecv op type
This will be used for cloning a âmyâ sub on scope entry.
I was going to use pp_padcv for this, but it would end up having a
top-level if/else.
M ext/Opcode/Opcode.pm
M opcode.h
M opnames.h
M pp.c
M pp_proto.h
M regen/opcode.pl
M regen/opcodes
commit 6ac765005713eba4a9fc8cdf0574bc6a3ee31c4c
Author: Father Chrysostomos <[email protected]>
Date: Thu Jul 26 18:21:02 2012 -0700
Add introcv op type
This will be used for introducing âmyâ subs on scope entry, by turning
off the stale flag.
M ext/Opcode/Opcode.pm
M opcode.h
M opnames.h
M pp.c
M pp_proto.h
M regen/opcode.pl
M regen/opcodes
commit f4b5f80bd9c9e4a196f82d1e06d294e95ac3e185
Author: Father Chrysostomos <[email protected]>
Date: Thu Jul 26 12:38:14 2012 -0700
Let state sub fwd decls and nested subs work in anons
I had this working:
state sub foo;
sub other {
sub foo { # defines the state sub declared outside
...
}
}
But it failed inside an anonymous subroutine:
sub {
state sub foo;
sub other {
sub foo { # defines the state sub declared outside
...
}
}
}
When an anonymous (or otherwise clonable) sub is cloned, any state
vars, and, likewise, any state subs, inside it are cloned, too.
In the first example above the state sub forward declaration creates
a subroutine stub. The âotherâ subâs âsub fooâ declaration
creates a
pad entry in otherâs pad that closes over the outer foo immediately,
so the same stub is visible in two pads. The sub foo {} declaration
uses that stub.
When the outer sub containing the forward declaration is clonable,
the pad entry is not closed over immediately at compile time, because
the pad entry is just a prototype, not the actual value that will be
shared by the clone and its nested subs. So the inner pad entry does
not contain the sub.
So the actual creation of the sub, if it only looks at the inner
pad (otherâs pad), will not see the stub, and will not attach a
body to it.
This was the result:
$ ./miniperl -e 'CORE::state sub foo; CORE::state sub bar { sub foo {warn
called} }; foo()'
called at -e line 1.
$ ./miniperl -e 'sub { CORE::state sub foo; CORE::state sub bar { sub foo
{warn called} }; foo() }->()'
Undefined subroutine &foo called at -e line 1.
This commit fixes that by having newMYSUB follow the CvOUTSIDE chain
to find the original pad entry where it defines the sub, if the for-
ward declaration is occurs outside and has not been closed over yet.
M op.c
M t/cmd/lexsub.t
commit 726a6f7ae91b1a1c986af30d5578ca485b1089aa
Author: Father Chrysostomos <[email protected]>
Date: Thu Jul 12 23:31:52 2012 -0700
Add proto magic type
This will be used for storing the prototype CV of a âmyâ sub. The
clone needs to occupy the pad entry so that padcv ops will be able to
find it. That means the clone has to displace its prototype. In case
the same sub is called recursively, we still need to be able to access
the prototype.
M mg_names.c
M mg_raw.h
M mg_vtable.h
M pod/perlguts.pod
M regen/mg_vtable.pl
commit 15ee9defa2bb72dd51aea95aa28c59835b3c532b
Author: Father Chrysostomos <[email protected]>
Date: Tue Jul 10 20:18:48 2012 -0700
First stab at my sub
This does just enough to get things to compile.
They currently do weird things in edge cases, including âBizarre
copy of CODEâ.
âmy subâ now produces a SUB token, and goes through the same grammar
rule as âstate subâ and just plain âsubâ. The separate MYSUB branch
of the barestmt rule will go soon, as it is now unused.
M op.c
M t/cmd/lexsub.t
M t/lib/croak/op
M toke.c
commit a010ee6415c1db4bef418d2c12dd0835d897d2c9
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 9 22:25:24 2012 -0700
op.c:newMYSUB: Pop scope after creating sub
I was popping the scope before creating the sub in order to expose the
parent pad, where the new sub is to be stored.
That can cause problems, since ops may still be created that get
attached to the new sub. Those ops will end up using the parent subâs
slab in that case. If the parent sub does not finish compiling, due
to an error, it may clean out its slab, freeing ops that the inner sub
is using, so the inner sub, when freed, will try to free ops that are
no longer in allocated memory, as the slab is gone. Most of the time,
the inner ops wonât have been reused for anything, so the op type will
still be OP_FREED, and op_free will do nothing (except a single bad
read). But debugging builds detect that and fail an assertion.
Popping the scope afterwards actually does simplify things, surpris-
ingly enough.
I was able to produce this bug with a one-liner, but it did not fail
as part of the test suite. So this fix includes no test.
Since the o variable in newMYSUB is a padop, it can only be freed when
its pad is active. It is created before the sub, so it cannot be
freed until the scope has been popped, so it has to go at the bot-
tom. If an error occurs during newMYSUB, opslab_force_free will take
care of it.
M op.c
commit 57fad87d4860683ac31ae68a42ac4c51d63ebe73
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 9 18:02:33 2012 -0700
dump.c: Dump CvNAME_HEK
M dump.c
commit 5b1d3a2cc9d7da25c67aaddf51a2841aed8478ac
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 9 13:00:28 2012 -0700
Remove & from redef warnings for lex subs
This is just for consistency with package subs.
M op.c
M t/cmd/lexsub.t
commit 0d876c25e910f10852674f65f8651322a4c3f2e6
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 9 12:52:48 2012 -0700
lexsub.t: Fix another test
The problem with writing to-do tests is that it is very easy to get
the tests wrong, such that they continue to fail even when the prob-
lems they test for are fixed.
M t/cmd/lexsub.t
commit 51fd7242e0e6713ab09a7c2eacee1a0f997c9328
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 9 06:29:09 2012 -0700
Clone state subs in anon subs
Since state variables are not shared between closures, but only
between invocations of the same closure, state subs should behave
the same way.
This was a little tricky. When we clone a sub, we now clone inner
state subs at the same time. When walking through the pad, cloning
items, we cannot simply clone the inner sub when we see it, because it
may close over things we havenât cloned yet:
sub {
state sub foo;
my $x
sub foo { $x }
}
We canât just delay cloning it and do it afterwards, because they may
be multiple subs closing over each other:
sub {
state sub foo;
state sub bar;
sub foo { \&bar }
sub bar { \&foo }
}
So *all* the entries in the new pad must be filled before any inner
subs can be cloned.
So what we do is put a stub in place of the cloned sub. And then
in a second pass clone the inner subs, reusing the stubs from the
first pass.
M pad.c
M perly.act
M perly.h
M perly.tab
M perly.y
M t/cmd/lexsub.t
commit 79c15cd414532019a9b07861e53386c75228eb9b
Author: Father Chrysostomos <[email protected]>
Date: Sun Jul 8 14:51:10 2012 -0700
perldiag: closure referents â closure references
This goes back to 2ba9eb46.
M pod/perldiag.pod
commit 3284102cbb28da7e59446fe760d99791f927f7a2
Author: Father Chrysostomos <[email protected]>
Date: Sun Jul 8 14:42:39 2012 -0700
Donât say âvariable &fooâ in warnings
It should be âsubroutine &fooâ. (It could be âsubroutine fooâ, but
we
use both forms elsewhere, and &foo is the easier to implement, the &
already being contained in the pad name.)
M pad.c
M pod/perldiag.pod
M t/cmd/lexsub.t
commit ffdf8036b5afc5733247adb899e2f0d65cfe6357
Author: Father Chrysostomos <[email protected]>
Date: Sun Jul 8 14:28:22 2012 -0700
lexsub.t: Fix some tests
I got this working a few commits ago, but the tests mentioned the
wrong sub name.
M t/cmd/lexsub.t
commit ce6fb864377875bb57595ab4f6fde9a746a914d6
Author: Father Chrysostomos <[email protected]>
Date: Sun Jul 8 14:18:43 2012 -0700
Make pad_fixup_inner_anons cope with closed-over subs
When a sub starts being parsed, a new CV is created. When it fin-
ishes, it is stored in its final location. If there is a stub there
already, the pad is copied to the stub and the body attached thereto.
Since there may be closures inside the sub whose CvOUTSIDE
pointers point to the temporary CV used during compilation,
pad_fixup_inner_anons is called, to reassign all those
CvOUTSIDE pointers.
This happens in cases like this:
sub f;
sub f { sub { } }
When a sub closes over a lexical item in an outer sub, the inner sub
gets its own pad entry with the same value as the outer pad entry.
This means that, now that we have lexical subs (currently just state
subs), we can end up with a pad entry (&s) holding a sub whose
CvOUTSIDE does not point to the sub (f) that owns the pad:
state sub s { }
sub f { s() }
If the f sub has to reuse a stub, then pad_fixup_inner_anons gets to
see that, and complains bitterly:
$ ./perl -Ilib -E 'state sub s; sub f; sub f { s() }'
Assertion failed: (CvOUTSIDE(innercv) == old_cv), function
Perl_pad_fixup_inner_anons, file pad.c, line 2095.
Abort trap
M pad.c
M t/cmd/lexsub.t
commit c17b10d7842e97705307d44b816d45778ab1840d
Author: Father Chrysostomos <[email protected]>
Date: Sat Jul 7 23:46:52 2012 -0700
âUndefined subroutine &foo calledâ for lex subs
instead of just âUndefined subroutine calledâ without the name.
M pp_hot.c
M t/cmd/lexsub.t
commit 90b6633a5fc87647fa0bb20888e9ea96dde9c14d
Author: Father Chrysostomos <[email protected]>
Date: Sat Jul 7 23:11:23 2012 -0700
op.c:newMYSUB: inline var used only once
as of the previous commit
M op.c
commit 8146f0374dbdc9552cea79137715ec96a930b6ba
Author: Father Chrysostomos <[email protected]>
Date: Sat Jul 7 23:07:55 2012 -0700
Lexical stubs should not AUTOLOAD
There is a feature that allows stubs to fall back to their GVsâ
CVs when called. If I reference a stub, e.g., \&bar, and then
bar is autoloaded, the AUTOLOAD sub assigning *bar = *foo or
*bar = sub {...}, I can still call the stub to which I have a refer-
ence, and it will fall back to the overloaded sub.
That is all fine and dandy, but it causes any stub that references a
GV via its CvGV pointer to call that GVâs CV. If we name a lexical
sub by pointing its CvGV pointer at the GV whose name we want it to
have, then the lexical sub, if undefined, will try to fall back to an
autoloaded sub.
That causes things to gang agley in cases like this:
use 5.01;
sub foo { } # package sub
state sub foo;
foo(); # calls lexical sub; falls back to package sub
While we could fix this by flagging the sub and checking for the flag
in pp_entersub (as we do with anonymous subs), it is better simply to
use a HEK, instead of a GV. Since a GV is quite heavyweight for stor-
ing just a name, I was going to do that anyway, eventually. Doing it
now fixes a bug.
M op.c
commit a0be14c81d6e0183f55983052b6b982c616cf0b2
Author: Father Chrysostomos <[email protected]>
Date: Sat Jul 7 17:35:10 2012 -0700
Allow CVs to point to HEKs rather than GVs
This will allow named lexical subs to exist independent of GVs.
M cv.h
M ext/B/B.xs
M gv.c
M pad.c
M pp.c
M sv.c
M sv.h
commit a7a10ab9234321aa2fafafb3e80ae745b794d09e
Author: Father Chrysostomos <[email protected]>
Date: Sat Jul 7 12:18:49 2012 -0700
Implement padcv
State subs can now be referenced and called. Most of the tests in
lexsub.t are now passing. I noticed mistakes in a couple of the
tests and corrected them. In doing so I got an assertion failure
during compilation, so the tests in question I wrapped in a skipped
string eval.
State subs are now mostly working, but there are a few things to
clean up still.
M op.c
M pp.c
M t/cmd/lexsub.t
commit 91ef309ca5703ba3d1641f4df7ea2d2f643c5d41
Author: Father Chrysostomos <[email protected]>
Date: Thu Jul 5 23:28:43 2012 -0700
Test state subs
Most of these tests are still to-do. The previous commit got every-
thing compiling at least. Then I went through putting eval{} around
all the dying tests and marking the failing tests as to-do.
At least this way I donât have to do everything at once (even though
that was how I wrote the tests).
About the only thing that works is constant inlining, of all things.
M t/cmd/lexsub.t
commit b5995b1cdda5c5cc2c7004167a20ce6f8b035721
Author: Father Chrysostomos <[email protected]>
Date: Fri Jul 6 23:35:15 2012 -0700
Look up state subs in the pad
This commit does just enough to get things compiling. The padcv op
is still unimplemented (in fact, converting the padany to a padcv is
still not done), so you canât actually run the code yet.
Bareword lookup in yylex now produces PRIVATEREF tokens for state
subs, so the grammar has been adjusted to accept a âsubnameâ in sub
calls (PRIVATEREF or WORD) where previously only a WORD was permitted.
M perly.act
M perly.h
M perly.tab
M perly.y
M toke.c
commit 4137dc9a0ce52cfc1fc994d73d38ffa875e8f6f3
Author: Father Chrysostomos <[email protected]>
Date: Fri Jul 6 14:31:31 2012 -0700
op.c:newMYSUB: disable stub optimisation
It will be a lot easier to get things working without this, for now.
It can be reënabled later. It might not be worth it, though, as
AUTOLOADing will ignore lexical subs, and this optimisation is mainly
for AUTOLOAD stubs that are rarely used.
M op.c
commit 0ff54ed177b07cf3c01b85b0f7f08a8650602b68
Author: Father Chrysostomos <[email protected]>
Date: Thu Jul 5 23:22:21 2012 -0700
Store state subs in the pad
In making âsub fooâ respect previous âour subâ declarations in a
recent commit, I actually made âstate sub fooâ into a syntax error.
(At the time, I patched up MYSUB in perly.y to keep the tests for â"my
sub" not yet implementedâ still working.) Basically, it was creat-
ing an empty pad entry, but returning something that perly.y was not
expecting.
This commit adjusts the grammar to allow the SUB branch of barestmt to
accept a PRIVATEREF for its subname, in addition to a WORD. It reuses
the subname rule that SUB used to use (before our subs were added),
gutting it to remove the special block handling, which SUB now tokes
care of. That means the MYSUB rule will no longer turn on CvSPECIAL
on the PL_compcv that is going to be thrown away anyway.
The code for special blocks (BEGIN, END, etc.) that turns on CvSPECIAL
now checks for state subs and skips those. It only applies to our
subs and package subs.
newMYSUB has now actually been written. It basically duplicates
newATTRSUB, except for GV-specific things. It does currently vivify a
GV and set CvGV, but I am hoping to change that later. I also hope to
merge some of the code later, too.
I changed the prototype of newMYSUB to make it easier to use. It is
not used anywhere on CPAN and has always simply died, so that should
be all right.
M embed.fnc
M embed.h
M op.c
M perly.act
M perly.h
M perly.tab
M perly.y
M proto.h
commit 94c1da126cbd5bba8cd2f3a7bd267658f85b935c
Author: Father Chrysostomos <[email protected]>
Date: Thu Jul 5 10:41:05 2012 -0700
lexsub.t: Add test name, test override from another pkg
The bareword logic in toke.c looks up GVs in various places. This
tests that we are bypassing those correctly.
M t/cmd/lexsub.t
commit ed1ceb5bbd9093f86c8b35d8e34bdb3d7fa3fc88
Author: Father Chrysostomos <[email protected]>
Date: Wed Jul 4 23:18:32 2012 -0700
Let barewords look up our subs
These take precedence over built-in keywords (just as my $AUTOLOAD
shadows the package var), but not the keyword plugin, as the latter
takes precedence over labels, and these donât.
M t/cmd/lexsub.t
M toke.c
commit 56e168315961440f9bd040fc9a6b32e128fa289b
Author: Father Chrysostomos <[email protected]>
Date: Wed Jul 4 14:09:46 2012 -0700
toke.c:yylex:KEY_sub can use PL_tokenbuf to begin with
There is no need to allocate a separate âtmpbufâ and then copy it into
PL_tokenbuf afterwards.
M toke.c
commit 5d0ed496948edc205a5ae83af962ce04a36d8618
Author: Father Chrysostomos <[email protected]>
Date: Wed Jul 4 09:13:17 2012 -0700
Make âsub foo{}â respect âour fooâ
This commit switches all sub definitions, whether with âourâ or not,
to using S_force_ident_maybe_lex (formerly known as S_pending_ident).
This means that an unqualified (no our/my/state or package prefix)
âsub fooâ declaration does a pad lookup, just like $foo.
It turns out that the vivification that I added to the then
S_pending_ident for CVs was unnecessary and actually buggy. We
*donât* want to autovivify GVs for CVs, because they might be con-
stants or forward declarations, which are stored in a simpler form.
I also had to change the subname rule used by MYSUB in perly.y, since
it can now be fed a PRIVATEREF, which it does not expect. This may
prove to be temporary, but it keeps current tests passing.
M perly.act
M perly.h
M perly.tab
M perly.y
M t/cmd/lexsub.t
M toke.c
commit 2ce90714e2874b79f7ec0985ba66197030724432
Author: Father Chrysostomos <[email protected]>
Date: Wed Jul 4 00:17:55 2012 -0700
Fix our sub with proto
yylex must emit exactly one token each time it is called. Some-
times yylex needs to parse several tokens at once. Thatâs what
the various force functions are for. But that is also what
PL_pending_ident is for.
The various force_next, force_word, force_ident, etc., functions keep
a stack of tokens (PL_nextval/PL_nexttype) that yylex will check imme-
diately when called.
PL_pending_ident is used to track a single identifier that yylex will
hand off to S_pending_ident to handle.
S_pending_ident is the only piece of code for resolving an identi-
fier that could be lexical but could also be a package variable.
force_ident assumes it is looking for a package variable.
force_* takes precedence over PL_pending_ident.
All this means that, if an identifier needs to be looked up in the pad
on the next yylex invocation, it has to use PL_pending_ident, and the
force_* functions cannot be used at the same time.
Not realising that, when I made âour sub fooâ store the sub in the
pad I also made âour sub foo ($)â into a syntax error, because it
was being parsed as âour sub ($) fooâ (the prototype being
âforcedâ);
i.e., the pending tokens were being pulled out of the âqueueâ in the
wrong order. (I put queue in quotes, because one queue and one unre-
lated buffer together donât exactly count as âa queueâ.)
Changing PL_pending_ident to have precedence over the force stack
breaks ext/XS-APItest/t/swaptwostmts.t, because the statement-parsing
interface does not localise PL_pending_ident. It could be changed to
do that, but I donât think it is the right solution.
Having two separate pending token mechanisms makes things need-
lessly fragile.
This commit eliminates the PL_pending_ident mechanism and
modifies S_pending_ident (renaming it in the process to
S_force_ident_maybe_lex) to work with the force mechanism. I was
going to merge it with force_ident, but the two make incompatible
assumptions that just complicate the code if merged. S_pending_ident
needs the sigil in the same string buffer, to pass to the pad inter-
face. force_ident needs to be able to work without a sigil present.
So now we only have one queue for pending tokens and the order is more
predictable.
M embed.fnc
M parser.h
M proto.h
M sv.c
M t/cmd/lexsub.t
M toke.c
commit 69d7f3de0ff45f75eacbb5db14d6a07430a83d00
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 2 21:26:13 2012 -0700
Make do sub() respect our declarations
M t/cmd/lexsub.t
M toke.c
commit 14e93a983d389937e0a2ea8473dba3eee96977de
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 2 12:29:48 2012 -0700
lexsub.t: Fix a test
This is not testing what I meant it to test: that âsub dâ will respect
a preceding âour sub d;â. If âsub dâ is in the same package, it
makes
no difference, so the test tests nothing.
It turns out this does not work yet.
M t/cmd/lexsub.t
commit 725dbe4eab4f39901d3f6a4be207b0c69c5b4294
Author: Father Chrysostomos <[email protected]>
Date: Mon Jul 2 09:07:31 2012 -0700
Use test.pl in lexsub.t
I thought cmd/ couldnât use test.pl, but was mistaken.
M t/cmd/lexsub.t
commit 48ce38d13fd615601d435d2ccbad4c28266fd6b2
Author: Father Chrysostomos <[email protected]>
Date: Sun Jul 1 22:53:41 2012 -0700
Make &foo respect our sub
This changes &foo to go through S_pending_ident (by setting
PL_pending_ident, which causes yylex to defer to S_pending_ident for
the next token) the way $foo and %foo do.
This necessitated reducing the maximum identifier length of &foo from
252 to 251, making it match @foo, $foo, etc. So somebodyâs JAPH might
break. :-)
M MANIFEST
A t/cmd/lexsub.t
M t/comp/parser.t
M toke.c
commit 6bef218b62c9160c6b4224d51b2ba3c16bb500bc
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 30 23:20:25 2012 -0700
Allocate âour subâ in the pad
Currently the name is only allocated there. Nothing fetches it yet.
Notes on the implementation:
S_pending_ident contains the logic for determining whether $foo or
@foo refers to a lexical or package variable.
yylex defers to S_pending_ident if PL_pending_ident is set.
The KEY_sub case in yylex is changed to set PL_pending_ident instead
of using force_word. For package variables (including our),
S_pending_ident returns a WORD token, which is the same thing that
force_word produces. So *that* aspect of this change does not affect
the grammar. However....
The barestmt ruleâs SUB branch begins with âSUB startsub subnameâ.
startsub is a null rule that creates a new sub in PL_compcv via
start_subparse(). subname is defined in terms of WORD and also checks
whether this is a special block, turning on CvSPECIAL(PL_compcv) if
it is. That flag has to be visible during compilation of the sub.
But for a lexical name, such as âour fooâ, to be allocated in the
right pad, it has to come *before* startsub, i.e., âSUB subname
startsubâ.
But subname needs to modify the sub that startsub created, set-
ting the flag.
So I copied (not moved, because MYSUB still uses it) the name-checking
code from the subname rule into the SUB branch of barestmt. Now that
uses WORD directly instead of invoking subname. That allows the code
there to set everything up in the right order.
M perly.act
M perly.h
M perly.tab
M perly.y
M toke.c
commit 0f87f0bd8093cee7b2dcd488b2a638661f65aacf
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 30 23:00:11 2012 -0700
Add padcv to Opcode.pm
M ext/Opcode/Opcode.pm
commit c2837f2b837d7e5c275a691d8c16fef000cc86b7
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 30 22:29:28 2012 -0700
padcv op type
M opcode.h
M opnames.h
M pp.c
M pp_proto.h
M regen/opcodes
commit 3eae315febf7fbcccbf667405550c6f479661ec1
Author: Father Chrysostomos <[email protected]>
Date: Sat Jun 30 17:31:32 2012 -0700
Donât allow name after our/state sub
It was a mistake that this was ever allowed.
M pod/perldiag.pod
M t/lib/croak/toke
M toke.c
-----------------------------------------------------------------------
--
Perl5 Master Repository
