In perl.git, the branch sprout/lexsub has been created

<http://perl5.git.perl.org/perl.git/commitdiff/03ae5dc9bae4a75570928ab2787efd4bd9c4684a?hp=0000000000000000000000000000000000000000>

        at  03ae5dc9bae4a75570928ab2787efd4bd9c4684a (commit)

- Log -----------------------------------------------------------------
commit 03ae5dc9bae4a75570928ab2787efd4bd9c4684a
Author: Father Chrysostomos <[email protected]>
Date:   Fri Aug 3 14:40:15 2012 -0700

    lexsub.t: Fix a test
    
    It was reusing an existing sub name resulting in a warning, and the
    test was for no warnings.

M       t/cmd/lexsub.t

commit 412d2ef53b8fe2a48f2c7b8cd34d78f9d1e8b0c6
Author: Father Chrysostomos <[email protected]>
Date:   Fri Aug 3 12:41:11 2012 -0700

    CvNAME_HEK_set

M       cv.h
M       op.c
M       pad.c
M       scope.c

commit b5ba58e6055e2ca75679ac1a7e4d635149fac73a
Author: Father Chrysostomos <[email protected]>
Date:   Fri Aug 3 09:23:15 2012 -0700

    Clone my subs on scope entry
    
    The pad slot for a my sub now holds a stub with a prototype CV
    attached to it by proto magic.
    
    The prototype is cloned on scope entry.  The stub in the pad is used
    when cloning, so any code that references the sub before scope entry
    will be able to see that stub become defined, making these behave
    similarly:
    
        our $x;
        BEGIN { $x = \&foo }
        sub foo { }
    
        our $x;
        my sub foo { }
        BEGIN { $x = \&foo }
    
    Constants are currently not cloned, but that may cause bugs in
    pad_push.  I’ll have to look into that.
    
    On scope exit, lexical CVs go through leave_scope’s SAVEt_CLEARSV sec-
    tion, like lexical variables.  If the sub is referenced elsewhere, it
    is abandoned, and its proto magic is stolen and attached to a new stub
    stored in the pad.  If the sub is not referenced elsewhere, it is
    undefined via cv_undef.
    
    To clone my subs on scope entry, we create a sequence of introcv and
    clonecv ops.  See the huge comment in block_end that explains why we
    need two separate ops for each CV.
    
    To allow my subs to be defined in inner subs (my sub foo; sub { sub
    foo {} }), pad_add_name_pvn and S_pad_findlex now upgrade the entry
    for a my sub to a CV to begin with, so that fake entries added to pads
    (fake entries are those that reference outer pads) can share the same
    CV.  Otherwise newMYSUB would have to add the CV to every pad that
    closes over the ‘my sub’ declaration.  newMYSUB no longer throws away
    the initial value replacing it with a new one.
    
    Prototypes are not currently visible to sub calls at compile time,
    because the lexer sees the empty stub.  A future commit will
    solve that.
    
    When I added name heks to CV’s I made mistakes in a few places, by not
    turning on the CVf_NAMED flag, or by not clearing the field when free-
    ing the hek.  Those code paths were not exercised enough by state
    subs, so the problems did not show up till now.  So this commit fixes
    those, too.
    
    One of the tests in lexsub.t, involving foreach loops, was incorrect,
    and has been fixed.  Another test has been added to the end for a par-
    ticular case of state subs closing over my subs that I broke when ini-
    tially trying to get sibling my subs to close over each other, before
    I had separate introcv and clonecv ops.

M       embed.fnc
M       embed.h
M       op.c
M       pad.c
M       perly.act
M       perly.h
M       perly.tab
M       perly.y
M       pp.c
M       proto.h
M       scope.c
M       t/cmd/lexsub.t

commit 8ff8f7f8f927bb907dc47abfa37e4ce57ebc27ac
Author: Father Chrysostomos <[email protected]>
Date:   Fri Aug 3 09:29:38 2012 -0700

    cv_clone: panic for no pad
    
    cv_clone has serendipitously gained the ability to clone CVs without
    pads.  It is not clear that we want to add this ability to this API
    function, because we would be stuck supporting it, even if we came up
    with a better interface.  It used to crash or fail an assertion if
    there was no pad.

M       pad.c

commit baafa594f6db07dea1bb2e2e338c176c1de9c4a5
Author: Father Chrysostomos <[email protected]>
Date:   Thu Aug 2 13:45:31 2012 -0700

    pad.c: Let S_cv_clone clone stubs
    
    This will be used by cv_clone_into (which does not exist yet) in a
    later commit.  pp_clonecv will use cv_clone_into.
    
    Teasing out the pad-related and non-pad-related parts of cv_clone
    was the easiest way to do this.  Now the pad stuff is in a separate
    function.

M       pad.c

commit 7d419db3bf36ffe06658b962ba97f8aa87034ed0
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 29 18:47:48 2012 -0700

    op.c: Remove proto storage optimisation for lex subs
    
    It was already #if 0’d out.  This optimisation, copied from package
    subs, only makes sense when there is autoloading, which lexical subs
    don’t do.  Hence, lexical stubs will be rare indeed, so having an
    optimisation for those just creates more nooks to hide bugs.

M       op.c

commit 92c280a618b98c98678a0a66841a63a90a75109b
Author: Father Chrysostomos <[email protected]>
Date:   Thu Aug 2 22:11:08 2012 -0700

    Add clonecv op type
    
    This will be used for cloning a ‘my’ sub on scope entry.
    I was going to use pp_padcv for this, but it would end up having a
    top-level if/else.

M       ext/Opcode/Opcode.pm
M       opcode.h
M       opnames.h
M       pp.c
M       pp_proto.h
M       regen/opcode.pl
M       regen/opcodes

commit b3e413c17be4c9665d7c645c5de1c01ce804585b
Author: Father Chrysostomos <[email protected]>
Date:   Thu Jul 26 18:21:02 2012 -0700

    Add introcv op type
    
    This will be used for introducing ‘my’ subs on scope entry, by turning
    off the stale flag.

M       ext/Opcode/Opcode.pm
M       opcode.h
M       opnames.h
M       pp.c
M       pp_proto.h
M       regen/opcode.pl
M       regen/opcodes

commit 65184031c375719183e7992ed1e30c43ada30e2a
Author: Father Chrysostomos <[email protected]>
Date:   Thu Jul 26 12:38:14 2012 -0700

    Let state sub fwd decls and nested subs work in anons
    
    I had this working:
    
    state sub foo;
    sub other {
        sub foo { # defines the state sub declared outside
            ...
        }
    }
    
    But it failed inside an anonymous subroutine:
    
    sub {
        state sub foo;
        sub other {
            sub foo { # defines the state sub declared outside
                ...
            }
        }
    }
    
    When an anonymous (or otherwise clonable) sub is cloned, any state
    vars, and, likewise, any state subs, inside it are cloned, too.
    
    In the first example above the state sub forward declaration creates
    a subroutine stub.  The ‘other’ sub’s ‘sub foo’ declaration 
creates a
    pad entry in other’s pad that closes over the outer foo immediately,
    so the same stub is visible in two pads.  The sub foo {} declaration
    uses that stub.
    
    When the outer sub containing the forward declaration is clonable,
    the pad entry is not closed over immediately at compile time, because
    the pad entry is just a prototype, not the actual value that will be
    shared by the clone and its nested subs.  So the inner pad entry does
    not contain the sub.
    
    So the actual creation of the sub, if it only looks at the inner
    pad (other’s pad), will not see the stub, and will not attach a
    body to it.
    
    This was the result:
    
    $ ./miniperl -e 'CORE::state sub foo; CORE::state sub bar { sub foo {warn 
called} }; foo()'
    called at -e line 1.
    $ ./miniperl -e 'sub { CORE::state sub foo; CORE::state sub bar { sub foo 
{warn called} }; foo() }->()'
    Undefined subroutine &foo called at -e line 1.
    
    This commit fixes that by having newMYSUB follow the CvOUTSIDE chain
    to find the original pad entry where it defines the sub, if the for-
    ward declaration is occurs outside and has not been closed over yet.

M       op.c
M       t/cmd/lexsub.t

commit e6afa685469024ec513ffa16c8df87c335e31c0f
Author: Father Chrysostomos <[email protected]>
Date:   Thu Jul 12 23:31:52 2012 -0700

    Add proto magic type
    
    This will be used for storing the prototype CV of a ‘my’ sub.  The
    clone needs to occupy the pad entry so that padcv ops will be able to
    find it.  That means the clone has to displace its prototype.  In case
    the same sub is called recursively, we still need to be able to access
    the prototype.

M       mg_names.c
M       mg_raw.h
M       mg_vtable.h
M       pod/perlguts.pod
M       regen/mg_vtable.pl

commit 4f6077b2d44cf887725d380194fbb4fe00df6c27
Author: Father Chrysostomos <[email protected]>
Date:   Tue Jul 10 20:18:48 2012 -0700

    First stab at my sub
    
    This does just enough to get things to compile.
    
    They currently do weird things in edge cases, including ‘Bizarre
    copy of CODE’.
    
    ‘my sub’ now produces a SUB token, and goes through the same grammar
    rule as ‘state sub’ and just plain ‘sub’.  The separate MYSUB branch
    of the barestmt rule will go soon, as it is now unused.

M       op.c
M       t/cmd/lexsub.t
M       t/lib/croak/op
M       toke.c

commit 46ac7d1ca6c499d680b6b8d53ee45e161db5730d
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 9 22:25:24 2012 -0700

    op.c:newMYSUB: Pop scope after creating sub
    
    I was popping the scope before creating the sub in order to expose the
    parent pad, where the new sub is to be stored.
    
    That can cause problems, since ops may still be created that get
    attached to the new sub.  Those ops will end up using the parent sub’s
    slab in that case.  If the parent sub does not finish compiling, due
    to an error, it may clean out its slab, freeing ops that the inner sub
    is using, so the inner sub, when freed, will try to free ops that are
    no longer in allocated memory, as the slab is gone.  Most of the time,
    the inner ops won’t have been reused for anything, so the op type will
    still be OP_FREED, and op_free will do nothing (except a single bad
    read).  But debugging builds detect that and fail an assertion.
    
    Popping the scope afterwards actually does simplify things, surpris-
    ingly enough.
    
    I was able to produce this bug with a one-liner, but it did not fail
    as part of the test suite.  So this fix includes no test.
    
    Since the o variable in newMYSUB is a padop, it can only be freed when
    its pad is active.  It is created before the sub, so it cannot be
    freed until the scope has been popped, so it has to go at the bot-
    tom.  If an error occurs during newMYSUB, opslab_force_free will take
    care of it.

M       op.c

commit f326050a6a5ed8db2d805b5623432fdff304e4c1
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 9 18:02:33 2012 -0700

    dump.c: Dump CvNAME_HEK

M       dump.c

commit b14db08e1778223cad0967bfb1f50c0858486a65
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 9 13:00:28 2012 -0700

    Remove & from redef warnings for lex subs
    
    I started to write this, creating a special SV to hold the name with-
    out the ampersand, but then never used that SV.
    
    This is just for consistency with package subs.
    
    I also made this slightly more efficient when warnings are off.

M       op.c
M       t/cmd/lexsub.t

commit 5cf9b6111c4077079d7276b578cc111f286a2fab
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 9 12:52:48 2012 -0700

    lexsub.t: Fix another test
    
    The problem with writing to-do tests is that it is very easy to get
    the tests wrong, such that they continue to fail even when the prob-
    lems they test for are fixed.

M       t/cmd/lexsub.t

commit 5e3a7a96c4805a6dd74259554151c17a38f32cbb
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 9 06:29:09 2012 -0700

    Clone state subs in anon subs
    
    Since state variables are not shared between closures, but only
    between invocations of the same closure, state subs should behave
    the same way.
    
    This was a little tricky.  When we clone a sub, we now clone inner
    state subs at the same time.  When walking through the pad, cloning
    items, we cannot simply clone the inner sub when we see it, because it
    may close over things we haven’t cloned yet:
    
        sub {
            state sub foo;
            my $x
            sub foo { $x }
        }
    
    We can’t just delay cloning it and do it afterwards, because they may
    be multiple subs closing over each other:
    
        sub {
           state sub foo;
           state sub bar;
           sub foo { \&bar }
           sub bar { \&foo }
        }
    
    So *all* the entries in the new pad must be filled before any inner
    subs can be cloned.
    
    So what we do is put a stub in place of the cloned sub.   And then
    in a second pass clone the inner subs, reusing the stubs from the
    first pass.

M       pad.c
M       perly.act
M       perly.h
M       perly.tab
M       perly.y
M       t/cmd/lexsub.t

commit be5865533a243f089c90cc02c03cab3a98f50ea4
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 8 14:51:10 2012 -0700

    perldiag: closure referents → closure references
    
    This goes back to 2ba9eb46.

M       pod/perldiag.pod

commit 52c657b41aeb1f4f7e3510c058105e4f9fe8c6da
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 8 14:42:39 2012 -0700

    Don’t say ‘variable &foo’ in warnings
    
    It should be ‘subroutine &foo’.  (It could be ‘subroutine foo’, but 
we
    use both forms elsewhere, and &foo is the easier to implement, the &
    already being contained in the pad name.)

M       pad.c
M       pod/perldiag.pod
M       t/cmd/lexsub.t

commit 60bfa5f6f9a69dfb6108b0ddbfccb9d457741fd0
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 8 14:28:22 2012 -0700

    lexsub.t: Fix some tests
    
    I got this working a few commits ago, but the tests mentioned the
    wrong sub name.

M       t/cmd/lexsub.t

commit 12a0ba88502f33390ae79c35aa21b8a34fa2e379
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 8 14:18:43 2012 -0700

    Make pad_fixup_inner_anons cope with closed-over subs
    
    When a sub starts being parsed, a new CV is created.  When it fin-
    ishes, it is stored in its final location.  If there is a stub there
    already, the pad is copied to the stub and the body attached thereto.
    
    Since there may be closures inside the sub whose CvOUTSIDE
    pointers point to the temporary CV used during compilation,
    pad_fixup_inner_anons is called, to reassign all those
    CvOUTSIDE pointers.
    
    This happens in cases like this:
    
        sub f;
        sub f { sub { } }
    
    When a sub closes over a lexical item in an outer sub, the inner sub
    gets its own pad entry with the same value as the outer pad entry.
    
    This means that, now that we have lexical subs (currently just state
    subs), we can end up with a pad entry (&s) holding a sub whose
    CvOUTSIDE does not point to the sub (f) that owns the pad:
    
        state sub s { }
        sub f { s() }
    
    If the f sub has to reuse a stub, then pad_fixup_inner_anons gets to
    see that, and complains bitterly:
    
    $ ./perl -Ilib -E 'state sub s; sub f; sub f { s() }'
    Assertion failed: (CvOUTSIDE(innercv) == old_cv), function 
Perl_pad_fixup_inner_anons, file pad.c, line 2095.
    Abort trap

M       pad.c
M       t/cmd/lexsub.t

commit bccecfb94f4c42e4a4feda79bc89344e9e5cb202
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 23:46:52 2012 -0700

    ‘Undefined subroutine &foo called’ for lex subs
    
    instead of just ‘Undefined subroutine called’ without the name.

M       pp_hot.c
M       t/cmd/lexsub.t

commit 04cd0dd04530b92cb39066a6a7c9c6d5724dddec
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 23:12:20 2012 -0700

    op.c:newMYSUB: Remove unused vars

M       op.c

commit c7f5c1dc8b951f18aa7e41e07b1d20a4b7cbe443
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 23:11:23 2012 -0700

    op.c:newMYSUB: inline var used only once
    
    as of the previous commit

M       op.c

commit 16a48fce28e27eb9d0f756f12e587f0a61e6b182
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 23:07:55 2012 -0700

    Lexical stubs should not AUTOLOAD
    
    There is a feature that allows stubs to fall back to their GVs’
    CVs when called.  If I reference a stub, e.g., \&bar, and then
    bar is autoloaded, the AUTOLOAD sub assigning *bar = *foo or
    *bar = sub {...}, I can still call the stub to which I have a refer-
    ence, and it will fall back to the overloaded sub.
    
    That is all fine and dandy, but it causes any stub that references a
    GV via its CvGV pointer to call that GV’s CV.  If we name a lexical
    sub by pointing its CvGV pointer at the GV whose name we want it to
    have, then the lexical sub, if undefined, will try to fall back to an
    autoloaded sub.
    
    That causes things to gang agley in cases like this:
    
        use 5.01;
        sub foo { } # package sub
        state sub foo;
        foo(); # calls lexical sub; falls back to package sub
    
    While we could fix this by flagging the sub and checking for the flag
    in pp_entersub (as we do with anonymous subs), it is better simply to
    use a HEK, instead of a GV.  Since a GV is quite heavyweight for stor-
    ing just a name, I was going to do that anyway, eventually.  Doing it
    now fixes a bug.

M       op.c

commit 60c7bd7ac4d05815ff248bf446b6f47527866dc4
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 18:22:11 2012 -0700

    Increase $B::VERSION to 1.37

M       ext/B/B.pm

commit 2a0f5435e81301a24f24e69ac4a968e467fb47de
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 17:35:10 2012 -0700

    Allow CVs to point to HEKs rather than GVs
    
    This will allow named lexical subs to exist independent of GVs.

M       cv.h
M       ext/B/B.xs
M       gv.c
M       pad.c
M       pp.c
M       sv.c
M       sv.h

commit bedb46ee62dea6af13d21407b4970569e8e357c9
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jul 7 12:18:49 2012 -0700

    Implement padcv
    
    State subs can now be referenced and called.  Most of the tests in
    lexsub.t are now passing.  I noticed mistakes in a couple of the
    tests and corrected them.  In doing so I got an assertion failure
    during compilation, so the tests in question I wrapped in a skipped
    string eval.
    
    State subs are now mostly working, but there are a few things to
    clean up still.

M       op.c
M       pp.c
M       t/cmd/lexsub.t

commit c78e0d4cc297a41e263f605811a2f3ee56ffa0b7
Author: Father Chrysostomos <[email protected]>
Date:   Thu Jul 5 23:28:43 2012 -0700

    Test state subs
    
    Most of these tests are still to-do.  The previous commit got every-
    thing compiling at least.  Then I went through putting eval{} around
    all the dying tests and marking the failing tests as to-do.
    
    At least this way I don’t have to do everything at once (even though
    that was how I wrote the tests).
    
    About the only thing that works is constant inlining, of all things.

M       t/cmd/lexsub.t

commit e4caa1413f32e80e83b4e3215a9375120432742d
Author: Father Chrysostomos <[email protected]>
Date:   Fri Jul 6 23:35:15 2012 -0700

    Look up state subs in the pad
    
    This commit does just enough to get things compiling.  The padcv op
    is still unimplemented (in fact, converting the padany to a padcv is
    still not done), so you can’t actually run the code yet.
    
    Bareword lookup in yylex now produces PRIVATEREF tokens for state
    subs, so the grammar has been adjusted to accept a ‘subname’ in sub
    calls (PRIVATEREF or WORD) where previously only a WORD was permitted.

M       perly.act
M       perly.h
M       perly.tab
M       perly.y
M       toke.c

commit e165a542ea9bf9511308f7756e47a619bbf0360e
Author: Father Chrysostomos <[email protected]>
Date:   Fri Jul 6 14:31:31 2012 -0700

    op.c:newMYSUB: disable stub optimisation
    
    It will be a lot easier to get things working without this, for now.
    It can be reënabled later.  It might not be worth it, though, as
    AUTOLOADing will ignore lexical subs, and this optimisation is mainly
    for AUTOLOAD stubs that are rarely used.

M       op.c

commit b2c201b7fd8923018500a0c4ef0375b98925b5fd
Author: Father Chrysostomos <[email protected]>
Date:   Thu Jul 5 23:22:21 2012 -0700

    Store state subs in the pad
    
    In making ‘sub foo’ respect previous ‘our sub’ declarations in a
    recent commit, I actually made ‘state sub foo’ into a syntax error.
    (At the time, I patched up MYSUB in perly.y to keep the tests for ‘"my
    sub" not yet implemented’ still working.)  Basically, it was creat-
    ing an empty pad entry, but returning something that perly.y was not
    expecting.
    
    This commit adjusts the grammar to allow the SUB branch of barestmt to
    accept a PRIVATEREF for its subname, in addition to a WORD.  It reuses
    the subname rule that SUB used to use (before our subs were added),
    gutting it to remove the special block handling, which SUB now tokes
    care of.  That means the MYSUB rule will no longer turn on CvSPECIAL
    on the PL_compcv that is going to be thrown away anyway.
    
    The code for special blocks (BEGIN, END, etc.) that turns on CvSPECIAL
    now checks for state subs and skips those.  It only applies to our
    subs and package subs.
    
    newMYSUB has now actually been written.  It basically duplicates
    newATTRSUB, except for GV-specific things.  It does currently vivify a
    GV and set CvGV, but I am hoping to change that later.  I also hope to
    merge some of the code later, too.
    
    I changed the prototype of newMYSUB to make it easier to use.  It is
    not used anywhere on CPAN and has always simply died, so that should
    be all right.

M       embed.fnc
M       embed.h
M       op.c
M       perly.act
M       perly.h
M       perly.tab
M       perly.y
M       proto.h

commit c215ec0e028ac68477f12760614f915c5ad6e2da
Author: Father Chrysostomos <[email protected]>
Date:   Thu Jul 5 10:41:05 2012 -0700

    lexsub.t: Add test name, test override from another pkg
    
    The bareword logic in toke.c looks up GVs in various places.  This
    tests that we are bypassing those correctly.

M       t/cmd/lexsub.t

commit 535614a0bed6e78da66a3586b7f0e9d0fbdceb64
Author: Father Chrysostomos <[email protected]>
Date:   Wed Jul 4 23:18:32 2012 -0700

    Let barewords look up our subs
    
    These take precedence over built-in keywords (just as my $AUTOLOAD
    shadows the package var), but not the keyword plugin, as the latter
    takes precedence over labels, and these don’t.

M       t/cmd/lexsub.t
M       toke.c

commit 0b0d0888352a350575f63576a901aa9b48fa7499
Author: Father Chrysostomos <[email protected]>
Date:   Wed Jul 4 14:09:46 2012 -0700

    toke.c:yylex:KEY_sub can use PL_tokenbuf to begin with
    
    There is no need to allocate a separate ‘tmpbuf’ and then copy it into
    PL_tokenbuf afterwards.

M       toke.c

commit c582081db5359fa8b1aec35ae7750ff75f979124
Author: Father Chrysostomos <[email protected]>
Date:   Wed Jul 4 09:13:17 2012 -0700

    Make ‘sub foo{}’ respect ‘our foo’
    
    This commit switches all sub definitions, whether with ‘our’ or not,
    to using S_force_ident_maybe_lex (formerly known as S_pending_ident).
    
    This means that an unqualified (no our/my/state or package prefix)
    ‘sub foo’ declaration does a pad lookup, just like $foo.
    
    It turns out that the vivification that I added to the then
    S_pending_ident for CVs was unnecessary and actually buggy.  We
    *don’t* want to autovivify GVs for CVs, because they might be con-
    stants or forward declarations, which are stored in a simpler form.
    
    I also had to change the subname rule used by MYSUB in perly.y, since
    it can now be fed a PRIVATEREF, which it does not expect.  This may
    prove to be temporary, but it keeps current tests passing.

M       perly.act
M       perly.h
M       perly.tab
M       perly.y
M       t/cmd/lexsub.t
M       toke.c

commit 41f73551a97148f8dc386abb30d9efd17d4d253b
Author: Father Chrysostomos <[email protected]>
Date:   Wed Jul 4 06:23:16 2012 -0700

    Test initial tick in sub declaration

M       t/comp/parser.t

commit 0b470a6dd2bac9f9433b3234cc020c57bdd3a969
Author: Father Chrysostomos <[email protected]>
Date:   Wed Jul 4 00:17:55 2012 -0700

    Fix our sub with proto
    
    yylex must emit exactly one token each time it is called.  Some-
    times yylex needs to parse several tokens at once.  That’s what
    the various force functions are for.  But that is also what
    PL_pending_ident is for.
    
    The various force_next, force_word, force_ident, etc., functions keep
    a stack of tokens (PL_nextval/PL_nexttype) that yylex will check imme-
    diately when called.
    
    PL_pending_ident is used to track a single identifier that yylex will
    hand off to S_pending_ident to handle.
    
    S_pending_ident is the only piece of code for resolving an identi-
    fier that could be lexical but could also be a package variable.
    force_ident assumes it is looking for a package variable.
    
    force_* takes precedence over PL_pending_ident.
    
    All this means that, if an identifier needs to be looked up in the pad
    on the next yylex invocation, it has to use PL_pending_ident, and the
    force_* functions cannot be used at the same time.
    
    Not realising that, when I made ‘our sub foo’ store the sub in the
    pad I also made ‘our sub foo ($)’ into a syntax error, because it
    was being parsed as ‘our sub ($) foo’ (the prototype being 
‘forced’);
    i.e., the pending tokens were being pulled out of the ‘queue’ in the
    wrong order.  (I put queue in quotes, because one queue and one unre-
    lated buffer together don’t exactly count as ‘a queue’.)
    
    Changing PL_pending_ident to have precedence over the force stack
    breaks ext/XS-APItest/t/swaptwostmts.t, because the statement-parsing
    interface does not localise PL_pending_ident.  It could be changed to
    do that, but I don’t think it is the right solution.
    
    Having two separate pending token mechanisms makes things need-
    lessly fragile.
    
    This commit eliminates the PL_pending_ident mechanism and
    modifies S_pending_ident (renaming it in the process to
    S_force_ident_maybe_lex) to work with the force mechanism.  I was
    going to merge it with force_ident, but the two make incompatible
    assumptions that just complicate the code if merged.  S_pending_ident
    needs the sigil in the same string buffer, to pass to the pad inter-
    face.  force_ident needs to be able to work without a sigil present.
    
    So now we only have one queue for pending tokens and the order is more
    predictable.

M       embed.fnc
M       parser.h
M       proto.h
M       sv.c
M       t/cmd/lexsub.t
M       toke.c

commit 454d86b8d33ec785e87b8fca858f2a076f39abd7
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 2 21:26:13 2012 -0700

    Make do sub() respect our declarations

M       t/cmd/lexsub.t
M       toke.c

commit 53de2e33b46920d1a4ab6af54edeca73b669488c
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 2 18:11:23 2012 -0700

    do-file should not force a bareword
    
    A word following do is forced to be a bareword for do-sub’s sake.  But
    if it is going to be interpreted as do-file after all, that does not
    make sense.  ‘do subname;’ should call the sub and run the file whose
    name it returns, instead of running the file named ‘subname’.

M       t/op/do.t
M       toke.c

commit 6d14859b39155be1c5a2227c29c5f0965e0da183
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 2 14:47:50 2012 -0700

    Let do.t run from the top level

M       t/op/do.t

commit 331075f35d4c48595c0c2ab8aa7282582d009204
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 2 14:47:34 2012 -0700

    do.t: Load test.pl at BEGIN time
    
    so that parentheses can be omitted.

M       t/op/do.t

commit a30a1d1e8ab783e165c8c577a11adb204a10fc69
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 2 12:29:48 2012 -0700

    lexsub.t: Fix a test
    
    This is not testing what I meant it to test: that ‘sub d’ will respect
    a preceding ‘our sub d;’.  If ‘sub d’ is in the same package, it 
makes
    no difference, so the test tests nothing.
    
    It turns out this does not work yet.

M       t/cmd/lexsub.t

commit f021c3bbae65632490fe9c9d2fc28c100dcb4ac8
Author: Father Chrysostomos <[email protected]>
Date:   Mon Jul 2 09:07:31 2012 -0700

    Use test.pl in lexsub.t
    
    I thought cmd/ couldn’t use test.pl, but was mistaken.

M       t/cmd/lexsub.t

commit 71eb32b5717f847ce5ef65351f5044150abd73d4
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 1 23:05:21 2012 -0700

    Allow test_bootstrap.t to run from the top level

M       t/porting/test_bootstrap.t

commit 01114d1db8188f7abf3ab5718c3eafc416298841
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 1 22:53:41 2012 -0700

    Make &foo respect our sub
    
    This changes &foo to go through S_pending_ident (by setting
    PL_pending_ident, which causes yylex to defer to S_pending_ident for
    the next token) the way $foo and %foo do.
    
    This necessitated reducing the maximum identifier length of &foo from
    252 to 251, making it match @foo, $foo, etc.  So somebody’s JAPH might
    break. :-)

M       MANIFEST
A       t/cmd/lexsub.t
M       t/comp/parser.t
M       toke.c

commit be836647134ab29f5d737173501150aa83cda6bf
Author: Father Chrysostomos <[email protected]>
Date:   Sun Jul 1 20:23:06 2012 -0700

    pad.c apidocs: Missing fullstop

M       pad.c

commit deb4ce0dd608992ed8d9a8e34d442f7a233cc897
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jun 30 23:20:25 2012 -0700

    Allocate ‘our sub’ in the pad
    
    Currently the name is only allocated there.  Nothing fetches it yet.
    
    Notes on the implementation:
    
    S_pending_ident contains the logic for determining whether $foo or
    @foo refers to a lexical or package variable.
    
    yylex defers to S_pending_ident if PL_pending_ident is set.
    
    The KEY_sub case in yylex is changed to set PL_pending_ident instead
    of using force_word.  For package variables (including our),
    S_pending_ident returns a WORD token, which is the same thing that
    force_word produces.  So *that* aspect of this change does not affect
    the grammar.  However....
    
    The barestmt rule’s SUB branch begins with ‘SUB startsub subname’.
    startsub is a null rule that creates a new sub in PL_compcv via
    start_subparse().  subname is defined in terms of WORD and also checks
    whether this is a special block, turning on CvSPECIAL(PL_compcv) if
    it is.  That flag has to be visible during compilation of the sub.
    
    But for a lexical name, such as ‘our foo’, to be allocated in the
    right pad, it has to come *before* startsub, i.e., ‘SUB subname
    startsub’.
    
    But subname needs to modify the sub that startsub created, set-
    ting the flag.
    
    So I copied (not moved, because MYSUB still uses it) the name-checking
    code from the subname rule into the SUB branch of barestmt.  Now that
    uses WORD directly instead of invoking subname.  That allows the code
    there to set everything up in the right order.

M       perly.act
M       perly.h
M       perly.tab
M       perly.y
M       toke.c

commit 9749fd948574b5fa59e16fc0fe96f18782bb9ffa
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jun 30 23:00:57 2012 -0700

    Increase $Opcode::VERSION to 1.24

M       ext/Opcode/Opcode.pm

commit 828dfe56f907adef77580d6f4111a862771c15f1
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jun 30 23:00:11 2012 -0700

    Add padcv to Opcode.pm

M       ext/Opcode/Opcode.pm

commit 00c8ff76e98389968f07a3b58da51cc6af78acf9
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jun 30 22:29:28 2012 -0700

    padcv op type

M       opcode.h
M       opnames.h
M       pp.c
M       pp_proto.h
M       regen/opcodes

commit 67df098a2a7a4ca545d4397190a71c6e43f1974f
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jun 30 17:31:32 2012 -0700

    Don’t allow name after our/state sub
    
    It was a mistake that this was ever allowed.

M       MANIFEST
M       pod/perldiag.pod
M       t/lib/croak/toke
M       toke.c

commit 8efad6839285a41e0f211df4dbfea6b3f3e989b4
Author: Father Chrysostomos <[email protected]>
Date:   Sat Jun 30 17:17:39 2012 -0700

    Test ‘Missing name in "my sub"’

M       MANIFEST
A       t/lib/croak/toke
-----------------------------------------------------------------------

--
Perl5 Master Repository

Reply via email to