At 11:18 AM 8/23/00 -0700, Larry Wall wrote:
>Dan Sugalski writes:
>: At 10:35 AM 8/19/00 +1000, Damian Conway wrote:
>: >However, for Perl 6 I'd really like to see run-time access to the
>: >Real Tokenizer (tm):
>: >
>: > use tokenizer;
>: >
>: > my $tree = tokenizer( $sourcecode );
>: >
>: >This would be dead handy for building source-code filtering modules,
>: >rather than relying on the pseudo-parsing tricks we're currently
>: >forced to use.
>:
>: Well, sure. You couldn't write an optimizer, bytecode compiler, or bytecode
>: execution engine in perl if you couldn't do that... :)
>
>I was wondering this morning whether we ought to write the Perl 6
>parser as a set of recursive regexes. Might make it easier to plug in
>new productions on the fly. And designing the parser around regexes
>might indicate ways in which Perl's regexes are not yet powerful
>enough.
Now that's an interesting idea. It might make it easier to extend than the
"build a static dispatch table" that seems to be the way yacc-ish things
go. It'd also give us a good reason to optimize the heck out of some parts
of the regex engine if we want any sort of speed from the thing.
I expect we'd want to have some sort of heavy-duty regex optimizer, then,
to detect common prefixes and subexpressions and suchlike things, otherwise
we end up with a rather monstrous alternation sequence...
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
[EMAIL PROTECTED] have teddy bears and even
teddy bears get drunk
