Re: Backtracking through the source

2000-11-29 Thread Dan Sugalski
At 01:38 PM 11/29/00 +, David Grove wrote: >That's basically where I've been talking about a "creole processor", which >would in these terms be a pre-preprocessor, I imagine. This was also my >original source of confusion, since I thought that this was the primary >goal of the "pre-processor".

Re: Backtracking through the source

2000-11-29 Thread Dan Sugalski
At 07:07 PM 11/29/00 +, Simon Cozens wrote: >On Wed, Nov 29, 2000 at 02:02:31PM -0500, Dan Sugalski wrote: > > I'm really thinking that the lexer, parser, and tokenizer can't be > anywhere > > near as separate as we'd like. I think we're going to end up with a rather > > odd mutant beast. Hop

Re: Backtracking through the source

2000-11-29 Thread David Grove
Simon Cozens <[EMAIL PROTECTED]> wrote: > On Wed, Nov 29, 2000 at 02:02:31PM -0500, Dan Sugalski wrote: > > I'm really thinking that the lexer, parser, and tokenizer can't be > anywhere > > near as separate as we'd like. I think we're going to end up with a > rather > > odd mutant beast. H

Re: Backtracking through the source

2000-11-29 Thread Steve Fink
Dan Sugalski wrote: > > At 04:16 PM 11/28/00 -0800, Steve Fink wrote: > >Perl5 is parseable with a single token of lookahead and lots of > >parser/lexer communication. Sort of. It would be nice to prevent it from > >getting any worse. > > I'm really thinking that the lexer, parser, and tokenizer

Re: Backtracking through the source

2000-11-29 Thread Simon Cozens
On Wed, Nov 29, 2000 at 02:02:31PM -0500, Dan Sugalski wrote: > I'm really thinking that the lexer, parser, and tokenizer can't be anywhere > near as separate as we'd like. I think we're going to end up with a rather > odd mutant beast. Hopefully one that's understandable by reasonably sane > p

Re: Backtracking through the source

2000-11-29 Thread Dan Sugalski
At 04:16 PM 11/28/00 -0800, Steve Fink wrote: >Perl5 is parseable with a single token of lookahead and lots of >parser/lexer communication. Sort of. It would be nice to prevent it from >getting any worse. I'm really thinking that the lexer, parser, and tokenizer can't be anywhere near as separat

Re: The external interface for the parser piece

2000-11-29 Thread Dan Sugalski
At 02:49 PM 11/29/00 +, Nicholas Clark wrote: >On Wed, Nov 29, 2000 at 09:38:44AM -0500, Dan Sugalski wrote: > > At 10:42 AM 11/29/00 +, Nick Ing-Simmons wrote: > > > >FILE * is not a good idea. PerlIO * is fine. > > > > The problem with that is we're potentially getting the filehandle fro

Re: The external interface for the parser piece

2000-11-29 Thread Simon Cozens
On Wed, Nov 29, 2000 at 09:51:07AM -0800, Dave Storrs wrote: > I have a feeling this is a stupid question, but I have to ask anyway. > Do we really need to pass in a PerlInterp pointer? Yes. Threads. There's a reason for all the PERL_EXPLICIT_CONTEXT anxiety. -- Old Japanese proverb: T

Re: Backtracking through the source

2000-11-29 Thread Dan Sugalski
At 09:58 AM 11/29/00 -0800, Steve Fink wrote: >As soon as the lexer sees "s#", it >starts treating # as a delimiter -- it doesn't need to conditionally >treat the # as either a delimiter or a comment. (Especially since >there's nothing following it that could resolve the ambiguity!) One thing I'v

Re: The external interface for the parser piece

2000-11-29 Thread Dan Sugalski
At 09:51 AM 11/29/00 -0800, Dave Storrs wrote: >I have a feeling this is a stupid question, but I have to ask anyway. > >Do we really need to pass in a PerlInterp pointer? Or can perl6_parse >just create one for itself if/when it needs one? If created, it could of >course be kept around so that

Re: The external interface for the parser piece

2000-11-29 Thread Dave Storrs
I have a feeling this is a stupid question, but I have to ask anyway. Do we really need to pass in a PerlInterp pointer? Or can perl6_parse just create one for itself if/when it needs one? If created, it could of course be kept around so that it didn't need to be re-created later.

Re: Backtracking through the source

2000-11-29 Thread Steve Fink
Chaim Frenkel wrote: > > > "SF" == Steve Fink <[EMAIL PROTECTED]> writes: > > SF> Handling the parser's state can be done in a backtracking DFA-like or a > SF> direct NFA-like way. The NFA way is to keep track of all possible parse > SF> states and advance each one in parallel based on the n

Re: The external interface for the parser piece

2000-11-29 Thread Chaim Frenkel
> "NC" == Nicholas Clark <[EMAIL PROTECTED]> writes: NC> But on the other hand I also liked Simon Cozen's argument that it should NC> be easy to embed somewhere else, in which case why do we want to make NC> the parser have a dependency on the IO library? I can't see anyway that the dependen

Re: The external interface for the parser piece

2000-11-29 Thread Nick Ing-Simmons
Tom Hughes <[EMAIL PROTECTED]> writes: >In message <[EMAIL PROTECTED]> >Dan Sugalski <[EMAIL PROTECTED]> wrote: > >> At 10:42 AM 11/29/00 +, Nick Ing-Simmons wrote: >> >> >FILE * is not a good idea. PerlIO * is fine. >> >> The problem with that is we're potentially getting the filehan

Re: Backtracking through the source

2000-11-29 Thread Chaim Frenkel
> "SF" == Steve Fink <[EMAIL PROTECTED]> writes: SF> Handling the parser's state can be done in a backtracking DFA-like or a SF> direct NFA-like way. The NFA way is to keep track of all possible parse SF> states and advance each one in parallel based on the next token. The DFA SF> way is recu

Re: The external interface for the parser piece

2000-11-29 Thread Nicholas Clark
On Wed, Nov 29, 2000 at 09:38:44AM -0500, Dan Sugalski wrote: > At 10:42 AM 11/29/00 +, Nick Ing-Simmons wrote: > >FILE * is not a good idea. PerlIO * is fine. > > The problem with that is we're potentially getting the filehandle from > something that isn't perl. Or so my thinking went at t

Re: The external interface for the parser piece

2000-11-29 Thread Tom Hughes
In message <[EMAIL PROTECTED]> Dan Sugalski <[EMAIL PROTECTED]> wrote: > At 10:42 AM 11/29/00 +, Nick Ing-Simmons wrote: > > >FILE * is not a good idea. PerlIO * is fine. > > The problem with that is we're potentially getting the filehandle from > something that isn't perl. Or so my

Re: The external interface for the parser piece

2000-11-29 Thread Dan Sugalski
At 10:42 AM 11/29/00 +, Nick Ing-Simmons wrote: >Dan Sugalski <[EMAIL PROTECTED]> writes: > >At 07:03 PM 11/28/00 +, Tom Hughes wrote: > >>In message <[EMAIL PROTECTED]> > >> Dan Sugalski <[EMAIL PROTECTED]> wrote: > >> > >> > The third parameter is the flags parameter, and it's

Re: The external interface for the parser piece

2000-11-29 Thread Nick Ing-Simmons
Dan Sugalski <[EMAIL PROTECTED]> writes: >At 07:03 PM 11/28/00 +, Tom Hughes wrote: >>In message <[EMAIL PROTECTED]> >> Dan Sugalski <[EMAIL PROTECTED]> wrote: >> >> > The third parameter is the flags parameter, and it's optional. If omitted >> > or set to PERL_CHAR_SOURCE, the secon