Re: [racket-users] Towards an Incremental Racket Parser for better IDE experience?

2020-12-14 Thread Hendrik Boom
On Mon, Dec 14, 2020 at 04:43:24PM -0700, William G Hatch wrote:
> On Wed, Dec 02, 2020 at 01:56:41PM -0500, Sam Tobin-Hochstadt wrote:
> > A few thoughts on these topics, which I've been thinking about for a while.
> > 
> > First, let's distinguish two things. One is an _incremental_ system,
> > such as a parser, which is one which does less work in response to a
> > small change than it would need to do from scratch. The other is a
> > system with _error recovery_, which is one where in the presence of
> > one error, the system can still provide a useful answer and/or
> > continue on to discover other errors. tree-sitter, for example, aims
> > to do both of these, but they're quite different.
> 
> 
> I thought I might as well chime in here that I'm currently working on
> an incremental parsing system for Racket that will support parser
> combinators, BNF, and arbitrary ad-hoc parsing procedures.  I already
> have a parsing system[1] that supports expressive parsing with any mix
> of those constructors that supports all context-free grammars and
> beyond, so that much is already “successful”.  Unfortunately, I
> haven't been able to make it as performant as I would like.  The
> original goal wasn't to support incremental parsing, and I didn't
> write incremental support into that implementation.  However, while
> trying to optimize it I also decided to figure out how to make it
> incremental, so now I have the solution for that too.
> 
> I'm now working on another similar parsing system that is a little
> less expressive (in particular it has more limited support for
> ambiguous grammars, and thus doesn't support the full class of
> context-free grammars, though it will still support useful things
> outside of CFG, such as non-CFG things that Racket and Scribble
> parsers do), but should be much faster (because it gets to avoid extra
> work supporting the possibility of ambiguity).  I'm adding my solution
> to incremental parsing support to this version.  If I can ever make
> the original algorithm faster I'll add incremental support to it too.
> Both systems leverage delimited continuations at their core -- it
> turns out they are really useful for parsing, both for improving
> expressiveness and for incrementality.
> 
> That said, I'm not currently aiming to solve the error recovery part
> of the problem.

Error recovery on reparsing??  Just use the parse you had from the
last good parse, but mark the parse tree where it doesn't match
the parsed string.

-- hendrik

> 
> [1] https://github.com/willghatch/racket-chido-parse
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to racket-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/racket-users/20201214234324.GA13213%40conspirator.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/20201215043507.fy6qf3boykhfcatp%40topoi.pooq.com.


Re: [racket-users] Towards an Incremental Racket Parser for better IDE experience?

2020-12-14 Thread William G Hatch

On Wed, Dec 02, 2020 at 01:56:41PM -0500, Sam Tobin-Hochstadt wrote:

A few thoughts on these topics, which I've been thinking about for a while.

First, let's distinguish two things. One is an _incremental_ system,
such as a parser, which is one which does less work in response to a
small change than it would need to do from scratch. The other is a
system with _error recovery_, which is one where in the presence of
one error, the system can still provide a useful answer and/or
continue on to discover other errors. tree-sitter, for example, aims
to do both of these, but they're quite different.



I thought I might as well chime in here that I'm currently working on
an incremental parsing system for Racket that will support parser
combinators, BNF, and arbitrary ad-hoc parsing procedures.  I already
have a parsing system[1] that supports expressive parsing with any mix
of those constructors that supports all context-free grammars and
beyond, so that much is already “successful”.  Unfortunately, I
haven't been able to make it as performant as I would like.  The
original goal wasn't to support incremental parsing, and I didn't
write incremental support into that implementation.  However, while
trying to optimize it I also decided to figure out how to make it
incremental, so now I have the solution for that too.

I'm now working on another similar parsing system that is a little
less expressive (in particular it has more limited support for
ambiguous grammars, and thus doesn't support the full class of
context-free grammars, though it will still support useful things
outside of CFG, such as non-CFG things that Racket and Scribble
parsers do), but should be much faster (because it gets to avoid extra
work supporting the possibility of ambiguity).  I'm adding my solution
to incremental parsing support to this version.  If I can ever make
the original algorithm faster I'll add incremental support to it too.
Both systems leverage delimited continuations at their core -- it
turns out they are really useful for parsing, both for improving
expressiveness and for incrementality.

That said, I'm not currently aiming to solve the error recovery part
of the problem.

[1] https://github.com/willghatch/racket-chido-parse

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/racket-users/20201214234324.GA13213%40conspirator.