On 18/01/2014 10:18 AM, Jonathan S. Shapiro wrote:
> The point is that I haven't seen any RE generator (unless it's hiding
> inside Antlr) that has this notion of multiple possible results and
> prioritized disambiguation.
Assuming I understand what you mean, it sounds like I have this in
Sasa.Parsing[1].The lexer is defined by interfaces ILexerBasic and
ILexer. These lexer interfaces are parameterized over ILexerState which
contains all position information about matching lexemes. Lexers that
make use of ILexerBasic.Alternate can have multiple matches, so that
method has the following signature:
public delegate void LexMerge<T>(ref T root, T original, ref T current)
where T : ILexerState
public interface ILexerBasic<T, TLexerState>
where TLexerState : ILexerState
{
...
T Alternate(LexMerge<TLexerState> merge, params T[] lexers)
...
}
So alternate lexes are subject to a customizable merge/choice function.
The library comes with single-match [2], longest-match [3], and
accumulate-match [4] delegates. ILexerState contains all the info about
match locations, and could contain lexer-level priorities if that's
something you need.
I've run into exactly your issue of disambiguating identifiers from
keywords, and the parser contains a simple Disambiguate declaration to
specify an ordering among operators. This is currently managed by an
ordering among parse rules, but given your description and what I
outlined above, I could actually push this into the lexer by specifying
priorities consulted by LexMerge. I'm not sure it would be cleaner though.
Sandro
[1] https://sourceforge.net/p/sasa/code/ci/default/tree/Sasa.Parsing/
[2]
https://sourceforge.net/p/sasa/code/ci/default/tree/Sasa.Parsing/Lexing/Lexers.cs#l41
[3]
https://sourceforge.net/p/sasa/code/ci/default/tree/Sasa.Parsing/Lexing/Lexers.cs#l61
[4]
https://sourceforge.net/p/sasa/code/ci/default/tree/Sasa.Parsing/Lexing/Lexers.cs#l103
_______________________________________________
bitc-dev mailing list
[email protected]
http://www.coyotos.org/mailman/listinfo/bitc-dev
