Thanks for the suggestions on this issue.
I'm not concerned about the line count - in fact, I want to know which physical
line a token is located for subsequent regeneration of the source. I'm using
this for a "syntax directed" editor. I just want to absorb the continuations
quietly.
I still can't figure out how to handle the case where continuation characters
(- and +) are embedded in prior to the end of line. A + or - is only a
continuation if the following character is an end of line. If this isn't true,
then the + or - is a valid character in an token.
My lexer rules look like this --
/*
LEXER RULES
*/
ID : Any+
| Quote (Any | Blank)* Quote
;
fragment
Blank : ' '
;
fragment
Any :( AlphaNum | Special | NATL )
;
fragment
Quote : '\''
;
fragment
Special : '_' | '-' | '=' | '+'
| '/' | '\\'
| ':' | ';'
| '<' | '>'
| '.' | ',' | '?' | '!'
| '~' | '%' | '^' | '&' | '*'
| '{' | '}' | '[' | ']' | '|'
;
fragment
AlphaNum: ALPHA|DIGIT;
fragment
DIGIT : ('0'..'9');
fragment
ALPHA
: ('a'..'z'|'A'..'Z')
;
fragment
NATL : ( '$' | '#' | '@')
;
EOS :
( '\r'
| '\n'
)+
;
CONTINUEMINUS
: '-\r'
| '-\n'
| '-\r\n'
{ $channel=HIDDEN; }
;
CONTINUEPLUS
: '+\r'
| '+\n'
| '+\r\n'
{ $channel=HIDDEN; }
;
WS :
( ' '
| '\t'
)+
{ $channel=HIDDEN; }
;
COMMENT
: '/*' (options {greedy=false;} : . )* '*/'
{ $channel=HIDDEN; }
;
I have a problem when I have a statement like the following --
Cmd parm1 parm2 verylong-
parm
The - at the end of the verylongparm is absorbed as part of the ID token.
The above works OK if there's WS between the last token and the -, but that't
not the syntax I have to conform to.
Thanks for any additional feedback.
-----Original Message-----
From: Johannes Luber [mailto:[email protected]]
Sent: Tuesday, January 13, 2009 9:13 AM
To: Gavin Lambert; Brisard, Fred D; [email protected]
Subject: Re: [antlr-interest] Handling explicit continuation characters
> At 21:05 13/01/2009, Johannes Luber wrote:
> >Wouldn't it be easier to create an own StringStream (dreived
> from
> >ANTLRStringStream) which silently swallows the + and - as well
> the
> >following newline? Then both lexer and parser are cleaner.
>
> That's certainly a possibility (and perhaps a good one), but
> that'd end up screwing up the line numbering, wouldn't it?
I don't see, why swallowing the two characters would prevent increasing the
line count. Tokens receive the line number from the stream itself and not
because the lexer counts newlines.
Johannes
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen:
http://www.gmx.net/de/go/multimessenger
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---
