Hi,
I tried everything I know so I am tending to you. I have a
context-sensitive construct in my lexer. I have two tokens where where
one is a superset of the other, but shall only be recognized after an
equals was found and is basically any character until a komma.
Therefore, I defined a predicate which is set to true once an equals is
reached which enables the gated superset token and when a comma is found
the predicate is set to false.
In order to allow the following:
key = value = 3
where key is key and everything after the '=' is a value until EOF or
',' ,i..e. in this case "value = 3". Yes I know this is very strange and
usually you would quote string, but we need it for legacy reasons,
mainly incoperating legacy hand-written parser funcitonality into ANTLR ;-)
But unfortunately I get a strange syntax error? Please see below: I also
attached the full grammar, in case you need that.
I include a snippet of the AST grammar, where the rule in question is used:
// parser...
properties
:
( property ( Delim property )* )? Delim?
-> property*
;
attributes
: // FIXME is this wanted h=(d=3 k=3) , i.e. with komma?
( attribute ( Delim attribute )* )* Delim?
-> ^( ATTRIBUTES attribute* )
;
list
:
( value ( Delim value )* )*
-> ^( TYPE LIST ) ^( VALUE value* )
;
// lexer...
Delim
:
','
{
unsetFoundFirstEqualsInPropert();
}
;
fragment USTRING_
:
{hasFoundFirstEqualsInProperty()}?=>
(~(' ' /*| '\t'*/ | '%' | Delim | '\n' | '[' | '{' | '"' | '\'' |
'`' | '<' | '\\' | '/') | HEXESCAPE | '%' ~HEXDIGIT_) (~('%' | Delim |
'\n' | '}' | ']' ) | HEXESCAPE | '%' ~HEXDIGIT_)*
;
The error is "syntax error: buildnfa: <AST>:227:2: expecting EOA, found '
unsetFoundFirstEqualsInPropert();
'".
I get from the list that EOA means End-Of-Alternative, but the rules
does not have an alternative. This may also be related to the bug
http://www.antlr.org/jira/browse/ANTLR-305 ? It seems very similar to my
problem.
Please help, we really need this. Also if you have another suggestion
for a solution to this problem.
Thanks,
Paul
ANTLR Parser Generator Version 3.1.1
error(100):
T:\Workspaces\paulb\AAA-nemip\ag\netzwert\nemip\StringProps.g:0:0:
syntax error: buildnfa: <AST>:227:2: expecting EOA, found '
unsetFoundFirstEqualsInPropert();
'
error(100):
T:\Workspaces\paulb\AAA-nemip\ag\netzwert\nemip\StringProps.g:0:0:
syntax error: buildnfa: <AST>: expecting EOA, found '<empty tree>'
error(100):
T:\Workspaces\paulb\AAA-nemip\ag\netzwert\nemip\StringProps.g:0:0:
syntax error: buildnfa: <AST>:227:2: expecting EOA, found '
unsetFoundFirstEqualsInPropert();
'
error(100):
T:\Workspaces\paulb\AAA-nemip\ag\netzwert\nemip\StringProps.g:0:0:
syntax error: buildnfa: <AST>: expecting EOA, found '<empty tree>'
error(10): internal error: Can't get property javaCompressedTransition
using method get/isJavaCompressedTransition from org.antlr.analysis.DFA
instance : java.lang.OutOfMemoryError: Java heap space
java.util.ArrayList.<init>(ArrayList.java:112)
org.antlr.analysis.DFA.getRunLengthEncoding(DFA.java:357)
org.antlr.analysis.DFA.getJavaCompressedTransition(DFA.java:333)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.antlr.stringtemplate.language.ASTExpr.invokeMethod(ASTExpr.java:563)
org.antlr.stringtemplate.language.ASTExpr.rawGetObjectProperty(ASTExpr.java:514)
org.antlr.stringtemplate.language.ASTExpr.getObjectProperty(ASTExpr.java:416)
org.antlr.stringtemplate.language.ActionEvaluator.attribute(ActionEvaluator.java:351)
org.antlr.stringtemplate.language.ActionEvaluator.expr(ActionEvaluator.java:136)
org.antlr.stringtemplate.language.ActionEvaluator.templateApplication(ActionEvaluator.java:216)
org.antlr.stringtemplate.language.ActionEvaluator.expr(ActionEvaluator.java:126)
org.antlr.stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:84)
org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:148)
org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:700)
org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:722)
org.antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java:659)
org.antlr.stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:86)
org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:148)
org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:700)
org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:722)
org.antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java:659)
org.antlr.stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java:86)
org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:148)
org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:700)
org.antlr.codegen.CodeGenerator.write(CodeGenerator.java:1275)
org.antlr.codegen.Target.genRecognizerFile(Target.java:94)
org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:460)
org.antlr.Tool.generateRecognizer(Tool.java:420)
org.antlr.Tool.process(Tool.java:342)
--
Paul Bouché
Voice: +49 30 590080-1284
Nokia Siemens Networks GmbH & Co. KG, An den Treptowers 1, 12435 Berlin, Germany
Sitz der Gesellschaft: München / Registered office: Munich
Registergericht: München / Commercial registry: Munich, HRA 88537
WEEE-Reg.-Nr.: DE 52984304
Persönlich haftende Gesellschafterin / General Partner: Nokia Siemens Networks
Management GmbH
Geschäftsleitung / Board of Directors: Lydia Sommer, Olaf Horsthemke
Vorsitzender des Aufsichtsrats / Chairman of supervisory board: Lauri Kivinen
Sitz der Gesellschaft: München / Registered office: Munich
Registergericht: München / Commercial registry: Munich, HRB 163416
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"il-antlr-interest" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en
-~----------~----~----~----~------~----~------~--~---
grammar StringProps;
options {
output = AST;
}
tokens {
PROPSMESSAGE;
PROPERTYNODE;
PROPERTIES;
PROPERTY;
ATTRIBUTES;
ATTRIBUTE;
KEY; // rename to NAME?
TYPE;
CAST;
VALUE;
LIST;
NUMBER;
HEX;
BUFREF;
TEXT;
UTEXT;
PASSWORD;
IPADDRESS;
PEERADDRESS;
BOOL;
SIGNAL;
Signal;
}
@members {
protected void mismatch(IntStream input, int ttype, BitSet follow)
throws RecognitionException {
MismatchedTokenException mte = new MismatchedTokenException(ttype,
input);
reportError(mte);
throw mte;
}
public void recoverFromMismatchedSet(IntStream input,
RecognitionException e,
BitSet follow)
throws RecognitionException {
throw e;
}
StringBuilder sb = new StringBuilder();
/**
* Provide a meaningful representation for the recognition
exception
*/
public String userErrorMessageFor(RecognitionException re) {
sb.setLength(0);
sb.append("Error in line ");
sb.append(re.line);
sb.append(" column ");
sb.append(re.charPositionInLine);
sb.append(": ");
sb.append(getErrorMessage(re, getTokenNames()));
return sb.toString();
}
}
@lexer::header {
package ag.netzwert.nemip;
}
@lexer::members {
private boolean foundFirstEqualsInProperty = false;
public void emitErrorMessage(String msg) {
System.err.println(msg);
}
private setFoundFirstEqualsInProperty() {
foundFirstEqualsInProperty = true;
}
private unsetFoundFirstEqualsInProperty() {
foundFirstEqualsInProperty = false;
}
private hasFoundFirstEqualsInProperty() {
foundFirstEqualsInProperty = true;
}
}
// Does not work with gunit
@rulecatch {
catch (RecognitionException e) {
reportError(e);
throw e;
}
}
start
:
propertynode EOF
;
propertynode
:
properties
-> ^( TYPE PROPERTYNODE ) properties
;
properties
:
( property ( Delim property )* )? Delim?
-> property*
;
property
:
name
( '.' cast )?
( '(' attributes ')' )?
EQ
value
-> ^( PROPERTY name cast? attributes? value )
;
attribute
:
name
( '.' cast )?
EQ
value
-> ^( ATTRIBUTE name cast? value )
;
name
:
( NAME
| NUMERIC
| PATH
| STRING
)
-> ^( KEY NAME? NUMERIC? PATH? STRING? )
;
cast
:
NAME
-> ^( CAST NAME )
;
attributes
: // FIXME is this wanted h=(d=3 k=3) , i.e. with komma?
( attribute ( Delim attribute )* )* Delim?
-> ^( ATTRIBUTES attribute* )
;
value
:
( plain
| '[' list ']'
| '{' propertynode '}'
)
-> plain? list? propertynode?
;
list
:
( value ( Delim value )* )*
-> ^( TYPE LIST ) ^( VALUE value* )
;
plain
:
( number
| hexdigit
| ustring
| string
| bufref
| ipaddress
| peeraddress
| password
| bool
) ;
number
: NUMERIC -> ^( TYPE NUMBER ) ^( VALUE NUMERIC );
hexdigit
: HEXDIGIT -> ^( TYPE HEX ) ^( VALUE HEXDIGIT );
string
: STRING -> ^(TYPE TEXT ) ^( VALUE STRING );
ustring
: (val=USTRING | val=NAME) -> ^(TYPE UTEXT ) ^( VALUE $val );
bufref
: BREF -> ^( TYPE BUFREF ) ^( VALUE BREF );
bool
: BOOLVAL -> ^( TYPE BOOL ) ^( VALUE BOOLVAL );
ipaddress
: IPADDR -> ^( TYPE IPADDRESS ) ^( VALUE IPADDR );
peeraddress
: PEERADDR -> ^( TYPE PEERADDRESS ) ^( VALUE PEERADDR );
password
: PWD -> ^( TYPE PASSWORD ) ^( VALUE PWD );
Delim
:
','
{
unsetFoundFirstEqualsInPropert();
}
;
EQ
:
'='
{
setFoundFirstEqualsInPropert();
}
;
BOOLVAL
:
( 'true'
| 'false'
) ;
IPADDR
:
( IP4
| IP6
) ;
PEERADDR
:
IP4 ':' DIGIT_+
;
fragment IP4
:
Quad4 '.' Quad4 '.' Quad4 '.' Quad4
;
fragment IP6
:
( '::' | Quad6 ':' ) Quad6 ( ':' Quad6 ( ':' Quad6 ( ':' Quad6 ( ':'
Quad6 ( ':' Quad6 ( ':' Quad6 )? )? )? )? )? )?
;
NUMERIC
:
SIGNS? DIGIT_+ ( '.' DIGIT_+ ( ( 'E' | 'e' ) SIGNS? DIGIT_+ )? ) ?
;
SIGNS
:
( '-' | '+' )
;
HEXDIGIT
:
( '0x' | '0X') HEXDIGIT_+
;
NAME
:
NAME_+
;
PATH
:
'/'? NAME ( '/' NAME )* ( '//' NAME )?
;
USTRING
:
USTRING_
;
STRING
:
( '\'' ( STRING_ | '`' | '"' | '\\' '\'' )* '\''
| '"' ( STRING_ | '`' | '\'' | '\\' '"' )* '"'
) ;
BREF
:
'`' ( STRING_ | '"' | '\'' | '\\' '`' )* '`'
;
PWD
:
'<' HEXDIGIT_+ '>'
// '*' HEXDIGIT_+
;
ESCAPE
:
'\\' ( UNICODE | 'a' | 'b' | 'e' | 'v' | 'r' | 't' | 'n' | '\\' )
;
WHITESPACE
:
( ' ' | '\t' | NEWLINE_ )+
{ skip(); }
;
fragment HEXESCAPE
:
'%' HEXDIGIT_ HEXDIGIT_
;
fragment NEWLINE_
:
( '\r' | '\r'? '\n' )
;
fragment NAME_
:
( NAMECHAR_
| DIGIT_
) ;
fragment UNICODE
:
'u' HEXDIGIT_ HEXDIGIT_ HEXDIGIT_ HEXDIGIT_
;
fragment HEXDIGIT_
:
( HEXCHAR_
| DIGIT_
) ;
fragment HEXCHAR_
:
( 'a'..'f'
| 'A'..'F'
) ;
fragment NAMECHAR_
:
( HEXCHAR_
| 'g'..'z'
| 'G'..'Z'
| '-' | '_' | ':' // need '<' & '>' but collides with PWD :(
) ;
fragment DIGIT_
:
'0'..'9'
;
fragment STRING_
:
( ~( '\u0000'..'\u001f' | '"' | '\'' | '`' | '%' )
| NEWLINE_ | '\t'
| ESCAPE
| HEXESCAPE
) ;
fragment USTRING_
:
{hasFoundFirstEqualsInProperty()}?=>
(~(' ' /*| '\t'*/ | '%' | Delim | '\n' | '[' | '{' | '"' | '\'' | '`' |
'<' | '\\' | '/') | HEXESCAPE | '%' ~HEXDIGIT_) (~('%' | Delim | '\n' | '}' |
']' ) | HEXESCAPE | '%' ~HEXDIGIT_)*
;
fragment Quad4
:
DIGIT_ ( DIGIT_ DIGIT_? )?
;
fragment Quad6
:
HEXDIGIT_ ( HEXDIGIT_ ( HEXDIGIT_ HEXDIGIT_? )? )?
;
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
