[il-antlr-interest: 31932] Re: [antlr-interest] antlr v4 wish list
Default values for action rule parameters. I often want to pass in some info from only one context. Its a pain tracking down all the other calls. I don't know how you would handle this for multiple parameters, but having it for just one parameter would be a big win. k On Tue, Mar 22, 2011 at 6:45 PM, Terence Parr pa...@cs.usfca.edu wrote: Howdy, I'm going to start augmenting ANTLR v3 significantly to create v4. The goal is backward compatibility; any new functionality, of course, will require altering or augmenting your grammars to take advantage of it. Here is my potential list of updates: http://www.antlr.org/wiki/display/ANTLR4/ANTLR+v4+Wish+list Anything to add or comment on? Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31934] Re: [antlr-interest] antlr v4 wish list
Howdy, I'm going to start augmenting ANTLR v3 significantly to create v4. The goal is backward compatibility; any new functionality, of course, will require altering or augmenting your grammars to take advantage of it. Here is my potential list of updates: http://www.antlr.org/wiki/display/ANTLR4/ANTLR+v4+Wish+list Anything to add or comment on? Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address A new error recovery mode for tree parsing: When parsing ASTs, the ordinary error recovery strategies based on token deletion/insertion are completely useless, because there are no man-made syntax errors. In my experience, what you really want to do is the following: assume that you have an error handler attached to some rule and an error happens somewhere in the subtree of the node parsed by that rule. When the handler catches an error, the parser must skip the remainder of that subtree, otherwise the parser position is not consistent with the grammar position anymore. In AST implementations that are based on pointers between nodes this happens automatically, but Antlr's representation as a flat list of nodes with UP and DOWN tokens makes it requires some work - the parser has to keep track of the current node's depth and skip the appropriate number of UP nodes whenever an error is caught. Iztok List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31936] [antlr-interest] [CSharp3] @namespace and composite grammars
To maximize reuse of our grammars we generally create separate lexer and parser
grammars which are target agnostic (i.e. contain no target specific code). We
then combine them into a composite grammar in which the target language and
other implementation specific details are specified
(header,members,namespace,etc.). This way, grammars can be used for multiple
targets and we have maximum freedom to combine multiple lexers/parsers into
larger ones.
If a composite grammar C imports a parser grammar P and a lexer grammar L, the
tool generates CParser, CLexer, C_P and C_L.
Adding @namespace { X } to the composite grammar should IMHO put all four
recognizers in namespace X. Currently, at least with the CSharp3 target, only
CParser is put into the specified namespace.
Specifying @lexer::namespace { X } to the composite grammar causes only the
outermost lexer CLexer to be added to the specified namespace, not the imported
lexer C_L. The same holds for @parser::namespace.
Btw, if I'm not mistaken, there's no way to put recognizers in packages with
the Java target. Is that correct?
Best regards,
Ranco Marcus
Epirion Knowledge Solutions B.V.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31939] Re: [antlr-interest] [CSharp3] @namespace and composite grammars
Hi Ranco,
As mentioned in the CSharp3 documentation, you need to use both
@lexer::namespace{...} and @parser::namespace{...} in a composite grammar.
If you use the templates included in my Visual Studio extension, then these
are automatically inserted for you (including using the correct namespace
based on the location in the project where you placed them).
I'll check into the namespace issue when it comes to imported grammars. The
expected functionality is this should work once you have
@lexer::namespace{...} and @parser::namespace{...} in the composite grammar
, plus @namespace{...} in each of P and L.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus
Sent: Thursday, March 24, 2011 5:55 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [CSharp3] @namespace and composite grammars
To maximize reuse of our grammars we generally create separate lexer and
parser grammars which are target agnostic (i.e. contain no target specific
code). We then combine them into a composite grammar in which the target
language and other implementation specific details are specified
(header,members,namespace,etc.). This way, grammars can be used for multiple
targets and we have maximum freedom to combine multiple lexers/parsers into
larger ones.
If a composite grammar C imports a parser grammar P and a lexer grammar L,
the tool generates CParser, CLexer, C_P and C_L.
Adding @namespace { X } to the composite grammar should IMHO put all four
recognizers in namespace X. Currently, at least with the CSharp3 target,
only CParser is put into the specified namespace.
Specifying @lexer::namespace { X } to the composite grammar causes only
the outermost lexer CLexer to be added to the specified namespace, not the
imported lexer C_L. The same holds for @parser::namespace.
Btw, if I'm not mistaken, there's no way to put recognizers in packages with
the Java target. Is that correct?
Best regards,
Ranco Marcus
Epirion Knowledge Solutions B.V.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31940] Re: [antlr-interest] [CSharp3] @namespace and composite grammars
@Sam, what is the name of the Visual Studio extension you are talking about?
I tried to search for anything with ANTLR in it and it couldn't find
anything.
On Thu, Mar 24, 2011 at 8:50 AM, Sam Harwell sharw...@pixelminegames.comwrote:
Hi Ranco,
As mentioned in the CSharp3 documentation, you need to use both
@lexer::namespace{...} and @parser::namespace{...} in a composite grammar.
If you use the templates included in my Visual Studio extension, then these
are automatically inserted for you (including using the correct namespace
based on the location in the project where you placed them).
I'll check into the namespace issue when it comes to imported grammars. The
expected functionality is this should work once you have
@lexer::namespace{...} and @parser::namespace{...} in the composite grammar
, plus @namespace{...} in each of P and L.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus
Sent: Thursday, March 24, 2011 5:55 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [CSharp3] @namespace and composite grammars
To maximize reuse of our grammars we generally create separate lexer and
parser grammars which are target agnostic (i.e. contain no target specific
code). We then combine them into a composite grammar in which the target
language and other implementation specific details are specified
(header,members,namespace,etc.). This way, grammars can be used for
multiple
targets and we have maximum freedom to combine multiple lexers/parsers into
larger ones.
If a composite grammar C imports a parser grammar P and a lexer grammar L,
the tool generates CParser, CLexer, C_P and C_L.
Adding @namespace { X } to the composite grammar should IMHO put all four
recognizers in namespace X. Currently, at least with the CSharp3 target,
only CParser is put into the specified namespace.
Specifying @lexer::namespace { X } to the composite grammar causes only
the outermost lexer CLexer to be added to the specified namespace, not the
imported lexer C_L. The same holds for @parser::namespace.
Btw, if I'm not mistaken, there's no way to put recognizers in packages
with
the Java target. Is that correct?
Best regards,
Ranco Marcus
Epirion Knowledge Solutions B.V.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31941] Re: [antlr-interest] [CSharp3] @namespace and composite grammars
Hi Kevin,
The plugin information is included in the Visual Studio and the ANTLR C#
Target documentation referenced on the ANTLR wiki:
http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases
Thanks,
Sam
From: Kevin Cherry [mailto:kche...@tigers.lsu.edu]
Sent: Thursday, March 24, 2011 9:12 AM
To: Sam Harwell
Cc: Ranco Marcus; antlr-interest@antlr.org
Subject: Re: [antlr-interest] [CSharp3] @namespace and composite grammars
@Sam, what is the name of the Visual Studio extension you are talking about?
I tried to search for anything with ANTLR in it and it couldn't find
anything.
On Thu, Mar 24, 2011 at 8:50 AM, Sam Harwell sharw...@pixelminegames.com
wrote:
Hi Ranco,
As mentioned in the CSharp3 documentation, you need to use both
@lexer::namespace{...} and @parser::namespace{...} in a composite grammar.
If you use the templates included in my Visual Studio extension, then these
are automatically inserted for you (including using the correct namespace
based on the location in the project where you placed them).
I'll check into the namespace issue when it comes to imported grammars. The
expected functionality is this should work once you have
@lexer::namespace{...} and @parser::namespace{...} in the composite grammar
, plus @namespace{...} in each of P and L.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of Ranco Marcus
Sent: Thursday, March 24, 2011 5:55 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] [CSharp3] @namespace and composite grammars
To maximize reuse of our grammars we generally create separate lexer and
parser grammars which are target agnostic (i.e. contain no target specific
code). We then combine them into a composite grammar in which the target
language and other implementation specific details are specified
(header,members,namespace,etc.). This way, grammars can be used for multiple
targets and we have maximum freedom to combine multiple lexers/parsers into
larger ones.
If a composite grammar C imports a parser grammar P and a lexer grammar L,
the tool generates CParser, CLexer, C_P and C_L.
Adding @namespace { X } to the composite grammar should IMHO put all four
recognizers in namespace X. Currently, at least with the CSharp3 target,
only CParser is put into the specified namespace.
Specifying @lexer::namespace { X } to the composite grammar causes only
the outermost lexer CLexer to be added to the specified namespace, not the
imported lexer C_L. The same holds for @parser::namespace.
Btw, if I'm not mistaken, there's no way to put recognizers in packages with
the Java target. Is that correct?
Best regards,
Ranco Marcus
Epirion Knowledge Solutions B.V.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31942] [antlr-interest] problems getting a simple grammar to accept it's input
Hi,
I'm having problems getting a (so far) very simple grammar to accept it's input:
-
grammar Simulink;
IDENTIFIER : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
;
INT : '0'..'9'+
;
FLOAT
: ('0'..'9')+ '.' ('0'..'9')* EXPONENT?
| '.' ('0'..'9')+ EXPONENT?
| ('0'..'9')+ EXPONENT
;
COMMENT
: '#' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;}
;
WS : ( ' '
| '\t'
| '\r'
| '\n'
) {$channel=HIDDEN;}
;
STRING
: '' ( ESC_SEQ | ~('\\'|'') )* ''
;
fragment
EXPONENT : ('e'|'E') ('+'|'-')? ('0'..'9')+ ;
fragment
HEX_DIGIT : ('0'..'9'|'a'..'f'|'A'..'F') ;
fragment
ESC_SEQ
: '\\' ('b'|'t'|'n'|'f'|'r'|'\'|'\''|'\\')
| UNICODE_ESC
| OCTAL_ESC
;
fragment
OCTAL_ESC
: '\\' ('0'..'3') ('0'..'7') ('0'..'7')
| '\\' ('0'..'7') ('0'..'7')
| '\\' ('0'..'7')
;
fragment
UNICODE_ESC
: '\\' 'u' HEX_DIGIT HEX_DIGIT HEX_DIGIT HEX_DIGIT
;
fragment
BLOCK_BEGIN
: '{'
;
fragment
BLOCK_END
: '}'
;
file: block+
;
block : IDENTIFIER BLOCK_BEGIN BLOCK_END
;
-
This is the input:
-
# bla
Model {
}
-
And here is what happens when I try to feed it to the grammar:
-
$ make smalltests
antlr3 -verbose -trace -report Simulink.g
ANTLR Parser Generator Version 3.3 Nov 30, 2010 12:50:56
Simulink.g
Simulink.file:65:8 decision 1: k=1
javac -classpath antlr/antlr-3.3-complete.jar:. SimulinkLexer.java
javac -classpath antlr/antlr-3.3-complete.jar:. SimulinkParser.java
javac -classpath antlr/antlr-3.3-complete.jar:. Test.java
cat testdata/empty.mdl| java -classpath
antlr/antlr-3.3-complete.jar:. Test
enter COMMENT # line=1:0
exit COMMENT M line=2:0
enter IDENTIFIER M line=2:0
exit IDENTIFIER line=2:5
enter file [@1,6:10='Model',4,2:0]
enter block [@1,6:10='Model',4,2:0]
enter WS line=2:5
exit WS { line=2:6
line 2:6 no viable alternative at character '{'
enter WS
line=2:7
exit WS } line=3:0
line 3:0 no viable alternative at character '}'
enter WS
line=3:1
exit WS
line=4:0
enter WS
line=4:0
exit WS line=5:0
line 5:0 mismatched input 'EOF' expecting BLOCK_BEGIN
exit block [@6,17:17='EOF',-1,5:0]
exit file [@6,17:17='EOF',-1,5:0]
-
As I understand it the parser consumes 'Model' as IDENTIFIER and goes into
state block. It ignores a WS, then finds a '{'. This should be recognized as
BLOCK_BEGIN, which is the next token expected in block---any idea what I'm
doing wrong?
best regards
Florian Franzmann
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31943] Re: [antlr-interest] problems getting a simple grammar to accept it's input
On 03/24/2011 11:08 AM, Florian Franzmann wrote:
Hi,
I'm having problems getting a (so far) very simple grammar to accept it's
input:
-
grammar Simulink;
IDENTIFIER : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
;
INT : '0'..'9'+
;
FLOAT
: ('0'..'9')+ '.' ('0'..'9')* EXPONENT?
| '.' ('0'..'9')+ EXPONENT?
| ('0'..'9')+ EXPONENT
;
COMMENT
: '#' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;}
;
WS : ( ' '
| '\t'
| '\r'
| '\n'
) {$channel=HIDDEN;}
;
STRING
: '' ( ESC_SEQ | ~('\\'|'') )* ''
;
fragment
EXPONENT : ('e'|'E') ('+'|'-')? ('0'..'9')+ ;
fragment
HEX_DIGIT : ('0'..'9'|'a'..'f'|'A'..'F') ;
fragment
ESC_SEQ
: '\\' ('b'|'t'|'n'|'f'|'r'|'\'|'\''|'\\')
| UNICODE_ESC
| OCTAL_ESC
;
fragment
OCTAL_ESC
: '\\' ('0'..'3') ('0'..'7') ('0'..'7')
| '\\' ('0'..'7') ('0'..'7')
| '\\' ('0'..'7')
;
fragment
UNICODE_ESC
: '\\' 'u' HEX_DIGIT HEX_DIGIT HEX_DIGIT HEX_DIGIT
;
fragment
BLOCK_BEGIN
: '{'
;
fragment
BLOCK_END
: '}'
;
file : block+
;
block : IDENTIFIER BLOCK_BEGIN BLOCK_END
;
Because you actually defined BLOCK_BEGIN and BLOCK_END as fragments,
those tokens are never actually created. Remove the fragment from the
TOKEN rules.
-
This is the input:
-
# bla
Model {
}
-
And here is what happens when I try to feed it to the grammar:
-
$ make smalltests
antlr3 -verbose -trace -report Simulink.g
ANTLR Parser Generator Version 3.3 Nov 30, 2010 12:50:56
Simulink.g
Simulink.file:65:8 decision 1: k=1
javac -classpath antlr/antlr-3.3-complete.jar:. SimulinkLexer.java
javac -classpath antlr/antlr-3.3-complete.jar:. SimulinkParser.java
javac -classpath antlr/antlr-3.3-complete.jar:. Test.java
cat testdata/empty.mdl| java -classpath
antlr/antlr-3.3-complete.jar:. Test
enter COMMENT # line=1:0
exit COMMENT M line=2:0
enter IDENTIFIER M line=2:0
exit IDENTIFIER line=2:5
enter file [@1,6:10='Model',4,2:0]
enter block [@1,6:10='Model',4,2:0]
enter WS line=2:5
exit WS { line=2:6
line 2:6 no viable alternative at character '{'
enter WS
line=2:7
exit WS } line=3:0
line 3:0 no viable alternative at character '}'
enter WS
line=3:1
exit WS
line=4:0
enter WS
line=4:0
exit WS line=5:0
line 5:0 mismatched input 'EOF' expecting BLOCK_BEGIN
exit block [@6,17:17='EOF',-1,5:0]
exit file [@6,17:17='EOF',-1,5:0]
-
As I understand it the parser consumes 'Model' as IDENTIFIER and goes into
state block. It ignores a WS, then finds a '{'. This should be recognized as
BLOCK_BEGIN, which is the next token expected in block---any idea what I'm
doing wrong?
fragment TOKENs are meant to only be recognized when creating further
tokens. Since your BLOCK_BEGIN and BLOCK_END are intended to be final
TOKENs (you use them in your parser's block rule), you should remove
the fragment from those token rules.
best regards
Florian Franzmann
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
Kevin J. Cummings
kjch...@verizon.net
cummi...@kjchome.homeip.net
cummi...@kjc386.framingham.ma.us
Registered Linux User #1232 (http://counter.li.org)
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31944] Re: [antlr-interest] problems getting a simple grammar to accept it's input
On Thu, Mar 24, 2011 at 4:08 PM, Florian Franzmann
siflf...@hawo.stw.uni-erlangen.de wrote:
...
As I understand it the parser consumes 'Model' as IDENTIFIER and goes into
state block. It ignores a WS, then finds a '{'. This should be recognized
as
BLOCK_BEGIN, which is the next token expected in block---any idea what I'm
doing wrong?
best regards
Florian Franzmann
Fragment rules can only be seen by other lexer rules. There will never be
a token `BLOCK_BEGIN ` created, so it can't be used inside parser rules.
Regards,
Bart.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31945] Re: [antlr-interest] problems getting a simple grammar to accept it's input
On Thu 2011-03-24 16:26:44 +0100, Bart Kiers wrote:
On Thu, Mar 24, 2011 at 4:08 PM, Florian Franzmann
[1]siflf...@hawo.stw.uni-erlangen.de wrote:
...
As I understand it the parser consumes 'Model' as IDENTIFIER and goes
into
state block. It ignores a WS, then finds a '{'. This should be
recognized as
BLOCK_BEGIN, which is the next token expected in block---any idea what
I'm
doing wrong?
best regards
Florian Franzmann
Fragment rules can only be seen by other lexer rules. There will never
be a token `BLOCK_BEGIN*` created, so it can't be used inside parser
rules.
Thanks, that did it.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31946] [antlr-interest] Unused rule trashes my grammar
Hi,
I don't understand what's happening here. I am generating an ANTLR
grammar with XText. Xtext introduces rules like shown below with
'someGeneratedRule1' and 'someGeneratedRule2', which I can't influence.
One of the generated rules (someGeneratedRule2) isn't actually used,
but seems to provoke an ambiguity. How could that be possible?
When checking this (stripped down) grammar ANTLR yields 'error(202):
grammar_min2.g:14:14: the decision cannot distinguish between
alternative(s) 1,2 for input such as RULE_INT '.' EOF EOF'
Obviously this has something to to with the EOF's Xtext introduces.
When I remove the rule someGeneratedRule2 the problem disappears.
{{{
grammar grammar_min2;
someGeneratedRule1: rulemodule EOF;
rulemodule:
'module' RULE_ID ruleCOLON
ruleunaryexpr
('end' 'module' |rulePERIOD)
;
// PROVOKES THE PROBLEM
someGeneratedRule2: ruleunaryexpr EOF;
ruleunaryexpr:
RULE_INT
|ruleDOUBLECONST
;
ruleEXPONENT:
('e' | 'E') ('+'|'-')? RULE_INT
;
ruleDOUBLECONST:
( RULE_INT '.' (RULE_INT)? (ruleEXPONENT)?)
| ('.' RULE_INT (ruleEXPONENT)?)
| (RULE_INTruleEXPONENT)
;
ruleCOLON:':';
rulePERIOD:'.';
// lexer
RULE_ID:('a' .. 'z' | 'A' .. 'Z' | '_' ) ( 'a' .. 'z' | 'A' .. 'Z' | '_'
| '0' .. '9' ) * ;
RULE_INT:('0' .. '9' ) + ;
RULE_WS:(' ' | '\t' | '\r' | '\n')+;
RULE_ML_COMMENT:'%{' (options {greedy = false; } : . ) * '}%' ;
RULE_SL_COMMENT : '%' ~ ( ( '\n' | '\r' ) ) * ( '\r' ? '\n' ) ? ;
}}}
Additionally ANTLRWorks doesn't visualize the error, just writes it to
the console.
Any ideas?
Thanks,
Martin
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31947] [antlr-interest] Help please
Hello,
My name is Wael Sellami, I am a computer science researcher in Redcad
laboratory, university of sfax, tunisia, master level.
I use ANTLR to transform xml code to PROMELA code as input language of the
model-checker SPIN.
I have a problem in defining of nested bloc in my output. In fact, I can not
organize the output code as the form of input code. Below, there is a
exemple of small grammar and the rule of translation:
grammar xml;
options {
output = AST;
ASTLabelType = CommonTree;
backtrack=true;
}
tokens {
ROOTDEF; NAMEDEF; XMLTAGDEF; TITLEDEF; BOOKDEF; AUTHORDEF; }
system : root+;
root : 'books' (xmltag)+ '/books'
- ^(ROOTDEF (xmltag)+);
xmltag : book - ^(XMLTAGDEF book) | title - ^(XMLTAGDEF title)|
author - ^(XMLTAGDEF author);
book : 'book' name '' (xmltag)+ '/book'
- ^(BOOKDEF name (xmltag)+);
title: 'title' name '' (ID)+ '/title'
- ^(TITLEDEF name (ID)+);
author: 'author' name '' (ID)+ '/author'
- ^(AUTHORDEF name (ID)+);
name : 'name=' ID ''
- ^(NAMEDEF ID);
ID : ('a'..'z'|'A'..'Z'|'0'..'9') ('a'..'z'|'A'..'Z'|'0'..'9')*;
WS : ( ' ' | '\t' | '\r' | '\n')* { $channel = HIDDEN; } ;
Then I have defined the tree walker:
tree grammar Translate;
options {
tokenVocab = xml;
ASTLabelType = CommonTree;
output=template;
}
@members {
ListString columnsactbpel = new ArrayListString();
}
system : root+;
root : ^(ROOTDEF (xmltag)+);
xmltag : ^(XMLTAGDEF book) | ^(XMLTAGDEF title) | ^(XMLTAGDEF author);
book : ^(BOOKDEF name (xmltag)+)
{
System.out.println(Begin book tag);
System.out.println(END book tag);
};
titlereturns [String tit]
: ^(TITLEDEF name (ID)+)
{ System.out.println($name.namevar);};
authorreturns [String auth]
: ^(AUTHORDEF name (ID)+)
{ System.out.println($name.namevar);};
name returns [String namevar]
: ^(NAMEDEF (ID)+)
{ $namevar = $ID.text;};
After that, I have tried with this input code:
books
title name=AAA Data base /title
author name=BBB XML /author
book name=a
title name=CCC Data base /title
author name=DDD XML /author
book name=a
title name=EEE Data base /title
author name=FFF XML /author
/book
/book
/books
So, the result of this translation is:
AAA
BBB
CCC
DDD
EEE
FFF
Begin book tag
END book tag
Begin book tag
END book tag
But normally, if we follow the input xml code the résult must be:
AAA
BBB
Begin book tag
CCC
DDD
Begin book tag
EEE
FFF
END book tag
END book tag
Please, can you help me to solve this problem.
Thank you in advance
Wael
--
-
* Mr. Wael Sellami
* Master student
* Research Unit on Development and Control of Distributed Applications
(ReDCAD)
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31948] Re: [antlr-interest] antlr v4 wish list
added * Tokens and Trees should both know their start/stop line, start/stop char position to make IDEs easier. Ter On Mar 23, 2011, at 2:17 PM, Matt Fowles wrote: Terence~ Better location management. Tokens and Trees should both have methods for: getStartLine() getEndLine() getStartCharPosition() getEndCharPosition() Trees should base them on the tokens that they include. Synthetic tokens should get them from the original token or the full production they match when no original token is provide. This is especially useful for IDEs that do several layers of rewrite before they produce an error, as it can be difficult to match the positions exactly if things like desugaring have happened before an error is produced. The methods for end position are important as IDEs use that to figure out how far to draw the squiggle. Matt On Tue, Mar 22, 2011 at 5:45 PM, Terence Parr pa...@cs.usfca.edu wrote: Howdy, I'm going to start augmenting ANTLR v3 significantly to create v4. The goal is backward compatibility; any new functionality, of course, will require altering or augmenting your grammars to take advantage of it. Here is my potential list of updates: http://www.antlr.org/wiki/display/ANTLR4/ANTLR+v4+Wish+list Anything to add or comment on? Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31949] Re: [antlr-interest] antlr v4 wish list
added * Default rule parameter values, same for scope fields. Ter On Mar 24, 2011, at 1:30 AM, Kevin Glynn wrote: Default values for action rule parameters. I often want to pass in some info from only one context. Its a pain tracking down all the other calls. I don't know how you would handle this for multiple parameters, but having it for just one parameter would be a big win. k On Tue, Mar 22, 2011 at 6:45 PM, Terence Parr pa...@cs.usfca.edu wrote: Howdy, I'm going to start augmenting ANTLR v3 significantly to create v4. The goal is backward compatibility; any new functionality, of course, will require altering or augmenting your grammars to take advantage of it. Here is my potential list of updates: http://www.antlr.org/wiki/display/ANTLR4/ANTLR+v4+Wish+list Anything to add or comment on? Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31950] Re: [antlr-interest] antlr v4 wish list
added * Tree parser error handling should skip subtrees not nodes; these are programming errors not input errors. The flat stream makes it hard to resync. Ter On Mar 24, 2011, at 2:07 AM, Iztok Kavkler wrote: Howdy, I'm going to start augmenting ANTLR v3 significantly to create v4. The goal is backward compatibility; any new functionality, of course, will require altering or augmenting your grammars to take advantage of it. Here is my potential list of updates: http://www.antlr.org/wiki/display/ANTLR4/ANTLR+v4+Wish+list Anything to add or comment on? Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address A new error recovery mode for tree parsing: When parsing ASTs, the ordinary error recovery strategies based on token deletion/insertion are completely useless, because there are no man-made syntax errors. In my experience, what you really want to do is the following: assume that you have an error handler attached to some rule and an error happens somewhere in the subtree of the node parsed by that rule. When the handler catches an error, the parser must skip the remainder of that subtree, otherwise the parser position is not consistent with the grammar position anymore. In AST implementations that are based on pointers between nodes this happens automatically, but Antlr's representation as a flat list of nodes with UP and DOWN tokens makes it requires some work - the parser has to keep track of the current node's depth and skip the appropriate number of UP nodes whenever an error is caught. Iztok List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31951] Re: [antlr-interest] Help please
Greetings!
On Thu, 2011-03-24 at 17:22 +0100, wael sellami wrote:
Hello,
...stuff snipped...
book : ^(BOOKDEF name (xmltag)+)
{
System.out.println(Begin book tag);
System.out.println(END book tag);
};
...snipped...
short answer, place actions properly:
book : ^(BOOKDEF name
{ System.out.println(Begin book tag); }
(xmltag)+
{ System.out.println(END book tag); }
) ;
preferred answer, use StringTemplate to compose your output.
hope this helps...
-jbb
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31952] Re: [antlr-interest] antlr v4 wish list
On Thu, Mar 24, 2011 at 1:23 PM, Terence Parr pa...@cs.usfca.edu wrote:
added
* Tree parser error handling should skip subtrees not nodes; these are
programming errors not input errors. The flat stream makes it hard to
resync.
Ter
On Mar 24, 2011, at 2:07 AM, Iztok Kavkler wrote:
Howdy, I'm going to start augmenting ANTLR v3 significantly to create
v4. The goal is backward compatibility; any new functionality, of course,
will require altering or augmenting your grammars to take advantage of it.
Here is my potential list of updates:
http://www.antlr.org/wiki/display/ANTLR4/ANTLR+v4+Wish+list
Anything to add or comment on?
Ter
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
A new error recovery mode for tree parsing:
When parsing ASTs, the ordinary error recovery strategies based on token
deletion/insertion are completely useless, because there are no man-made
syntax errors. In my experience, what you really want to do is the
following: assume that you have an error handler attached to some rule
and an error happens somewhere in the subtree of the node parsed by that
rule. When the handler catches an error, the parser must skip the
remainder of that subtree, otherwise the parser position is not
consistent with the grammar position anymore. In AST implementations
that are based on pointers between nodes this happens automatically, but
Antlr's representation as a flat list of nodes with UP and DOWN tokens
makes it requires some work - the parser has to keep track of the
current node's depth and skip the appropriate number of UP nodes
whenever an error is caught.
Iztok
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
1. If my concept of scannerless parsing is the same as yours, then in the
generated code for a rule allow the true for do { rule code }
while(true) to be an attribute of the rule, i.e exit. Obviously the value
would be true unless changed by a user.This would allow the user to have
control of when to exit the rule. By turning true into a attribute of the
rule, this allows for more control than gated semantic predicates.
Based on by concept of scannerless parsing, there is no lexer and the parser
drives the reading of the tokens from the intput stream. The input stream
does not generate the tokens ahead of time but only when needed. In a quick
proof of concept I had the token type passed from the parser as a generic
parameter, allowing the redefinition of the token returned by the token
stream. There were no pre-defined tokens values; they were dynamically
generated.To get the proof of concept to work required having a
cross-reference table between token types and token values.
2. If ANTLR 4 will allow the reading of binary data streams, then please
don't put char and line pos in a base class. There could be one inherited
classes that defines line and char pos, and another inherited class that
defines offset.
Thanks
Eric
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31953] [antlr-interest] Problem with ANTLR3.3 and Python 2.7
Hi all, I am attempting to use ANTLRWorks with Python2.7 on Windows 7. I am using the latest ANTLRWorks release which I think includes ANTLR3.3 I obtained the python runtime from the antlr-3.3.tar.gz file specified on this webpage: http://www.antlr.org/wiki/display/ANTLR3/Python+runtime (the Complete ANTLR 3.x Java binaries jar file mentioned on the webpage actually contained no Python runtime.) I moved the antlr3 folder to my LIB directory under my python2.7 installation directory as have for every other library I use. I built and compiled a grammar without difficulty and tried to run the resultant parser and I get the following error: RuntimeError: ANTLR version mismatch: The recognizer has been generated by V3.3 Nov 30, 2010 12:45:30, but this runtime is V3.1.3. Please use the V3.3 Nov 30, 2010 12:45:30 runtime or higher. I have been searching for answers to this problem for a couple of days and have come up empty. Does anyone here have any suggestions? Best regards, Jason Doege List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31954] [antlr-interest] in v3.3, does the token stream mechanism slurp in the whole file?
Hello, According to: http://v2kparse.blogspot.com/2008/06/first-pass-uploaded-to-sourceforce.html the ANTLR v3.0 token stream mechanism tries to slurp in the whole source file, which is problematic with gigantic source files. 1) Does ANTLR v3.3 still try to read all the input at once? 2) Is this what BufferedTokenStream fixes? Thanks, Martin List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
