[il-antlr-interest: 31834] Re: [antlr-interest] lexer rule precedence
True is first. I also tried adding True and Identifier to the tokens list, (with True first), but that doesn't make any difference. I also added an ERR: . token like you suggested and that doesn't help. -- Ryan Brown On Fri, Mar 18, 2011 at 8:18 PM, Jim Idle j...@temporal-wave.com wrote: List it before IDENTIFIER as you have written it here and add a ERR : . ; as the last rule to catch non-matches and issue an error. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Ryan Brown Sent: Friday, March 18, 2011 8:15 PM To: antlr-interest@antlr.org Subject: [antlr-interest] lexer rule precedence I have a lexer with rules like this: TRUE : 'true'; IDENTIFIER : 'a'..'z'+; The text true parses as IDENTIFIER. How do I get it to parse as TRUE? Do I need to add not and all my keywords to the IDENTIFIER rule? I don't see anything like that in the java grammar. -- Ryan Brown List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31836] Re: [antlr-interest] Read multiple times
Thanks John for a prompt reply. Appreciate your response but your example
was bit hard for me to understand though 'Putting it in the Loop' made
sense.
So this is what I did... now I can sleep like a baby :)
parse
: ('Basket' basket)*
;
basket
@init {
Basket basket;
String state= ;
String employee= ;
String phone= ;
String zip= ;
}
@after {
basket= new Basket(state, employee);
basket.setPhone(phone);
this.somelist.addBakets(basket);
print(basket.toString());
//start again
//parse(); ? Is this right approach /// Not a right approach
}
: ('state'st=TOKEN { state= $st.text; }
| 'employee' et=TOKEN { employee= $et.text; }
| 'phone' pt=TOKEN { phone= $pt.text; }
| 'zip')* //For looop
;
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31840] Re: [antlr-interest] Problem with grammar
Wojciech~
Try pulling out the common left factor from your term expression:
term : factor (('*'|'/'|'*') term)?;
Matt
On Sat, Mar 19, 2011 at 12:18 PM, Wojciech Tomasz Cichon
wtcic...@googlemail.com wrote:
i’m struggling with this grammar for a few days now, and nothing really
working,
so i started from beginning and i have :
grammar myGrammar;
options {
language = Java;
k=1;
}
rule: term ;
factor : '-'? NUMBER
;
term : factor '*' term
| factor '/' term
| factor '%' term
| factor;
NUMBER : '0'..'9'+
;
WS : ( ' '
| '\t'
| '\r'
| '\n'
) {$channel=HIDDEN;}
;
and i got errors:
warning(200): /ANTLR_TEST/myGrammar.g:13:7: Decision can match input such as
'-' using multiple alternatives: 1, 2, 3, 4
As a result, alternative(s) 2,3,4 were disabled for that input
|--- term : factor '*' term
warning(200): /ANTLR_TEST/myGrammar.g:13:7: Decision can match input such as
NUMBER using multiple alternatives: 1, 2, 3, 4
As a result, alternative(s) 2,3,4 were disabled for that input
|--- term : factor '*' term
error(201): /ANTLR_TEST/myGrammar.g:13:7: The following alternatives can
never be matched: 2,3,4
|--- term : factor '*' term
can anyone tell me what i’m doing wrong
regards
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31841] Re: [antlr-interest] LIP pg. 285 11.4 Rewriter. How do you do it with ANLTR 3?
On Thu, Mar 17, 2011 at 5:54 PM, The Researcher researcher0...@gmail.comwrote:
On Thu, Mar 17, 2011 at 8:56 AM, Stephen Tuttlebee
themightystep...@googlemail.com wrote:
Hi Eric
I don't know if I understand your question completely (I haven't read
the previous questions you mentioned), but are you simply asking how you
rewrite a tree using ANTLR? Or were you actually asking how ANTLR
actually IMPLEMENTS this rewriting under the hood (i.e. what code does
it generate to do the rewriting)? Or something else?
If you were asking the first question, then I can provide a fairly
simple example from a couple of grammars I have been working on recently
(I don't know if the example is too simple since it only alters the tree
in a very minor way, and maybe you're thinking along the lines of larger
and more substantial changes to the tree).
I have the 'phase 1 grammar' (JavaBPhase1Parser.g) that does the parsing
and has tree rewrite rules to build an AST. This grammar has the
following options:
options {
language = Java;
output = AST;
tokenVocab = JavaBLexer;
}
This grammar just builds a tree in the normal way.
My 'phase 2 grammar' (JavaBPhase2WalkerSem1.g) is a tree grammar that
reads that AST, performs some semantic analysis of the input and spits
out an almost identical AST. It has similar options as before but the
really important one here is the rewrite=true option:
options {
language = Java;
output = AST; // since we are performing a tree rewrite, as well as
reading in an AST, I will also be outputting an AST
rewrite = true; // this is the key option here
tokenVocab = JavaBPhase1Parser;
ASTLabelType = CommonTree; // this option here because we're
reading in a tree and ANTLR would like to know the type of the nodes
}
The rewrite=true causes the tree to be spat out just the same as the
input tree except where you specify otherwise. For example, in one rule
I perform some semantic analysis that informs me on how to rewrite the
tree:
compositionExpression
@init {
boolean compositionComponent = false; // this is only here rather
than further below where I would want to use it because in the code
generated by ANTLR the variable is out of scope if I declare it down there
}
: ... // other tree rule
| ... // other tree rule
| ^(PLAIN_OR_COMPOSITION_COMPONENT ^(IDENT IDENTIFIER))
{ some actions that perform semantic analysis (look up in
the symbol table) to decide if compositionComponent is true or false }
- {compositionComponent}? ^(COMPOSITION_COMPONENT ^(IDENT
IDENTIFIER))
-
^(PLAIN_COMPONENT ^(IDENT IDENTIFIER))
In this example I am performing a very simple tree rewrite by simply
changing the imaginary token of a subtree's root node, but you could
change the entire subtree by changing the rewrite rule(s) to produce a
completely different subtree.
For larger, more 'structural' changes to the tree, I would imagine you
need to do a fair amount of passing subtrees around as parameters /
return values between rules so that you have access to the subtrees in
the rules that need them. p170 of the ANTLR reference book has an
example of this kind of thing (example is used in doing tree
construction in a *parser* grammar but you can do exactly the same in a
tree grammar).
You can also have semantic predicates (I think that's the proper name)
before multiple rewrite rules to rewrite the tree in different ways
depending on what the boolean expression in the semantic predicates
evaluates to. See page 181 of ANTLR reference for an example in the
variableDefinition rule:
variableDefinition
: modifiers type ID ('=' expression)? ';'
- {inMethod}? ^(VARIABLE ID modifier* type expression?) //
semantic predicate used here
-^(FIELD ID modifier* type expression?)
;
Again, that example is used in doing tree construction in a parser
grammar but you can do exactly the same in a tree grammar.
Bear in mind though (at least I think this is the case...), that with
these semantic predicates you have to ensure that there is a default
case which specifies the tree to be built if all the other previous
semantic predicates evaluate to false.
Hope this helps.
Stephen,
Thanks for the big road sign. I tried this originally but failed. I am
working on it again but am sure to have more questions.
Thanks, Eric
Stephen
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
Stephen,
It now works. My problem was in my driver/test harness and not the rewrite
rules.
For others with similar problems; the best example of this can be found on
the tree construction wiki page which I just now read.
http://www.antlr.org/wiki/display/ANTLR3/Tree+construction
Yes I spent to much time focused on the books and just needed to look at the
[il-antlr-interest: 31842] Re: [antlr-interest] lexer rule precedence
You need to post your grammar and make sure that you have not used ‘literals’ in the parser. Jim *From:* Ryan Brown [mailto:rib...@google.com] *Sent:* Friday, March 18, 2011 10:59 PM *To:* Jim Idle *Cc:* antlr-interest@antlr.org *Subject:* Re: [antlr-interest] lexer rule precedence True is first. I also tried adding True and Identifier to the tokens list, (with True first), but that doesn't make any difference. I also added an ERR: . token like you suggested and that doesn't help. -- Ryan Brown On Fri, Mar 18, 2011 at 8:18 PM, Jim Idle j...@temporal-wave.com wrote: List it before IDENTIFIER as you have written it here and add a ERR : . ; as the last rule to catch non-matches and issue an error. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Ryan Brown Sent: Friday, March 18, 2011 8:15 PM To: antlr-interest@antlr.org Subject: [antlr-interest] lexer rule precedence I have a lexer with rules like this: TRUE : 'true'; IDENTIFIER : 'a'..'z'+; The text true parses as IDENTIFIER. How do I get it to parse as TRUE? Do I need to add not and all my keywords to the IDENTIFIER rule? I don't see anything like that in the java grammar. -- Ryan Brown List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31843] Re: [antlr-interest] Problem with grammar
You need to construct the correct LL tree form of the rules. Always copy
from a pre-written grammar in the examples or read the book and you will not
struggle so much:
factor
: addexp (('+'|'-')* addexp
;
addexp
: mulexp (('*'|'/') mulexp)*
;
mulexp
: '-' mulexp
| atom
;
atom
: NUMBER
| '(' factor ')'
;
Jim
-Original Message-
From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-
boun...@antlr.org] On Behalf Of Wojciech Tomasz Cichon
Sent: Saturday, March 19, 2011 5:19 AM
To: antlr-interest@antlr.org
Subject: [antlr-interest] Problem with grammar
i’m struggling with this grammar for a few days now, and nothing really
working, so i started from beginning and i have :
grammar myGrammar;
options {
language = Java;
k=1;
}
rule: term ;
factor : '-'? NUMBER
;
term : factor '*' term
| factor '/' term
| factor '%' term
| factor;
NUMBER : '0'..'9'+
;
WS : ( ' '
| '\t'
| '\r'
| '\n'
) {$channel=HIDDEN;}
;
and i got errors:
warning(200): /ANTLR_TEST/myGrammar.g:13:7: Decision can match input
such as '-' using multiple alternatives: 1, 2, 3, 4 As a result,
alternative(s) 2,3,4 were disabled for that input
|--- term : factor '*' term
warning(200): /ANTLR_TEST/myGrammar.g:13:7: Decision can match input
such as NUMBER using multiple alternatives: 1, 2, 3, 4 As a result,
alternative(s) 2,3,4 were disabled for that input
|--- term : factor '*' term
error(201): /ANTLR_TEST/myGrammar.g:13:7: The following alternatives
can never be matched: 2,3,4
|--- term : factor '*' term
can anyone tell me what i’m doing wrong
regards
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31844] Re: [antlr-interest] Problem with grammar
On 03/19/2011 08:18 AM, Wojciech Tomasz Cichon wrote:
i’m struggling with this grammar for a few days now, and nothing really
working,
so i started from beginning and i have :
grammar myGrammar;
options {
language = Java;
k=1;
}
rule: term ;
factor : '-'? NUMBER
;
term : factor '*' term
| factor '/' term
| factor '%' term
| factor;
This is the classic case where the following is probably the proper left
factored LL(1) for you:
term : factor ( ( '*' | '/' | '%' ) factor )*
;
Now you have a single factor that will always match at the beginning
of your term rule, and not have to chose which of the 4 choices to
associate a '-' with without changing to k=2 or k=3, or turning on
backtracking.
NUMBER : '0'..'9'+
;
WS : ( ' '
| '\t'
| '\r'
| '\n'
) {$channel=HIDDEN;}
;
and i got errors:
warning(200): /ANTLR_TEST/myGrammar.g:13:7: Decision can match input such as
'-' using multiple alternatives: 1, 2, 3, 4
As a result, alternative(s) 2,3,4 were disabled for that input
|--- term : factor '*' term
It can't tell which of your 4 term alternatives to choose based on a '-'
look-ahead character
warning(200): /ANTLR_TEST/myGrammar.g:13:7: Decision can match input such as
NUMBER using multiple alternatives: 1, 2, 3, 4
As a result, alternative(s) 2,3,4 were disabled for that input
|--- term : factor '*' term
The same here, which of the 4 term alternatives does it choose when the
look-ahead character is a NUMBER?
error(201): /ANTLR_TEST/myGrammar.g:13:7: The following alternatives can
never be matched: 2,3,4
|--- term : factor '*' term
Because it chose the first alternative for the 2 cases above, it will
never try a division, modulus, or simple factor.
can anyone tell me what i’m doing wrong
left factor your grammar
regards
--
Kevin J. Cummings
kjch...@verizon.net
cummi...@kjchome.homeip.net
cummi...@kjc386.framingham.ma.us
Registered Linux User #1232 (http://counter.li.org)
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31845] [antlr-interest] problem with characters in grammar
i decided to extend my grammar for characters and i added: factor : '-'? (NUMBER ) | character ; character : AP (LETTER | NUMBER) AP; AP : '\''; LETTER : 'A'..'Z' | 'a'..'Z'; NUMBER : '0'..'9'+; it's compile with no error which is good, i running program and i’m sending 2+3*3 = 'a' + 2 and that gave me (= (+ 2 (* 3 3)) (+ ' ' 2)) line 1:10 no viable alternative at character 'a' clearly this grammar can’t handle characters can somebody tell me what i’m doing wrong List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31846] Re: [antlr-interest] problem with characters in grammar
On Sat, Mar 19, 2011 at 8:14 PM, Wojciech Tomasz Cichon wtcic...@googlemail.com wrote: i decided to extend my grammar for characters and i added: factor : '-'? (NUMBER ) | character ; character : AP (LETTER | NUMBER) AP; AP : '\''; LETTER : 'A'..'Z' | 'a'..'Z'; There's a capital Z after the `a`.. Try a lower case. Regards, Bart. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31849] Re: [antlr-interest] @scopeinit, can find the post but nothing else.
Hi Eric, In the C# 3 documentation, section 4.4 discusses partial methods called when a scope object is created. http://www.antlr.org/download/CSharp3.pdf Thanks, Sam -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of The Researcher Sent: Thursday, March 03, 2011 2:49 PM To: antlr-interest@antlr.org Subject: Re: [antlr-interest] @scopeinit, can find the post but nothing else. This appears to be on the To Implement list constructors or init for global scopes http://www.antlr.org/wiki/display/ANTLR3/constructors+or+init+for+global+sco pes Eric On Wed, Mar 2, 2011 at 12:26 PM, The Researcher researcher0...@gmail.comwrote: Sam, I saw your post about @scopeinit which I take was related to Trouble with syntactic predicates that reference semantic predicates that rely on a current dynamic attribute scope. Did @scopeinit make it into a released C# version? If so could you point me to how it is used? I can get by without it, but my current solution is a crutch at best. Thanks Eric. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 31850] Re: [antlr-interest] C# tree parser memoize issue?
Hi David,
Is there any way you can send me the full grammar you're using (you could
send it to just me if you don't want to send it to the public list)? I don't
use memoization in any of my grammars, so I don't have a way to test this
feature extensively.
Thanks,
Sam
-Original Message-
From: antlr-interest-boun...@antlr.org
[mailto:antlr-interest-boun...@antlr.org] On Behalf Of David Daeschler
Sent: Thursday, March 03, 2011 1:45 PM
To: antlr-interest@antlr.org
Subject: [antlr-interest] C# tree parser memoize issue?
I have a grammar that contains a 3d vector type.
Parsing the grammar for my test case yields the following AST:
http://graph.gafol.net/dZxqGkwww
http://graph.gafol.net/dZxqGkwwwDuring the code generation step I am using
a tree grammar with backtracking and memoization turned on.
With memoization the following generates an error:
float f;
vector v = 1.0,1.0,1.0 * f,f,f;
^Z
Gen.g: node from line 2:28 mismatched tree node: f expecting FLOAT_LITERAL
Turning off memoization fixes the issue.
The grammar fragments involved are (trying to cut down as much as I can):
expression
@init {MyAst t = (MyAst )input.LT(1);}
@after { $expression.st = DoPromotion(t, $expression.st); }
: ^(EXPR expr) - {$expr.st}
;
expr
: multiplicativeExpression- {$multiplicativeExpression.st} ;
multiplicativeExpression
: mult - {$mult.st}
| primary - {$primary.st}
;
mult
: ^(op='*' l=multiplicativeExpression r=multiplicativeExpression)
-
mul(subtemplate={TemplateMapping.Multiplication[$l.start.evalType.TypeIndex,
$r.start.evalType.TypeIndex]}, lexpr={$l.st}, rexpr={$r.st}) ; primary @init
{MyAst t = (MyAst)input.LT(1);} @after { $primary.st = DoPromotion(t,
$primary.st); }
| vecConst - {$vecConst.st}
| vecLiteral - {$vecLiteral.st}
| expression - {$expression.st}
;
vecConst
: ^(VECTOR_LITERAL x=FLOAT_LITERAL y=FLOAT_LITERAL z=FLOAT_LITERAL) -
vconst(x={$x.text}, y={$y.text}, z={$z.text}) ;
vecLiteral
: ^(VECTOR_LITERAL x=expr y=expr z=expr) - buildvec(x={$x.st}, y={$y.st},
z={$z.st}) ;
Sorry for the verbosity of this post, and thank you ahead of time for any
help.
--
David Daeschler
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
