[il-antlr-interest: 28550] Re: [antlr-interest] Non-determinism (was: Can I force a token to have precendence in the lexer?)
On Wed, Apr 21, 2010 at 2:41 AM, Andy Hull an...@sunrunhome.com wrote:
Wow, thanks for the article. I was able to redefine the language to avoid
the problem in order to keep the parser as simple as possible (now using
to instead of ... ).
My parser needs to be able to handle nested array expressions like so
{1,2,{5 to 10}, {3,6,9}, 4}
I have the following grammar:
arrayExpression
: LEFT_BRACKET! arrayInitializer? RIGHT_BRACKET!;
arrayInitializer
: (e+=expression (',' e+=expression)*)+ - ^(ELEMENTLIST $e*)
| expression AUTO expression - ^(AUTO expression expression)
;
expression
: arrayExpression
/* | other types of expression */
;
with the expected non-LL(*) grammar because arrayInitializer depends on
the recursive rule expression. Setting backtrack to true doesn't resolve
this as I expected.
x={1,2,3,4};
yields the correct tree but...
x={1 to 3};
yields the error:
BR.recoverFromMismatchedToken
line 1:5 mismatched input 'to' expecting RIGHT_BRACKET
arrayInitializer behaves as expected when it contains only a single subrule
(either the element list or the range initializer).
Is backtracking the right solution to the non-determinism? I am doing
something wrong?
How about something like this:
grammar Test;
parse
: array ';' EOF
;
array
: '{' (arrayAtom (',' arrayAtom)*)? '}'
;
arrayAtom
: Number
| array
| range
;
range
: Number 'to' Number
;
Number
: '0'..'9'+
;
Space
: (' ' | '\t' | '\r' | '\n') {skip();}
;
Regards,
Bart.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28551] Re: [antlr-interest] Runtime Library of v 3.2 for C#
Johannes Luber, maintainer of the CSharp2 target, wrote in December 2009: quote I'm working on the 3.2 release. The major issue I'm tackling is to make the APIs of CSharp2 and CSharp3 source compatible and I have to coordinate this with the other maintainer. I'll release a beta once I've finished this. /quote I don't know if it is viable to just pull the sources and compile it yourself. I have not yet tried that. If the changes were trivial, I expect it would have been in beta stage already. Until that time, I will try and keep my patience... Best regards, Ranco Marcus -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Nazim Oztahtaci Sent: dinsdag 20 april 2010 13:42 To: antlr-interest@antlr.org Subject: [antlr-interest] Runtime Library of v 3.2 for C# Hello, I am sorry if I repeat this question. I have checked the history of mail list to find out current situation of C# library for v 3.2 Currently I use Java for the implementation of my grammar because when I apply theorems(DeMorgan, Distribution etc) on the tree, BottomUp function is called to apply them. However in C# v.3.1.3 there is no support and I guess I need version 3.2 library. I read that version 3.2 for C# can be found in repository. Is that true? I just wanted to confirm that. I assume that to use BottomUp function, it is necessary to have TreeFilter class. If I cant use C# v 3.2 library yet and I have to implement my own TreeFilter class which has BottomUp function, which files in C# version 3.1.3 source code I should make changes?Is it too complex to implement just bottomup function manually? If it is, is there any implementation of C# v 3.2 runtime dlls on the Net for Antlr? BEst regards __ ___ Yeni Windows 7: Gündelik işlerinizi basitleştirin. Size en uygun bilgisayarı bulun. http://windows.microsoft.com/shop List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28552] Re: [antlr-interest] Runtime Library of v 3.2 for C#
Actually I have compiled yesterday but I get errors when I use CSharp2 version, regarding overridable set accessors in Antlr.Runtime.RuleReturnScope class. I did not dig in yet in source code. Actually all I need is tree filter class. Maybe I can just use that part in my source code and´use CSharp v 3.1.3 instead. But I assume there is no schedule or exact time for the release of library yet. Best regards From: ranco.mar...@epirion.nl To: nazim_oztaht...@hotmail.com; antlr-interest@antlr.org Subject: RE: [antlr-interest] Runtime Library of v 3.2 for C# Date: Wed, 21 Apr 2010 06:40:00 + Johannes Luber, maintainer of the CSharp2 target, wrote in December 2009: quote I'm working on the 3.2 release. The major issue I'm tackling is to make the APIs of CSharp2 and CSharp3 source compatible and I have to coordinate this with the other maintainer. I'll release a beta once I've finished this. /quote I don't know if it is viable to just pull the sources and compile it yourself. I have not yet tried that. If the changes were trivial, I expect it would have been in beta stage already. Until that time, I will try and keep my patience... Best regards, Ranco Marcus -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Nazim Oztahtaci Sent: dinsdag 20 april 2010 13:42 To: antlr-interest@antlr.org Subject: [antlr-interest] Runtime Library of v 3.2 for C# Hello, I am sorry if I repeat this question. I have checked the history of mail list to find out current situation of C# library for v 3.2 Currently I use Java for the implementation of my grammar because when I apply theorems(DeMorgan, Distribution etc) on the tree, BottomUp function is called to apply them. However in C# v.3.1.3 there is no support and I guess I need version 3.2 library. I read that version 3.2 for C# can be found in repository. Is that true? I just wanted to confirm that. I assume that to use BottomUp function, it is necessary to have TreeFilter class. If I cant use C# v 3.2 library yet and I have to implement my own TreeFilter class which has BottomUp function, which files in C# version 3.1.3 source code I should make changes?Is it too complex to implement just bottomup function manually? If it is, is there any implementation of C# v 3.2 runtime dlls on the Net for Antlr? BEst regards __ ___ Yeni Windows 7: Gündelik işlerinizi basitleştirin. Size en uygun bilgisayarı bulun. http://windows.microsoft.com/shop List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address _ Windows Live: Arkadaşlarınız size e-posta gönderdiklerinde Flickr, Twitter ve Digg'deki hareketlerinizi görürler. http://www.microsoft.com/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:tr-tr:SI_SB_3:092010 List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28553] [antlr-interest] help debegger ANTLRWorks
I have try an other grammar, all is well, without when I makes the debugger it shows me: cannot launch the debugger. Time-out waiting to connect to the remote parser Or sometimes it shows me: Compiler exception: java.io.IOException: Cannot run programjavac: CreateProcess errors=2, Le fichier spécifié est introuvable What is the wrong?? An other question please , when I use ANTLRWorks it is necessary to install ANTLR and how? Just add ANTLR.jar in Classpath? Thanks for your help. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28555] Re: [antlr-interest] mild simplification and tree grammars
Another extension of the example shows another ANTLR feature,
with associated subtle syntax. Say the conditionalExpression
was being called with TheActualASTRoot being optional, then
ANTLR supports this with semantic predicates on the tree generation
options:
protected
conditionalExpression[CommonTree TheActualASTRoot]
: QMARK t=expression COMMA f=expression
- {(TheActualASTRoot != null)}? ^(ITE {$TheActualASTRoot}
$t $f)
- {(TheActualASTRoot == null)}? ^(ITE $t $f)
-
|
;
Nice.
I didn't get why you were telling me that, but now I see: this way it is
easier to avoid non-LLR notations.
Just, I can't turn on my shc (SomethingHasChanged) flag anymore: a rule
like:
condExpr
:QMARK c=orExpression t=condExpr f=condExpr
- {($c.tree.getType()==TRUE)}? {shc=true;} $t
- {($c.tree.getType()==FALSE)}? {shc=true;} $f
-
;
Produces a java source with errors, because {shc=true;} is now interpreted
like a tree node reference, not like a java statement to be passed to the
compiler verbatim.
I had to use a function's side-effect to keep track of tree modifications:
@members {
private boolean shc = false;
private boolean sshc(boolean cond) {
shc |= cond;
return(cond);
}
}
...
protected
condExpr
: QMARK c=orExpression t=condExpr f=condExpr
- {sshc($c.tree.getType()==TRUE)}? $t
- {sshc($c.tree.getType()==FALSE)}?$f
- QMARK $c $t $f
| additiveExpression
;
Also note the last '-' term is non-empty, because I'm not rewriting (the
ANTLR compiler says rewrite mode implies backtrack=true).
Do you think is it fine this way or there is some workaround I can implement
to avoid the need of a side-effect function?
Giampaolo
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28556] Re: [antlr-interest] mild simplification and tree grammars
On Wed, Apr 21, 2010 at 12:32:58PM +0200, Giampaolo Tomassoni wrote:
Another extension of the example shows another ANTLR feature,
with associated subtle syntax. Say the conditionalExpression
was being called with TheActualASTRoot being optional, then
ANTLR supports this with semantic predicates on the tree generation
options:
protected
conditionalExpression[CommonTree TheActualASTRoot]
: QMARK t=expression COMMA f=expression
- {(TheActualASTRoot != null)}? ^(ITE {$TheActualASTRoot}
$t $f)
- {(TheActualASTRoot == null)}? ^(ITE $t $f)
-
|
;
Nice.
I didn't get why you were telling me that, but now I see: this way it is
easier to avoid non-LLR notations.
Just, I can't turn on my shc (SomethingHasChanged) flag anymore: a rule
like:
condExpr
:QMARK c=orExpression t=condExpr f=condExpr
- {($c.tree.getType()==TRUE)}? {shc=true;} $t
- {($c.tree.getType()==FALSE)}? {shc=true;} $f
-
;
Produces a java source with errors, because {shc=true;} is now interpreted
like a tree node reference, not like a java statement to be passed to the
compiler verbatim.
I had to use a function's side-effect to keep track of tree modifications:
@members {
private boolean shc = false;
private boolean sshc(boolean cond) {
shc |= cond;
return(cond);
}
}
...
protected
condExpr
: QMARK c=orExpression t=condExpr f=condExpr
- {sshc($c.tree.getType()==TRUE)}? $t
- {sshc($c.tree.getType()==FALSE)}?$f
- QMARK $c $t $f
| additiveExpression
;
Also note the last '-' term is non-empty, because I'm not rewriting (the
ANTLR compiler says rewrite mode implies backtrack=true).
Do you think is it fine this way or there is some workaround I can implement
to avoid the need of a side-effect function?
Giampaolo
Predicates should not have observable* side effects.
The solution seems simple: the actions should be called before the
tree rewrites, like:
condExpr
:QMARK c=orExpression t=condExpr f=condExpr
{shc=true;}
- {($c.tree.getType()==TRUE)}? $t
- {($c.tree.getType()==FALSE)}? $f
-
;
Regards, Mark
PS * since disambiguating semantic predicates may be called tens or
hundreds of times at the same input file position in complex
predicated parsers by antlr generated code, for performance
reasons it may be necessary for complex semantic predicates to calculate the
result once at each input file position, and cache the result, to
avoid re-computing it over and over again.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28557] Re: [antlr-interest] mild simplification and tree grammars
Do you think is it fine this way or there is some workaround I can
implement
to avoid the need of a side-effect function?
Giampaolo
Predicates should not have observable* side effects.
The solution seems simple: the actions should be called before the
tree rewrites, like:
condExpr
:QMARK c=orExpression t=condExpr f=condExpr
{shc=true;}
- {($c.tree.getType()==TRUE)}? $t
- {($c.tree.getType()==FALSE)}? $f
-
;
Regards, Mark
Well, I attempted that, but then the shc var is set true even when there are
no rewrites (the empty rewrite case in your example). This is probably
because the disambiguating predicates are on the right side of the rewrite
operator, while the {shc=true;} seems an action to me. So, it is of course
to its left.
I don't see a way to put an action to the right side of the rewrite
operator, since a {...} notation would be interpreted like a tree reference,
if I understand it right.
Maybe all this means there is space for an enhancement in ANTLR? ;)
Giampaolo
PS * since disambiguating semantic predicates may be called tens or
hundreds of times at the same input file position in complex
predicated parsers by antlr generated code, for performance
reasons it may be necessary for complex semantic predicates to
calculate the
result once at each input file position, and cache the result, to
avoid re-computing it over and over again.
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28559] Re: [antlr-interest] parser runtime error listeners
The parser itself could register a listener, which calls reportError. -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest-boun...@antlr.org] On Behalf Of Terence Parr Sent: April 20, 2010 5:45 PM To: ANTLR interest Subject: [antlr-interest] parser runtime error listeners Hiya. GeraldHas brought up an excellent point about our runtime. Currently, parsing errors get reported through method reportError. To do something different, you have to override that method. Gerald is suggesting that we move to an error listener mechanism so that multiple listeners can attach to a running parser and receive error events (kind of like what we do for debugging events). I'm all for this idea, but what about backward compatibility? a lot of people have overridden reportError in their parsers or tree parsers (like me) and we want that to continue to work. How do we integrate this with a listener mechanism? Also, Gerald points out that even if someone overrides that method, his tool and other tools for integration with eclipse need to get those errors regardless of whether someone has overridden reportError. Otherwise, the ANTLR IDE couldn't get syntax error messages when it ran. He suggested Preferably, the listener interface would be above anything that the end user could override. Stuff that now calls reportError should instead call some private reportError_intern, which fires the event listeners and then calls the public/overrideable reportError. any comments on the approach or what we want? The minute I get v4 running in v3, I want to turn around and have v4 generate its own parsers. Backward compatibility will also be important for me. Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28560] [antlr-interest] Decision can match input using multiple alternatives
I am trying to implement a LOGO grammar, but the langage is a little bit
ambiguous and lacks several parenthesis usage.
Here is my expr grammar:
or : and (OR^ and)*;
and : eq (AND^ eq)*;
eq : rel ((EQ|NOTEQ)^ rel)*;
rel : add ((LT|GT|LTEQ|GTEQ)^ add)*;
add : mul ((PLUS|MINUS)^ mul)*;
mul : atom ((MULT|DIV|MOD)^ atom)*;
atom : INT | FUNCTION_ID ({needMoreArgs()}?or)*;
Everything goes well until I add function call grammar signaled by
FUNCTION_ID and the list of args
And here are my warnings:
[19:09:37] warning(200): Logo.g:215:18: Decision can match input such as
OR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[19:09:37] warning(200): Logo.g:216:18: Decision can match input such as
AND using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[19:09:37] warning(200): Logo.g:217:26: Decision can match input such as
EQ..NOTEQ using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[19:09:37] warning(200): Logo.g:218:34: Decision can match input such as
LT..GTEQ using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[19:09:37] warning(200): Logo.g:219:29: Decision can match input such as
PLUS..MINUS using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
[19:09:37] warning(200): Logo.g:220:33: Decision can match input such as
MULT..MOD using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address
--
You received this message because you are subscribed to the Google Groups
il-antlr-interest group.
To post to this group, send email to il-antlr-inter...@googlegroups.com.
To unsubscribe from this group, send email to
il-antlr-interest+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/il-antlr-interest?hl=en.
