[il-antlr-interest: 32030] [antlr-interest] Fwd: F77 grammar
Hi, I have problem with running this grammar (downloaded from ANTLR website). Someone familiar with it? Marcin List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address f77.g Description: Binary data -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32031] Re: [antlr-interest] Fwd: F77 grammar
On Thu, Mar 31, 2011 at 9:24 AM, piet...@agh.edu.pl wrote: Hi, I have problem with running this grammar (downloaded from ANTLR website). Someone familiar with it? I'm not, but the first line of the grammar says: *Fortran 77 grammar for ANTLR 2.7.5. *So, my question to you is: are you uainf ANTLR v2.7? ANTLR 3.x will not cope with the syntax from that grammar. If you *are* using version 2.7, I suggest you explain your actual problem. Regards, Bart. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32032] Re: [antlr-interest] Fwd: F77 grammar
You probably want to check out http://fortran-parser.sourceforge.net/. --Loring - Original Message From: piet...@agh.edu.pl piet...@agh.edu.pl To: antlr-interest@antlr.org Sent: Thu, March 31, 2011 12:24:19 AM Subject: [antlr-interest] Fwd: F77 grammar Hi, I have problem with running this grammar (downloaded from ANTLR website). Someone familiar with it? Marcin List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32035] Re: [antlr-interest] Release of StringTemplate 4.0
Shall I create the Maven build or are you throwing Maven out? ;-) Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Terence Parr Sent: Sunday, March 27, 2011 12:55 PM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: [antlr-interest] Release of StringTemplate 4.0 Get it while it's hot. 4.0 is a complete rewrite as a bytecode interpreter. Is much cleaner and faster than 3.0. enjoy. http://www.stringtemplate.org/download.html http://www.antlr.org/wiki/display/ST4/StringTemplate+4+Documentation http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4 The GUI Visualizer / inspector is very useful; check it out: http://www.antlr.org/wiki/display/ST4/StringTemplate+Inspector+GUI Ports to C#, Python, Objective-C on the way thanks to Sam Harwell, Benjamin Niemann, and Alan Condit. A big thanks to Udo Borkowski for his incredible bug catching abilities and his suggestions! Terence ANTLR project supreme dictator for life List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32036] Re: [antlr-interest] Fwd: F77 grammar
You are going to have to ask a better question than that. I have problems with VCU screwing my bracket - any ideas? Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of piet...@agh.edu.pl Sent: Thursday, March 31, 2011 12:24 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Fwd: F77 grammar Hi, I have problem with running this grammar (downloaded from ANTLR website). Someone familiar with it? Marcin List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32037] [antlr-interest] Problem and commentary on semantic predicates and prediction
I'm trying to use semantic predicates in a lexer. I'll try to explain everything as I walk through the things I tried. I started with this: TEXT : ( ~('}'|''|'\n'|'\r'|'\\'|''|'') | '\\' . | {AnonymousTemplateLevel==0}? '}' | {Outermost!=String}? '' | {Outermost!=BigString || input.LA(2)!=''}? '' )+ ; The TEXT token continues to consume characters until the end of the TEXT section is complete. Depending on context (stored as fields in the lexer), the text section could end with '}', '', or ''. When I used the above form of the rule, the prediction section of mTokens() never evaluated my predicates, so when the next character of the input stream was any of '}', '', or '', it would predict the TEXT rule. Here's a shortened form of the prediction. For simplicity, I'll use (x in SetA) as syntax for x in ~('}'|''|'\n'|'\r'|''|''), which is a character from the first alt of TEXT or the '\' character of the second alt. I'll use SetB for ~(''|'\n'|'\r'), which is any possible character in TEXT without considering the predicates. if ((LA5_0 in SetB)) { alt5 = 5; // TEXT } I realized shortly after that no other rule in the lexer could start with these characters, so by process of elimination I supposed this kind of prediction is a reasonable result. I tried to correct the situation by adding the following rule after TEXT: ANYCHAR : . ; This changed the prediction in mTokens, but certainly didn't fix it. if ((LA5_0=='\')) { int LA5_6 = input.LA(2); if (LA5_6 in SetB) { alt5 = 5; } else if (((Outermost!=String))) { alt5 = 5; } else if ((true)) { alt5 = 6; } } Next, looking at the ACyclicDFACodeGenerator in the tool, I noticed that predicates are only emitted in the prediction phase if they are gated (assigned to the predicates attribute of the dfaEdge template in WalkFixedDFAGeneratingStateMachine). So I added gates to my predicates. In this case, the prediction is correct, but the Boolean expression is expressed as ((A B) || B), which reduces to simply (B). In this case, the expense of computing B (a semantic predicate) has no bearing on the result because there is no way to avoid evaluating it. if ((LA5_0=='\')) { int LA5_6 = input.LA(2); if ((LA5_6 in SetB) ((Outermost!=String))) { alt5 = 5; } else if (((Outermost!=String))) { alt5 = 5; } else if ((true)) { alt5 = 6; } } I originally didn't see the correctness of this test, so I went on to try removing the ANYCHAR rule while keeping the gates on the predicates. The results are interesting, so I'll note them here. The result in mTokens looked like this: ... } else if (LA5_0 in SetA) { alt5 = 5; } else if ((LA5_0=='}') ((AnonymousTemplateLevel==0))) { alt5 = 5; } else if ((LA5_0=='\') ((Outermost!=String))) { alt5 = 5; } else if ((LA5_0=='') ((Outermost!=BigString || input.LA(2)!=''))) { alt5 = 5; } else ... In this case I observed the following: 1. The LL(1) prediction that now appears is faster. 2. This lexer is used as a syntax highlighting component, where removing the ANYCHAR rule can lead to substantial performance degradation. However, since the Tokens rule is implicitly created by the grammar compiler, I have no way to specify that the decision on TEXT can/should be treated as LL(1). 3. The check (LA5_0 in SetA) is overly restrictive because an conditions check for (LA_5 in (''|'\n'|'\r')). Relaxing the test reduces the number of comparisons in the false case (say (LA5_0=='{')) from 14 to 8. 4. The test (LA5_0 in SetB) expands to the following. Edit here: I meant to show this for SetA, but since I'll accidentally used SetB, we'll go with that. ((LA5_5='\u' LA5_5='\t')||(LA5_5='\u000B' LA5_5='\f')||(LA5_5='\u000E' LA5_5=';')||(LA5_5='=' LA5_5='\u')) A simple transform (moving a few parenthesis) of this already ordered expression gives: ((LA5_5 = '\u' (LA5_5 = '\t' || (LA5_5 = '\u000B' (LA5_5 = '\f' || (LA5_5 = '\u000E' (LA5_5 = ';' || (LA5_5 = '=' LA5_5 = '\u' Which reduces the number of comparisons from 5 to 3 for eliminating '\n', 6 to 5 for eliminating '\r', and remains at 7 for eliminating '}'. For sets with a very large number of ranges such as the ID start character in the Java grammar, the time savings for this change could be tremendous considering nearly all identifiers start with a character in the range 0..127 which is tested in the early portion of the expression. Thanks, Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this
[il-antlr-interest: 32038] Re: [antlr-interest] [stringtemplate-interest] Release of StringTemplate 4.0
I already created one (attached) if you want to use it as a start point. :) It uses build-helper-maven-plugin to avoid having to restructure the directories in P4. Sam -Original Message- From: stringtemplate-interest-boun...@antlr.org [mailto:stringtemplate-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: Thursday, March 31, 2011 8:49 AM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: Re: [stringtemplate-interest] [antlr-interest] Release of StringTemplate 4.0 Shall I create the Maven build or are you throwing Maven out? ;-) Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Terence Parr Sent: Sunday, March 27, 2011 12:55 PM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: [antlr-interest] Release of StringTemplate 4.0 Get it while it's hot. 4.0 is a complete rewrite as a bytecode interpreter. Is much cleaner and faster than 3.0. enjoy. http://www.stringtemplate.org/download.html http://www.antlr.org/wiki/display/ST4/StringTemplate+4+Documentation http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4 The GUI Visualizer / inspector is very useful; check it out: http://www.antlr.org/wiki/display/ST4/StringTemplate+Inspector+GUI Ports to C#, Python, Objective-C on the way thanks to Sam Harwell, Benjamin Niemann, and Alan Condit. A big thanks to Udo Borkowski for his incredible bug catching abilities and his suggestions! Terence ANTLR project supreme dictator for life List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address ___ stringtemplate-interest mailing list stringtemplate-inter...@antlr.org http://www.antlr.org/mailman/listinfo/stringtemplate-interest List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address stringtemplate.config Description: XML document -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32039] Re: [antlr-interest] [stringtemplate-interest] Release of StringTemplate 4.0
Cheers, it saves me configuring build helper. However, it isn't so much that, as whether we are supporting Maven philosophically or not. If Ter does not want to build with it, then I will be forever playing catch up and wrangling builds together. The build helper is fine but then we are already heading away from the structure etc. I like Maven a lot, but you have to buy in to it really. So, I wonder if we don't just create the artifacts directly from the Jars the Ter builds and deploy them. We lose the jar with dependencies though, which is a shame. So, if we could decide not to randomly change the build around and add weird scripts to ANT, then I can keep it going. But even keeping the plugin going is going to be a pain if I have to keep deploying the jars, and then because we are not eating our own dog food, we will keep breaking the plugin by changing the options without changing the plugin and so on etc... However, we should move to maven 3 for version 4 of ST and ANTLR if we are going to keep going. Jim -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Thursday, March 31, 2011 8:21 AM To: 'Jim Idle'; 'stringtemplate-interest List' Cc: 'antlr-interest Interest' Subject: RE: [stringtemplate-interest] [antlr-interest] Release of StringTemplate 4.0 I already created one (attached) if you want to use it as a start point. :) It uses build-helper-maven-plugin to avoid having to restructure the directories in P4. Sam -Original Message- From: stringtemplate-interest-boun...@antlr.org [mailto:stringtemplate-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: Thursday, March 31, 2011 8:49 AM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: Re: [stringtemplate-interest] [antlr-interest] Release of StringTemplate 4.0 Shall I create the Maven build or are you throwing Maven out? ;-) Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Terence Parr Sent: Sunday, March 27, 2011 12:55 PM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: [antlr-interest] Release of StringTemplate 4.0 Get it while it's hot. 4.0 is a complete rewrite as a bytecode interpreter. Is much cleaner and faster than 3.0. enjoy. http://www.stringtemplate.org/download.html http://www.antlr.org/wiki/display/ST4/StringTemplate+4+Documentation http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4 The GUI Visualizer / inspector is very useful; check it out: http://www.antlr.org/wiki/display/ST4/StringTemplate+Inspector+GUI Ports to C#, Python, Objective-C on the way thanks to Sam Harwell, Benjamin Niemann, and Alan Condit. A big thanks to Udo Borkowski for his incredible bug catching abilities and his suggestions! Terence ANTLR project supreme dictator for life List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address ___ stringtemplate-interest mailing list stringtemplate-inter...@antlr.org http://www.antlr.org/mailman/listinfo/stringtemplate-interest List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32040] Re: [antlr-interest] Problem and commentary on semantic predicates and prediction
By reordering on binary search basis, the number of comparisons becomes 3 for '\n', 3 for '\r', and 3 for ''. In other words, the way comparisons are expressed even for a set with only 4 sub-ranges results in 7 comparisons for characters where only 3 are actually needed. This is O(log n) instead of O(n), so as the interval set gets more ranges the discrepancy quickly grows - for 8 sub-ranges used in testing SetA from before, 14 comparisons are used where only 4 are needed. (LA5_0 = '\f' (LA5_0 = '\u000B' || (LA5_0 = '\t' LA5_0 = '\u'))) || ((LA5_0 = ';' LA5_0 = '\u000E') || (LA5_0 = '=' LA5_0 = '\u')) Sam -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Thursday, March 31, 2011 10:15 AM To: antlr-interest@antlr.org Subject: Problem and commentary on semantic predicates and prediction 4. The test (LA5_0 in SetB) expands to the following. Edit here: I meant to show this for SetA, but since I'll accidentally used SetB, we'll go with that. ((LA5_5='\u' LA5_5='\t')||(LA5_5='\u000B' LA5_5='\f')||(LA5_5='\u000E' LA5_5=';')||(LA5_5='=' LA5_5='\u')) A simple transform (moving a few parenthesis) of this already ordered expression gives: ((LA5_5 = '\u' (LA5_5 = '\t' || (LA5_5 = '\u000B' (LA5_5 = '\f' || (LA5_5 = '\u000E' (LA5_5 = ';' || (LA5_5 = '=' LA5_5 = '\u' Which reduces the number of comparisons from 5 to 3 for eliminating '\n', 6 to 5 for eliminating '\r', and remains at 7 for eliminating ''. For sets with a very large number of ranges such as the ID start character in the Java grammar, the time savings for this change could be tremendous considering nearly all identifiers start with a character in the range 0..127 which is tested in the early portion of the expression. Thanks, Sam List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32041] [antlr-interest] Imaginary node modifies tree unexpectedly. Bug, Grrr, or not what I expected?
Imaginary node modifies tree unexpectedly. Bug, Grrr, or not what I expected? In the following grammar, rule a invokes rule b twice sequentially. The first b is captured as b1, and the second b is captured as b2. If rule b does not create an imaginary token, then both the trees for b1 and b2 are created as expected. However, if rule b creates an imaginary token, then tree b1 is created as expected at first. When b2 is created, b1 is changed and now holds the value of both b1 and b2, when only b1 is expected. grammar InvalidTreeBuild001; options { language = 'CSharp2'; output = AST; } // Imaginary Tokens tokens { I1; I2; } a : b1=b b2=b ; // No imaginary created //b // : IntegerDigit - ^(I1 IntegerDigit) // ; // Imaginary created b : IntegerDigit - ^(I1 IntegerDigit) ^(I2 IntegerDigit[0]) ; WS : (' ' | '\r' | '\t' | '\u000C' | '\n') {$channel=HIDDEN;} ; IntegerDigit : ('0'..'9') ; = Input 1 2 - For No imaginary created Before b2=b b1 = (I1 1) b2 = nil After b2=b b1 = (I1 1) b2 = (I1 2) - For Imaginary created Before b2=b b1 1 = (I1 1) (I2 0) b2 = nil After b2=b b1 = (I1 1) (I2 0) (I1 2) (I2 0) - Why did (I1 2) (I2 0) get added to b1? b2 = (I1 2) (I2 0) List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32042] Re: [antlr-interest] [stringtemplate-interest] Release of StringTemplate 4.0
Hello all, I'm following this thread with interest and keeping my fingers crossed that this isn't the start of the end for Maven support. Michael On 1 April 2011 02:44, Jim Idle j...@temporal-wave.com wrote: Cheers, it saves me configuring build helper. However, it isn't so much that, as whether we are supporting Maven philosophically or not. If Ter does not want to build with it, then I will be forever playing catch up and wrangling builds together. The build helper is fine but then we are already heading away from the structure etc. I like Maven a lot, but you have to buy in to it really. So, I wonder if we don't just create the artifacts directly from the Jars the Ter builds and deploy them. We lose the jar with dependencies though, which is a shame. So, if we could decide not to randomly change the build around and add weird scripts to ANT, then I can keep it going. But even keeping the plugin going is going to be a pain if I have to keep deploying the jars, and then because we are not eating our own dog food, we will keep breaking the plugin by changing the options without changing the plugin and so on etc... However, we should move to maven 3 for version 4 of ST and ANTLR if we are going to keep going. Jim -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Thursday, March 31, 2011 8:21 AM To: 'Jim Idle'; 'stringtemplate-interest List' Cc: 'antlr-interest Interest' Subject: RE: [stringtemplate-interest] [antlr-interest] Release of StringTemplate 4.0 I already created one (attached) if you want to use it as a start point. :) It uses build-helper-maven-plugin to avoid having to restructure the directories in P4. Sam -Original Message- From: stringtemplate-interest-boun...@antlr.org [mailto:stringtemplate-interest-boun...@antlr.org] On Behalf Of Jim Idle Sent: Thursday, March 31, 2011 8:49 AM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: Re: [stringtemplate-interest] [antlr-interest] Release of StringTemplate 4.0 Shall I create the Maven build or are you throwing Maven out? ;-) Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Terence Parr Sent: Sunday, March 27, 2011 12:55 PM To: stringtemplate-interest List Cc: antlr-interest Interest Subject: [antlr-interest] Release of StringTemplate 4.0 Get it while it's hot. 4.0 is a complete rewrite as a bytecode interpreter. Is much cleaner and faster than 3.0. enjoy. http://www.stringtemplate.org/download.html http://www.antlr.org/wiki/display/ST4/StringTemplate+4+Documentation http://www.antlr.org/wiki/display/ST4/Differences+between+v3+and+v4 The GUI Visualizer / inspector is very useful; check it out: http://www.antlr.org/wiki/display/ST4/StringTemplate+Inspector+GUI Ports to C#, Python, Objective-C on the way thanks to Sam Harwell, Benjamin Niemann, and Alan Condit. A big thanks to Udo Borkowski for his incredible bug catching abilities and his suggestions! Terence ANTLR project supreme dictator for life List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your- email-address ___ stringtemplate-interest mailing list stringtemplate-inter...@antlr.org http://www.antlr.org/mailman/listinfo/stringtemplate-interest List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.