[il-antlr-interest: 32682] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
Yes, that sounds like a good idea. I would definitely be in favour of merging the grammars before generating the code. My only concern is that this approach would deviate from the general ANTLR approach. In general, I have found ANTLR to be a great tool for parser generation, but never really liked the way target specific actions are mixed with the grammar definition. Ideally, I would like my grammars to be _completely_ target agnostic (no actions, no visibility modifiers, members, headers, superClass definitions, etc.) and have an abstract mechanism that we can use to hookup actions and implementation specific stuff to the generated grammar. Do you know if there are plans to redesign the composite grammar feature in v4? Best regards, Ranco -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Sunday, May 29, 2011 11:08 AM To: Ranco Marcus; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars I'm not going to be able to address this issue until the second week of June. That said, it seems the best way to handle all these issues with delegate grammars is to inline their rules before code generation. Suppose you have grammar C importing A and B, and you also have D importing A and B. The code generation will result in classes C, C_A, C_B, D, D_A, and D_B. Clearly the independent generation of C_A and D_A during code generation does not allow a single instance of the imported A grammar to be shared by C and D. If we instead flatten the imported grammar hierarchy and only generate classes C and D, then everything behaves like it was written in a single grammar. Do you see any immediate problems with this potential approach? -Original Message- From: Ranco Marcus [mailto:ranco.mar...@epirion.nl] Sent: Wednesday, May 25, 2011 4:26 PM To: Sam Harwell; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi Sam/all, When a (tree) grammar C imports (tree) grammars A and B, where grammar A calls a rule R from grammar B, a call is being made from delegate parser C_A to a delegate rule R (targeting C_B) in the composite parser C (its parent). Now that the visibility of the delegate rules in C match the visibility of the imported grammar, the rule R has to be made public for the above to work. In our grammars, we build up internal structures that are subsequently processed. In our case, that means that all those internal structures have to be made public as well. This could be solved by allowing ANTLR rules to have 'internal' visibility. Also, imported grammars can probably remain internal as well. What are your thoughts on this? Thanks, Ranco -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: Thursday, April 07, 2011 3:57 PM To: Ranco Marcus; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars The visibility of delegated rules now matches the way they're declared in the imported grammar. Since some of them return nested classes declared in the delegate grammar, I had to make the delegate grammar 'public' and all its rules 'internal'. I fixed the reserved names issue right after releasing 3.3.1, so it'll be included when I release 3.3.2. Note that rule parameters and return values need to include an explicit '@' in their declarations in the grammar, as follows. Labels and rule names do not need the '@'. namespace[int @in] returns [int @out] : as=A B; Sam -Original Message- From: Ranco Marcus [mailto:ranco.mar...@epirion.nl] Sent: Thursday, April 07, 2011 6:49 AM To: Sam Harwell; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Sounds great, Sam! What will be the visibility of the methods in the 'Delegated rules' region now that the class of the imported grammar is internal? Private or internal? Another small suggestion is to prefix variable names (at least those that are reserved in C#) with an at-sign. The generated class for the grammar below has a compile error since 'as' is not a valid variable name while '@as' is. name : as=firstName lastName; Are you planning to release new binaries any time soon? Thanks again, Ranco Marcus -Original Message- From: Sam Harwell [mailto:sharw...@pixelminegames.com] Sent: woensdag 6 april 2011 17:20 To: Ranco Marcus; antlr-interest@antlr.org Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite grammars Hi Ranco, I made the following changes so far: * An imported grammar is always generated as an internal class instead of a public class. * Rules within the imported grammar are always declared public since they are only called by the root grammar. How does that
[il-antlr-interest: 32683] [antlr-interest] AST with optional parameters
I'm parsing a grammar in which the semicolon on the end of a line is optional. So two of the statement rules might be: | (ID GETS expr SEMI) = ID GETS expr SEMI - ^(GETS ID expr SEMI) | (ID GETS) = ID GETS expr - ^(GETS ID expr) Since this occurs with a number of different assignment statements, I would really like to collapse this into one rule that looks something like this: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s) but every implementation I can think of either refuses to generate the grammar or, as in the case above, generates the grammar but decides that the variable 's' is unknown. Is there a any way to achieve this? DMS David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR-33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32684] Re: [antlr-interest] AST with optional parameters
Greetings! On Tue, 2011-06-07 at 07:41 -0400, David Smith wrote: I'm parsing a grammar in which the semicolon on the end of a line is optional. So two of the statement rules might be: | (ID GETS expr SEMI) = ID GETS expr SEMI - ^(GETS ID expr SEMI) | (ID GETS) = ID GETS expr - ^(GETS ID expr) Since this occurs with a number of different assignment statements, I would really like to collapse this into one rule that looks something like this: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s) but every implementation I can think of either refuses to generate the grammar or, as in the case above, generates the grammar but decides that the variable 's' is unknown. Is there a any way to achieve this? Have you tried: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s?) is the semicolon really necessary in the tree? (e.g. does the presence of the semicolon actually change the meaning of of the statement?) perhaps: | (ID GETS) = ID GETS^ expr SEMI!? Hope this helps... -jbb List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32685] Re: [antlr-interest] AST with optional parameters
Hi David, Try this: (ID GETS) = ID GETS expr SEMI? - ^(GETS ID expr SEMI?) Regards, Bart. On Tue, Jun 7, 2011 at 1:41 PM, David Smith david.sm...@cc.gatech.eduwrote: I'm parsing a grammar in which the semicolon on the end of a line is optional. So two of the statement rules might be: | (ID GETS expr SEMI) = ID GETS expr SEMI - ^(GETS ID expr SEMI) | (ID GETS) = ID GETS expr - ^(GETS ID expr) Since this occurs with a number of different assignment statements, I would really like to collapse this into one rule that looks something like this: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s) but every implementation I can think of either refuses to generate the grammar or, as in the case above, generates the grammar but decides that the variable 's' is unknown. Is there a any way to achieve this? DMS David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR-33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32691] Re: [antlr-interest] AST with optional parameters
Yes, the language is Matlab and a semicolon on the end of an assignment expression suppresses display of the result of the assignment. Bart, I appreciate the suggestion; I haven't tried that (much simpler) solution - I'll let you know. DMS At 08:26 AM 6/7/2011, Bart Kiers wrote: Hi John, Have you tried: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s?) is the semicolon really necessary in the tree? (e.g. does the presence of the semicolon actually change the meaning of of the statement?) Exactly my thought after I sent my previous e-mail! perhaps: | (ID GETS) = ID GETS^ expr SEMI!? Or the equivalent rewrite rule: (ID GETS) = ID GETS expr SEMI? - ^(GETS ID expr) Regards, Bart. Hope this helps... -jbb List: http://www.antlr.org/mailman/listinfo/antlr-interesthttp://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-addresshttp://www.antlr.org/mailman/options/antlr-interest/your-email-address David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR-33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32692] Re: [antlr-interest] AST with optional parameters
On Tue, 2011-06-07 at 10:33 -0400, David Smith wrote: Yes, the language is Matlab and a semicolon on the end of an assignment expression suppresses display of the result of the assignment. Bart, I appreciate the suggestion; I haven't tried that (much simpler) solution - I'll let you know. you didn't ask about this but may i suggest an alternative tree form in this case. i believe you said that the optional semicolon occurs on many statements (i am not that familiar with Matlab so sorry if this is obvious). so rather than having the semicolon on every statement in the tree forcing the tree processor to check for it on every statement; make the display operator explicit in the tree at parse-time. something like ... | (ID GETS) = gets ... ; gets : (ID GETS expr - ^(GETS ID expr)) (SEMI - ^(DISPLAY[$SEMI] gets))? ; where DISPLAY is an imaginary token (i may have the meta-syntax wrong but i hope you get the idea). now you have separated the notion of assignment from the notion of displaying a result. and thereby, i suspect, life becomes easier. Just my unsolicited 2 cents worth... -jbb At 08:26 AM 6/7/2011, Bart Kiers wrote: Hi John, Have you tried: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s?) is the semicolon really necessary in the tree? (e.g. does the presence of the semicolon actually change the meaning of of the statement?) Exactly my thought after I sent my previous e-mail! perhaps: | (ID GETS) = ID GETS^ expr SEMI!? Or the equivalent rewrite rule: (ID GETS) = ID GETS expr SEMI? - ^(GETS ID expr) Regards, Bart. Hope this helps... -jbb List: http://www.antlr.org/mailman/listinfo/antlr-interesthttp://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-addresshttp://www.antlr.org/mailman/options/antlr-interest/your-email-address David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR-33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32694] Re: [antlr-interest] [CSharp3] rule visibility in composite grammars
2 months ago I submitted a feature request for an Antlr built-in symbol
table to support the common
requirements of the majority of block-structured languages. By making the
SymbolTable part of the
Antlr grammar language the interface can be much cleaner. The implementation
of the SymbolTable
classes would be part of the target runtime(s).
i.e.
new_variable_name:
(Identifier.IsNewSymbol()) = Identifier.AddSymbol();
SymbolTable - NameSpace - SymbolScope - Symbol - Attribute-List {
optional for structs - NameSpace }
On Tue, Jun 7, 2011 at 5:56 AM, Ranco Marcus ranco.mar...@epirion.nlwrote:
Yes, that sounds like a good idea. I would definitely be in favour of
merging the grammars before generating the code. My only concern is that
this approach would deviate from the general ANTLR approach.
In general, I have found ANTLR to be a great tool for parser generation,
but never really liked the way target specific actions are mixed with the
grammar definition. Ideally, I would like my grammars to be _completely_
target agnostic (no actions, no visibility modifiers, members, headers,
superClass definitions, etc.) and have an abstract mechanism that we can use
to hookup actions and implementation specific stuff to the generated
grammar.
Do you know if there are plans to redesign the composite grammar feature in
v4?
Best regards,
Ranco
-Original Message-
From: Sam Harwell [mailto:sharw...@pixelminegames.com]
Sent: Sunday, May 29, 2011 11:08 AM
To: Ranco Marcus; antlr-interest@antlr.org
Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
I'm not going to be able to address this issue until the second week of
June.
That said, it seems the best way to handle all these issues with delegate
grammars is to inline their rules before code generation. Suppose you
have
grammar C importing A and B, and you also have D importing A and B. The
code generation will result in classes C, C_A, C_B, D, D_A, and D_B.
Clearly
the independent generation of C_A and D_A during code generation does
not allow a single instance of the imported A grammar to be shared by C
and
D. If we instead flatten the imported grammar hierarchy and only
generate
classes C and D, then everything behaves like it was written in a single
grammar. Do you see any immediate problems with this potential approach?
-Original Message-
From: Ranco Marcus [mailto:ranco.mar...@epirion.nl]
Sent: Wednesday, May 25, 2011 4:26 PM
To: Sam Harwell; antlr-interest@antlr.org
Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
Hi Sam/all,
When a (tree) grammar C imports (tree) grammars A and B, where grammar
A calls a rule R from grammar B, a call is being made from delegate
parser C_A
to a delegate rule R (targeting C_B) in the composite parser C (its
parent).
Now that the visibility of the delegate rules in C match the visibility
of the
imported grammar, the rule R has to be made public for the above to work.
In our grammars, we build up internal structures that are subsequently
processed. In our case, that means that all those internal structures
have to
be made public as well. This could be solved by allowing ANTLR rules to
have
'internal' visibility. Also, imported grammars can probably remain
internal as
well.
What are your thoughts on this?
Thanks, Ranco
-Original Message-
From: Sam Harwell [mailto:sharw...@pixelminegames.com]
Sent: Thursday, April 07, 2011 3:57 PM
To: Ranco Marcus; antlr-interest@antlr.org
Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
The visibility of delegated rules now matches the way they're declared
in the imported grammar. Since some of them return nested classes
declared in the delegate grammar, I had to make the delegate grammar
'public' and all its rules 'internal'.
I fixed the reserved names issue right after releasing 3.3.1, so it'll
be included when I release 3.3.2. Note that rule parameters and return
values need to include an explicit '@' in their declarations in the
grammar, as follows. Labels and rule names do not need the '@'.
namespace[int @in] returns [int @out] : as=A B;
Sam
-Original Message-
From: Ranco Marcus [mailto:ranco.mar...@epirion.nl]
Sent: Thursday, April 07, 2011 6:49 AM
To: Sam Harwell; antlr-interest@antlr.org
Subject: RE: [antlr-interest] [CSharp3] rule visibility in composite
grammars
Sounds great, Sam!
What will be the visibility of the methods in the 'Delegated rules'
region now that the class of the imported grammar is internal? Private
or
internal?
Another small suggestion is to prefix variable names (at least those
that are reserved in C#) with an at-sign. The generated class for the
grammar below has a compile error since 'as' is not a valid variable
name
while
[il-antlr-interest: 32697] Re: [antlr-interest] AST with optional parameters
Jim, From an earlier message, David wrote: *Yes, the language is Matlab and a semicolon on the end of an assignment expression suppresses display of the result of the assignment. ...* Bart. On Tue, Jun 7, 2011 at 6:30 PM, Jim Idle j...@temporal-wave.com wrote: Why do you want the SEMI in your AST? Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of David Smith Sent: Tuesday, June 07, 2011 4:41 AM To: antlr-interest@antlr.org Subject: [antlr-interest] AST with optional parameters I'm parsing a grammar in which the semicolon on the end of a line is optional. So two of the statement rules might be: | (ID GETS expr SEMI) = ID GETS expr SEMI - ^(GETS ID expr SEMI) | (ID GETS) = ID GETS expr - ^(GETS ID expr) Since this occurs with a number of different assignment statements, I would really like to collapse this into one rule that looks something like this: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s) but every implementation I can think of either refuses to generate the grammar or, as in the case above, generates the grammar but decides that the variable 's' is unknown. Is there a any way to achieve this? DMS David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR- 33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 32698] Re: [antlr-interest] AST with optional parameters
It was not in any earlier message than the one I replied to. MATLAB is such an abysmal language ;-) Jim *From:* Bart Kiers [mailto:bki...@gmail.com] *Sent:* Tuesday, June 07, 2011 10:14 AM *To:* Jim Idle *Cc:* antlr-interest@antlr.org *Subject:* Re: [antlr-interest] AST with optional parameters Jim, From an earlier message, David wrote: ***Yes, the language is Matlab and a semicolon on the end of an assignment expression suppresses display of the result of the assignment. ...*** Bart. On Tue, Jun 7, 2011 at 6:30 PM, Jim Idle j...@temporal-wave.com wrote: Why do you want the SEMI in your AST? Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of David Smith Sent: Tuesday, June 07, 2011 4:41 AM To: antlr-interest@antlr.org Subject: [antlr-interest] AST with optional parameters I'm parsing a grammar in which the semicolon on the end of a line is optional. So two of the statement rules might be: | (ID GETS expr SEMI) = ID GETS expr SEMI - ^(GETS ID expr SEMI) | (ID GETS) = ID GETS expr - ^(GETS ID expr) Since this occurs with a number of different assignment statements, I would really like to collapse this into one rule that looks something like this: | (ID GETS) = ID GETS e=expr (s=SEMI)? - ^(GETS ID $e $s) but every implementation I can think of either refuses to generate the grammar or, as in the case above, generates the grammar but decides that the variable 's' is unknown. Is there a any way to achieve this? DMS David M. Smith http://www.cc.gatech.edu/fac/David.Smith Georgia Institute of Technology, College of Computing Sent from my ASR- 33 Teletype List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
