[il-antlr-interest: 28323] Re: [antlr-interest] Help with semantic actions commit
hi. antlr shouldn't see past actions to sem preds. are you sure that the define sym action is in every path? Do you get an error about insufficiently uncovered alts? Ter On Mar 18, 2010, at 4:54 PM, swalton wrote: I derived my grammar from Terry's C grammar. I have added a lot of semantic actions and kept the typedef symbol definition. I have run into a problem and need help. Consider the following: typedef int uint32_t; typedef uint32_t result_t; I can have any number of these typedefs, but this case is sufficient to illustrate the problem I am encountering. The grammar looks ahead beyond the first semicolon before committing the 'uint32_t' in the scope table. Needless to say, the second line always fails. I watched step-by- step, placing System.err.print()'s along the way to find out when the '$Symbols::types.add(name);' is called. The test (boolean isTypeName(String name)) always occurs *before* 'uint32_t' is stored. I added a { addName(name) }? test (which always returns true) to see if I could force an execution of the semantic action without success. How do I get the symbol committed? How do I keep ANTLR from looking so far ahead? Any help is appreciated. This is a major roadblock. -Sean Walton PhD Student, SoC, U of U List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28324] [antlr-interest] Problem removing warning
Having a bit of difficulty in figuring out how to unambiguously parse this into an AST. Order of the elements is significant, the parens are significant, and the leading dot is significant. (.buf_unittest.complex_opt1).foo; .buf_unittest.complex_opt1.fum; (buf_unittest.complex_opt1).(.buf_unittest.quux); (.buf_unittest.complex_opt1).(buf_unittest.corge).qux; (complex_opt2).baz; (complex_opt2).(grault); (complex_opt2).bar.foo; (complex_opt2).bar.(quux); (complex_opt2).bar.(buf_unittest.corge).qux; (complex_opt2).(garply).foo; (complex_opt2).(garply).(.buf_unittest.quux); (complex_opt2).(buf_unittest.garply).(corge).qux; (ComplexOptionType2.ComplexOptionType4.complex_opt4).waldo; (complex_opt2).fred.waldo; (buf_unittest.complex_opt3).qux; (complex_opt3).complexoptiontype5.plugh; (complexopt6).xyzzy; The rule ident_parens following appears to work, but Antlr is complaining Decision can match input such as '.' ID using multiple alternatives: 1, 2 on both identN and identO. I can see the theoretical overlap, but cannot tell if the warning is actually significant or how to fix the rules to avoid the warning. ident_parens : (identM | identN | identO ) ('.' ( identM | identO ) )* ; identM : '(' '.' ipd+=ID ( '.' ipd+=ID )* ')' - ^( IDENT_PARENSDOT $ipd+ ) | '(' ip+=ID ( '.' ip+=ID )* ')' - ^( IDENT_PARENS $ip+ ) ; identN : '.' id+=ID ( '.' id+=ID )* - ^( IDENT_DOT $id+ ) ; identO : i+=ID ( '.' i+=ID )*- ^( IDENT $i+ ) ; Thanks... grammar PBTest ; options { ASTLabelType = CommonTree; output = AST; } tokens { MAIN; IDENT = 'ident'; IDENT_DOT = 'ident_dot'; IDENT_PARENS = 'ident_parens'; IDENT_PARENSDOT = 'ident_parens_dot'; } // / @parser::header { package net.certiv.core.parser.gen; } @lexer::header { package net.certiv.core.parser.gen; } ident : ident_parens+ EOF - ^( MAIN ident_parens+ ) ; ident_parens : ( identM | identN | identO ) ( '.' ( identM | identO ) )* ';' ; identM : '(' '.' ipd+=ID ( '.' ipd+=ID )* ')' - ^( IDENT_PARENSDOT $ipd+ ) | '(' ip+=ID ( '.' ip+=ID )* ')'- ^( IDENT_PARENS $ip+ ) ; identN : '.' id+=ID ( '.' id+=ID )* - ^( IDENT_DOT $id+ ) ; identO : i+=ID ( '.' i+=ID )* - ^( IDENT $i+ ) ; // /// ID : LETTER ( LETTER | '_' | DIGIT )* ; fragment LETTER : UPPER | LOWER ; fragment DIGIT : '0'..'9' ; fragment LOWER : 'a'..'z' ; fragment UPPER : 'A'..'Z' ; WS : ( ' ' | '\r' | '\t' | '\u000C' | '\n' ) { $channel = HIDDEN; } ; (.buf_unittest.complex_opt1).foo; .buf_unittest.complex_opt1.fum; (buf_unittest.complex_opt1).(.buf_unittest.quux); (.buf_unittest.complex_opt1).(buf_unittest.corge).qux; (complex_opt2).baz; (complex_opt2).(grault); (complex_opt2).bar.foo; (complex_opt2).bar.(quux); (complex_opt2).bar.(buf_unittest.corge).qux; (complex_opt2).(garply).foo; (complex_opt2).(garply).(.buf_unittest.quux); (complex_opt2).(buf_unittest.garply).(corge).qux; (ComplexOptionType2.ComplexOptionType4.complex_opt4).waldo; (complex_opt2).fred.waldo; (buf_unittest.complex_opt3).qux; (complex_opt3).complexoptiontype5.plugh; (complexopt6).xyzzy; List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28326] Re: [antlr-interest] Problem removing warning
That will give a tree that isn't very useful ;-) You need to express this in LL form such that the things that can be elements of your compound appear at the bottom of the tree. Then use DOT and not '.', make that be the root node and do not try to impose any semantic verification via syntactical specifications. You get: grammar T; options { output=AST; } tokens {EXPR; FUNC;} aago:(expr NL)+ EOF -^(EXPR expr)+ ; expr : element (DOT^ element)* ; element : LPAREN! expr RPAREN! // Sometimes you want to preserve the LPAREN here | ID (LPAREN^ expr RPAREN)? // function call ; LPAREN :'('; RPAREN :')'; ID :('A'..'Z')+; DOT : '.' ; NL : ('\n'|'\r')+; ANY :. {skip();}; -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Gerald Rosenberg Sent: Friday, March 19, 2010 10:01 AM To: antlr-interest interest Subject: [antlr-interest] Problem removing warning Having a bit of difficulty in figuring out how to unambiguously parse this into an AST. Order of the elements is significant, the parens are significant, and the leading dot is significant. (.buf_unittest.complex_opt1).foo; .buf_unittest.complex_opt1.fum; (buf_unittest.complex_opt1).(.buf_unittest.quux); (.buf_unittest.complex_opt1).(buf_unittest.corge).qux; (complex_opt2).baz; (complex_opt2).(grault); (complex_opt2).bar.foo; (complex_opt2).bar.(quux); (complex_opt2).bar.(buf_unittest.corge).qux; (complex_opt2).(garply).foo; (complex_opt2).(garply).(.buf_unittest.quux); (complex_opt2).(buf_unittest.garply).(corge).qux; (ComplexOptionType2.ComplexOptionType4.complex_opt4).waldo; (complex_opt2).fred.waldo; (buf_unittest.complex_opt3).qux; (complex_opt3).complexoptiontype5.plugh; (complexopt6).xyzzy; The rule ident_parens following appears to work, but Antlr is complaining Decision can match input such as '.' ID using multiple alternatives: 1, 2 on both identN and identO. I can see the theoretical overlap, but cannot tell if the warning is actually significant or how to fix the rules to avoid the warning. ident_parens : (identM | identN | identO ) ('.' ( identM | identO ) )* ; identM : '(' '.' ipd+=ID ( '.' ipd+=ID )* ')' - ^( IDENT_PARENSDOT $ipd+ ) | '(' ip+=ID ( '.' ip+=ID )* ')' - ^( IDENT_PARENS $ip+ ) ; identN : '.' id+=ID ( '.' id+=ID )* - ^( IDENT_DOT $id+ ) ; identO : i+=ID ( '.' i+=ID )*- ^( IDENT $i+ ) ; Thanks... List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28327] Re: [antlr-interest] sematic analysis ?
On Fri, Mar 19, 2010 at 10:37 AM, Kaleb Pederson kaleb.peder...@gmail.com wrote: Venkat, One way to do what you've mentioned is to create a tree walker which walks the AST. Then, within the AST, you'll need to do a couple of things. First, you need to populate a symbol table. That symbol table will store the different objects/variables that are declared. In your example, it would store a and b, their values, if known, and their types. Second, you need to do an analysis of the structure while using the symbol table. Here's a fragment from one of my tree walking grammars: plusMinusExpression returns [Type type] : ^(PLUS lhs=baseExpression rhs=baseExpression) { typeChecker.assertIsNumericType($lhs.type); typeChecker.assertIsNumericType($rhs.type); typeChecker.assertEqualTypes($lhs.type, $rhs.type); $type = $lhs.type; } /* ... */ ; I realized that doesn't fully explain everything. In the various expressions, I resolve any necessary symbols and then resolve their types by: * using the symbol table if it's a variable, OR * using the type of the literal I then return the type of the expression so I can verify that the types match throughout the entire expression. -- Kaleb Pederson http://kalebpederson.com http://twitter.com/kalebpederson List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28328] Re: [antlr-interest] Problem removing warning
Thanks, Jim. Please understand though that I dumbed down the test grammar to minimally illustrate the problem I was asking about - my actual grammar shapes a much more complex tree and uses tidy tokens;). The revision you suggest unfortunately looses a bit too much intrinsic information that would be difficult to infer if at all in a later pass. As I mentioned, the order, existence and position of the parens and leading dots are significant - (foo), .foo, (.foo), and foo are not equivalent and need to be distinctly represented in the tree. On 3/19/2010 10:41 AM, Jim Idle wrote: That will give a tree that isn't very useful ;-) You need to express this in LL form such that the things that can be elements of your compound appear at the bottom of the tree. Then use DOT and not '.', make that be the root node and do not try to impose any semantic verification via syntactical specifications. You get: grammar T; options { output=AST; } tokens { EXPR; FUNC;} aago :(expr NL)+ EOF -^(EXPR expr)+ ; expr : element (DOT^ element)* ; element : LPAREN! expr RPAREN! // Sometimes you want to preserve the LPAREN here | ID (LPAREN^ expr RPAREN)? // function call ; LPAREN:'('; RPAREN : ')'; ID:('A'..'Z')+; DOT : '.' ; NL: ('\n'|'\r')+; ANY :. {skip();}; -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Gerald Rosenberg Sent: Friday, March 19, 2010 10:01 AM To: antlr-interest interest Subject: [antlr-interest] Problem removing warning Having a bit of difficulty in figuring out how to unambiguously parse this into an AST. Order of the elements is significant, the parens are significant, and the leading dot is significant. (.buf_unittest.complex_opt1).foo; .buf_unittest.complex_opt1.fum; (buf_unittest.complex_opt1).(.buf_unittest.quux); (.buf_unittest.complex_opt1).(buf_unittest.corge).qux; (complex_opt2).baz; (complex_opt2).(grault); (complex_opt2).bar.foo; (complex_opt2).bar.(quux); (complex_opt2).bar.(buf_unittest.corge).qux; (complex_opt2).(garply).foo; (complex_opt2).(garply).(.buf_unittest.quux); (complex_opt2).(buf_unittest.garply).(corge).qux; (ComplexOptionType2.ComplexOptionType4.complex_opt4).waldo; (complex_opt2).fred.waldo; (buf_unittest.complex_opt3).qux; (complex_opt3).complexoptiontype5.plugh; (complexopt6).xyzzy; The rule ident_parens following appears to work, but Antlr is complaining Decision can match input such as '.' ID using multiple alternatives: 1, 2 on both identN and identO. I can see the theoretical overlap, but cannot tell if the warning is actually significant or how to fix the rules to avoid the warning. ident_parens : (identM | identN | identO ) ('.' ( identM | identO ) )* ; identM : '(' '.' ipd+=ID ( '.' ipd+=ID )* ')' - ^( IDENT_PARENSDOT $ipd+ ) | '(' ip+=ID ( '.' ip+=ID )* ')' - ^( IDENT_PARENS $ip+ ) ; identN : '.' id+=ID ( '.' id+=ID )* - ^( IDENT_DOT $id+ ) ; identO : i+=ID ( '.' i+=ID )*- ^( IDENT $i+ ) ; Thanks... List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28329] Re: [antlr-interest] Problem removing warning
Yes - you can retain all that stuff, but if you try to do it the way you have it, you won't get a tree that makes sense. You can see a much more complicated example using my C# parser, which is online at: http://www.temporal-wave.com/index.php?option=com_psrrunview=psrrunItemid=58 All you need do is not kill the information I am killing. (and add | DOT element to the expr). However, you know what you want :-) jim -Original Message- From: Gerald Rosenberg [mailto:ger...@certiv.net] Sent: Friday, March 19, 2010 11:24 AM To: Jim Idle Cc: antlr-interest interest Subject: Re: [antlr-interest] Problem removing warning Thanks, Jim. Please understand though that I dumbed down the test grammar to minimally illustrate the problem I was asking about - my actual grammar shapes a much more complex tree and uses tidy tokens;). The revision you suggest unfortunately looses a bit too much intrinsic information that would be difficult to infer if at all in a later pass. As I mentioned, the order, existence and position of the parens and leading dots are significant - (foo), .foo, (.foo), and foo are not equivalent and need to be distinctly represented in the tree. On 3/19/2010 10:41 AM, Jim Idle wrote: That will give a tree that isn't very useful ;-) You need to express this in LL form such that the things that can be elements of your compound appear at the bottom of the tree. Then use DOT and not '.', make that be the root node and do not try to impose any semantic verification via syntactical specifications. You get: grammar T; options { output=AST; } tokens {EXPR; FUNC;} aago:(expr NL)+ EOF -^(EXPR expr)+ ; expr : element (DOT^ element)* ; element : LPAREN! expr RPAREN! // Sometimes you want to preserve the LPAREN here | ID (LPAREN^ expr RPAREN)? // function call ; LPAREN :'('; RPAREN :')'; ID :('A'..'Z')+; DOT : '.' ; NL : ('\n'|'\r')+; ANY :. {skip();}; -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Gerald Rosenberg Sent: Friday, March 19, 2010 10:01 AM To: antlr-interest interest Subject: [antlr-interest] Problem removing warning Having a bit of difficulty in figuring out how to unambiguously parse this into an AST. Order of the elements is significant, the parens are significant, and the leading dot is significant. (.buf_unittest.complex_opt1).foo; .buf_unittest.complex_opt1.fum; (buf_unittest.complex_opt1).(.buf_unittest.quux); (.buf_unittest.complex_opt1).(buf_unittest.corge).qux; (complex_opt2).baz; (complex_opt2).(grault); (complex_opt2).bar.foo; (complex_opt2).bar.(quux); (complex_opt2).bar.(buf_unittest.corge).qux; (complex_opt2).(garply).foo; (complex_opt2).(garply).(.buf_unittest.quux); (complex_opt2).(buf_unittest.garply).(corge).qux; (ComplexOptionType2.ComplexOptionType4.complex_opt4).waldo; (complex_opt2).fred.waldo; (buf_unittest.complex_opt3).qux; (complex_opt3).complexoptiontype5.plugh; (complexopt6).xyzzy; The rule ident_parens following appears to work, but Antlr is complaining Decision can match input such as '.' ID using multiple alternatives: 1, 2 on both identN and identO. I can see the theoretical overlap, but cannot tell if the warning is actually significant or how to fix the rules to avoid the warning. ident_parens : (identM | identN | identO ) ('.' ( identM | identO ) )* ; identM : '(' '.' ipd+=ID ( '.' ipd+=ID )* ')' - ^( IDENT_PARENSDOT $ipd+ ) | '(' ip+=ID ( '.' ip+=ID )* ')' - ^( IDENT_PARENS $ip+ ) ; identN : '.' id+=ID ( '.' id+=ID )* - ^( IDENT_DOT $id+ ) ; identO : i+=ID ( '.' i+=ID )*- ^( IDENT $i+ ) ; Thanks... List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr- interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28331] Re: [antlr-interest] THelp w/ Tree Grammer - Rule Action Error
Greetings! On Fri, 2010-03-19 at 15:26 -0400, William Koscho wrote: Hi All, I have a tree grammar, and am trying to just print out some information from the tree. This works fine for matching tokens, but not when matching the rules. I'm hoping someone can help explain why this is giving me a NullPointerException and how to correct it: Tree Grammar: // the $i.text causes the NullPointerException, when I $i.text, it works fine interfaceDeclaration: ^(i=interfaceType ID) { System.out.println($i.text + : + $ID.text); }; interfaceType: PROVIDES | REQUIRES; Corresponding Parser Grammar: interfaceDeclaration: interfaceType ID - ^(interfaceType ID); interfaceType: 'provides' - PROVIDES | 'requires' - REQUIRES; I believe your interfaceType parser rule does not initialize the text of the token it creates for the tree node. You should be able to inspect the generated code and verify this I believe you want your interfaceType parser rule to be: interfaceType : ( k='provides' - PROVIDES[$k] ) | ( k='requires' - REQUIRES[$k] ) ; if i recall correctly, using the [] stuff sets both the text and the location of the generated virtual token. and oh by the way are you sure you really need the virtual tokens? hope this helps... -jbb List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28332] Re: [antlr-interest] Problem removing warning
List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28333] [antlr-interest] semantic analysis
Hi, I have a simple grammar for which I was able to walk through the AST and printout the input text to the console. I learned that we need to put the information in the symbol table and do the semantic analysis from there on. How , what information should I put into the symbol table and how do I check for type errors ? Let say, I have something like a + b, given that I am able to walk through the AST, how do I do the type checking for this little case? Any simple code snippets would be appreciated. Thanks. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28334] Re: [antlr-interest] semantic analysis
Greetings! On Fri, 2010-03-19 at 17:01 -0400, venkat medhaj wrote: Hi, I have a simple grammar for which I was able to walk through the AST and printout the input text to the console. I learned that we need to put the information in the symbol table and do the semantic analysis from there on. How , what information should I put into the symbol table and how do I check for type errors ? Let say, I have something like a + b, given that I am able to walk through the AST, how do I do the type checking for this little case? Any simple code snippets would be appreciated. Attached please find an example Expression parsing suite for ANTLRv3. I intended for this to be a simple tutorial example for tree rewriting. It is complete, but not documented. So, as is, it is not a very good example, but hopefully you can see one possibility for your solution (i hope). It has 4 phases: 1) parsing, 2) type checking, 3) instruction selection, and 4) evaluation. There is a grammar that drives each phase. Probably start by looking at the Main.java and try to figure out where we go from there... (sorry for the lack of commentary/documentation) Hope this helps -jbb List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address expressions.tgz Description: application/compressed-tar -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.