[il-antlr-interest: 28869] [antlr-interest] Find X :-)
Dear All, I have in my grammar: fragment UNIT_NAME :; ID : ('a'..'z'|'_')* { if ( myLookUpMethod( *X* ) ) $type=UNIT_NAME; } ; === Instead of *X*, I wanna refer to the text that is in red.. For example, the token w_aw_s is ID. I want to take the text of this token w_aw_s and look it up in a certain array and if it is in this array then I set its type to UNIT_NAME. P.S.: I tried to replace *X* with *$text* but I receive the error: [16:15:56] symbol : method myLookUpMethod(java.lang.String) [16:15:56] location: class SimpleTestLexer [16:15:56] if(myLookUpMethod(getText())) _type=UNIT_NAME; [16:15:56]^ [16:15:56] 1 error Find X :-) Many Thanks ! -- Sameh W. Zaky List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28870] Re: [antlr-interest] Find X :-)
OK, Sorry for that! Just after I have sent this message I realized that the problem is that ANTLR cannot find my method myLookUpMethod.. I have defined it in @members{} part of the grammar file.. But ANTLR still cannot find it.. Any idea? On Tue, May 18, 2010 at 4:35 PM, Sameh W. Zaky sameh...@gmail.com wrote: Dear All, I have in my grammar: fragment UNIT_NAME :; ID : ('a'..'z'|'_')* { if ( myLookUpMethod( *X* ) ) $type=UNIT_NAME; } ; === Instead of *X*, I wanna refer to the text that is in red.. For example, the token w_aw_s is ID. I want to take the text of this token w_aw_s and look it up in a certain array and if it is in this array then I set its type to UNIT_NAME. P.S.: I tried to replace *X* with *$text* but I receive the error: [16:15:56] symbol : method myLookUpMethod(java.lang.String) [16:15:56] location: class SimpleTestLexer [16:15:56] if(myLookUpMethod(getText())) _type=UNIT_NAME; [16:15:56]^ [16:15:56] 1 error Find X :-) Many Thanks ! -- Sameh W. Zaky -- Sameh W. Zaky List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28871] Re: [antlr-interest] Find X :-)
On Tue, 2010-05-18 at 17:10 +0200, Sameh W. Zaky wrote: OK, Sorry for that! Just after I have sent this message I realized that the problem is that ANTLR cannot find my method myLookUpMethod.. I have defined it in @members{} part of the grammar file.. But ANTLR still cannot find it.. Any idea? since it is in the lexer, the method needs to be defined in a @lexer::members{} section @members{} defaults to the parser when placed in a combined grammar. On Tue, May 18, 2010 at 4:35 PM, Sameh W. Zaky sameh...@gmail.com wrote: Dear All, I have in my grammar: fragment UNIT_NAME :; ID : ('a'..'z'|'_')* { if ( myLookUpMethod( *X* ) ) $type=UNIT_NAME; } ; === Instead of *X*, I wanna refer to the text that is in red.. For example, the token w_aw_s is ID. I want to take the text of this token w_aw_s and look it up in a certain array and if it is in this array then I set its type to UNIT_NAME. P.S.: I tried to replace *X* with *$text* but I receive the error: [16:15:56] symbol : method myLookUpMethod(java.lang.String) [16:15:56] location: class SimpleTestLexer [16:15:56] if(myLookUpMethod(getText())) _type=UNIT_NAME; [16:15:56]^ [16:15:56] 1 error Find X :-) Many Thanks ! -- Sameh W. Zaky List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28872] Re: [antlr-interest] ANTLR Two Simple Questions :-)
Well, err the method inArray() is an imaginary method that you will write. It just needs to take a string and see if that string is currently something that should be a UNIT_NAME. If it is real simple then you could just use a standard List object of your choice. Please replay to the list and not my personal email - thanks J Jim From: Sameh W. Zaky [mailto:sameh...@gmail.com] Sent: Tuesday, May 18, 2010 7:21 AM To: Jim Idle Subject: Re: [antlr-interest] ANTLR Two Simple Questions :-) Dear Jim, thanks for your reply. Concerning point (1): I did as you said: fragment UNIT_NAME :; ID : ('a'..'z'|'_')* { if(inArray($text)) $type=UNIT_NAME; } ; But when I click 'run' to run a test I get the error: D:\My Documents\Downloads\Simple Test Grammar\output\SimpleTestLexer.java:604: cannot find symbol [16:15:56] symbol : method inArray(java.lang.String) [16:15:56] location: class SimpleTestLexer [16:15:56] if(inArray(getText())) _type=UNIT_NAME; Any idea, how I can overcome this? Thanks again for your time :-) On Mon, May 17, 2010 at 6:54 PM, Jim Idle j...@temporal-wave.com wrote: -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Sameh W. Zaky Sent: Monday, May 17, 2010 7:33 AM To: antlr-interest@antlr.org Subject: [antlr-interest] ANTLR Two Simple Questions :-) Dear All, I am a Masters student, I use Java, and I am trying to make my parser with the help of ANTLR. I need this for my Masters project. I am using ANTLRWorks 1.3.1 I had 2 questions concerning it: 1) How can I have a dynamic token name (or dynamic lexer)? *Example: (the symbol SERVICE, will have different token values, depending on certain list (array) whose contents change over time)* Match the possible sequences generally, then change the token type if the strings match your dynamic list: fragment SERVICE : ; ID : ('a'..'z')+ { If (myLookupMethod($text)) { $type = SERVICE; } } ; So the token ID will be returned if the matched text is not in your list and the token SERVICE will be returned if it is. 2) Given a certain grammar, how can I (or: is it possible to) get the list of all possible next tokens? Read: http://www.antlr.org/wiki/display/ANTLR3/Custom+Syntax+Error+Recovery Do I get a masters too? ;-) Jim List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- Sameh W. Zaky List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28877] Re: [antlr-interest] Custom lexer and parser constructors
On Tue, May 18, 2010 at 6:04 PM, Cameron Ross cr...@kojeware.com wrote: Hi, Is there a way to define custom constructors for ANTLR lexers and parsers? Sure, you can create custom methods, c-tors and attributes in: @parser::members { // here } or @lexer::members { // here } Regards, Bart Kiers. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28878] Re: [antlr-interest] Custom lexer and parser constructors
OK - just add the reporter to the paresr members and create a setErrorReporter() method. Jim From: Cameron Ross [mailto:cr...@kojeware.com] Sent: Tuesday, May 18, 2010 9:19 AM To: Jim Idle Cc: antlr-interest@antlr.org Subject: Re: [antlr-interest] Custom lexer and parser constructors Hi Jim, My previous positing went unanswered so I thought I'd rephrase it to be more generic: http://www.antlr.org/pipermail/antlr-interest/2010-May/038661.html. Thanks for the lead on the superClass option... I'll look into it. Cameron. On Tue, May 18, 2010 at 12:15 PM, Jim Idle j...@temporal-wave.com wrote: Using a superClass=xyz option will probably get you where you want to be, but you might be better phrasing your question in terms of what you want to achieve rather than whether you can change the constructors. However, it is just as easy to add members to the class and supply setter methods. Jim -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Cameron Ross Sent: Tuesday, May 18, 2010 9:04 AM To: antlr-interest@antlr.org Subject: [antlr-interest] Custom lexer and parser constructors Hi, Is there a way to define custom constructors for ANTLR lexers and parsers? Thanks, Cameon. List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- Kojeware Corporation List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28881] Re: [antlr-interest] ANTLR Two Simple Questions :-)
That was helpful, thanks :-) On Tue, May 18, 2010 at 5:31 PM, Jim Idle j...@temporal-wave.com wrote: Well, err the method inArray() is an imaginary method that you will write. It just needs to take a string and see if that string is currently something that should be a UNIT_NAME. If it is real simple then you could just use a standard List object of your choice. Please replay to the list and not my personal email - thanks J Jim From: Sameh W. Zaky [mailto:sameh...@gmail.com] Sent: Tuesday, May 18, 2010 7:21 AM To: Jim Idle Subject: Re: [antlr-interest] ANTLR Two Simple Questions :-) Dear Jim, thanks for your reply. Concerning point (1): I did as you said: fragment UNIT_NAME :; ID : ('a'..'z'|'_')* { if(inArray($text)) $type=UNIT_NAME; } ; But when I click 'run' to run a test I get the error: D:\My Documents\Downloads\Simple Test Grammar\output\SimpleTestLexer.java:604: cannot find symbol [16:15:56] symbol : method inArray(java.lang.String) [16:15:56] location: class SimpleTestLexer [16:15:56] if(inArray(getText())) _type=UNIT_NAME; Any idea, how I can overcome this? Thanks again for your time :-) On Mon, May 17, 2010 at 6:54 PM, Jim Idle j...@temporal-wave.com wrote: -Original Message- From: antlr-interest-boun...@antlr.org [mailto:antlr-interest- boun...@antlr.org] On Behalf Of Sameh W. Zaky Sent: Monday, May 17, 2010 7:33 AM To: antlr-interest@antlr.org Subject: [antlr-interest] ANTLR Two Simple Questions :-) Dear All, I am a Masters student, I use Java, and I am trying to make my parser with the help of ANTLR. I need this for my Masters project. I am using ANTLRWorks 1.3.1 I had 2 questions concerning it: 1) How can I have a dynamic token name (or dynamic lexer)? *Example: (the symbol SERVICE, will have different token values, depending on certain list (array) whose contents change over time)* Match the possible sequences generally, then change the token type if the strings match your dynamic list: fragment SERVICE : ; ID : ('a'..'z')+ { If (myLookupMethod($text)) { $type = SERVICE; } } ; So the token ID will be returned if the matched text is not in your list and the token SERVICE will be returned if it is. 2) Given a certain grammar, how can I (or: is it possible to) get the list of all possible next tokens? Read: http://www.antlr.org/wiki/display/ANTLR3/Custom+Syntax+Error+Recovery Do I get a masters too? ;-) Jim List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- Sameh W. Zaky List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- Sameh W. Zaky List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28882] [antlr-interest] enums in v4 ANTLR Java code generation considered useless
A lot of you have wondered why I'd don't use enum types for token types in the generated Java-based parsers: enum TokenType { ID(3), INT(4) }; I gave it my best shot but once again I found that enums just don't seem to work for me. For example, without inheritance how can I define the match method? void match(TokenType t) {} Unfortunately this is in the superclass not the generated class so TokenType is not yet defined. Without inheritance, I have to use match(int). That means that there's no point in using the enum. But, you say, I should be using generics on the parser to pass in the token type enum type. Fair enough, And I'll try once I tried to generic'ize the v3 runtime for v4. We've had some discussion of this on the dev list. Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28885] Re: [antlr-interest] enums in v4 ANTLR Java code generation considered useless
There are several advantages to enums: * there is a discrete set of values that can be used (no accidental 42's passed in when 42 isn't a token type) * the enum value can carry extra information * the enum values can override methods differently OH - one of the things that's clouding this is that you really don't need the numeric type identifers anymore. You can just have public enum TokenType { IDENT, INT ...; } then in your match method: void match(TokenType type) { if (LA(1).getType() == type) { ... } } And you can use the types in a switch statement: switch(type) { case INT: case IDENT: ... } No more magic numbers! Woohoo! -- Scott Scott Stanchfield http://javadude.com On Tue, May 18, 2010 at 4:52 PM, Terence Parr pa...@cs.usfca.edu wrote: hi Edgar :) Don't i have to pass objects around then instead of ints with extra method calls etc...? void match(TokenType t) { int ttype = t.getType(); } no autoboxing etc... match(GeneratedTokenType.ID); Also i need Parser.GeneratedTokenType.ID in outside code. pretty ugly. I guess i don't see enum advantages for token types and they cause confusion. enum GeneratedTokenType implements TokenType vs int, in other words. Ter On May 18, 2010, at 1:47 PM, Edgar Espina wrote: Hi, TokenType can be an interface. Then your generated tokens can implemented this interface. enum GeneratedTokenType implements TokenType { ID(3); } Thanks, edgar On Tue, May 18, 2010 at 5:42 PM, Terence Parr pa...@cs.usfca.edu wrote: A lot of you have wondered why I'd don't use enum types for token types in the generated Java-based parsers: enum TokenType { ID(3), INT(4) }; I gave it my best shot but once again I found that enums just don't seem to work for me. For example, without inheritance how can I define the match method? void match(TokenType t) {} Unfortunately this is in the superclass not the generated class so TokenType is not yet defined. Without inheritance, I have to use match(int). That means that there's no point in using the enum. But, you say, I should be using generics on the parser to pass in the token type enum type. Fair enough, And I'll try once I tried to generic'ize the v3 runtime for v4. We've had some discussion of this on the dev list. Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- edgar List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
[il-antlr-interest: 28886] Re: [antlr-interest] enums in v4 ANTLR Java code generation considered useless
A little clarification... Start with (as Edgar said) public interface TokenType { } public enum CommonTypes implements TokenType { EOF; // others? } public class Token { private TokenType type; public TokenType getType() {return type;} ... } public class Parser ... { void match(TokenType type) { if (LA(1).getType() == type) { ... } } ... } then for the specific parser, generate public enum FooParserTokenType implements TokenType { IDENT, INT, ...; } What might be nice would be to have a description available for the type, something like public interface TokenType { String getDescription(); } public enum FooParserTokenType implements TokenType { IDENT(identifier (lower camel case word of doom)), INT(integer (digits, no decimal points), ...; private String description; private FooParserTokenType(String description) { this.description = description; } public String getDescription() { return description; } } this could be fed by something like IDENT options {description=identifier (lower camel case word of doom)} : ... and could be used to really help error messages by using token.getType().getDescription(). This is where Java enums get really cool -- carrying extra data. I've gotta upload an enumeration talk I gave at work (which described other cool things you can do, like the GoF state pattern and something kinda close to function pointers in java) Hope this helps! -- Scott Scott Stanchfield http://javadude.com On Tue, May 18, 2010 at 5:58 PM, Scott Stanchfield sc...@javadude.com wrote: There are several advantages to enums: * there is a discrete set of values that can be used (no accidental 42's passed in when 42 isn't a token type) * the enum value can carry extra information * the enum values can override methods differently OH - one of the things that's clouding this is that you really don't need the numeric type identifers anymore. You can just have public enum TokenType { IDENT, INT ...; } then in your match method: void match(TokenType type) { if (LA(1).getType() == type) { ... } } And you can use the types in a switch statement: switch(type) { case INT: case IDENT: ... } No more magic numbers! Woohoo! -- Scott Scott Stanchfield http://javadude.com On Tue, May 18, 2010 at 4:52 PM, Terence Parr pa...@cs.usfca.edu wrote: hi Edgar :) Don't i have to pass objects around then instead of ints with extra method calls etc...? void match(TokenType t) { int ttype = t.getType(); } no autoboxing etc... match(GeneratedTokenType.ID); Also i need Parser.GeneratedTokenType.ID in outside code. pretty ugly. I guess i don't see enum advantages for token types and they cause confusion. enum GeneratedTokenType implements TokenType vs int, in other words. Ter On May 18, 2010, at 1:47 PM, Edgar Espina wrote: Hi, TokenType can be an interface. Then your generated tokens can implemented this interface. enum GeneratedTokenType implements TokenType { ID(3); } Thanks, edgar On Tue, May 18, 2010 at 5:42 PM, Terence Parr pa...@cs.usfca.edu wrote: A lot of you have wondered why I'd don't use enum types for token types in the generated Java-based parsers: enum TokenType { ID(3), INT(4) }; I gave it my best shot but once again I found that enums just don't seem to work for me. For example, without inheritance how can I define the match method? void match(TokenType t) {} Unfortunately this is in the superclass not the generated class so TokenType is not yet defined. Without inheritance, I have to use match(int). That means that there's no point in using the enum. But, you say, I should be using generics on the parser to pass in the token type enum type. Fair enough, And I'll try once I tried to generic'ize the v3 runtime for v4. We've had some discussion of this on the dev list. Ter List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- edgar List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com.
[il-antlr-interest: 28887] [antlr-interest] help please
hello everybody my name is Ernesto and i am calling for help on antlr programming, i am a newby in this and i am in my second semester master , after 12 years ago that finish my degree in computer science but because the circumstance never work in the computer field, but planning to get into, so this semester i am taking compiler, and my first programming assignment was really bad because i am not clear how put together the java with the antlr. i know how Java work because my first semester i took Java and i used it with eclipse . Now i thing i have properly installed the antlr 3.2 nevertheless i do not know if i have to install antlrwork because the IDE , i was trying to do the main java with eclipse to invoke antlr but never work . so i feel lost in the sea and i have the antlr book but look like is the old version. my computer is Mac . i would appreciate the help thanks List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups il-antlr-interest group. To post to this group, send email to il-antlr-inter...@googlegroups.com. To unsubscribe from this group, send email to il-antlr-interest+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.