The pointer to the token (unless it is an imaginary token produced by the parser) is in the token field of pANTLR3_COMMON_TREE. So, take the pointer to the base tree that the reference in the tree gives you and cast the super field in there to (pANTLR3_COMMON_TREE) and then the token field is pANTLR3_COMMON_TOKEN. However, the getToken() method of pANTLR3_COMMON_TREE will do that for you. Look at the methods in antlr3commontree.c.
Remember that the tree parser only deals with pointers to the lowest basic structure which is pANTLR3_BASE_TREE and that has a pointer 'super' to the structure that contains it (normally pANTLR3_COMMON_TREE) which also has a suprt pointer in case you encapsulate it further (usually too much hassle to be worth it). For a node with children then follow the lists recursively. The code that produces dot files for an arbitrary tree is a good place to look for hints as this traverses pANTLR3_BASE_TREE and looks for the text that represents it. You will find that in antlr3basetreeadapator.c Jim > -----Original Message----- > From: [email protected] [mailto:antlr-interest- > [email protected]] On Behalf Of Karim Chichakly > Sent: Monday, January 31, 2011 10:53 AM > To: [email protected] interest > Subject: Re: [antlr-interest] Memory management of C target > > Hi Jim, > > It is clear that for tokens in the parser, you can use getStartIndex > and getStopIndex directly to avoid using $text. How can you do this > for an arbitrary tree node when walking the tree? It appears in this > case that you also need the token stream (to ask for the token using > get()). Is there any way to get the token stream from the tree node or > is there another way to get the text associated with the node? > > Thanks, > > Karim > > > On Mon, Jan 31, 2011 at 12:43 PM, Jim Idle <[email protected]> > wrote: > > > The C target will be a lot faster than the Java target, but the > > objects that are created are probably bigger. For v4 I plan to reduce > that a lot. > > It is probably better to reduce the input though. 530,000 lines of C > > code as input seems a bit of a tall order for anything, even if you > parse it. > > The individual input files would be better. > > > > Also, I think you were using $text references in your parser and > these > > will create hundreds of thousands of string objects that will not be > > released until you release the parser. To use the text of an object > it > > is better to get the pointer to the input from that object and use > the > > length (start and end pointer are stored in the object) so that you > > make no copies or memory allocations. The $text (in the C target) is > a > > convenience method that is relatively slow and inefficient; it is > just > > there when you don't really care that much about those factors. This > > catches so many people that I may abandon it in v4, in favor of > > functions/macros that give you the information. > > > > You can also try 64bit mode, which will raise the 2GB bar. > > > > Jim > > > > > > > > > -----Original Message----- > > > From: [email protected] [mailto:antlr-interest- > > > [email protected]] On Behalf Of Marco Trudel > > > Sent: Monday, January 31, 2011 5:37 AM > > > To: [email protected] > > > Subject: [antlr-interest] Memory management of C target > > > > > > Dear all > > > > > > Does anyone know how the C target handles memory? I noticed that > > > with very big input (e.g. 530.000 lines of C code) it crashes > > > because it hits the 2gb process memory limit. Is there something I > > > can tweak to make it work or do I have to split the input? > > > > > > The Java target manages to parse the input if I give the process > 1gb. > > > It even requires only 20 seconds. > > > Would be great if the C target could also do that. Even better it > > > the required time would be about half of the one of the Java target > > > (as I'm used to when the C target can handle the input). > > > > > > Thanks > > > Marco > > > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > > > Unsubscribe: > > > http://www.antlr.org/mailman/options/antlr-interest/your- > > > email-address > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > > Unsubscribe: > > http://www.antlr.org/mailman/options/antlr-interest/your-email- > address > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- > email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
