Hi Jim, It is clear that for tokens in the parser, you can use getStartIndex and getStopIndex directly to avoid using $text. How can you do this for an arbitrary tree node when walking the tree? It appears in this case that you also need the token stream (to ask for the token using get()). Is there any way to get the token stream from the tree node or is there another way to get the text associated with the node?
Thanks, Karim On Mon, Jan 31, 2011 at 12:43 PM, Jim Idle <[email protected]> wrote: > The C target will be a lot faster than the Java target, but the objects > that are created are probably bigger. For v4 I plan to reduce that a lot. > It is probably better to reduce the input though. 530,000 lines of C code > as input seems a bit of a tall order for anything, even if you parse it. > The individual input files would be better. > > Also, I think you were using $text references in your parser and these > will create hundreds of thousands of string objects that will not be > released until you release the parser. To use the text of an object it is > better to get the pointer to the input from that object and use the length > (start and end pointer are stored in the object) so that you make no > copies or memory allocations. The $text (in the C target) is a convenience > method that is relatively slow and inefficient; it is just there when you > don't really care that much about those factors. This catches so many > people that I may abandon it in v4, in favor of functions/macros that give > you the information. > > You can also try 64bit mode, which will raise the 2GB bar. > > Jim > > > > > -----Original Message----- > > From: [email protected] [mailto:antlr-interest- > > [email protected]] On Behalf Of Marco Trudel > > Sent: Monday, January 31, 2011 5:37 AM > > To: [email protected] > > Subject: [antlr-interest] Memory management of C target > > > > Dear all > > > > Does anyone know how the C target handles memory? I noticed that with > > very big input (e.g. 530.000 lines of C code) it crashes because it > > hits the 2gb process memory limit. Is there something I can tweak to > > make it work or do I have to split the input? > > > > The Java target manages to parse the input if I give the process 1gb. > > It even requires only 20 seconds. > > Would be great if the C target could also do that. Even better it the > > required time would be about half of the one of the Java target (as I'm > > used to when the C target can handle the input). > > > > Thanks > > Marco > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > > Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- > > email-address > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: > http://www.antlr.org/mailman/options/antlr-interest/your-email-address > List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
