On 02.02.2011 18:01, Jim Idle wrote: > I think that the fix is in there, which means that your input is too big > to build the tree in the way it is being done. Write an input stream > wrapper that splits the input by just returning EOF at the split point > then resets to the next unit.
Actually I already rewrote my splitter. Now I get almost exactly the same performance for parsing the files splitted and as a whole. Also the overall memory consumption is much better this way since I actually build an AST in another framework. I only use antlr to parse the input. I just wanted to compare the performance of different approaches. I now also tried to use the start and end pointers to the input instead of using $text. Doesn't make a difference in memory consumption or speed. So I stick with the convenient $text. > How are you ending up with 640,000 lines of C input? I'm doing some program analysis and preprocess the C programms with CIL [1]. vim and libgsl for instance then are about half a million lines of code. Thanks for your time Marco [1] http://hal.cs.berkeley.edu/cil/ > Jim > > >> -----Original Message----- >> From: [email protected] [mailto:antlr-interest- >> [email protected]] On Behalf Of Marco Trudel >> Sent: Wednesday, February 02, 2011 8:20 AM >> To: [email protected] >> Subject: Re: [antlr-interest] Memory management of C target >> >> Dear Jim >> >> On 01.02.2011 18:15, Jim Idle wrote: >>> <snip> >>> >>> I think >>> that in 3.3 I have fixed a bug that was not releasing memory >>> references when building a tree until the tree was freed. Try making >> a >>> version that does not build a tree and see how it differs. >>> >>> <snip> >> >> Ok, made a version of my grammar that doesn't build a tree: >> >> libantlr3c-3.2, no tree output: works, uses about 600mb memory >> libantlr3c-3.2, tree output: crash, wants more than 2gb >> >> libantlr3c-3.3-SNAPSHOT, no tree output: works, uses about 630mb memory >> libantlr3c-3.3-SNAPSHOT, tree output: crash, wants more than 2gb >> >> The only changes to the grammar where removing/adding "output = AST;". >> Is your fix already in the current SNAPSHOT? >> >> >> Thanks >> Marco >> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest >> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your- >> email-address > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: > http://www.antlr.org/mailman/options/antlr-interest/your-email-address List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
