An important note about using ST: don't translate toString and then emit. Send in a Writer to write(). No need to buffer in memory. CC'ing the other list. Ter
On Oct 17, 2009, at 9:05 PM, Jim Idle wrote: > Run Java with - Xmx1500M > > Or some other option that increases available memory size. However > this could point to something that can be better organized in terms > of recursion or whatever. > > You may also consider running Java with a better garbage collector > such as: > > -Xincgc > > JIm > > > From: [email protected] > [mailto:[email protected] > ] On Behalf Of Hieu Phung > Sent: Friday, October 16, 2009 4:03 PM > To: [email protected] > Subject: [antlr-interest] [Antlr + ST] Huge output string - > OutOfMemoryError > > Hi all, > > I am writing a program using Antlr and StringTemplate to translate > an input file to an XML file. Input is small but output is huge: > 720KB input file will generate about 100MB output file. When my > input reached 7608KB, I got OOM error > > time taken in second: 18 - file size in KB: 4804 > time taken in second: 20 - file size in KB: 5205 > time taken in second: 20 - file size in KB: 5605 > time taken in second: 21 - file size in KB: 6006 > time taken in second: 23 - file size in KB: 6406 > time taken in second: 25 - file size in KB: 6806 > time taken in second: 31 - file size in KB: 7207 > > at size 7608KB: > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:2882) > at > java > .lang > .AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) > at > java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572) > at java.lang.StringBuffer.append(StringBuffer.java:320) > at java.io.StringWriter.write(StringWriter.java:60) > at > org > .antlr.stringtemplate.AutoIndentWriter.write(AutoIndentWriter.java: > 172) > at > org.antlr.stringtemplate.language.ASTExpr.writePOJO(ASTExpr.java:702) > at > org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:683) > at > org > .antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java: > 660) > at > org > .antlr > .stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java: > 86) > at > org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149) > at > org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705) > at > org.antlr.stringtemplate.language.ASTExpr.writeTemplate(ASTExpr.java: > 750) > at > org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:680) > at > org > .antlr > .stringtemplate.language.ASTExpr.writeIterableValue(ASTExpr.java:792) > at > org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:682) > at > org > .antlr.stringtemplate.language.ASTExpr.writeAttribute(ASTExpr.java: > 660) > at > org > .antlr > .stringtemplate.language.ActionEvaluator.action(ActionEvaluator.java: > 86) > at > org.antlr.stringtemplate.language.ASTExpr.write(ASTExpr.java:149) > at > org.antlr.stringtemplate.StringTemplate.write(StringTemplate.java:705) > at > org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java: > 1670) > at > org.antlr.stringtemplate.StringTemplate.toString(StringTemplate.java: > 1661) > at FFMParser.file(FFMParser.java:127) > at CSVTest.main(CSVTest.java:33) > > Is there any suggestion so that program can generate 1GB file? > > 1) Having multiple templates, each template generates small > string, then write the small strings to file. > My input has format: ((Part1) (Part2)+ (Part3)). For each part2, > there is an xml element in the output. I can have 3 templates here. > In Antlr action, I will write each part to file when parsing is done > for the part. > > 2) Any template engine that could write to file directly, > instead of returning a string...? (This might not be proper to > discussed here :D) > > I hope problem is stated clearly. Please let me know if you need > more information. Any suggestion is much appreciated. Thank you. > Regards, > Helen > > > > > > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: > http://www.antlr.org/mailman/options/antlr-interest/your-email-address --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en -~----------~----~----~----~------~----~------~--~---
List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
