>- see footer for list info -< Simon, you're the man. Cheers, that's invaluable.
tbh, its got to be done in CF as there's so much that has to be done to the data before it can be written to the XML, which is why we're doing it - we have to call web services to validate the data coming out of SQL etc. But this string builder stuff should do the trick. I finally got it working by ramping the JVM max and min mem settings a la: <jvm-arg>-Xmx1024M</jvm-arg> <jvm-arg>-Xms1024M</jvm-arg> <jvm-arg>-Xmn512M</jvm-arg> in the resin.conf file, and that seemed to get through the whole batch without error, but if your code can save me a whole shedload of garbage collection, so much the better. I was having to batch the whole thing so each row (which in turn had child rows that ran their own reports) slept for a period of time before the next iteration. Will report back. Tanks, Rich 2009/12/6 Simon Baynes <[email protected]> > >- see footer for list info -< > > To be honest this something you should just get SQL server to do without > CFML. However, if you are definitely going down this path then there are a > few things you should know. In Java Strings are immutable this means that > you cannot change a string once it has been assigned. So if you do things > like:- > > <cfscript> > myString = "Simon"; > myString = myString & " Baynes"; > writeoutput(myString); > </cfscript> > > This doesn't actually update the variable but create a new variable in > memory that is the result of the second assignment. The old value will sit > in memory waiting to be Garbage Collected. I am sure you can now see that if > you are looping through a query doing loads of append statements then it's > going to create lots of information in memory. > > There is a Java class for this very type of operation called a > StringBuilder that manages this internally to reduce the memory usage. > > <cfscript> > oXml = createObject("java","java.lang.StringBuilder").init(); > oXml.append("Simon"); > oXml.append(" Baynes"); > writeoutput(oXml.toString()); > </cfscript> > > When using this for multiple append statements it'll save you lots of > memory! > > Having said that creating a 50Meg xml file will use at the very least 50Meg > of memory for that request which is not really what App Servers like Railo > or CF are for hence the advice of doing the whole job in SQL server. > > Hope that helps! > > Simon > Simon Baynes > www.simonbaynes.com > Free online storage you can access anywhere:- > http://www.simonbaynes.com/getsugarsync > LinkedIn profile:- http://www.linkedin.com/in/simonbaynes > > -----Original Message----- > From: Rich Wild <[email protected]> > Date: Sun, 6 Dec 2009 19:55:30 > To: Coldfusion Development<[email protected]> > Subject: [CF-Dev] Railo, Resin and large text variables = out of memory > > >- see footer for list info -< > Wotcha, > > I'm using Railo 3.1.2 + Resin 3.1 on an II6 Server. Its all been brill on > toast, thus far. > > Now, on this particular website, they have a requirement to build large XML > files produced from SQL Server databases. These XML files can be over 50Mb > in size. > > I'm running my SQL statements and building the XML using cfsavecontent > before saving out using cffile and zipping with cfzip. I figured producing > them as appended string variables as opposed to XML variables would save > time, processing and memory. > > However, I inconsistently get errors whilst building these files: > > Railo 3.1.2.001 Error (Java.lang.outofmemoryerror) - Java heap space > > Hmmm. That's not good. > > I tracked down the resin.conf file and changed > > <jvm-arg>-Xmx256m</jvm-arg> > to > <jvm-arg>-Xmx512m</jvm-arg> > > to try and increase the memory available. I still get the errors. > > Anyone have any clues how I can make this go away? I'm sure a comparable > CF8/9 server on the same hardware config machine wouldn't have this > problem. > They're not THAT big XML files. > > Cheers, > Rich > _______________________________________________ > > For details on ALL mailing lists and for joining or leaving lists, go to > http://list.cfdeveloper.co.uk/mailman/listinfo > > -- > CFDeveloper Sponsors:- > >- cfdeveloper Hosting provided by www.cfmxhosting.co.uk -< > >- Lists hosted by www.Gradwell.com -< > >- CFdeveloper is run by Russ Michaels, feel free to volunteer your help -< > > _______________________________________________ > > For details on ALL mailing lists and for joining or leaving lists, go to > http://list.cfdeveloper.co.uk/mailman/listinfo > > -- > CFDeveloper Sponsors:- > >- cfdeveloper Hosting provided by www.cfmxhosting.co.uk -< > >- Lists hosted by www.Gradwell.com -< > >- CFdeveloper is run by Russ Michaels, feel free to volunteer your help -< > _______________________________________________ For details on ALL mailing lists and for joining or leaving lists, go to http://list.cfdeveloper.co.uk/mailman/listinfo -- CFDeveloper Sponsors:- >- cfdeveloper Hosting provided by www.cfmxhosting.co.uk -< >- Lists hosted by www.Gradwell.com -< >- CFdeveloper is run by Russ Michaels, feel free to volunteer your help -<
