Thanks man. I'll pass this on to the person doing the coding. It looks interesting.
On 12/26/06, Ben Nadel <[EMAIL PROTECTED]> wrote: > Chris, > > I threw together a well-commented example of something that might > accomplish what you are talking about. It uses smaller record sets and > buffered output via the Java buffered output object. I am no Java coder, > so I am not always 100% sure what I am doing, but this might help point > you in a particular direction: > > > http://www.bennadel.com/blog/446-Writing-Enormous-Files-Based-On-Massive > -Record-Sets-In-ColdFusion.htm > (OR http://bennadel.com/index.cfm?dax=blog:446.view) > > Let me know if you have any questions. > > ...................... > Ben Nadel > Certified Advanced ColdFusion MX7 Developer > www.bennadel.com > > Need ColdFusion Help? > www.bennadel.com/ask-ben/ > > > -----Original Message----- > From: Chris Ditty [mailto:[EMAIL PROTECTED] > Sent: Tuesday, December 26, 2006 2:53 PM > To: CF-Talk > Subject: Re: Creating text files w/ 1mil + lines > > WOW. All I can say is duhhhh. This is way over my little CF head. :) > > On 12/26/06, Jochem van Dieten <[EMAIL PROTECTED]> wrote: > > Katz, Dov B \(IT\) wrote: > > > You shouldn't need to take memory into consideration if you use a > > > java.io.BufferedOutputStream and java.io.PrintWriter etc... > > > > Yes you should. Let's say we are writing 1.3 million, 20 character > records from the database to a .csv. First CF queries the database and > buffers the resultset (26 MB). In the proces it needs to convert from > the database charset to its internal charset (26 MB, maybe 52 MB). Then > you loop over the resultset and generate the .csv which you hold in > memory (26 MB). In the process of your 6 million loops you generate 8 > bytes of whitespace for each iteration (10.4 MB). Then you write it out. > > > > Of the total memory consumption of 94.4 MB (120.4 MB if you are > unlucky with the charsets), using Java to write it out saves you 26 MB. > And that is when we ignore the overhead of the different datatypes, with > the overhead the numbers would probably be worse. > > > > Jochem > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Create robust enterprise, web RIAs. Upgrade & integrate Adobe Coldfusion MX7 with Flex 2 http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:265042 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
