Will, Glad to help. Good luck with the kids. My brother just had his second girl in October; not twins, but a handful non-the-less. Anyway, let me know if you have any questions regarding the code I wrote.
...................... Ben Nadel Certified Advanced ColdFusion MX7 Developer www.bennadel.com Need ColdFusion Help? www.bennadel.com/ask-ben/ -----Original Message----- From: Will Swain [mailto:[EMAIL PROTECTED] Sent: Saturday, December 16, 2006 7:02 AM To: CF-Talk Subject: RE: splitting large csv file into smaller parts That's great Ben, just what I needed. Funnily enough, I'd made a start on this last night in between baby feeds (I have 3 month old twins) but that looks great. Cheers, and have a great weekend! Will -----Original Message----- From: Ben Nadel [mailto:[EMAIL PROTECTED] Sent: 16 December 2006 00:22 To: CF-Talk Subject: RE: splitting large csv file into smaller parts Will, I had some time to throw something together. This should take care of the out of memory issues as it doesn't read in the whole file at once: http://www.bennadel.com/blog/436-Breaking-Enormous-CSV-Files-Into-Smalle r-CSV-Files.htm (OR http://bennadel.com/index.cfm?dax=blog:436.view) This splits the CSV file into many smaller files. Then, you would run your originally parser on the smaller files, perhaps one per page request. This should not kill your memory at all. Just remember to delete the smaller CSV files once you are done with them (or they might corrupt the next night's CSV data). Have a great weekend. ........................ Ben Nadel Certified Advanced ColdFusion MX7 Developer www.bennadel.com Need ColdFusion Help? www.bennadel.com/ask-ben/ -----Original Message----- From: Will Swain [mailto:[EMAIL PROTECTED] Sent: Friday, December 15, 2006 4:23 PM To: CF-Talk Subject: RE: splitting large csv file into smaller parts Well, my test has just run into problems and thrown a java.lang.OutOfMemoryError error. I'm thinking though maybe I can use the LineNumberReader code to split the csv file into smaller parts. Maybe take 5,000 rows at a time and write them out to a new csv file. Then I can just run the processing code that I already have on each smaller file. Does that logic seem sound to you? Cheers Will ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Create robust enterprise, web RIAs. Upgrade & integrate Adobe Coldfusion MX7 with Flex 2 http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:264291 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
