Perhaps you could use a Java buffered reader or a line reader and read in a line at a time. This will stop the server from having to load an entire file at one time. Then for each line you can parse out the fields and apply to the DB or what ever.
I have some sample LineNumberReader code here: http://www.bennadel.com/blog/274-Java-Exploration-in-ColdFusion-java-io- LineNumberReader.htm (OR http://www.bennadel.com/index.cfm?dax=blog:274.view) This would create a LONG running page perhaps... Not sure how to deal with that. ...................... Ben Nadel Certified Advanced ColdFusion MX7 Developer www.bennadel.com Need ColdFusion Help? www.bennadel.com/ask-ben/ -----Original Message----- From: Will Swain [mailto:[EMAIL PROTECTED] Sent: Friday, December 15, 2006 3:36 PM To: CF-Talk Subject: splitting large csv file into smaller parts Hi, I have a tool that reads a nightly uploaded CSV file, and imports the data in it into a mySQL database. This works fine, after some messing around with the code to allow for consecutive blank fields in the CSV and stuff like that. Only problem is, some of these CSV files can be quite large, upto 8MB, with maybe 50,000 plus records, and these are giving CF server a bit of a headache. Is there a way I can split the CSV file after upload into, lets say, 8 1Mb files instead, and then process each one in turn. Maybe using CFFILE to read a limited number of rows, write them into another file and then delete them from the original. I can't think of a way to do this. Cheers Will ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Create robust enterprise, web RIAs. Upgrade & integrate Adobe Coldfusion MX7 with Flex 2 http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU Archive: http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:264186 Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
