25 MB CSV is a _lot_ for CF to work with. If the possibility exists that CF wil have to write out 50MB files on a production server, I would look into other ways of doing this. ESPECIALLY if more than one person would have access to this module at a time.
If you are forced to work with CF on this, try parsing each file line by line and writing the output to a temp file. This way, you can avoid performance issues and not lock up the server. Mike -----Original Message----- From: Rosa, Issac [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 06, 2002 10:30 AM To: CF-Talk Subject: Parsing and combining large files I have a project where I need to extract information from many comma delimited text files and Excel files, do some replacements, then combine the information in another comma delimited text file(s). The files are anywhere from 100KB to 25MB. Can CF handle this or should I be looking at a different solution? Any suggestions would be great. Here's an example. I have 45 or more files with this type of information. The rows repeat for many transactions with first 4 rows always being there, the 5th row (BatchCount) may or may not be be there and then at the end of the file, the last 4 rows will always be there. Filename = D0624MFM.102001 Sample file with only 2 entries and totals. D0624,,,,,,,,PAYMENTDATE,11/20/2001,10/20/2001 D0624,,,,,,,,CREATEDATE,11/20/2001,10/20/2001 D0624,,,,,,,,BATCHNUMBER,112101BP,10/20/2001 D0624,,,,,,,,BATCHTOTAL,25857.33,10/20/2001 D0624,,,,,,,,BATCHCOUNT,00047,10/20/2001 D0624,,,,,,,,PAYMENTDATE,11/20/2001,10/20/2001 D0624,,,,,,,,CREATEDATE,11/20/2001,10/20/2001 D0624,,,,,,,,BATCHNUMBER,P1117LE,10/20/2001 D0624,,,,,,,,BATCHTOTAL,200.00,10/20/2001 D0624,,,,,,,,UNBILLED,416486.69,10/20/2001 D0624,,,,,,,,BILLED,4406920.01,10/20/2001 D0624,,,,,,,,GRAND,4823406.70,10/20/2001 D0624,,,,,,,,TOTALCOUNT,02432,10/20/2001 I need to create the following comma delimited file based on the layout below and the above information. RECORD TYPE = 'D' (hardcoded) IMPORT ACCT ID = 'D0624SE' (hardcoded based on MFM = SE) POST DATE = Date after PAYMENTDATE (format = yyyymmdd) EFFECTIVE DATE = Date after PAYMENTDATE (format = yyyymmdd) TRANSACTION TYPE = based on the characters in BATCHNUMBER (ie P, BP) CREDIT/DEBIT = 'C' or 'D' hardcoded based on BATCHTOTAL AMOUNT = BATCHTOTAL REF 3 = BATCHNUMBER REF 4 = 3 characters before . in file name (ie MFM in D0624MFM.102001) D,D0624SE,20011020,20011020,TN,D,200.00,P1117LE,MFM D,D0624SE,20011020,20011020,BP,D,25857.33,112101BP,MFM Thanks, ______________________________________________________________________ Dedicated Windows 2000 Server PIII 800 / 256 MB RAM / 40 GB HD / 20 GB MO/XFER Instant Activation � $99/Month � Free Setup http://www.pennyhost.com/redirect.cfm?adcode=coldfusiona FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Archives: http://www.mail-archive.com/[email protected]/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
