>- see footer for list info -< Perhaps the code I gave you for csv2query can be improved, I only used it for small files so it was fine for me. There is another one at cflib.org maybe it will perform better. Although manually building queries is never fast. Of course it would be better to convert from CSV directly to XML and cut out that step completely, so you could always modify the query2xml code to work with the csv instead.
Russ -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Lee Fortnam Sent: 10 September 2006 14:30 To: 'Coldfusion Development'; 'database programming' Subject: [CF-Dev] Import of 10k records from a tab delimited txt >- see footer for list info -< Okay, So I thought I was getting somewhere. The following page http://www.appointmentm8.com/samplescripts/test/xmlStoredProcInsert2.cfm is the one I am playing with. The problem seems to be that the CSVToQuery is taking a long time (for 1054 records approx 10 or 11 seconds), the number manipulation that follows, i.e. converting 0121 287 2343 to 441212872343 takes only a second considering it is looping through each of the records in the query and performing an update. The conversion to XML is taking approx 3 or 4 seconds per 1000 rows so that is not too bad and the DB insert is taking very little. I have been setting the timeout of the page using cfsetting to just over the 20 second mark which is the rough time to cater for 1000 records. If there are more records in the file, this timeout setting is automatically adjusted to go to 40 seconds for 2000 records etc. I would therefore expect the page to take about 200 seconds for 10K records but the page is failing if I use a file this big. Have I just reached a limitation or is there something I am doing wrong. This really is proving to more of a pain in the bum than I expected. Lee Fortnam _______________________________________________ For details on ALL mailing lists and for joining or leaving lists, go to http://list.cfdeveloper.co.uk/mailman/listinfo -- CFDeveloper Sponsors:- >- cfdeveloper Hosting provided by www.cfmxhosting.co.uk -< >- Lists hosted by www.Gradwell.com -< >- CFdeveloper is run by Russ Michaels, feel free to volunteer your help >-< _______________________________________________ For details on ALL mailing lists and for joining or leaving lists, go to http://list.cfdeveloper.co.uk/mailman/listinfo -- CFDeveloper Sponsors:- >- cfdeveloper Hosting provided by www.cfmxhosting.co.uk -< >- Lists hosted by www.Gradwell.com -< >- CFdeveloper is run by Russ Michaels, feel free to volunteer your help -<
