On Tuesday 26 May 2009, Jason Durham wrote:
I have been tasked with providing a mechanism to upload/parse/perform a
couple of checks/save CSV data. The current test file is 28,000 records
This comes up now and again on the list, so I bloged my response to keep it to
hand :
I have found that using the Java Line Reader works great. It doesn't
load a text file into memory but will read it line by line which is
perfect for a CSV.
There is a CFC out there called filereader.cfc that should get you started.
J.J.
On Tue, May 26, 2009 at 10:52 AM, Jason Durham
Check out: http://msdn.microsoft.com/en-us/library/ms188365.aspx
For information about preparing data for bulk import, such as the
requirements for importing data from a CSV data file, see...
On Tue, May 26, 2009 at 11:52 AM, Jason Durham jdur...@cti-stl.com wrote:
I have been tasked with
Jason,
This is not what you want to hear but
My advice would be don't use CF to do this. DTS (or SSIS) is quite capable
of doing this in about 1/30 the time that CF is able to do this. One of our
DTS packages consumes a file of over 300k rows in less than 15 seconds every
few minutes. You
Jason Durham wrote:
What do the CF wizards suggest?
Push this work into the database using stored procedures and bulk
loaders. Use ColdFusion to get the file, move it somewhere the database
bulk loader can get to it, fire off the stored procedure, and notify the
user of any results or
Mark Kruger wrote:
Jason,
But if you insist on doing it with CF, don't use CFFILE. Instead read the
lines in one at a time using a line reader technique - that will allow you
to handle one line at a time instead of reding the entire file into memory.
I also recommended pushing the heavy
My advice would be don't use CF to do this. DTS (or SSIS) is quite capable
of doing this in about 1/30 the time that CF is able to do this. One of our
DTS packages consumes a file of over 300k rows in less than 15 seconds every
few minutes. You can't get that with CF no matter how hard you try :)
To: cf-talk
Subject: Re: Parsing large CSVs and inserting in MSSQL
Mark Kruger wrote:
Jason,
But if you insist on doing it with CF, don't use CFFILE. Instead read
the lines in one at a time using a line reader technique - that will
allow you to handle one line at a time instead of reding
+2 for DTS/SSIS
Case in point. Back in the CF 5 (or 6) days I tried to import a 20+ meg MLS
xml file into SQL server using CF (Yeah I know... I had no idea at the
time). The first run just about killed my dev server. After 30-60 seconds
the task manager showed that the machine was using 750 megs
Thanks to everybody who has replied. It appears I need to do some
homework on MS SQL. :)
-Original Message-
From: Mark Kruger [mailto:mkru...@cfwebtools.com]
Sent: Tuesday, May 26, 2009 11:03 AM
To: cf-talk
Subject: RE: Parsing large CSVs and inserting in MSSQL
Jason
) 408-3733 ext 105
www.cfwebtools.com
www.coldfusionmuse.com
www.necfug.com
-Original Message-
From: Jason Durham [mailto:jdur...@cti-stl.com]
Sent: Tuesday, May 26, 2009 1:27 PM
To: cf-talk
Subject: RE: Parsing large CSVs and inserting in MSSQL
Thanks to everybody who has replied
, May 26, 2009 1:37 PM
To: cf-talk
Subject: RE: Parsing large CSVs and inserting in MSSQL
Jason,
Here's a big uh oh for you If you are using MSSQL express you
don't
have the SSIS services that come with it. DTS may be possible through
installing the active X controls on the same server - but you
12 matches
Mail list logo
