Hi Martin -
I completely agree with the other comments that you should be using a DB to
this task. However, I wanted to mention that today I cobbled together a
blazing fast way of reading and appending to large text files with CF + Java.
<!--- ************** JAVA Reader + CF Array
*************************************************** --->
<!--- fastest way to read and append in CF as of 2007-01-29
only tested on CFMX 6.1
--->
<cfscript>
// set it to whatever file name you want (include the full path)
myFile = myFile;
startTime = GetTickCount();
// init file reader and buffered reader to read file line by line
// 'lifted' some excellent code by Greg Lively -
http://www.informationsavvy.com/coldfusion/
variables.joInputStreamReader =
CreateObject("java","java.io.FileReader").init(javaCast("string", myFile));
variables.joBufferedReader =
CreateObject("java","java.io.BufferedReader").init(variables.joInputStreamReader,
javaCast("int", 1024));
initFileReaderTime = GetTickCount() - startTime;
// create an array for appending to
// (fastest append method in CF - see
http://www.bennadel.com/blog/275-Passing-Arrays-By-Reference-In-ColdFusion-SWEEET-.htm)
arrFileOut = ArrayNew(1);
preLoopOfFileReaderTime = GetTickCount();
// loop through until readLine() fails due to being null, which would
throw a coldfusion.runtime.UndefinedVariableException
// exception when setting variables.inStr = variables.inStr_good;
isEOF = false;
while ( not isEOF ) {
try {
variables.inStr_good = variables.joBufferedReader.readLine();
variables.inStr = variables.inStr_good;
ArrayAppend(arrFileOut, variables.inStr);
ArrayAppend(arrFileOut, "???<br>");
}
catch(Any excpt) {
isEOF = true;
}
}
loopOfFileReaderTime = GetTickCount() - preLoopOfFileReaderTime;
// always remember to close your Java objects
variables.joBufferedReader.close();
variables.joInputStreamReader.close();
totalJavaTime = GetTickCount() - startTime;
preArrayToListTime = GetTickCount();
// convert it back to a string
fileOutputFromList = ArrayToList(arrFileOut, "");
arrayToListTime = GetTickCount() - preArrayToListTime;
</cfscript>
<!--- <cfoutput>#fileOutputFromList#</cfoutput> --->
<cfoutput>initFileReaderTime: #initFileReaderTime# ms</cfoutput><br>
<cfoutput>loopOfFileReaderTime: #loopOfFileReaderTime# ms</cfoutput><br>
<!--- <cfoutput>ArrayToList time: #arrayToListTime# ms</cfoutput><br> --->
<h2><cfoutput>JAVA Reader + CF Array: #totalJavaTime# ms</cfoutput></h2>
<!--- /************** JAVA Reader + CF Array
*************************************************** --->
>Hello all.
>
>
>
>I am uploading a 74 + MB tab delimited text file that I am then reading
>and inserting the values into a database. The problem is it always
>times out, or just takes too long (did not finish over night!!!!) to
>read the file. It is uploaded fine.
>
>
>
>Any suggestions about how I may approach this to make it work with such
>large files?
>
>
>
>I looked at a bit of Java code someone had posted here but it went no
>quicker really.
>
>
>
>I was thinking of maybe chopping the file into slices and then
>processing but not too sure how I would approach this.
>
>
>
>Any opinions or ideas/tips would be gratefully accepted.
>
>
>
>Thanks for reading.
>
>Cheers
>
>Martin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Upgrade to Adobe ColdFusion MX7
Experience Flex 2 & MX7 integration & create powerful cross-platform RIAs
http:http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU
Archive:
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:267956
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe:
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
