Hi Terry,Any thoughts on how this is possible and what we can learn from it when making programs?
Before dealing specifically with 1 @ 60 hours vs 60 @ 2 minutes, I need to know if you are actually deleting each duplicate line on the fly? Have you tried building a new list instead:
functon purgeDuplicates @textData
put empty into newData
put numToChar(30) into lastLine -- any char not in the first line of textData
repeat for each line thisLine in textData
if thisLine = lastLine then next repeat
put thisLine&return after newData
put thisLine into lastLine
end repeat
return newData -- or write as a command and "put newData into textData"
end purgeDuplicates
I'd be curious to know what algorithm you used and what times the above handler produces.
Other things to look at:
1. Is it possible you are maxed out in actual RAM and spending a lot of time reading from/writing to virtual memory?
2. Are you passing the 55K lines of text by value or reference?
3. Have you tried writing your handler inline with the handler that reads in the data so it needn't be passed at all?
Eg:
put get URL (whatever) into textData
put empty into newData
put numToChar(30) into lastLine -- any char not in the first line
repeat for each line thisLine in textData
if thisLine = lastLine then next repeat
put thisLine&return after newData
put thisLine into lastLine
end repeat
put newData into textData
instead of
put get URL (whatever) into textData
put purgeDuplicates(textData) into textData
(although if textData is passed by reference, the impact of item three is negligible).
--
Rob Cozens
CCW, Serendipity Software Company
http://www.oenolog.com/who.htm
"And I, which was two fooles, do so grow three;
Who are a little wise, the best fooles bee."
from "The Triple Foole" by John Donne (1572-1631)
_______________________________________________
use-revolution mailing list
[EMAIL PROTECTED]
http://lists.runrev.com/mailman/listinfo/use-revolution
