I think that when you refer to each line in a field as part of the
repeat command, like
repeat for each line tLine in field "someField"
you are effectively placing the field's text into some internal
variable that is part of the engine's looping mechanism. I don't know
this for sure, but it seems as if something like that is happening.
What tends to really slow things down in a long loop is referring to
a field (or a chunk of a field) inside a loop, like
repeat 10000
put field "someField" into someVar
end repeat
And in the light of what I just found with building big lists, how
big is your final output file? I think the speed gain from 'append'
only applies once the list gets really big.
best,
Mark
On 16 Jul 2007, at 22:35, Beynon, Rob wrote:
Another round of interesting suggestions...thanks to all...
Apart from the database solution, I tried some of the others which
I had the skill to implement -
1. Write data to a file instead of a variable
2. Read search data into a variable from a field
I set up a search of 10,000 queries against 16,000 targets - maybe
100million comparisons
Using a file append rather than putting data after a variable:
4min 57sec using a variable (put)
4min 31sec using a file (write)
Reading the input search list from a field into a variable
4min 57sec using a variable (put) with no pre-read of input field
4min 57sec using a variable (put) with pre-read of input field
So, reading from a field makes no difference, and the gain in
writing to a file rather than pitting to a variable is also modest
(about 10% gain)
Apart from the fact that I'm still processing about 20million
comparisons a minute, which is impressive anyway, it seems as
though there are no immediate ways of enhancing processing speed.
Off now to read up on SQL databases...
Rob
_______________________________________________
use-revolution mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution
_______________________________________________
use-revolution mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution
