Thank you Richard. Great feedback. That makes complete sense as to why repeat for each line is lightning fast
On Wednesday, April 09, 2008, at 06:36PM, "Richard Gaskin" <[EMAIL PROTECTED]> wrote: >Jim Schaubeck wrote: > > Very good feedback for me. You are correct , the method I was > > using was very slow (I had no idea). > >Superficially the two main forms of repeat look very similar, but under >the hood they do very different things. > >When you do this: > >repeat with i = 1 to the number of lines of tData > get line i of tData >end repeat > >...that second line has to count the lines from 1 to i each time through >the loop. That's why you saw the increasing slowdown the farther it got >into the data. > >But when you do this: > >repeat for each line tLine in tData > get tLine >end repeat > >...then the engine makes the assumption that the data in tData won't be >changing, so it doesn't need to count as it goes. Instead it parses as >it goes, automatically putting the value of each line into tLine each >time through the loop. > >For data sets with just 5000 records, or even 50,000 records, you >probably don't need an RDBMS to handle them. > >In cases where you're processing all records in sequence you probably >don't even need an array. Arrays are lightning fast for random access, >such as when you have a list of keys and you need to retrieve their >values. But the split and combine commands are very computationally >intensive, so for sequential processing of the full data set the >overhead of split and combine usually benchmarks as taking longer than >simply using "repeat for each" on the delimited text. > >My WebMerge customers regularly process data sets in the hundreds of >thousands of lines, and write me happy notes about how good the >performance is. :) > >I wish I could take credit for it, but really it's all Scott Raney, the >fella who owned the engine at the time the "repeat for each" form was >added. It's a godsend. > >-- > Richard Gaskin > Managing Editor, revJournal > _______________________________________________________ > Rev tips, tutorials and more: http://www.revJournal.com >_______________________________________________ >use-revolution mailing list >[email protected] >Please visit this url to subscribe, unsubscribe and manage your subscription >preferences: >http://lists.runrev.com/mailman/listinfo/use-revolution > > _______________________________________________ use-revolution mailing list [email protected] Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-revolution
