my guess is that You use sequential IO on the CSV-file and that is what eats time.
If You have memory enough to read the entire file inte memory and REMOVE lines
instead of READSEQ them You'll see a _dramatic_ performance increase.
Else split Your case into say four cases of ruoghly the same length with all lastnames sorting before say 'F' in the first one and those between F and M in the
second etc . This approach will reduce the time in the CASE constructs by a factor four.
Wich of course may turn out to be only some percent of total time :-(
/Mats
George Gallen wrote:
I can't setup any indexs to speed this up. Basically I'm scanning a CSV file for names to remove and set the flag of KICK=1 to remove it (creating a new CSV file at the same time).
Keep in mind the ".." are people's last names, or zip codes, or part of their address, changed them to ".." to protect the unwanting...
Right now, I do a series of CASE's ... Now, it's not a major problem as I'm only checking for 20 or so names, but as more and more people request to be removed (and we don't have access to the creation of the list). this could get quite slow over 50 or 60 thousand lines of checking.
LIN is one line of the CSV file, the INDEX is checking for a last name & a zip code and sometimes part of the address line.
Any Ideas?
Remember, we can't change the source of the file, it will always be a CSV, being read line by line
KICK=0
BEGIN CASE
CASE -1
KICK=1
BEGIN CASE
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 AND
INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0
CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 AND
INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE INDEX(LIN,"..",1)#0 AND INDEX(LIN,"..",1)#0 CASE -1
KICK=0
END CASE
END CASE
George Gallen Senior Programmer/Analyst Accounting/Data Division [EMAIL PROTECTED] ph:856.848.1000 Ext 220
SLACK Incorporated - An innovative information, education and management company http://www.slackinc.com
_______________________________________________ u2-users mailing list [EMAIL PROTECTED] http://www.oliver.com/mailman/listinfo/u2-users
_______________________________________________ u2-users mailing list [EMAIL PROTECTED] http://www.oliver.com/mailman/listinfo/u2-users
