Title: Message
|
If you get the Query to do the concatenation - 1
AsString per row.
Can you do what you require in a StoredProc and
return 1 large string which you then deal to (I can't remember what you were
doing with it).
Regards
Paul McKenzie
Analyst Programmer
SMSS Ltd.
----- Original Message -----
Sent: Friday, August 09, 2002 8:45
AM
Subject: RE: RE: [DUG]: File Export
Speed
I
did this and we got a marginal speed improvement of about 10 seconds (out of
180).
Doing further testing I found that the .AsString takes the majority of
the time.
Stacey
Corey wrote:
----- Original Message -----
Sent: Wednesday, August 07, 2002
10:20 AM
Subject: RE: [DUG]: File Export
Speed
LRow :=
'';
for i := 0 to PQuery.FieldCount -
1 do begin
LRow := LRow +
PQuery.Fields[i].AsString + '|';
end;
LRow := LRow +
#13#10;
This string concatenation
will be very slow - it is a N squared algorithm for the number of bytes
copied ie bad news.
Not quite that
bad. AFAIK Strings allocate memory a block at a time... but
it's still pretty bad. I wrote a program that passed Strings down a
recursive tree search, then rewrote it using PChars and got a major speed
increase... run time went from an average of 2 minutes to a little
over 18 seconds. Definitely worth the extra coding
effort.
Yes, it is an O(N
squared) algorithm since each string concatenation will require an extension
which means allocating a new bigger string and copying the result over.
Copying the string will take an amount of time proportional to the current
length (a function of N) and this will be done a number of times that is a
function of N. Hence, overall O(N squared) where N is Rows *
Fields.
Also, the Delphi
memory manager does not handle the continual expansion access pattern very
well and gets heavily fragmented which means even the allocation step
may well start taking time proportional to some log of N. So it might even
get up to O(N squared log N). Nasty!
Change the code to
write each string field independently (i.e. avoid concatenations) and the
algorithm will run much faster and still offer text output. Some buffering
on the TFileStream may help as well to avoid many small writes but the OS
should do a fair job of that for you.
TTFN,
Paul.
|
