Koen,
This is a question of theory. I'm not going to lay out the entire set
of procedures that get run here. Presumptuous of you to make that
"unacceptable" statement without knowing what actually happens behind
the scenes. I could be talking about a process that does 20 different
things, checking this, that, and another thing, writing here, writing
there. As another mentioned, the bulk of time was the API (outside of
my actual process), anyway.
On 8/1/2020 3:30 PM, Koen Piller wrote:
Also missing the code for your procedure ProcessRecord()
Op za 1 aug. 2020 om 20:05 schreef Koen Piller <[email protected]>:
Hi,
64000 records in 17 hours???? That is way unacceptable. To run the code
you published would take normally around minutes max. I suppose the most
of the time is not by your code but the API call to get the data into the
cursor ListOfRecsToProces, advise you to implement Codereference to find
the time consuming bottle-neck.
Instead of scan..endscan you may also try to make use of the SQL Update()
function, this should be faster.
Koen
Op za 1 aug. 2020 om 19:07 schreef MB Software Solutions, LLC <
[email protected]>:
I've got a regular process that runs, basically using key information to
grab data from an API and then update the local VFP database. There are
maybe 64000 records to process, and each record to update through this
process takes about a second, so to process this group would take over
17 hours. Each record could be processed on its own; there are no
relationships between each.
I don't want to start it and run the 64000 in a row for 17+ hours. I
would like to design the app to use the table, RLOCK the row I'm
processing, and the UNLOCK the row when I'm done. I figure with this
design, I could run multiple instances of the MyProgram.exe (similar to
how WestWind Web Connection allows you to run multiple instances) to
process the batch maybe 4x faster (if I launched 4 instances of
MyProgram.exe). The basic construct would be as follows:
USE ListOfRecsToProcess IN 0 SHARED Alias MyList && record is PK (to
process) i, tProcessed t, tError t, cSession c(10)
SCAN FOR EMPTY(cSession) AND RLOCK('MyList')
IF ProcessRecord(MyList.ID) THEN
REPLACE tProcessed WITH DATETIME(), cSession WITH this.cSession
IN MyList
ELSE
REPLACE tError WITH DATETIME(), cSession WITH this.cSession IN
MyList
ENDIF
ENDSCAN
Does anybody see any problems with that general design? The
ProcessRecord method calls an API to get values and then updates the
local VFP record accordingly.
tia,
--Mike
--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
[excessive quoting removed by server]
_______________________________________________
Post Messages to: [email protected]
Subscription Maintenance: https://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: https://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: https://leafe.com/archives
This message:
https://leafe.com/archives/byMID/[email protected]
** All postings, unless explicitly stated otherwise, are the opinions of the
author, and do not constitute legal or medical advice. This statement is added
to the messages for those lawyers who are too stupid to see the obvious.
