I have a DBase3 table (don't ask!) that is read only (to my app) and
changes several times daily. So I have need to frequently import it.
(This is a TG front end to an MS Goldmine database. Hopefully the
first step to replacing Goldmine with a TG app.)
At any rate, it has about 85000 records and I am disappointed with the
import speed and puzzled about this behavior. The controller method is
below. I am using ActiveMapper.
When the method runs, it goes through the while loop rather quickly.
(about a minute.) Then python/TG/SQLAlchemy sits and churns for a few
minutes doing I'm not sure what. So far, there has been absolutely no
mysqld activity and the db file sizes have not changed.
Then I start seeing mysqld activity. About 7% of processor, with
python taking up the other 93% and records get inserted into the
database at a rate of about 200/sec, so this takes several minutes to
complete.
>From the description in RWAwTG (THE BOOK) I thought that each run
through the while loop, TG/ActiveMapper was explicitly doing a
session.save() and session.flush(). So I would expect to start seeing
MySQL activity immediately, and for the whole thing to complete and
display the template as soon as the loop finished.
What is going on? And how can I speed this thing up?
I don't really need transactions, as there should never be a problem
importing this, and if there is, I would run the whole import again
after fixing the problem.
Thanks,
Steve
=============
def getcontact2(self):
db = dBase3('./Contact2.DBF')
i = 1
while True:
try:
contact = db.GetRecordAsDictionary(i)
for key, value in contact.iteritems():
if isinstance(value, str):
contact[key] = value.strip()
c = Contact2(**contact)
except:
break
i += 1
return i
==============
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"TurboGears" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/turbogears?hl=en
-~----------~----~----~----~------~----~------~--~---
