I am written a program that load mysql database from a flat file.
The table I am loading has a large number of columns about 800.
Here is simple version of what I am doing:
records = []
for line in lines:
record = {}
for col in columns:
record[col] = line[info[col].start: info[col].end]
records.append(record)
if count % 1000 == 0 and count > 1:
table.insert().execute(records)
records = []
I do a commit when all records have been loaded.
I have put print statement with times before and after the insert /
execute. I am seeing about 10 seconds to build records and 45 seconds
around the insert/execute.
Yes I known that insert is needed after the loop and commits are also
needed but I am trying to get a very simple example.
Is there a faster way to get SQLAlchemy to insert the records into the
database ?
Trying session and creating the class object to call save with was very
slow.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sqlalchemy
-~----------~----~----~----~------~----~------~--~---
