I Known there are faster ways of loading the data. I am looking for the fastest way under SQLAlchemy. If the loads become far too slow I will look into using methods outside of SQLAlchemy. At this stage there are about 10,000 records being loaded and things are slow workable. I am just checking to make sure that I am not overlooking something.
Robin Munn wrote: > On 10/25/06, Mike Bernson <[EMAIL PROTECTED]> wrote: >> I am written a program that load mysql database from a flat file. >> The table I am loading has a large number of columns about 800. > > 800 columns, and, judging from the code snippet you posted, a row > count that numbers in the thousands? > > You don't want INSERT statements here. You want to use a bulk-load > statement. For MySQL, looks like that's "LOAD DATA INFILE": > http://www.mysql.org/doc/refman/5.0/en/load-data.html > > Even if you first have to massage the data into a format that MySQL's > LOAD DATA INFILE likes, you'll still get much better performance than > by trying the "lots of INSERT statements" approach. > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy -~----------~----~----~----~------~----~------~--~---
