run it against a straight dbapi program, and youll have an idea of the absolute fastest it can happen through python.
On Oct 25, 2006, at 6:49 PM, Mike Bernson wrote: > > I Known there are faster ways of loading the data. I am looking for > the > fastest way under SQLAlchemy. If the loads become far too slow I will > look into using methods outside of SQLAlchemy. At this stage there are > about 10,000 records being loaded and things are slow workable. I am > just checking to make sure that I am not overlooking something. > > Robin Munn wrote: >> On 10/25/06, Mike Bernson <[EMAIL PROTECTED]> wrote: >>> I am written a program that load mysql database from a flat file. >>> The table I am loading has a large number of columns about 800. >> >> 800 columns, and, judging from the code snippet you posted, a row >> count that numbers in the thousands? >> >> You don't want INSERT statements here. You want to use a bulk-load >> statement. For MySQL, looks like that's "LOAD DATA INFILE": >> http://www.mysql.org/doc/refman/5.0/en/load-data.html >> >> Even if you first have to massage the data into a format that MySQL's >> LOAD DATA INFILE likes, you'll still get much better performance than >> by trying the "lots of INSERT statements" approach. >> > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy -~----------~----~----~----~------~----~------~--~---
