I should preface this by saying I'm a huge fan of SQLAlchemy. Huge! However, when trying to extol the virtues of SQLAlchemy I inevitably run into this issue:
"But it's slow!" My usual response to that is that, yes, it is somewhat slower than raw MySQL or PostgreSQL or whatever DB-API you are using, but that performance hit (if it's present at all) is outweighed by <the eleventy billion awesome things listed here>. I wrote some code (which I can send if it's useful) that compares fetching 10 million rows from a table with 2 integer columns. The comparison is between raw psycopg2 and two variations using SQLAlchemy: stream=False, stream=True. raw psycopg2 is consistently in the 5.5 to 6.0 second range SQLAlchemy is in the 20 second range. The C extensions are in use, and this is with SQALchemy 1.0.6. The run looks like this: SQLAlchemy version: 1.0.6 psycopg2 version: 2.5.2 (dt dec pq3 ext) PostgreSQL version: PostgreSQL 9.3.6 on x86_64-suse-linux-gnu, compiled by gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064], 64-bit sqlalchemy.cresultproxy is available. using psycopg2: 10000000 rows in 5.78s (1729246.93 row/s) using sqlalchemy, with stream=False: 10000000 rows in 17.71s (564795.64 row/s) using sqlalchemy, with stream=True: 10000000 rows in 20.65s (484226.33 row/s) I would wager that any low-hanging performance fruit has already been picked, but what might a plan of attack look like here? -- Jon Nelson Dyn / Senior Software Engineer p. +1 (603) 263-8029 -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
