On 4/22/15 12:07 PM, John Doe wrote:
On Wednesday, April 22, 2015 at 3:20:13 PM UTC+1, Michael Bayer wrote:
a COMMIT every time would slow it down, yes, but then in your
profiling you'd see the do_commit() method taking up that time as
well.
if you can get your logging going, you'll see what SQL its
emitting, and you can also set up the formatting to have
timestamps so you can measure the time spent between queries.
Also, if you can, try out the pymssql driver, instead of pyodbc.
pymssql is much better maintained these days.
Will try that, thanks.
In the meanwhile, I have tried your suggestions. The to_sql method
creates a single insert statement and then commits at the end. I can't
see anything which would justify such a low speed. The mistery remains!
If I can abuse your patience a bit more, is there a way to:
1) display the entire SQL statements created? The method you suggested
shows only 10 records, and adds the text "displaying 10 of 1000 total
bound parameter sets" when run on a 1000-record table
use an event listener and do the logging, printing, and/or debugging
that you want. I recommend before_cursor_execute():
http://docs.sqlalchemy.org/en/rel_1_0/core/events.html?highlight=before_cursor_execute#sqlalchemy.events.ConnectionEvents.before_cursor_execute
2) generate the SQL statements but without connecting to the server?
If I can get a text file with the insert statement, then I can run
that from within my database client, and it will be faster
Stick with #1. This can be more challenging because the script needs
to only emit statements that don't expect results. There is a tool
called the "mock strategy" that will do this which has a short example
at
http://docs.sqlalchemy.org/en/rel_1_0/faq/metadata_schema.html#how-can-i-get-the-create-table-drop-table-output-as-a-string.
I have also been very surprised by the lack of info on the net.
Either I am the only one using the to_sql method, or I am the
only one who finds it so slow!
--
You received this message because you are subscribed to the
Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected] <javascript:>.
To post to this group, send email to [email protected]
<javascript:>.
Visit this group at http://groups.google.com/group/sqlalchemy
<http://groups.google.com/group/sqlalchemy>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
