On Apr 12, 2010, at 1:00 AM, nekto0n wrote: > Wow. That was fast =) > Thanks for the fix. I'm a bit confused with results on 0.6beta3. Timer > showed significant speed boost and profiler (and breakpoint in > _CursorFairy) didn't show attempts to get rowcount. Am I doing > something wrong?
OK, something I missed before, we are only calling rowcount with an UPDATE or DELETE, as well as after any DDL (which is pointless), but its not called after an INSERT (this is new in 0.6). I've added a test to ensure this remains the case. I did some profiling and saw that FB's cursor.rowcount is slightly expensive (about 4% of a particular test run that did 1000 UPDATEs). I've changed the enable_rowcount option to be True by default, since its impact is not as great now, only executing after UPDATE and DELETE. You can still disable the usage of rowcount across the board if desired. > > On Apr 12, 2:39 am, Michael Bayer <[email protected]> wrote: >> On Apr 9, 2010, at 4:51 PM, nekto0n wrote: >> >>> Hi there! >>> I'm now in the process of migrating from pure kinterbasdb to >>> SQLAlchemy. Everything is fine except for speed of insertions and >>> maybe updates. After some time of profiling I found out that a >>> significant amount of time is spent on getting "rowcount" after >>> execution. It takes about 5 seconds vs 41 spent on 3000 insertions >>> themselves. That is 12%. Not much, but still. >>> Digging further showed that rowcount attribute of kinterbasdb.Cursor >>> is a getter, which remotely calles to Firebird Database. Here is the >>> comment from _kicore_cursor.c source file: "Determining rowcount is a >>> fairly expensive operation that requires an isc_dsql_sql_info call". >>> I wonder if there's any possible way to make getting "rowcount" >>> optional or lazy? >> >> done some more digging here. The unfortunate thing is that our result >> object autocloses after any non-result returning operation, and as such it >> unconditionally retrieves the rowcount, typically a cheap/free operation, so >> that it is availble for those DBAPIs like kinterbasdb who require the cursor >> to be available. >> >> This means we can't make "rowcount" lazy. I can add an option to the >> kinterbasdb dialect to just turn off rowcount entirely, though. There >> seems to be ongoing debate about whether or not "rowcount" works at all with >> kinterbasdb. It seems to work perfectly over here, so it will just be up to >> the flag. >> >> Here's the CHANGES for that change: >> >> - The functionality of result.rowcount is now disabled >> by default, and can be re-enabled using the 'enable_rowcount' >> flag with create_engine(), as well as the 'enable_rowcount' >> execution context flag on a per-execute basis. This because >> cursor.rowcount requires cursor access (can't be evaluated >> lazily since the result auto-closes) and also incurs an >> expensive round-trip. >> >> so if you wanted to use the "versioning" feature with the ORM, you'd use an >> engine that has the flag turned on with that application. otherwise the ORM >> knows not to use rowcount. >> >> >> >> >> >>> Thanks! >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "sqlalchemy" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group >>> athttp://groups.google.com/group/sqlalchemy?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
