nekto0n wrote: > Wow. That was fast =) > Thanks for the fix. I'm a bit confused with results on 0.6beta3. Timer > showed significant speed boost and profiler (and breakpoint in > _CursorFairy) didn't show attempts to get rowcount. Am I doing > something wrong?
0.6 is faster overall but I checked the source code - it is definitely calling rowcount in beta3 before the result object is returned, and the mapper is also calling rowcount unconditionally for updates. What is strange is that the firebird dialect isn't caching the result or anything which suggests that FB's Cursor object might do this after rowcount is called the first time. So perhaps I need to check more deeply to see what kinterbasdbs requirements are here - can cursor.rowcount be called after the cursor is closed ? Or can we do that, only if its been called at least once before it was closed ? If for some reason the previous speed issue doesn't apply, I may have acted too quickly disabling rowcount by default. Most helpful would be if you could try the latest tip and see if turning on/off the enable_rowcount flag has an effect on speed. > > On Apr 12, 2:39 am, Michael Bayer <[email protected]> wrote: >> On Apr 9, 2010, at 4:51 PM, nekto0n wrote: >> >> > Hi there! >> > I'm now in the process of migrating from pure kinterbasdb to >> > SQLAlchemy. Everything is fine except for speed of insertions and >> > maybe updates. After some time of profiling I found out that a >> > significant amount of time is spent on getting "rowcount" after >> > execution. It takes about 5 seconds vs 41 spent on 3000 insertions >> > themselves. That is 12%. Not much, but still. >> > Digging further showed that rowcount attribute of kinterbasdb.Cursor >> > is a getter, which remotely calles to Firebird Database. Here is the >> > comment from _kicore_cursor.c source file: "Determining rowcount is a >> > fairly expensive operation that requires an isc_dsql_sql_info call". >> > I wonder if there's any possible way to make getting "rowcount" >> > optional or lazy? >> >> done some more digging here. The unfortunate thing is that our result >> object autocloses after any non-result returning operation, and as such >> it unconditionally retrieves the rowcount, typically a cheap/free >> operation, so that it is availble for those DBAPIs like kinterbasdb who >> require the cursor to be available. >> >> This means we can't make "rowcount" lazy. I can add an option to the >> kinterbasdb dialect to just turn off rowcount entirely, though. >> There seems to be ongoing debate about whether or not "rowcount" works >> at all with kinterbasdb. It seems to work perfectly over here, so it >> will just be up to the flag. >> >> Here's the CHANGES for that change: >> >> - The functionality of result.rowcount is now disabled >> by default, and can be re-enabled using the 'enable_rowcount' >> flag with create_engine(), as well as the 'enable_rowcount' >> execution context flag on a per-execute basis. This because >> cursor.rowcount requires cursor access (can't be evaluated >> lazily since the result auto-closes) and also incurs an >> expensive round-trip. >> >> so if you wanted to use the "versioning" feature with the ORM, you'd use >> an engine that has the flag turned on with that application. otherwise >> the ORM knows not to use rowcount. >> >> >> >> >> >> > Thanks! >> >> > -- >> > You received this message because you are subscribed to the Google >> Groups "sqlalchemy" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> [email protected]. >> > For more options, visit this group >> athttp://groups.google.com/group/sqlalchemy?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
