I did need the objects, not just the raw data, otherwise I'd had to duplicate a bunch of existing code which expected full-blown objects to operate on. Modifying the mapper is not really an option unless the majority of the users have the same requirements, otherwise I end up having to add a huge amount of undefer() calls everywhere else (and the query storm caused by missing an undefer() would be much more painful). Maybe the documentation of defer() could mention that the use of the option can in fact reduce performance, because it's not intuitive that loading unneeded data is cheaper than not loading it.
From: sqlalchemy@googlegroups.com [mailto:sqlalchemy@googlegroups.com] On Behalf Of Michael Bayer Sent: 11 July 2013 15:34 To: sqlalchemy@googlegroups.com Subject: Re: [sqlalchemy] The cost of defer() the path would be to figure out if the logic of a per-query defer option can somehow be linked to the attribute when it hits its normal refresh logic - if those attribute were set up as "deferred" at the mapper config level (where deferred is usually used), you wouldn't see this overhead since the deferred loader would be the default callable. But if you only need a few attributes why not just go the other way and query for those attributes directly? that would save you way more overhead than even if we removed all overhead from defer(), since the most expensive thing is all the mapper identity map logic that takes place when full entities are loaded. On Jul 11, 2013, at 4:02 AM, "Gombas, Gabor" <gabor.gom...@morganstanley.com<mailto:gabor.gom...@morganstanley.com>> wrote: Hi, I wrote a query joining a couple of tables, returning over a hundred thousand rows. Since I only needed to access a couple of the attributes of the returned objects for this specific use case, I thought to use a dozen or so Query.options(defer(...)) calls to avoid loading the unneeded columns. But to my surprise, the query became much slower. Profiling attributed almost all the extra time to set_deferred_for_local_state() and LoadDeferredColumns.__init__(): 827855 15.668 0.000 27.068 0.000 .../SQLAlchemy-0.8.1-py2.6-linux-x86_64.egg/sqlalchemy/orm/strategies.py:184(set_deferred_for_local_state) 827855 10.524 0.000 10.524 0.000 .../SQLAlchemy-0.8.1-py2.6-linux-x86_64.egg/sqlalchemy/orm/strategies.py:259(__init__) If defer() is so expensive, then it is not very useful. Would it be possible to make it cheaper? Gabor ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com<mailto:sqlalchemy+unsubscr...@googlegroups.com>. To post to this group, send email to sqlalchemy@googlegroups.com<mailto:sqlalchemy@googlegroups.com>. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com<mailto:sqlalchemy+unsubscr...@googlegroups.com>. To post to this group, send email to sqlalchemy@googlegroups.com<mailto:sqlalchemy@googlegroups.com>. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
