[sqlalchemy] The cost of defer()

2013-07-11 Thread Gombas, Gabor
Hi, I wrote a query joining a couple of tables, returning over a hundred thousand rows. Since I only needed to access a couple of the attributes of the returned objects for this specific use case, I thought to use a dozen or so Query.options(defer(...)) calls to avoid loading the unneeded

Re: [sqlalchemy] The cost of defer()

2013-07-11 Thread Michael Bayer
the path would be to figure out if the logic of a per-query defer option can somehow be linked to the attribute when it hits its normal refresh logic - if those attribute were set up as deferred at the mapper config level (where deferred is usually used), you wouldn't see this overhead since

RE: [sqlalchemy] The cost of defer()

2013-07-11 Thread Gombas, Gabor
I did need the objects, not just the raw data, otherwise I'd had to duplicate a bunch of existing code which expected full-blown objects to operate on. Modifying the mapper is not really an option unless the majority of the users have the same requirements, otherwise I end up having to add a

Re: [sqlalchemy] SQLAlchemy 0.8.2 released

2013-07-11 Thread Werner
Michael, On 03/07/2013 22:20, Michael Bayer wrote: Hey all - SQLAlchemy release 0.8.2 is now available. 0.8.2 includes several dozen bug fixes and new features, including refinement of some of the new features introduced in 0.8. Areas of improvement include Core, ORM, as well as specific

[sqlalchemy] Generated update command is alway schosing alternative field names

2013-07-11 Thread Richard Gomes
hello, I've previously defined inserts and updates by hand in my application, which is working fine, not using SQLAlchemy at the moment. At this point, I'd like to employ SQLAlchemy to generate these inserts and updates for me. And that's all. I mean: just generate the queries for me. I'm *not*

[sqlalchemy] Dogpile caching: can't pickle function objects

2013-07-11 Thread Amir Elaguizy
If I do a query like this: return PcpPostModel.query.filter_by(id=post_id).options( FromCache(default) ) and then later I do another query like this: PcpPostModel.query.options(FromCache(default)).all() Any models that were returned by the first query are

Re: [sqlalchemy] SQLAlchemy 0.8.2 released

2013-07-11 Thread Jonathan Vanasco
On a tangent... I just noticed that there are several dozen (if not hundreds) of SqlAlchemy projects on PyPi Perhaps SqlAlchemy is now large enough that it should have it's own classifier ? Something like... Topic :: Database :: Front-Ends :: SqlAlchemy Topic :: Database ::

[sqlalchemy] Re: Dogpile caching: can't pickle function objects

2013-07-11 Thread Jonathan Vanasco
I serialize all my cached data into a dict or json before caching, then unserialize into whatever object i need. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: [sqlalchemy] The cost of defer()

2013-07-11 Thread Michael Bayer
well what kind of data are we talking about? defer()'s use case was for binary large objects and such, fields that are many K/Megs in size. if you're deferring a bunch of ints, then yes it's not optimized very well for that. Half of the overhead could be easily fixed here, creating those

[sqlalchemy] Join order determinism

2013-07-11 Thread Amir Elaguizy
I noticed that between runs my cache hit rate using dogpile query caching could change without any of the underlying data structures changing, after digging in what I found was the join order on my polymorphic classes is not deterministic. Is there any way to ensure a deterministic join order

Re: [sqlalchemy] The cost of defer()

2013-07-11 Thread Michael Bayer
please try out this patch: http://www.sqlalchemy.org/trac/attachment/ticket/2778/2778.patch which refactors this particular system to not require the production of a new object per instance, which is the slowest part of this, and also inlines the work of assembling the callable. This should

Re: [sqlalchemy] Generated update command is alway schosing alternative field names

2013-07-11 Thread Simon King
On Thu, Jul 11, 2013 at 4:30 PM, Richard Gomes rgomes.i...@gmail.com wrote: hello, I've previously defined inserts and updates by hand in my application, which is working fine, not using SQLAlchemy at the moment. At this point, I'd like to employ SQLAlchemy to generate these inserts and

Re: [sqlalchemy] Generated update command is alway schosing alternative field names

2013-07-11 Thread Michael Bayer
On Jul 11, 2013, at 11:30 AM, Richard Gomes rgomes.i...@gmail.com wrote: hello, I've previously defined inserts and updates by hand in my application, which is working fine, not using SQLAlchemy at the moment. At this point, I'd like to employ SQLAlchemy to generate these inserts and

Re: [sqlalchemy] Dogpile caching: can't pickle function objects

2013-07-11 Thread Michael Bayer
On Jul 11, 2013, at 11:43 AM, Amir Elaguizy aelag...@gmail.com wrote: If I do a query like this: return PcpPostModel.query.filter_by(id=post_id).options( FromCache(default) ) and then later I do another query like this:

Re: [sqlalchemy] SQLAlchemy 0.8.2 released

2013-07-11 Thread Michael Bayer
hey i like that idea. have any friends at pypi ? On Jul 11, 2013, at 1:13 PM, Jonathan Vanasco jvana...@gmail.com wrote: On a tangent... I just noticed that there are several dozen (if not hundreds) of SqlAlchemy projects on PyPi Perhaps SqlAlchemy is now large enough that it should

Re: [sqlalchemy] Dogpile caching: can't pickle function objects

2013-07-11 Thread Amir Elaguizy
Michael, Thanks for the reply. I understand what you're saying and can go search for that. I wonder if you could take a look at my question about join order determinism in polymorphic queries? Thanks, Amir On Thursday, July 11, 2013 11:09:50 AM UTC-7, Michael Bayer wrote: On Jul 11, 2013,

Re: [sqlalchemy] Join order determinism

2013-07-11 Thread Michael Bayer
when you say between runs, you mean whole new processes with new mappers, right? there are some memoized sets involved in polymorphic loading, those sets should not change order as the program runs but across runs there may be some changes in order.to improve this I'd need you to provide a

Re: [sqlalchemy] Join order determinism

2013-07-11 Thread Amir Elaguizy
Michael, That works! Amir On Thursday, July 11, 2013 11:17:27 AM UTC-7, Michael Bayer wrote: when you say between runs, you mean whole new processes with new mappers, right? there are some memoized sets involved in polymorphic loading, those sets should not change order as the program

Re: [sqlalchemy] Join order determinism

2013-07-11 Thread Michael Bayer
just that, huh. the tricky thing is its difficult to ensure that a set() doesn't find its way in there at some point and mess the order up again. open up a ticket for this one I'd need to come up with a test. On Jul 11, 2013, at 2:19 PM, Amir Elaguizy aelag...@gmail.com wrote:

Re: [sqlalchemy] Join order determinism

2013-07-11 Thread Amir Elaguizy
http://www.sqlalchemy.org/trac/ticket/2779 On Thursday, July 11, 2013 11:23:32 AM UTC-7, Michael Bayer wrote: just that, huh. the tricky thing is its difficult to ensure that a set() doesn't find its way in there at some point and mess the order up again. open up a ticket for this one

Re: [sqlalchemy] Generated update command is alway schosing alternative field names

2013-07-11 Thread Richard Gomes
Hello Michael, Thanks a lot for your help :) I've followed your directions. it works. Regarding the reserved column names (now I remember I saw this yesterday) ... it does not happen because I'm restricting the field names which appear in the SET clause, so that there's no collision between

[sqlalchemy] Re: best practice for SqlAlchemy and webapps ?

2013-07-11 Thread Jonathan Vanasco
Mike, thanks again. I finally found time to integrate your recommendations. https://github.com/jvanasco/pyramid_sqlassist -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an

[sqlalchemy] Retreiving datetime(6) value using sqlalchemy

2013-07-11 Thread Matt
Hi all, I have been writing to my database using func.now(6) for datetime(6) valued columns and it has worked perfectly. However, I've recently run into an issue with querying for these values. It would appear that every time I query for the values in datetime(6) columns it returns None.

[sqlalchemy] Re: Retreiving datetime(6) value using sqlalchemy

2013-07-11 Thread Matt
Sorry, forgot to mention this is for mysql. http://dev.mysql.com/doc/refman/5.6/en/fractional-seconds.html On Thursday, July 11, 2013 4:38:03 PM UTC-4, Matt wrote: Hi all, I have been writing to my database using func.now(6) for datetime(6) valued columns and it has worked perfectly.