Re: Optimization of a Large QuerySet iteration

2011-07-27 Thread bruno desthuilliers
On Jul 26, 4:59 pm, nixlists wrote: > On Tue, Jul 26, 2011 at 7:42 AM, Roman Klesel > wrote: > > ... > > > The main question you may ask yourself may be whether or not you > > really want to have django do the whole calculation thing. Many > > database engines have very powerful aggregation capab

Re: Optimization of a Large QuerySet iteration

2011-07-27 Thread Roman Klesel
2011/7/26 nixlists : > Is it a best practice to take a bunch of code out to the stored > procedures as much as possible, or the opposite is true with Django? I don't want to comment on whether it's best practice or not ... I was thinking about efficiency: If the data processing gets too slow doi

Re: Optimization of a Large QuerySet iteration

2011-07-26 Thread nixlists
On Tue, Jul 26, 2011 at 7:42 AM, Roman Klesel wrote: ... > The main question you may ask yourself may be whether or not you > really want to have django do the whole calculation thing. Many > database engines have very powerful aggregation capabilities, support > for stored procedures, functions

Re: Optimization of a Large QuerySet iteration

2011-07-26 Thread nixlists
On Tue, Jul 26, 2011 at 9:52 AM, Tom Evans wrote: >  p_dict = dict(contract_products.values_list('product', 'wac__wac')) > > This should cut down the number of queries in the view significantly. Thanks! This is much better than what I had. -- You received this message because you are subscribed

Re: Optimization of a Large QuerySet iteration

2011-07-26 Thread Tom Evans
On Mon, Jul 25, 2011 at 4:57 AM, nixlists wrote: > Hi. I am a newbie  :) > > Is there a way to rewrite the dictionary code above to make it more > readable, and/or rewrite everything in a more efficient manner in > general? It still takes a bit of time to run (but much faster than > doing queries

Re: Optimization of a Large QuerySet iteration

2011-07-26 Thread Roman Klesel
Hello, 2011/7/25 nixlists : > Is there a way to rewrite the dictionary code above to make it more > readable, and/or rewrite everything in a more efficient manner in > general? I also consider myself a beginner in programming but I've been using django for about 1 year now. To me the code looks

Optimization of a Large QuerySet iteration

2011-07-24 Thread nixlists
Hi. I am a newbie :) This is probably not a very Django-like design at this point, but the goal is to move towards it from a legacy-type design DB. My models are currently set up something like the following (since there is quite a bit of trial-and-error database design and data import work to s

Re: queryset iteration

2009-02-04 Thread Casey
On Feb 3, 6:19 pm, Malcolm Tredinnick wrote: > "Default" iteration over a queryset (using __iter__) also caches the > instances in memory so that they can be reused without requiring extra > database queries. Normally, the memory overhead is negligible, since the > queryset is being used to put r

Re: queryset iteration

2009-02-03 Thread Malcolm Tredinnick
On Tue, 2009-02-03 at 15:06 -0800, Casey Deccio wrote: > Hi, > > I'm using django 1.0.2. I have an model with a large number of > records. I was hoping that the built-in iteration on queryset objects > would reduce the memory requirement, but it is using up all my memory > (and swap), and I'm

queryset iteration

2009-02-03 Thread Casey Deccio
Hi, I'm using django 1.0.2. I have an model with a large number of records. I was hoping that the built-in iteration on queryset objects would reduce the memory requirement, but it is using up all my memory (and swap), and I'm forced to break up the chunks manually using queryset slices. Iterat