Interaction of annotate() and values()

[Russell Keith-Magee]

Hi all,

I've been working my way through a couple of aggregation tickets, and
I've hit a bit of an inconsistency that I think may need to be
addressed. The problem lies in the interaction of values() and
annotate(), especially as compared to the interaction of extra() and
values().

Ordinarily, values() directs the ORM to return the raw values, rather
than an object instance. values('x','y','z') directs the ORM to return
only the named columns x,y, and z.

If the query has an extra clause(), values('x','y','z') will only
return the nominated columns. If an extra select column isn't
mentioned specifically in the values clause, it isn't included in the
result set.

However, if the query contains an annotate() clause, the annotated
value is always returned in the values() set, regardless of whether it
is mentioned in the values clause or not.

Aside from this being inconsistent, it's a little inconvenient when it
interacts with the behaviour introduced in [9701] - the ability to use
querysets as rvalues for __in queries. If you have a query with an
annotate in it, you essentially can't use it as an rvalue, because it
will always have a minimum of 2 columns, which means you can't use a
query with an annotation in an __in clause (which require a single
column).

So - the initial suggestion here is to modify ValuesQuerySet so that
annotations aren't automatically included in the result set - you
would need to specifically mention the annotate column in order for it
to be returned.

There is a slight complication, though. The ordering of values() and
annotate() is significant - values() controls the grouping of
annotated values if it precedes the annotate(). However, you can't
mention the annotated column in a values() clause before it is
specified, so are left in a situation where you can't actually return
the annotated value. For example in the query:

>>> Book.values('title','isbn').annotate(n_authors=Count('authors'))

n_authors wouldn't be returned in the result set, and there isn't any
way to add it to the returned values.

So - some options:

1) Leave things as-is. Annotated columns always appear in the result
set. This is inconsistent with extra(), and means you can't use
annotated queries in __in clauses.

2) Modify things slightly - annotate() calls before a values() call
aren't automatically added to the results, but annotate() calls after
values() are included automatically. This would allow annotations to
always be returned, optionally under those circumstances that allow
(i.e., annotate() before values()).

3) Rely on multiple calls to values() to get annotate columns into the
result set. i.e., the previous example would need to become something
like:

>>> Book.values('title','isbn').annotate(n_authors=Count('authors')).values('title','isbn','n_authors')

This has the potential to be a bit fragile - we would need to put in
all sorts of checks for changes in column lists between successive
calls to values() - but it would allow all combinations of annotation.

Opinions? Any other options I may have missed?

Yours
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Testing framework inflexibilities

[Russell Keith-Magee]

On Sat, Feb 7, 2009 at 8:17 PM, Ludvig Ericson  wrote:
>
> Hi,
> I realize this topic is in a gray zone between django-users and
> django-developers, but I thought it'd be more fitting to post it here.
>
> I needed to inject things into the session, and I'm not using regular
> sessions.
> I have my own session framework*, but the testing framework is rather
> tightly
> coupled with the built-in Django sessions.
>
> All I want to do with it - for now - is to have a simple dict as the
> session of
> the testing client.
>
> Normally, I'd just subclass django.test.TestCase, and change the part
> I need to
> change - simple programming! But this is more or less impossible with
> TestCase,
> because the part that sets up the client object is TestCase.__call__,
> which also
> contains other logics.
>
> So for me to be able to override the class used for the client object,
> I would
> have to *copy* the __call__ method from the actual .py file, paste and
> rewrite.
> IMHO that's a pretty strong indication that modularization isn't at
> its best.
> :-)
>
> I'm not sure how to modularize this either. A first step would be
> moving the
> line `self.client = Client()` from __call__ to _pre_setup. That way,
> one could
> subclass and upcall, then replace. That's virtually impossible
> currently.

First off - it isn't impossible to do what you are describing with the
existing setup. There is no reason you couldn't override _pre_setup()
in your subclass and either re-instantiate self.client, or modify the
self.client instance that has already been created. This isn't
necessarily clean, but it would work.

This area could certainly be cleaned up, though. Moving the
instantiation of Client into _pre_setup() would be one approach.
Another would be to parameterize the class that is instantiated when
the client is created - i.e., rather than always instantiating
django.test.client.Client, we provide a customization hook that lets
subclasses provide their own Client class.

Patches welcome. :-)

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Custom FilterSpecs #5833 planned for Django 1.1?

[Alex Gaynor]

On Sat, Feb 7, 2009 at 8:27 AM, Ben Gerdemann  wrote:

>
> Hello,
>
> I'd like to ask what the status of ticked #5833
> http://code.djangoproject.com/ticket/5833
> is. I see that it's listed as a "maybe" feature for 1.1. The patch is
> marked "needs improvement," but it's not clear to me from the bug
> discussion what improvement is being requested. I (gerdemb) have
> updated the original patch submitted by fas to work with Django 1.0.2
> and have included one small bug-fix. So, what is the next step?
>
> I've never contributed to an open-source project, but I've been using
> Python for several years and Django for past several months. Is there
> a way that I can help here?
>
> Also, wanted to take this opportunity to thank all the developers.
> This is my first web project using Django and it's been a joy to work
> with. :)
>
> Cheers,
> Ben
>
> >
>
A couple things, first the patch still has a pair of TODO comments, so
either those comments are no longer applicable, or what they refer to should
be fixed.  Secondly, it needs docs and tests.  I'm not a core dev so I can't
speak to whether the approach taken is the right one(I notice it changed
radically from Honza's original patches), however those items are going to
be a baseline for whatever patch is accepted.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." --Voltaire
"The people's good is the highest law."--Cicero

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Custom date formats and admin

[David Larlet]

Hello,

Today I tried to handle custom dates formats in admin and there are a  
couple of issues related to that.

First on rendering, AdminDateWidget doesn't inherit from DateTimeInput  
which seems more logic to me. I eventually update the patch #7656 and  
it works. Is there any reason against that approach before I add tests  
and documentation?

Then on validation, my first idea was to allow  
form.fields.DEFAULT_DATE/TIME_INPUT_FORMATS to be overridden by  
settings and I think it makes sense because from my experience, when  
you decide to create a website (let's say, in French) you do not want  
to pass your custom input_formats argument to all your fields. I know,  
I can add a FRDate/TimeField in localflavor but it doesn't solve the  
admin issue either. Are there drawbacks to this approach?

I know that the timing isn't that good because you are working on big  
1.1 features but I need it now and I'll work on it so feedback is  
appreciated before I submit complete patches.

Regards,
David

ps: looking for tickets on this topic, I found #8962 which is  
interesting in terms of consistency, note that the author discuss the  
interest of settings in comments too.

http://code.djangoproject.com/ticket/7656
http://code.djangoproject.com/ticket/8962


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Testing framework inflexibilities

[Ludvig Ericson]

Hi,
I realize this topic is in a gray zone between django-users and
django-developers, but I thought it'd be more fitting to post it here.

I needed to inject things into the session, and I'm not using regular  
sessions.
I have my own session framework*, but the testing framework is rather  
tightly
coupled with the built-in Django sessions.

All I want to do with it - for now - is to have a simple dict as the  
session of
the testing client.

Normally, I'd just subclass django.test.TestCase, and change the part  
I need to
change - simple programming! But this is more or less impossible with  
TestCase,
because the part that sets up the client object is TestCase.__call__,  
which also
contains other logics.

So for me to be able to override the class used for the client object,  
I would
have to *copy* the __call__ method from the actual .py file, paste and  
rewrite.
IMHO that's a pretty strong indication that modularization isn't at  
its best.
:-)

I'm not sure how to modularize this either. A first step would be  
moving the
line `self.client = Client()` from __call__ to _pre_setup. That way,  
one could
subclass and upcall, then replace. That's virtually impossible  
currently.

- Ludvig

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---